u-boot.git/test, branch v2016.03-rc2

test/py: put "Starting U-Boot" into separate log section

2016-02-15T20:58:29+00:00

The initial boot of U-Boot happens within the context of the first test
that needs to access the U-Boot console when there is no existing
connection. This keeps all activity nestled within test execution, which
fits well into the pytest model. However, this mingles the U-Boot startup
logs with the execution of some test(s), which hides find the boundary
between the two.

To solve this, wrap the "Starting U-Boot" logic into a separate log
section. If the user wishes, they can simply collapse this log section
when viewing the HTML log, to concentrate purely on the test's own
interaction.

Signed-off-by: Stephen Warren 
Acked-by: Simon Glass

test/py: handle exceptions in console creation

2016-02-15T20:58:29+00:00

u_boot_console.exec_attach.get_spawn() performs two steps:
1) Spawn a process to communicate with the serial console.
2) Reset the board so that U-Boot starts running from scratch.

Currently, if an exception happens in step (2), no cleanup is performed on
the process created in step (1). That process stays running and may e.g.
hold serial port locks, or simply continue to read data from the serial
port, thus preventing it from reaching any other process that attempts to
read from the same serial port later. While there is error cleanup code in
u_boot_console_base.ensure_spawned(), this is not triggered since the
exception prevents assignment to self.p there, and hence the exception
handler has no object to operate upon in cleanup_spawn().

Solve this by enhancing u_boot_console.exec_attach.get_spawn() to clean
up any objects it has created.

In theory, u_boot_spawn.Spawn's constructor has a similar issue, so fix
this too.

Signed-off-by: Stephen Warren 
Acked-by: Simon Glass

test/py: print summary in test order

2016-02-15T20:58:28+00:00

Use lists rather than sets to record the status of tests. This causes
the test summary in the HTML file to be generated in the same order as
the tests are (or would have been) run. This makes it easier to locate
the first failed test. The log for this test might have interesting
first clues re: interaction with the environment (e.g. hardware flashing,
serial console, ...) and may help tracking down external issues.

Signed-off-by: Stephen Warren 
Acked-by: Simon Glass

test/py: fix CONFIG_SPL test

2016-02-15T20:58:28+00:00

The Python ini file parser that's used to parse .config converts all keys
to lower-case. Hence, all queries against the results must use lower-case.
Fix u_boot_console.ensure_spawned() to test CONFIG_SPL correctly, or the
connection will fail for boards that have SPL.

Signed-off-by: Stephen Warren 
Acked-by: Simon Glass

test/py: don't import pexpect

2016-02-15T20:58:27+00:00

The code replaced pexpect with custom code long ago. Don't import the
unused module so it doesn't need to be installed.

Signed-off-by: Stephen Warren 
Acked-by: Simon Glass

test/py: add docs for gdbserver and pytest options

2016-02-15T20:58:27+00:00

Add documentation describing the new --gdbserver feature, and some common
pytest options.

Signed-off-by: Stephen Warren 
Acked-by: Simon Glass

test/py: run all "ut" subtests

2016-02-15T20:58:26+00:00

Invoke each "ut"-based unit test as a separate pytest.

Now that the DM unit test runs under test/py, remove the manual shell
script that invokes it.

Signed-off-by: Stephen Warren 
Acked-by: Simon Glass 
Tested-by: Simon Glass  # v2, on sandbox

test/py: capture the entire U-Boot version at boot

2016-02-09T22:41:19+00:00

The existing regex simply ensures that the captured version string doesn't
go past the end of a line. We really want to grab as much as possible. Do
this by explicitly including a ) character at the end of the regex to
match the last character of the version test.

Signed-off-by: Stephen Warren

test/py: fix off-by-one error in spawn matching code

2016-02-09T22:41:19+00:00

A regex match object's .end() value is already the index after the match,
not the index of the last character in the match, so there's no need to
add 1 to point past the match.

Signed-off-by: Stephen Warren

test/py: HTML awesome!

2016-02-09T22:41:19+00:00

Implement three improvements to the HTML log file:
- Ability to expand/contract sections. All passing sections are contracted
  at file load time so the user can concentrate on issues requiring
  action.
- The overall status report is copied to the top of the log for easy
  access.
- Add links from the status report to the test logs, for easy navigation.

This all relies on Javascript and the jquery library. If the user doesn't
have Javascript enabled, or jquery can't be downloaded, the log should
look and behave identically to how it did before this patch.

A few notes on the diff:

- A few more 'with log.section("xxx")' were added, so that all stream
  blocks are kept within a section block for consistent HTML entity
  nesting structure. This changed indentation in a few places, making
  the diff look slightly larger.
- HTML entity IDs are cleaned up. We assign simple incrementing integer
  IDs now, rather than using mangled test names which were possibly
  invalid.
- Sections and streams now use common CSS class names (in addition to the
  current separate class names) to more easily share the new behaviour.
  This also reduces the CSS file size since rules don't need to be
  duplicated.
- An "OK" status is logged after some external command executions so that
  make and flash steps are auto-contracted at log file load time, assuming
  they passed.

Signed-off-by: Stephen Warren