Fix a typo in the docstring for run_build() where source_dir was
misspelled.

Signed-off-by: David Lechner <dlechner@baylibre.com>

Set ubconfig to None and add a check in the show_timings() function of
test/py/test.py to ensure that the global ubconfig variable was actually
initialized before access attributes.

If tests fail early, e.g. because --build failed, ubconfig may not have
been initialized yet and results in an exception in an atexit handler.
Adding this check avoids unnecessary noise in the output.

    Exception ignored in atexit callback: <function cleanup at 0x7de475ea6b60>
    Traceback (most recent call last):
    File "u-boot/test/py/conftest.py", line 669, in cleanup
        show_timings()
    File "u-boot/test/py/conftest.py", line 616, in show_timings
        if ubconfig.timing:
        ^^^^^^^^
    NameError: name 'ubconfig' is not defined

Tested-by: Mattijs Korpershoek <mkorpershoek@kernel.org> # sandbox
Reviewed-by: Mattijs Korpershoek <mkorpershoek@kernel.org>
Signed-off-by: David Lechner <dlechner@baylibre.com>

If an Unexpected exception is thrown in a test, an undefined variable
error is reported. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: 85d7dae377a ("test: Detect dead connections")
Reviewed-by: Mattijs Korpershoek <mkorpershoek@kernel.org>

If a BootFail exception is thrown in a test, it is not handled
correctly. Use the correct fixture variable 'ubman_fix' to resolve this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: d9ed4b75add ("test/py: Drop u_boot_ prefix on test files")
Reviewed-by: Mattijs Korpershoek <mkorpershoek@kernel.org>

In some cases we have alternative configuration options that supply the
same functionality, e.g CONFIG_NET and CONFIG_NET_LWIP.

Allow to specify all of them as arguments for buildconfigspec() and execute
the text if any of these is fulfilled, e.g.

    @pytest.mark.buildconfigspec('net', 'net_lwip')

Update the documentation.

Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>

Some test can only be run by a particular board in a lab, e.g. because
they are loaded with an OS image used by the test. Add a way to specify
this in tests.

Signed-off-by: Simon Glass <sjg@chromium.org>

It is sometimes tricky to figure out what modules test.py is loading
when it starts up. The result can be a silent failure with no clue as to
what when wrong.

Add a section which lists the modules loaded as well as those not
found.

Signed-off-by: Simon Glass <sjg@chromium.org>

We know this is U-Boot so the prefix serves no purpose other than to
make things longer and harder to read. Drop it and rename the files.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Mattijs Korpershoek <mkorpershoek@baylibre.com> # test_android / test_dfu

This fixture name is quite long and results in lots of verbose code.
We know this is U-Boot so the 'u_boot_' part is not necessary.

But it is also a bit of a misnomer, since it provides access to all the
information available to tests. It is not just the console.

It would be too confusing to use con as it would be confused with
config and it is probably too short.

So shorten it to 'ubman'.

Signed-off-by: Simon Glass <sjg@chromium.org>
Link: https://lore.kernel.org/u-boot/CAFLszTgPa4aT_J9h9pqeTtLCVn4x2JvLWRcWRD8NaN3uoSAtyA@mail.gmail.com/

Execution time varies widely with the existing tests. Provides a way to
produce a summary of the time taken for each test, along with a
histogram.

This is enabled with the --timing flag.

Enable it for sandbox in CI.

Example:

   Duration : Number of tests
   ======== : ========================================
       <1ms :  1
       <8ms :  1
      <20ms : # 20
      <30ms : ######## 127
      <50ms : ######################################## 582
      <75ms : ####### 102
     <100ms : ## 39
     <200ms : ##### 86
     <300ms : # 29
     <500ms : ## 42
     <750ms : # 16
      <1.0s : # 15
      <2.0s : # 23
      <3.0s :  13
      <5.0s :  9
      <7.5s :  1
     <10.0s :  6
     <20.0s :  12

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>