u-boot.git/test/py/conftest.py, branch v2025.01

test/py: Always use the current dir as the source tree

2024-12-17T15:11:43+00:00

The logic in get_details() retrieves the default source directory from
the Labgrid settings. This is convenient for interactive use, since it
allows pytests to be run from any directory and still find the source
tree.

However, it is not actually correct.

Gitlab sets the current directory to the source tree and expects that to
be used. At present it is ignored. The result is that Gitlab builds
whatever happens to be in the default source directory, ignoring the
tree it is supposed to be building.

Fix this by using the directory of the source tree, always. This is
obtained by looking at the grandparent of the conftest.py file.

Signed-off-by: Simon Glass 
Reported-by: Tom Rini 
Fixes: bf89a8f1fc2 ("test: Introduce the concept of a role")
Tested-by: Tom Rini

test: Support testing with two board-builds

2024-11-13T18:01:35+00:00

The Beagleplay board uses an SoC from the TI K3 family. This has both a
Cortex-R core and a Cortex-A core and the R core needs to come up before
the A core. In both cases we have U-Boot SPL then U-Boot proper being
used.

In practice this means we need two entirely separate builds to produce
an image.

Handle this in test.py by adding more parameters.

Signed-off-by: Simon Glass

test: Introduce the concept of a role

2024-11-13T18:01:35+00:00

In Labgrid there is the concept of a 'role', which is similar to the
U-Boot board ID in U-Boot's pytest subsystem.

The role indicates both the target and information about the U-Boot
build to use. It can also provide any amount of other configuration.
The information is obtained using the 'labgrid-client query' operation.

Using this role, all required configuration for the board is stored
within the Labgrid environment, with pytest simply querying it. This
allows connecting to boards using an interactive console, something that
isn't possible without some kind of mapping. It also means that we don't
need to replicate the pytest functionality in tbot, since Labgrid can
handle the console and kick off builds as needed.

Make use of this in tests, so that only the role is required in gitlab
and other situations. The board type and other things can be queried
as needed.

Use a new 'u-boot-test-getrole' script to obtain the requested
information.

With this it is possible to run lab tests in gitlab with just a single
'ROLE' variable for each board.

Note that, without this feature:
- interactive use of boards with Labgrid-sjg would require repeating the
  id/board in a separate configuration file
- Gitlab yaml file would need to specify both the id and board

This feature is entirely optional, however, with the code gracefully
falling back to using a separate ID and board.

Link: https://tbot.tools

Signed-off-by: Simon Glass

test: Allow connecting to a running board

2024-11-13T18:01:35+00:00

Sometimes we know that the board is already running the right software,
so provide an option to allow running of tests directly, without first
resetting the board.

This saves time when re-running a test where only the Python code is
changing.

Note that this feature is open to errors, since the user must know that
the board is in a fit state to execute tests. It is useful for repeated
iteration on a particular test, where it can save quite a bit of time.

Signed-off-by: Simon Glass

test: Detect dead connections

2024-10-15T16:24:27+00:00

When the connection to a board dies, assume it is dead forever until
some user action is taken. Skip all remaining tests. This avoids CI
runs taking an hour, with hundreds of 30-second timeouts all to no
avail.

Signed-off-by: Simon Glass

test: Create a common function to get the config

2024-10-15T16:24:27+00:00

The settings are decoded in two places. Combine them into a new
function, before (in a future patch) expanding the number of items.

Signed-off-by: Simon Glass

test: Support tests which can only be run manually

2022-10-31T15:02:44+00:00

At present we normally write tests either in Python or in C. But most
Python tests end up doing a lot of checks which would be better done in C.
Checks done in C are orders of magnitude faster and it is possible to get
full access to U-Boot's internal workings, rather than just relying on
the command line.

The model is to have a Python test set up some things and then use C code
(in a unit test) to check that they were done correctly. But we don't want
those checks to happen as part of normal test running, since each C unit
tests is dependent on the associate Python tests, so cannot run without
it.

To acheive this, add a new UT_TESTF_MANUAL flag to use with the C 'check'
tests, so that they can be skipped by default when the 'ut' command is
used. Require that tests have a name ending with '_norun', so that pytest
knows to skip them.

Signed-off-by: Simon Glass

test/py: Support --build when running tests in parallel

2022-09-12T22:06:36+00:00

At present when -n is used, all workers try to build U-Boot at once.
Add a lock to ensure that only one of them builds, with the others using
the build that is produced.

The lock file is removed on startup.

Signed-off-by: Simon Glass

test/py: Move U-Boot building into a function

2022-09-12T22:06:36+00:00

This is a lot of code in a function that is too long. Split out the
building code.

Signed-off-by: Simon Glass

test/py: Allow tests to be marked single-threaded only

2022-09-12T22:06:36+00:00

Add a new 'singlethread' marker to allow tests to be skipped when running
in parallel.

Signed-off-by: Simon Glass