From 6e5d3e6e0569f5b9df0f891b1fdab8e3dac6291e Mon Sep 17 00:00:00 2001 From: Heinrich Schuchardt Date: Sat, 3 May 2025 12:12:37 +0200 Subject: doc: u-boot-test-reset: mention power cycling Using power cycling is a valid option to implement u-boot-test-reset. Reviewed-by: Mattijs Korpershoek Signed-off-by: Heinrich Schuchardt --- doc/develop/py_testing.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'doc/develop') diff --git a/doc/develop/py_testing.rst b/doc/develop/py_testing.rst index 217ae447035..5ad7f576ccd 100644 --- a/doc/develop/py_testing.rst +++ b/doc/develop/py_testing.rst @@ -377,7 +377,8 @@ this script again to restore U-Boot to an operational state before running the next test function. This script will likely be implemented by communicating with some form of -relay or electronic switch attached to the board's reset signal. +relay or electronic switch attached to the board's reset signal. Power cycling +is another option. The semantics of this script require that when it is executed, U-Boot will start running from scratch. If the U-Boot binary to be tested has been written -- cgit v1.2.3 From de56e5c9ebe65c1888c3b6f711223cf3a1e1038f Mon Sep 17 00:00:00 2001 From: Heinrich Schuchardt Date: Sat, 3 May 2025 12:14:24 +0200 Subject: doc: describe u-boot-test-release The scripts u-boot-test-release is called at the end of testing. Describe it. Reviewed-by: Mattijs Korpershoek Reviewed-by: Simon Glass Signed-off-by: Heinrich Schuchardt --- doc/develop/py_testing.rst | 7 +++++++ 1 file changed, 7 insertions(+) (limited to 'doc/develop') diff --git a/doc/develop/py_testing.rst b/doc/develop/py_testing.rst index 5ad7f576ccd..deea048f771 100644 --- a/doc/develop/py_testing.rst +++ b/doc/develop/py_testing.rst @@ -389,6 +389,13 @@ to download the U-Boot binary directly into RAM and execute it. This would avoid the need for `u-boot-test-flash` to actually write U-Boot to flash, thus saving wear on the flash chip(s). +u-boot-test-release +''''''''''''''''''' + +When all tests for the board have been executed, this script is called. + +The board can be switched off now. + Examples '''''''' -- cgit v1.2.3 From be2bdfd0199f37fed0ce19f90840f79d185e20d1 Mon Sep 17 00:00:00 2001 From: Tom Rini Date: Wed, 7 May 2025 16:08:17 -0600 Subject: doc: Start improving our pytest documentation Begin the work of documenting all of our pytests. To do this, we should have a directory under develop for it as there will be a large number of new files. As the current document is referenced externally in a number of locations, add the sphinx_reredirects module so that we can redirect from the old location to the new. Signed-off-by: Tom Rini Reviewed-by: Simon Glass --- doc/develop/ci_testing.rst | 2 +- doc/develop/index.rst | 2 +- doc/develop/py_testing.rst | 524 ------------------------------------------ doc/develop/pytest/index.rst | 12 + doc/develop/pytest/usage.rst | 524 ++++++++++++++++++++++++++++++++++++++++++ doc/develop/testing.rst | 4 +- doc/develop/tests_writing.rst | 2 +- 7 files changed, 541 insertions(+), 529 deletions(-) delete mode 100644 doc/develop/py_testing.rst create mode 100644 doc/develop/pytest/index.rst create mode 100644 doc/develop/pytest/usage.rst (limited to 'doc/develop') diff --git a/doc/develop/ci_testing.rst b/doc/develop/ci_testing.rst index ffaacedc3d8..2033e3447e9 100644 --- a/doc/develop/ci_testing.rst +++ b/doc/develop/ci_testing.rst @@ -18,7 +18,7 @@ be built with mingw to run on Windows. Each of the pipelines is written in such as way as to be a "world build" style test and as such we try and build all possible platforms. In addition, for all platforms that support being run in QEMU we run them in QEMU and use our pytest -suite. See :doc:`py_testing` for more information about those tests. +suite. See :doc:`pytest/usage` for more information about those tests. Azure Pipelines --------------- diff --git a/doc/develop/index.rst b/doc/develop/index.rst index c907f8c9c2c..0c83ef109ab 100644 --- a/doc/develop/index.rst +++ b/doc/develop/index.rst @@ -82,7 +82,7 @@ Testing :maxdepth: 1 testing - py_testing + pytest/index tests_writing tests_sandbox binman_tests diff --git a/doc/develop/py_testing.rst b/doc/develop/py_testing.rst deleted file mode 100644 index deea048f771..00000000000 --- a/doc/develop/py_testing.rst +++ /dev/null @@ -1,524 +0,0 @@ -U-Boot pytest suite -=================== - -Introduction ------------- - -This tool aims to test U-Boot by executing U-Boot shell commands using the -console interface. A single top-level script exists to execute or attach to the -U-Boot console, run the entire script of tests against it, and summarize the -results. Advantages of this approach are: - -- Testing is performed in the same way a user or script would interact with - U-Boot; there can be no disconnect. -- There is no need to write or embed test-related code into U-Boot itself. - It is asserted that writing test-related code in Python is simpler and more - flexible than writing it all in C. But see :doc:`tests_writing` for caveats - and more discussion / analysis. -- It is reasonably simple to interact with U-Boot in this way. - -Requirements ------------- - -The test suite is implemented using pytest. Interaction with the U-Boot console -involves executing some binary and interacting with its stdin/stdout. You will -need to implement various "hook" scripts that are called by the test suite at -the appropriate time. - -In order to run the test suite at a minimum we require that both Python 3 and -pip for Python 3 are installed. All of the required python modules are -described in the requirements.txt file in the /test/py/ directory and can be -installed via the command - -.. code-block:: bash - - pip install -r requirements.txt - -In order to execute certain tests on their supported platforms other tools -will be required. The following is an incomplete list: - -* gdisk -* dfu-util -* dtc -* openssl -* e2fsprogs -* util-linux -* coreutils -* dosfstools -* efitools -* mount -* mtools -* sbsigntool -* udisks2 - -Please use the appropriate commands for your distribution to match these tools -up with the package that provides them. - -The test script supports either: - -- Executing a sandbox port of U-Boot on the local machine as a sub-process, - and interacting with it over stdin/stdout. -- Executing an external "hook" scripts to flash a U-Boot binary onto a - physical board, attach to the board's console stream, and reset the board. - Further details are described later. - -The usage of the command 'sudo' is not allowed in tests. Using elevated -priviledges can lead to security concerns. Furthermore not all users may have -administrator rights. Therefore the command 'sudo' must not be used in tests. -To create disk images we have helper functions located in -`test/py/tests/fs_helper.py` which shall be used in any tests that require -creating disk images. - -Using a Python sandbox to provide requirements -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The recommended way to run the test suite, in order to ensure reproducibility -is to use a Python sandbox such as `python -m venv` to set up the necessary -environment. This can be done via the following commands: - - -.. code-block:: console - - $ cd /path/to/u-boot - $ sudo apt-get install python3 python3-venv - $ python3 -m venv venv - $ . ./venv/bin/activate - $ pip install -r test/py/requirements.txt - -Testing sandbox ---------------- - -To run the test suite on the sandbox port (U-Boot built as a native user-space -application), simply execute: - -.. code-block:: bash - - ./test/py/test.py --bd sandbox --build - -The `--bd` option tells the test suite which board type is being tested. This -lets the test suite know which features the board has, and hence exactly what -can be tested. - -The `--build` option tells U-Boot to compile U-Boot. Alternatively, you may -omit this option and build U-Boot yourself, in whatever way you choose, before -running the test script. - -The test script will attach to U-Boot, execute all valid tests for the board, -then print a summary of the test process. A complete log of the test session -will be written to `${build_dir}/test-log.html`. This is best viewed in a web -browser, but may be read directly as plain text, perhaps with the aid of the -`html2text` utility. - -If sandbox crashes (e.g. with a segfault) you will see message like this:: - - - test/py/spawn.py:171: in expect - c = os.read(self.fd, 1024).decode(errors='replace') - E ValueError: U-Boot exited with signal 11 (Signals.SIGSEGV) - - -Controlling output -~~~~~~~~~~~~~~~~~~ - -By default a short backtrace is reported. If you would like a longer one, -pass ``--tb=long`` when running the test. See the pytest documentation for -more options. - -Running tests in parallel -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Note: Not all tests can run in parallel at present, so the usual approach is -to just run those that can. - -First install support for parallel tests:: - - sudo apt install python3-pytest-xdist - -or::: - - pip3 install pytest-xdist - -Then run the tests in parallel using the -n flag:: - - test/py/test.py -B sandbox --build --build-dir /tmp/b/sandbox -q -k \ - 'not slow and not bootstd and not spi_flash' -n16 - -You can also use `make pcheck` to run all tests in parallel. This uses a maximum -of 16 threads, since the setup time is significant and there are under 1000 -tests. - -Note that the `test-log.html` output does not work correctly at present with -parallel testing. All the threads write to it at once, so it is garbled. - -Note that the `tools/` tests still run each tool's tests once after the other, -although within that, they do run in parallel. So for example, the buildman -tests run in parallel, then the binman tests run in parallel. There would be a -significant advantage to running them all in parallel together, but that would -require a large amount of refactoring, e.g. with more use of pytest fixtures. -The code-coverage tests are omitted since they cannot run in parallel due to a -Python limitation. - - -Testing under a debugger -~~~~~~~~~~~~~~~~~~~~~~~~ - -If you need to run sandbox under a debugger, you may pass the command-line -option `--gdbserver COMM`. This causes two things to happens: - -- Instead of running U-Boot directly, it will be run under gdbserver, with - debug communication via the channel `COMM`. You can attach a debugger to the - sandbox process in order to debug it. See `man gdbserver` and the example - below for details of valid values for `COMM`. -- All timeouts in tests are disabled, allowing U-Boot an arbitrary amount of - time to execute commands. This is useful if U-Boot is stopped at a breakpoint - during debugging. - -A usage example is: - -Window 1: - -.. code-block:: bash - - ./test/py/test.py --bd sandbox --gdbserver localhost:1234 - -Window 2: - -.. code-block:: bash - - gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234' - -Alternatively, you could leave off the `-ex` option and type the command -manually into gdb once it starts. - -You can use any debugger you wish, as long as it speaks the gdb remote -protocol, or any graphical wrapper around gdb. - -Some tests deliberately cause the sandbox process to exit, e.g. to test the -reset command, or sandbox's CTRL-C handling. When this happens, you will need -to attach the debugger to the new sandbox instance. If these tests are not -relevant to your debugging session, you can skip them using pytest's -k -command-line option; see the next section. - -Command-line options --------------------- - ---board-type, --bd, -B - set the type of the board to be tested. For example, `sandbox` or `seaboard`. - ---board-identity`, --id - sets the identity of the board to be tested. This allows differentiation - between multiple instances of the same type of physical board that are - attached to the same host machine. This parameter is not interpreted by th - test script in any way, but rather is simply passed to the hook scripts - described below, and may be used in any site-specific way deemed necessary. - ---build - indicates that the test script should compile U-Boot itself before running - the tests. If using this option, make sure that any environment variables - required by the build process are already set, such as `$CROSS_COMPILE`. - ---buildman - indicates that `--build` should use buildman to build U-Boot. There is no need - to set $CROSS_COMPILE` in this case since buildman handles it. - ---build-dir - sets the directory containing the compiled U-Boot binaries. If omitted, this - is `${source_dir}/build-${board_type}`. - ---result-dir - sets the directory to write results, such as log files, into. - If omitted, the build directory is used. - ---persistent-data-dir - sets the directory used to store persistent test data. This is test data that - may be re-used across test runs, such as file-system images. - ---timing - shows a histogram of test duration, at the end of the run. The columns are: - - Duration - the duration-bucket that this test was in - - Total - total time of all tests in this bucket - - Number of tests - graph showing the number of tests in this bucket, with the actual number - shown at the end - - Example:: - - Duration : Total | Number of tests - ======== : ======= |======================================== - <20ms : 418ms |## 23 - <30ms : 9.1s |######################################## 347 - <40ms : 10.0s |################################# 294 - <50ms : 3.1s |####### 69 - <75ms : 2.6s |#### 43 - <100ms : 1.7s |## 19 - <200ms : 3.0s |## 22 - <300ms : 1.7s | 7 - <400ms : 675ms | 2 - <500ms : 2.2s | 5 - <750ms : 8.3s |# 13 - <1.0s : 1.6s | 2 - <2.0s : 9.4s | 7 - <3.0s : 2.4s | 1 - <7.5s : 6.1s | 1 - -`pytest` also implements a number of its own command-line options. Commonly used -options are mentioned below. Please see `pytest` documentation for complete -details. Execute `py.test --version` for a brief summary. Note that U-Boot's -test.py script passes all command-line arguments directly to `pytest` for -processing. - --k - selects which tests to run. The default is to run all known tests. This - option takes a single argument which is used to filter test names. Simple - logical operators are supported. For example: - - - `'-k ums'` runs only tests with "ums" in their name. - - `'-k ut_dm'` runs only tests with "ut_dm" in their name. Note that in this - case, "ut_dm" is a parameter to a test rather than the test name. The full - test name is e.g. "test_ut[ut_dm_leak]". - - `'-k not reset'` runs everything except tests with "reset" in their name. - - `'-k ut or hush'` runs only tests with "ut" or "hush" in their name. - - `'-k not (ut or hush)'` runs everything except tests with "ut" or "hush" in - their name. - --s - prevents pytest from hiding a test's stdout. This allows you to see - U-Boot's console log in real time on pytest's stdout. - -Testing real hardware ---------------------- - -The tools and techniques used to interact with real hardware will vary -radically between different host and target systems, and the whims of the user. -For this reason, the test suite does not attempt to directly interact with real -hardware in any way. Rather, it executes a standardized set of "hook" scripts -via `$PATH`. These scripts implement certain actions on behalf of the test -suite. This keeps the test suite simple and isolated from system variances -unrelated to U-Boot features. - -Hook scripts -~~~~~~~~~~~~ - -Environment variables -''''''''''''''''''''' - -The following environment variables are set when running hook scripts: - -- `UBOOT_BOARD_TYPE` the board type being tested. -- `UBOOT_BOARD_IDENTITY` the board identity being tested, or `na` if none was - specified. -- `UBOOT_SOURCE_DIR` the U-Boot source directory. -- `UBOOT_TEST_PY_DIR` the full path to `test/py/` in the source directory. -- `UBOOT_BUILD_DIR` the U-Boot build directory. -- `UBOOT_RESULT_DIR` the test result directory. -- `UBOOT_PERSISTENT_DATA_DIR` the test persistent data directory. - -u-boot-test-console -''''''''''''''''''' - -This script provides access to the U-Boot console. The script's stdin/stdout -should be connected to the board's console. This process should continue to run -indefinitely, until killed. The test suite will run this script in parallel -with all other hooks. - -This script may be implemented e.g. by executing `cu`, `kermit`, `conmux`, etc. -via exec(). - -If you are able to run U-Boot under a hardware simulator such as QEMU, then -you would likely spawn that simulator from this script. However, note that -`u-boot-test-reset` may be called multiple times per test script run, and must -cause U-Boot to start execution from scratch each time. Hopefully your -simulator includes a virtual reset button! If not, you can launch the -simulator from `u-boot-test-reset` instead, while arranging for this console -process to always communicate with the current simulator instance. - -u-boot-test-flash -''''''''''''''''' - -Prior to running the test suite against a board, some arrangement must be made -so that the board executes the particular U-Boot binary to be tested. Often -this involves writing the U-Boot binary to the board's flash ROM. The test -suite calls this hook script for that purpose. - -This script should perform the entire flashing process synchronously; the -script should only exit once flashing is complete, and a board reset will -cause the newly flashed U-Boot binary to be executed. - -It is conceivable that this script will do nothing. This might be useful in -the following cases: - -- Some other process has already written the desired U-Boot binary into the - board's flash prior to running the test suite. -- The board allows U-Boot to be downloaded directly into RAM, and executed - from there. Use of this feature will reduce wear on the board's flash, so - may be preferable if available, and if cold boot testing of U-Boot is not - required. If this feature is used, the `u-boot-test-reset` script should - perform this download, since the board could conceivably be reset multiple - times in a single test run. - -It is up to the user to determine if those situations exist, and to code this -hook script appropriately. - -This script will typically be implemented by calling out to some SoC- or -board-specific vendor flashing utility. - -u-boot-test-reset -''''''''''''''''' - -Whenever the test suite needs to reset the target board, this script is -executed. This is guaranteed to happen at least once, prior to executing the -first test function. If any test fails, the test infra-structure will execute -this script again to restore U-Boot to an operational state before running the -next test function. - -This script will likely be implemented by communicating with some form of -relay or electronic switch attached to the board's reset signal. Power cycling -is another option. - -The semantics of this script require that when it is executed, U-Boot will -start running from scratch. If the U-Boot binary to be tested has been written -to flash, pulsing the board's reset signal is likely all this script needs to -do. However, in some scenarios, this script may perform other actions. For -example, it may call out to some SoC- or board-specific vendor utility in order -to download the U-Boot binary directly into RAM and execute it. This would -avoid the need for `u-boot-test-flash` to actually write U-Boot to flash, thus -saving wear on the flash chip(s). - -u-boot-test-release -''''''''''''''''''' - -When all tests for the board have been executed, this script is called. - -The board can be switched off now. - -Examples -'''''''' - -https://source.denx.de/u-boot/u-boot-test-hooks contains some working example hook -scripts, and may be useful as a reference when implementing hook scripts for -your platform. These scripts are not considered part of U-Boot itself. - -Board-type-specific configuration -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Each board has a different configuration and behaviour. Many of these -differences can be automatically detected by parsing the `.config` file in the -build directory. However, some differences can't yet be handled automatically. - -For each board, an optional Python module `u_boot_board_${board_type}` may exist -to provide board-specific information to the test script. Any global value -defined in these modules is available for use by any test function. The data -contained in these scripts must be purely derived from U-Boot source code. -Hence, these configuration files are part of the U-Boot source tree too. - -Execution environment configuration -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Each user's hardware setup may enable testing different subsets of the features -implemented by a particular board's configuration of U-Boot. For example, a -U-Boot configuration may support USB device mode and USB Mass Storage, but this -can only be tested if a USB cable is connected between the board and the host -machine running the test script. - -For each board, optional Python modules `u_boot_boardenv_${board_type}` and -`u_boot_boardenv_${board_type}_${board_identity}` may exist to provide -board-specific and board-identity-specific information to the test script. Any -global value defined in these modules is available for use by any test -function. The data contained in these is specific to a particular user's -hardware configuration. Hence, these configuration files are not part of the -U-Boot source tree, and should be installed outside of the source tree. Users -should set `$PYTHONPATH` prior to running the test script to allow these -modules to be loaded. - -Board module parameter usage -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The test scripts rely on the following variables being defined by the board -module: - -- none at present - -U-Boot `.config` feature usage -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The test scripts rely on various U-Boot `.config` features, either directly in -order to test those features, or indirectly in order to query information from -the running U-Boot instance in order to test other features. - -One example is that testing of the `md` command requires knowledge of a RAM -address to use for the test. This data is parsed from the output of the -`bdinfo` command, and hence relies on CONFIG_CMD_BDI being enabled. - -For a complete list of dependencies, please search the test scripts for -instances of: - -- `buildconfig.get(...` -- `@pytest.mark.buildconfigspec(...` -- `@pytest.mark.notbuildconfigspec(...` - -Complete invocation example -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Assuming that you have installed the hook scripts into $HOME/ubtest/bin, and -any required environment configuration Python modules into $HOME/ubtest/py, -then you would likely invoke the test script as follows: - -If U-Boot has already been built: - -.. code-block:: bash - - PATH=$HOME/ubtest/bin:$PATH \ - PYTHONPATH=${HOME}/ubtest/py/${HOSTNAME}:${PYTHONPATH} \ - ./test/py/test.py --bd seaboard - -If you want the test script to compile U-Boot for you too, then you likely -need to set `$CROSS_COMPILE` to allow this, and invoke the test script as -follows: - -.. code-block:: bash - - CROSS_COMPILE=arm-none-eabi- \ - PATH=$HOME/ubtest/bin:$PATH \ - PYTHONPATH=${HOME}/ubtest/py/${HOSTNAME}:${PYTHONPATH} \ - ./test/py/test.py --bd seaboard --build - -or, using buildman to handle it: - -.. code-block:: bash - - PATH=$HOME/ubtest/bin:$PATH \ - PYTHONPATH=${HOME}/ubtest/py/${HOSTNAME}:${PYTHONPATH} \ - ./test/py/test.py --bd seaboard --build --buildman - -Writing tests -------------- - -Please refer to the pytest documentation for details of writing pytest tests. -Details specific to the U-Boot test suite are described below. - -A test fixture named `ubman` should be used by each test function. This -provides the means to interact with the U-Boot console, and retrieve board and -environment configuration information. - -The function `ubman.run_command()` executes a shell command on the -U-Boot console, and returns all output from that command. This allows -validation or interpretation of the command output. This function validates -that certain strings are not seen on the U-Boot console. These include shell -error messages and the U-Boot sign-on message (in order to detect unexpected -board resets). See the source of `console_base.py` for a complete list of -"bad" strings. Some test scenarios are expected to trigger these strings. Use -`ubman.disable_check()` to temporarily disable checking for specific -strings. See `test_unknown_cmd.py` for an example. - -Board- and board-environment configuration values may be accessed as sub-fields -of the `ubman.config` object, for example -`ubman.config.ram_base`. - -Build configuration values (from `.config`) may be accessed via the dictionary -`ubman.config.buildconfig`, with keys equal to the Kconfig variable -names. diff --git a/doc/develop/pytest/index.rst b/doc/develop/pytest/index.rst new file mode 100644 index 00000000000..435d84fc619 --- /dev/null +++ b/doc/develop/pytest/index.rst @@ -0,0 +1,12 @@ +.. SPDX-License-Identifier: GPL-2.0+ + +pytest Framework within U-Boot +============================== + +General +------- + +.. toctree:: + :maxdepth: 1 + + usage diff --git a/doc/develop/pytest/usage.rst b/doc/develop/pytest/usage.rst new file mode 100644 index 00000000000..49d269d66a7 --- /dev/null +++ b/doc/develop/pytest/usage.rst @@ -0,0 +1,524 @@ +U-Boot pytest suite +=================== + +Introduction +------------ + +This tool aims to test U-Boot by executing U-Boot shell commands using the +console interface. A single top-level script exists to execute or attach to the +U-Boot console, run the entire script of tests against it, and summarize the +results. Advantages of this approach are: + +- Testing is performed in the same way a user or script would interact with + U-Boot; there can be no disconnect. +- There is no need to write or embed test-related code into U-Boot itself. + It is asserted that writing test-related code in Python is simpler and more + flexible than writing it all in C. But see :doc:`../tests_writing` for caveats + and more discussion / analysis. +- It is reasonably simple to interact with U-Boot in this way. + +Requirements +------------ + +The test suite is implemented using pytest. Interaction with the U-Boot console +involves executing some binary and interacting with its stdin/stdout. You will +need to implement various "hook" scripts that are called by the test suite at +the appropriate time. + +In order to run the test suite at a minimum we require that both Python 3 and +pip for Python 3 are installed. All of the required python modules are +described in the requirements.txt file in the /test/py/ directory and can be +installed via the command + +.. code-block:: bash + + pip install -r requirements.txt + +In order to execute certain tests on their supported platforms other tools +will be required. The following is an incomplete list: + +* gdisk +* dfu-util +* dtc +* openssl +* e2fsprogs +* util-linux +* coreutils +* dosfstools +* efitools +* mount +* mtools +* sbsigntool +* udisks2 + +Please use the appropriate commands for your distribution to match these tools +up with the package that provides them. + +The test script supports either: + +- Executing a sandbox port of U-Boot on the local machine as a sub-process, + and interacting with it over stdin/stdout. +- Executing an external "hook" scripts to flash a U-Boot binary onto a + physical board, attach to the board's console stream, and reset the board. + Further details are described later. + +The usage of the command 'sudo' is not allowed in tests. Using elevated +priviledges can lead to security concerns. Furthermore not all users may have +administrator rights. Therefore the command 'sudo' must not be used in tests. +To create disk images we have helper functions located in +`test/py/tests/fs_helper.py` which shall be used in any tests that require +creating disk images. + +Using a Python sandbox to provide requirements +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The recommended way to run the test suite, in order to ensure reproducibility +is to use a Python sandbox such as `python -m venv` to set up the necessary +environment. This can be done via the following commands: + + +.. code-block:: console + + $ cd /path/to/u-boot + $ sudo apt-get install python3 python3-venv + $ python3 -m venv venv + $ . ./venv/bin/activate + $ pip install -r test/py/requirements.txt + +Testing sandbox +--------------- + +To run the test suite on the sandbox port (U-Boot built as a native user-space +application), simply execute: + +.. code-block:: bash + + ./test/py/test.py --bd sandbox --build + +The `--bd` option tells the test suite which board type is being tested. This +lets the test suite know which features the board has, and hence exactly what +can be tested. + +The `--build` option tells U-Boot to compile U-Boot. Alternatively, you may +omit this option and build U-Boot yourself, in whatever way you choose, before +running the test script. + +The test script will attach to U-Boot, execute all valid tests for the board, +then print a summary of the test process. A complete log of the test session +will be written to `${build_dir}/test-log.html`. This is best viewed in a web +browser, but may be read directly as plain text, perhaps with the aid of the +`html2text` utility. + +If sandbox crashes (e.g. with a segfault) you will see message like this:: + + + test/py/spawn.py:171: in expect + c = os.read(self.fd, 1024).decode(errors='replace') + E ValueError: U-Boot exited with signal 11 (Signals.SIGSEGV) + + +Controlling output +~~~~~~~~~~~~~~~~~~ + +By default a short backtrace is reported. If you would like a longer one, +pass ``--tb=long`` when running the test. See the pytest documentation for +more options. + +Running tests in parallel +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Note: Not all tests can run in parallel at present, so the usual approach is +to just run those that can. + +First install support for parallel tests:: + + sudo apt install python3-pytest-xdist + +or::: + + pip3 install pytest-xdist + +Then run the tests in parallel using the -n flag:: + + test/py/test.py -B sandbox --build --build-dir /tmp/b/sandbox -q -k \ + 'not slow and not bootstd and not spi_flash' -n16 + +You can also use `make pcheck` to run all tests in parallel. This uses a maximum +of 16 threads, since the setup time is significant and there are under 1000 +tests. + +Note that the `test-log.html` output does not work correctly at present with +parallel testing. All the threads write to it at once, so it is garbled. + +Note that the `tools/` tests still run each tool's tests once after the other, +although within that, they do run in parallel. So for example, the buildman +tests run in parallel, then the binman tests run in parallel. There would be a +significant advantage to running them all in parallel together, but that would +require a large amount of refactoring, e.g. with more use of pytest fixtures. +The code-coverage tests are omitted since they cannot run in parallel due to a +Python limitation. + + +Testing under a debugger +~~~~~~~~~~~~~~~~~~~~~~~~ + +If you need to run sandbox under a debugger, you may pass the command-line +option `--gdbserver COMM`. This causes two things to happens: + +- Instead of running U-Boot directly, it will be run under gdbserver, with + debug communication via the channel `COMM`. You can attach a debugger to the + sandbox process in order to debug it. See `man gdbserver` and the example + below for details of valid values for `COMM`. +- All timeouts in tests are disabled, allowing U-Boot an arbitrary amount of + time to execute commands. This is useful if U-Boot is stopped at a breakpoint + during debugging. + +A usage example is: + +Window 1: + +.. code-block:: bash + + ./test/py/test.py --bd sandbox --gdbserver localhost:1234 + +Window 2: + +.. code-block:: bash + + gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234' + +Alternatively, you could leave off the `-ex` option and type the command +manually into gdb once it starts. + +You can use any debugger you wish, as long as it speaks the gdb remote +protocol, or any graphical wrapper around gdb. + +Some tests deliberately cause the sandbox process to exit, e.g. to test the +reset command, or sandbox's CTRL-C handling. When this happens, you will need +to attach the debugger to the new sandbox instance. If these tests are not +relevant to your debugging session, you can skip them using pytest's -k +command-line option; see the next section. + +Command-line options +-------------------- + +--board-type, --bd, -B + set the type of the board to be tested. For example, `sandbox` or `seaboard`. + +--board-identity`, --id + sets the identity of the board to be tested. This allows differentiation + between multiple instances of the same type of physical board that are + attached to the same host machine. This parameter is not interpreted by th + test script in any way, but rather is simply passed to the hook scripts + described below, and may be used in any site-specific way deemed necessary. + +--build + indicates that the test script should compile U-Boot itself before running + the tests. If using this option, make sure that any environment variables + required by the build process are already set, such as `$CROSS_COMPILE`. + +--buildman + indicates that `--build` should use buildman to build U-Boot. There is no need + to set $CROSS_COMPILE` in this case since buildman handles it. + +--build-dir + sets the directory containing the compiled U-Boot binaries. If omitted, this + is `${source_dir}/build-${board_type}`. + +--result-dir + sets the directory to write results, such as log files, into. + If omitted, the build directory is used. + +--persistent-data-dir + sets the directory used to store persistent test data. This is test data that + may be re-used across test runs, such as file-system images. + +--timing + shows a histogram of test duration, at the end of the run. The columns are: + + Duration + the duration-bucket that this test was in + + Total + total time of all tests in this bucket + + Number of tests + graph showing the number of tests in this bucket, with the actual number + shown at the end + + Example:: + + Duration : Total | Number of tests + ======== : ======= |======================================== + <20ms : 418ms |## 23 + <30ms : 9.1s |######################################## 347 + <40ms : 10.0s |################################# 294 + <50ms : 3.1s |####### 69 + <75ms : 2.6s |#### 43 + <100ms : 1.7s |## 19 + <200ms : 3.0s |## 22 + <300ms : 1.7s | 7 + <400ms : 675ms | 2 + <500ms : 2.2s | 5 + <750ms : 8.3s |# 13 + <1.0s : 1.6s | 2 + <2.0s : 9.4s | 7 + <3.0s : 2.4s | 1 + <7.5s : 6.1s | 1 + +`pytest` also implements a number of its own command-line options. Commonly used +options are mentioned below. Please see `pytest` documentation for complete +details. Execute `py.test --version` for a brief summary. Note that U-Boot's +test.py script passes all command-line arguments directly to `pytest` for +processing. + +-k + selects which tests to run. The default is to run all known tests. This + option takes a single argument which is used to filter test names. Simple + logical operators are supported. For example: + + - `'-k ums'` runs only tests with "ums" in their name. + - `'-k ut_dm'` runs only tests with "ut_dm" in their name. Note that in this + case, "ut_dm" is a parameter to a test rather than the test name. The full + test name is e.g. "test_ut[ut_dm_leak]". + - `'-k not reset'` runs everything except tests with "reset" in their name. + - `'-k ut or hush'` runs only tests with "ut" or "hush" in their name. + - `'-k not (ut or hush)'` runs everything except tests with "ut" or "hush" in + their name. + +-s + prevents pytest from hiding a test's stdout. This allows you to see + U-Boot's console log in real time on pytest's stdout. + +Testing real hardware +--------------------- + +The tools and techniques used to interact with real hardware will vary +radically between different host and target systems, and the whims of the user. +For this reason, the test suite does not attempt to directly interact with real +hardware in any way. Rather, it executes a standardized set of "hook" scripts +via `$PATH`. These scripts implement certain actions on behalf of the test +suite. This keeps the test suite simple and isolated from system variances +unrelated to U-Boot features. + +Hook scripts +~~~~~~~~~~~~ + +Environment variables +''''''''''''''''''''' + +The following environment variables are set when running hook scripts: + +- `UBOOT_BOARD_TYPE` the board type being tested. +- `UBOOT_BOARD_IDENTITY` the board identity being tested, or `na` if none was + specified. +- `UBOOT_SOURCE_DIR` the U-Boot source directory. +- `UBOOT_TEST_PY_DIR` the full path to `test/py/` in the source directory. +- `UBOOT_BUILD_DIR` the U-Boot build directory. +- `UBOOT_RESULT_DIR` the test result directory. +- `UBOOT_PERSISTENT_DATA_DIR` the test persistent data directory. + +u-boot-test-console +''''''''''''''''''' + +This script provides access to the U-Boot console. The script's stdin/stdout +should be connected to the board's console. This process should continue to run +indefinitely, until killed. The test suite will run this script in parallel +with all other hooks. + +This script may be implemented e.g. by executing `cu`, `kermit`, `conmux`, etc. +via exec(). + +If you are able to run U-Boot under a hardware simulator such as QEMU, then +you would likely spawn that simulator from this script. However, note that +`u-boot-test-reset` may be called multiple times per test script run, and must +cause U-Boot to start execution from scratch each time. Hopefully your +simulator includes a virtual reset button! If not, you can launch the +simulator from `u-boot-test-reset` instead, while arranging for this console +process to always communicate with the current simulator instance. + +u-boot-test-flash +''''''''''''''''' + +Prior to running the test suite against a board, some arrangement must be made +so that the board executes the particular U-Boot binary to be tested. Often +this involves writing the U-Boot binary to the board's flash ROM. The test +suite calls this hook script for that purpose. + +This script should perform the entire flashing process synchronously; the +script should only exit once flashing is complete, and a board reset will +cause the newly flashed U-Boot binary to be executed. + +It is conceivable that this script will do nothing. This might be useful in +the following cases: + +- Some other process has already written the desired U-Boot binary into the + board's flash prior to running the test suite. +- The board allows U-Boot to be downloaded directly into RAM, and executed + from there. Use of this feature will reduce wear on the board's flash, so + may be preferable if available, and if cold boot testing of U-Boot is not + required. If this feature is used, the `u-boot-test-reset` script should + perform this download, since the board could conceivably be reset multiple + times in a single test run. + +It is up to the user to determine if those situations exist, and to code this +hook script appropriately. + +This script will typically be implemented by calling out to some SoC- or +board-specific vendor flashing utility. + +u-boot-test-reset +''''''''''''''''' + +Whenever the test suite needs to reset the target board, this script is +executed. This is guaranteed to happen at least once, prior to executing the +first test function. If any test fails, the test infra-structure will execute +this script again to restore U-Boot to an operational state before running the +next test function. + +This script will likely be implemented by communicating with some form of +relay or electronic switch attached to the board's reset signal. Power cycling +is another option. + +The semantics of this script require that when it is executed, U-Boot will +start running from scratch. If the U-Boot binary to be tested has been written +to flash, pulsing the board's reset signal is likely all this script needs to +do. However, in some scenarios, this script may perform other actions. For +example, it may call out to some SoC- or board-specific vendor utility in order +to download the U-Boot binary directly into RAM and execute it. This would +avoid the need for `u-boot-test-flash` to actually write U-Boot to flash, thus +saving wear on the flash chip(s). + +u-boot-test-release +''''''''''''''''''' + +When all tests for the board have been executed, this script is called. + +The board can be switched off now. + +Examples +'''''''' + +https://source.denx.de/u-boot/u-boot-test-hooks contains some working example hook +scripts, and may be useful as a reference when implementing hook scripts for +your platform. These scripts are not considered part of U-Boot itself. + +Board-type-specific configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Each board has a different configuration and behaviour. Many of these +differences can be automatically detected by parsing the `.config` file in the +build directory. However, some differences can't yet be handled automatically. + +For each board, an optional Python module `u_boot_board_${board_type}` may exist +to provide board-specific information to the test script. Any global value +defined in these modules is available for use by any test function. The data +contained in these scripts must be purely derived from U-Boot source code. +Hence, these configuration files are part of the U-Boot source tree too. + +Execution environment configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Each user's hardware setup may enable testing different subsets of the features +implemented by a particular board's configuration of U-Boot. For example, a +U-Boot configuration may support USB device mode and USB Mass Storage, but this +can only be tested if a USB cable is connected between the board and the host +machine running the test script. + +For each board, optional Python modules `u_boot_boardenv_${board_type}` and +`u_boot_boardenv_${board_type}_${board_identity}` may exist to provide +board-specific and board-identity-specific information to the test script. Any +global value defined in these modules is available for use by any test +function. The data contained in these is specific to a particular user's +hardware configuration. Hence, these configuration files are not part of the +U-Boot source tree, and should be installed outside of the source tree. Users +should set `$PYTHONPATH` prior to running the test script to allow these +modules to be loaded. + +Board module parameter usage +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The test scripts rely on the following variables being defined by the board +module: + +- none at present + +U-Boot `.config` feature usage +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The test scripts rely on various U-Boot `.config` features, either directly in +order to test those features, or indirectly in order to query information from +the running U-Boot instance in order to test other features. + +One example is that testing of the `md` command requires knowledge of a RAM +address to use for the test. This data is parsed from the output of the +`bdinfo` command, and hence relies on CONFIG_CMD_BDI being enabled. + +For a complete list of dependencies, please search the test scripts for +instances of: + +- `buildconfig.get(...` +- `@pytest.mark.buildconfigspec(...` +- `@pytest.mark.notbuildconfigspec(...` + +Complete invocation example +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Assuming that you have installed the hook scripts into $HOME/ubtest/bin, and +any required environment configuration Python modules into $HOME/ubtest/py, +then you would likely invoke the test script as follows: + +If U-Boot has already been built: + +.. code-block:: bash + + PATH=$HOME/ubtest/bin:$PATH \ + PYTHONPATH=${HOME}/ubtest/py/${HOSTNAME}:${PYTHONPATH} \ + ./test/py/test.py --bd seaboard + +If you want the test script to compile U-Boot for you too, then you likely +need to set `$CROSS_COMPILE` to allow this, and invoke the test script as +follows: + +.. code-block:: bash + + CROSS_COMPILE=arm-none-eabi- \ + PATH=$HOME/ubtest/bin:$PATH \ + PYTHONPATH=${HOME}/ubtest/py/${HOSTNAME}:${PYTHONPATH} \ + ./test/py/test.py --bd seaboard --build + +or, using buildman to handle it: + +.. code-block:: bash + + PATH=$HOME/ubtest/bin:$PATH \ + PYTHONPATH=${HOME}/ubtest/py/${HOSTNAME}:${PYTHONPATH} \ + ./test/py/test.py --bd seaboard --build --buildman + +Writing tests +------------- + +Please refer to the pytest documentation for details of writing pytest tests. +Details specific to the U-Boot test suite are described below. + +A test fixture named `ubman` should be used by each test function. This +provides the means to interact with the U-Boot console, and retrieve board and +environment configuration information. + +The function `ubman.run_command()` executes a shell command on the +U-Boot console, and returns all output from that command. This allows +validation or interpretation of the command output. This function validates +that certain strings are not seen on the U-Boot console. These include shell +error messages and the U-Boot sign-on message (in order to detect unexpected +board resets). See the source of `console_base.py` for a complete list of +"bad" strings. Some test scenarios are expected to trigger these strings. Use +`ubman.disable_check()` to temporarily disable checking for specific +strings. See `test_unknown_cmd.py` for an example. + +Board- and board-environment configuration values may be accessed as sub-fields +of the `ubman.config` object, for example +`ubman.config.ram_base`. + +Build configuration values (from `.config`) may be accessed via the dictionary +`ubman.config.buildconfig`, with keys equal to the Kconfig variable +names. diff --git a/doc/develop/testing.rst b/doc/develop/testing.rst index 9114d11ad26..aa7786c99fd 100644 --- a/doc/develop/testing.rst +++ b/doc/develop/testing.rst @@ -33,7 +33,7 @@ You can also run a selection tests in parallel with:: make pcheck All of the above use the test/run script with a paremeter to select which tests -are run. See :doc:`py_testing` for more information. +are run. See :doc:`pytest/usage` for more information. Sandbox @@ -69,7 +69,7 @@ build:: ./test/py/test.py --bd sandbox_spl --build -k test_spl -See :doc:`py_testing` for more information about the pytest suite. +See :doc:`pytest/usage` for more information about the pytest suite. See :doc:`tests_sandbox` for how to run tests directly (not through pytest). diff --git a/doc/develop/tests_writing.rst b/doc/develop/tests_writing.rst index f6f852c297d..062194659b5 100644 --- a/doc/develop/tests_writing.rst +++ b/doc/develop/tests_writing.rst @@ -376,5 +376,5 @@ An example SPL test is spl_test_load(). Writing Python tests -------------------- -See :doc:`py_testing` for brief notes how to write Python tests. You +See :doc:`pytest/usage` for brief notes how to write Python tests. You should be able to use the existing tests in test/py/tests as examples. -- cgit v1.2.3 From a865d1d25410af97376fd3c822bac560b99fd84b Mon Sep 17 00:00:00 2001 From: Tom Rini Date: Wed, 7 May 2025 16:08:18 -0600 Subject: doc: pytest: Framework for documenting tests and document test_000_version In order to easily document pytests, we need to include the autodoc extension. We also need to make sure that for building the docs, CI includes pytest and that we have PYTHONPATH configured such that it will find all of the tests and related files. Finally, we need to have our comments in the test file by in proper pydoc format in order to be included in the output. Signed-off-by: Tom Rini Reviewed-by: Simon Glass --- doc/develop/pytest/index.rst | 8 ++++++++ doc/develop/pytest/test_000_version.rst | 8 ++++++++ 2 files changed, 16 insertions(+) create mode 100644 doc/develop/pytest/test_000_version.rst (limited to 'doc/develop') diff --git a/doc/develop/pytest/index.rst b/doc/develop/pytest/index.rst index 435d84fc619..ca45e157d3b 100644 --- a/doc/develop/pytest/index.rst +++ b/doc/develop/pytest/index.rst @@ -10,3 +10,11 @@ General :maxdepth: 1 usage + +Individual tests +---------------- + +.. toctree:: + :maxdepth: 1 + + test_000_version diff --git a/doc/develop/pytest/test_000_version.rst b/doc/develop/pytest/test_000_version.rst new file mode 100644 index 00000000000..09e9a361729 --- /dev/null +++ b/doc/develop/pytest/test_000_version.rst @@ -0,0 +1,8 @@ +test_000_version +================ + +.. automodule:: test_000_version + :synopsis: + :member-order: bysource + :members: + :undoc-members: -- cgit v1.2.3 From 8f8f0f0ed9d0f9d4ac385da735cef1eedd891499 Mon Sep 17 00:00:00 2001 From: Tom Rini Date: Wed, 7 May 2025 16:08:19 -0600 Subject: doc: pytest: Document the test_net_boot test Add the test_net_boot.py test to the generated documentation. While most of this was already commented correctly for inclusion the biggest problem was examples of code without a code-block notation. This in turn broke parsing. Add the missing notations. We also must have the comment prior to any import lines or it will not be seen as a comment on the overall file and thus not included. Signed-off-by: Tom Rini Reviewed-by: Simon Glass --- doc/develop/pytest/index.rst | 1 + doc/develop/pytest/test_net_boot.rst | 8 ++++++++ 2 files changed, 9 insertions(+) create mode 100644 doc/develop/pytest/test_net_boot.rst (limited to 'doc/develop') diff --git a/doc/develop/pytest/index.rst b/doc/develop/pytest/index.rst index ca45e157d3b..36083882584 100644 --- a/doc/develop/pytest/index.rst +++ b/doc/develop/pytest/index.rst @@ -18,3 +18,4 @@ Individual tests :maxdepth: 1 test_000_version + test_net_boot diff --git a/doc/develop/pytest/test_net_boot.rst b/doc/develop/pytest/test_net_boot.rst new file mode 100644 index 00000000000..369bd3b809c --- /dev/null +++ b/doc/develop/pytest/test_net_boot.rst @@ -0,0 +1,8 @@ +test_net_boot +============= + +.. automodule:: test_net_boot + :synopsis: + :member-order: bysource + :members: + :undoc-members: -- cgit v1.2.3 From 45d325ba0993f60b7b81bae1efde0bc324d49400 Mon Sep 17 00:00:00 2001 From: Tom Rini Date: Wed, 7 May 2025 17:23:00 -0600 Subject: doc: pytest: Document the test_net test Add this test to the documentation. While the diff appears large at first, the only changes within the test are to move the imports to follow the pydoc comment and then to code-block and indent the example configuration. Signed-off-by: Tom Rini Reviewed-by: Simon Glass --- doc/develop/pytest/index.rst | 1 + doc/develop/pytest/test_net.rst | 8 ++++++++ 2 files changed, 9 insertions(+) create mode 100644 doc/develop/pytest/test_net.rst (limited to 'doc/develop') diff --git a/doc/develop/pytest/index.rst b/doc/develop/pytest/index.rst index 36083882584..e2131a2f4a6 100644 --- a/doc/develop/pytest/index.rst +++ b/doc/develop/pytest/index.rst @@ -18,4 +18,5 @@ Individual tests :maxdepth: 1 test_000_version + test_net test_net_boot diff --git a/doc/develop/pytest/test_net.rst b/doc/develop/pytest/test_net.rst new file mode 100644 index 00000000000..571179f7b9d --- /dev/null +++ b/doc/develop/pytest/test_net.rst @@ -0,0 +1,8 @@ +test_net +======== + +.. automodule:: test_net + :synopsis: + :member-order: bysource + :members: + :undoc-members: -- cgit v1.2.3 From 58b2f0895bcbe24aad86c14813911f04b35cc5fb Mon Sep 17 00:00:00 2001 From: Tom Rini Date: Wed, 7 May 2025 17:23:01 -0600 Subject: doc: pytest: Document the test_bootstage test Add this test to the documentation. We need to move the import to follow the main comment so that it renders correctly, and add a code-block annotation to the example and indent it correctly. Next, neither of the functions had comments themselves, so document them now. Signed-off-by: Tom Rini Reviewed-by: Simon Glass --- doc/develop/pytest/index.rst | 1 + doc/develop/pytest/test_bootstage.rst | 8 ++++++++ 2 files changed, 9 insertions(+) create mode 100644 doc/develop/pytest/test_bootstage.rst (limited to 'doc/develop') diff --git a/doc/develop/pytest/index.rst b/doc/develop/pytest/index.rst index e2131a2f4a6..5bcda1f0952 100644 --- a/doc/develop/pytest/index.rst +++ b/doc/develop/pytest/index.rst @@ -18,5 +18,6 @@ Individual tests :maxdepth: 1 test_000_version + test_bootstage test_net test_net_boot diff --git a/doc/develop/pytest/test_bootstage.rst b/doc/develop/pytest/test_bootstage.rst new file mode 100644 index 00000000000..f8f10e96a42 --- /dev/null +++ b/doc/develop/pytest/test_bootstage.rst @@ -0,0 +1,8 @@ +test_bootstage +============== + +.. automodule:: test_bootstage + :synopsis: + :member-order: bysource + :members: + :undoc-members: -- cgit v1.2.3 From 6c3c08c4789b482e810dc4b60bfee914b063b232 Mon Sep 17 00:00:00 2001 From: Tom Rini Date: Wed, 7 May 2025 17:23:02 -0600 Subject: doc: pytest: Document the test_efi_loader test Add this test to the documentation. We need to add a code-block annotation to the example and indent it correctly. We also need to document the do_test_efi_helloworld_net function and that in turn means changing the documentation to test_efi_helloworld_net_http and test_efi_helloworld_net_tftp to reflect what is and isn't done in those functions themselves now. Signed-off-by: Tom Rini Reviewed-by: Simon Glass --- doc/develop/pytest/index.rst | 1 + doc/develop/pytest/test_efi_loader.rst | 8 ++++++++ 2 files changed, 9 insertions(+) create mode 100644 doc/develop/pytest/test_efi_loader.rst (limited to 'doc/develop') diff --git a/doc/develop/pytest/index.rst b/doc/develop/pytest/index.rst index 5bcda1f0952..3265eb97c7e 100644 --- a/doc/develop/pytest/index.rst +++ b/doc/develop/pytest/index.rst @@ -19,5 +19,6 @@ Individual tests test_000_version test_bootstage + test_efi_loader test_net test_net_boot diff --git a/doc/develop/pytest/test_efi_loader.rst b/doc/develop/pytest/test_efi_loader.rst new file mode 100644 index 00000000000..da2fa7e6ec2 --- /dev/null +++ b/doc/develop/pytest/test_efi_loader.rst @@ -0,0 +1,8 @@ +test_efi_loader +=============== + +.. automodule:: test_efi_loader + :synopsis: + :member-order: bysource + :members: + :undoc-members: -- cgit v1.2.3 From 4964cc9caa885dbcdc028ecff5e86678ef3ea8af Mon Sep 17 00:00:00 2001 From: Tom Rini Date: Thu, 8 May 2025 15:34:43 -0600 Subject: doc: pytest: Document the test_bind test Add this test to the documentation. None of the functions had comments, so attempt to explain what each does. Signed-off-by: Tom Rini Reviewed-by: Simon Glass --- doc/develop/pytest/index.rst | 1 + doc/develop/pytest/test_bind.rst | 8 ++++++++ 2 files changed, 9 insertions(+) create mode 100644 doc/develop/pytest/test_bind.rst (limited to 'doc/develop') diff --git a/doc/develop/pytest/index.rst b/doc/develop/pytest/index.rst index 3265eb97c7e..cd6eb5f038f 100644 --- a/doc/develop/pytest/index.rst +++ b/doc/develop/pytest/index.rst @@ -18,6 +18,7 @@ Individual tests :maxdepth: 1 test_000_version + test_bind test_bootstage test_efi_loader test_net diff --git a/doc/develop/pytest/test_bind.rst b/doc/develop/pytest/test_bind.rst new file mode 100644 index 00000000000..0f636a9cc5a --- /dev/null +++ b/doc/develop/pytest/test_bind.rst @@ -0,0 +1,8 @@ +test_bind +========= + +.. automodule:: test_bind + :synopsis: + :member-order: bysource + :members: + :undoc-members: -- cgit v1.2.3 From e1076c6619194ddd82f8e364d8cafd907da40616 Mon Sep 17 00:00:00 2001 From: Tom Rini Date: Thu, 8 May 2025 15:34:44 -0600 Subject: doc: pytest: Document the test_bootmenu test Add this test to the documentation. There was already a function comment that included the argument, so convert it to the right style to be rendered correctly in output. Signed-off-by: Tom Rini Reviewed-by: Simon Glass --- doc/develop/pytest/index.rst | 1 + doc/develop/pytest/test_bootmenu.rst | 8 ++++++++ 2 files changed, 9 insertions(+) create mode 100644 doc/develop/pytest/test_bootmenu.rst (limited to 'doc/develop') diff --git a/doc/develop/pytest/index.rst b/doc/develop/pytest/index.rst index cd6eb5f038f..4d949cfd1d2 100644 --- a/doc/develop/pytest/index.rst +++ b/doc/develop/pytest/index.rst @@ -19,6 +19,7 @@ Individual tests test_000_version test_bind + test_bootmenu test_bootstage test_efi_loader test_net diff --git a/doc/develop/pytest/test_bootmenu.rst b/doc/develop/pytest/test_bootmenu.rst new file mode 100644 index 00000000000..e2ff5c4c201 --- /dev/null +++ b/doc/develop/pytest/test_bootmenu.rst @@ -0,0 +1,8 @@ +test_bootmenu +============= + +.. automodule:: test_bootmenu + :synopsis: + :member-order: bysource + :members: + :undoc-members: -- cgit v1.2.3 From 02e5d344a0d75a4f05682b3438354748dc83d941 Mon Sep 17 00:00:00 2001 From: Tom Rini Date: Thu, 8 May 2025 15:34:45 -0600 Subject: doc: pytest: Document the test_button test Add this test to the documentation. No changes to the test itself were required. Signed-off-by: Tom Rini Reviewed-by: Simon Glass --- doc/develop/pytest/index.rst | 1 + doc/develop/pytest/test_button.rst | 8 ++++++++ 2 files changed, 9 insertions(+) create mode 100644 doc/develop/pytest/test_button.rst (limited to 'doc/develop') diff --git a/doc/develop/pytest/index.rst b/doc/develop/pytest/index.rst index 4d949cfd1d2..0a7c1b21a24 100644 --- a/doc/develop/pytest/index.rst +++ b/doc/develop/pytest/index.rst @@ -21,6 +21,7 @@ Individual tests test_bind test_bootmenu test_bootstage + test_button test_efi_loader test_net test_net_boot diff --git a/doc/develop/pytest/test_button.rst b/doc/develop/pytest/test_button.rst new file mode 100644 index 00000000000..4e5a29f20a7 --- /dev/null +++ b/doc/develop/pytest/test_button.rst @@ -0,0 +1,8 @@ +test_button +=========== + +.. automodule:: test_button + :synopsis: + :member-order: bysource + :members: + :undoc-members: -- cgit v1.2.3 From b043852a789ec90e60b45012bc8cda29815cb44b Mon Sep 17 00:00:00 2001 From: Heinrich Schuchardt Date: Fri, 9 May 2025 08:39:04 +0200 Subject: doc: correct link to QEMU %s/hhttps:/https:/ Reviewed-by: Quentin Schulz Signed-off-by: Heinrich Schuchardt --- doc/develop/bootstd/qfw.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/develop') diff --git a/doc/develop/bootstd/qfw.rst b/doc/develop/bootstd/qfw.rst index 70086ad1817..fdc3ff71619 100644 --- a/doc/develop/bootstd/qfw.rst +++ b/doc/develop/bootstd/qfw.rst @@ -3,7 +3,7 @@ QFW Bootmeth ============ -`QEMU `_ is a system emulator which is able to boot +`QEMU `_ is a system emulator which is able to boot Operating Systems. QEMU provides specific support for booting an OS image provided on the QEMU command line. -- cgit v1.2.3