Begin the work of documenting all of our pytests. To do this, we should
have a directory under develop for it as there will be a large number of
new files. As the current document is referenced externally in a number
of locations, add the sphinx_reredirects module so that we can redirect
from the old location to the new.

Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>

The scripts u-boot-test-release is called at the end of testing.
Describe it.

Reviewed-by: Mattijs Korpershoek <mkorpershoek@kernel.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>

Using power cycling is a valid option to implement u-boot-test-reset.

Reviewed-by: Mattijs Korpershoek <mkorpershoek@kernel.org>
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>

Using some form of sandbox with Python modules is a long standing best
practice with the language. There are a number of ways to have a Python
sandbox be created. At this point in time, it seems the Python community
is moving towards using the "venv" module provided with Python rather
than a separate tool. To match that we make the following changes:

- Refer to a "Python sandbox" rather than virtualenv in comments, etc.
- Install the python3-venv module in our container and not virtualenv.
- In our CI files, invoke "python -m venv" rather than "virtualenv".
- In documentation, tell users to install python3-venv and not
  virtualenv.

Signed-off-by: Tom Rini <trini@konsulko.com>

Now that we have no users of "virt-make-fs" nor users of "sudo" for
creating disk images update the documentation. We remove packages that
are no longer required (and related text) as well as be firm in our
wording around not using "sudo".

Signed-off-by: Tom Rini <trini@konsulko.com>

We know this is U-Boot so the prefix serves no purpose other than to
make things longer and harder to read. Drop it and rename the files.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Mattijs Korpershoek <mkorpershoek@baylibre.com> # test_android / test_dfu

This fixture name is quite long and results in lots of verbose code.
We know this is U-Boot so the 'u_boot_' part is not necessary.

But it is also a bit of a misnomer, since it provides access to all the
information available to tests. It is not just the console.

It would be too confusing to use con as it would be confused with
config and it is probably too short.

So shorten it to 'ubman'.

Signed-off-by: Simon Glass <sjg@chromium.org>
Link: https://lore.kernel.org/u-boot/CAFLszTgPa4aT_J9h9pqeTtLCVn4x2JvLWRcWRD8NaN3uoSAtyA@mail.gmail.com/

Execution time varies widely with the existing tests. Provides a way to
produce a summary of the time taken for each test, along with a
histogram.

This is enabled with the --timing flag.

Enable it for sandbox in CI.

Example:

   Duration : Number of tests
   ======== : ========================================
       <1ms :  1
       <8ms :  1
      <20ms : # 20
      <30ms : ######## 127
      <50ms : ######################################## 582
      <75ms : ####### 102
     <100ms : ## 39
     <200ms : ##### 86
     <300ms : # 29
     <500ms : ## 42
     <750ms : # 16
      <1.0s : # 15
      <2.0s : # 23
      <3.0s :  13
      <5.0s :  9
      <7.5s :  1
     <10.0s :  6
     <20.0s :  12

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>

We want tests to avoid the usage of sudo. Describe that virt-make-fs can
generate disk images without being root.

Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>

Running tests in parallel is much faster, e.g. 15 seconds to run the tests
on sandbox (only), instead of 100 seconds (on a 16-core machine). Add a
'make pcheck' option to access this feature.

Note that the tools/ tests still run each tool's tests once after the
other, although within that, they do run in parallel. So for example,
the buildman tests run in parallel, then the binman tests run in
parallel. There would be a signiificant advantage to running them all
in parallel together, but that would require a large amount of
refactoring, e.g. with more use of pytest fixtures.

Update the documentation to represent the current state.

Signed-off-by: Simon Glass <sjg@chromium.org>