summaryrefslogtreecommitdiff
path: root/tools/buildman
AgeCommit message (Collapse)Author
2025-03-04Merge patch series "tools: Minor clean-ups for the command library"Tom Rini
Simon Glass <[email protected]> says: This series adds comments and fixes pylint warnings in the command library. It also introduces a new, simpler way of running a single command. Link: https://lore.kernel.org/r/[email protected]
2025-03-04u_boot_pylib: Add a function to run a single commandSimon Glass
Add a helper to avoid needing to use a list within a list for this simple case. Update existing users of runpipe() to use this where possible. Signed-off-by: Simon Glass <[email protected]>
2025-03-04u_boot_pylib: Correct case for test_resultSimon Glass
This should be in capitals and defined at the start of the file. Update it. Signed-off-by: Simon Glass <[email protected]>
2025-02-21buildman: Update tests for newer filelock moduleSimon Glass
Recent versions of this module call time.perf_counter() so add a patch for this also. Signed-off-by: Simon Glass <[email protected]> Reported-by: Tom Rini <[email protected]>
2025-02-21tools: Fix pylint 3.3.4 errorsSimon Glass
This newer pylint produces errors about variables possibly being used before being set. Adjust the code to pass these checks. Signed-off-by: Simon Glass <[email protected]> Reported-by: Tom Rini <[email protected]>
2025-02-17u_boot_pylib: Move gitutil into the librarySimon Glass
Move this file into U-Boot's Python library, so that it is no-longer part of patman. This makes a start on: https://source.denx.de/u-boot/custodians/u-boot-dm/-/issues/35 Signed-off-by: Simon Glass <[email protected]>
2025-02-14python: Recreate test/py and tools/buildman requirements.txt filesTom Rini
Use the "pipreqs" tool to re-create these files, with a few manual corrections. We still need to include pytest-xdist which the tool does not detect. We also for now don't upgrade most of the required tools as that creates problems with various tests, which should be resolved independently. Signed-off-by: Tom Rini <[email protected]>
2025-01-25buildman: Record an error if a toolchain is missingSimon Glass
Buildman has always treated the lack of a toolchain as an infrastructure problem rather than a build failure. However the logic for this is not correct, since it does not write a 'done' file in this case. As a result, one of two things can happen. 1. If a previous build ran in the same (output) directory, the outcome of *that* build is recorded as the outcome of this one 2. Otherwise, no outcome is recorded Obviously this inconsistency is not ideal. While (2) is rare, it can be very confusing as the build sort-of fails but does not produce any summary output with 'buildman -s' Overall it seems better to attribute a toolchain issue to the boards that it affects. This results in clear failures which can be examined, no matter what happened in the .bm-work directory previously. So write a 'done' file for each build when a toolchain is missing. The end result of this patch is to make missing toolchains much more obvious. It should be things a bit easier for novice users. Signed-off-by: Simon Glass <[email protected]>
2025-01-25buildman: Drop unused OUTPUT_FILE constantSimon Glass
This is not actually used but its presence suggests that it is the filename for the board database. Drop it to avoid confusion. Signed-off-by: Simon Glass <[email protected]>
2024-11-19buildman: Support a tilde to represent the home directorySimon Glass
It is convenient to use ~ to represent the home directory in the settings file. Add support for this. Signed-off-by: Simon Glass <[email protected]>
2024-11-19buildman: Correct logic for adding a toolchainSimon Glass
If the toolchain is bad, it should not be added. Correct the logic for this. Signed-off-by: Simon Glass <[email protected]>
2024-11-19buildman: Document the toolchain-prefix sectionSimon Glass
This is mentioned in examples but should have its own mention in the 'settings' documentation. Add it. Signed-off-by: Simon Glass <[email protected]>
2024-11-19buildman: Correct the indentation in the setting-up sectionSimon Glass
The example settings file should be indented so that it shows correctly in the output. Fix it. Signed-off-by: Simon Glass <[email protected]>
2024-11-19buildman: Support #include files in defconfigsSimon Glass
This is used by some boards in U-Boot and is a convenient way to deal with common settings where using a Kconfig files is not desirable. Detect #include files and process them as if they were part of the original file. Signed-off-by: Simon Glass <[email protected]> Fixes: https://source.denx.de/u-boot/custodians/u-boot-dm/-/issues/30
2024-11-19buildman: Set up the tout librarySimon Glass
Use this to control info and debugging messages. Enable them if -v is given. Signed-off-by: Simon Glass <[email protected]>
2024-11-19buildman: Add a lower-level test for KconfigScannerSimon Glass
This code is tested by test_scan_defconfigs() but it is useful to have some specific tests for the KconfigScanner's operation in U-Boot. Add a test which checks that the values are obtained correctly. Signed-off-by: Simon Glass <[email protected]>
2024-10-18buildman: Correct some warnings about regex stringsSimon Glass
With Python 3.12 some warnings have cropped up. Fix them. Signed-off-by: Simon Glass <[email protected]>
2024-09-26buildman: Allow skipping the dtc buildSimon Glass
For most boards, the device-tree compiler is built in-tree, ignoring the system version. Add a special option to skip this build. This can be useful when the system dtc is up-to-date, as it speeds up the build. Signed-off-by: Simon Glass <[email protected]>
2024-09-26buildman: Retry the build for current sourceSimon Glass
Buildman retries a failed build when processing a branch, but does not do this when building current source. It is useful to do this retry in both cases, so add the logic for it. Signed-off-by: Simon Glass <[email protected]>
2024-09-24buildman/toolchain.py: do not set CROSS_COMPILE for sandboxJerome Forissier
When building for sandbox, self.cross is empty. In MakeEnvironment(), CROSS_COMPILE is defined to be self.cross (with or without a full path), optionally prefixed by the toolchain wrapper defined in ~/.buildman. This is fine when self.cross is not empty, but it doesn't make sense when it is: - Either there is no wrapper and we end up with an empty CROSS_COMPILE which is the same as not defining it (the host compiler will be used), - Or there is a wrapper and CROSS_COMPILE will contain only the wrapper which obviously is not a valid compiler, hence an error. Test case: $ sudo apt install ccache $ grep -q toolchain-wrapper ~/.buildman || \ printf "[toolchain-wrapper]\nwrapper = ccache\n" >>~/.buildman $ make mrproper $ ./tools/buildman/buildman sandbox_noinst $ ./tools/buildman/buildman sandbox_noinst Building current source for 1 boards (1 thread, 24 jobs per thread) sandbox: + sandbox_noinst +arch/sandbox/lib/reloc_sandbox_efi.c:10:15: error: operator '==' has no left operand + 10 | #if HOST_ARCH == HOST_ARCH_X86_64 + | ^~ [...] The GetEnvArgs function is modified too, since the VAR_CROSS_COMPILE case has the same issue. In tools/buildman/test.py, testGetEnvArgs is extended and testMakeEnvironment is added. They check the 'arm' and 'sandbox' toolchains, with and without a wrapper. Signed-off-by: Jerome Forissier <[email protected]> Reviewed-by: Simon Glass <[email protected]>
2024-09-06buildman: Support building within a Python venvSimon Glass
The Python virtualenv tool sets up a few things in the environment, putting its path first in the PATH environment variable and setting up a sys.prefix different from the sys.base_prefix value. At present buildman puts the toolchain path first in PATH so that it can be found easily during the build. For sandbox this causes problems since /usr/bin/gcc (for example) results in '/usr/bin' being prepended to the PATH variable. As a result, the venv is partially disabled. The result is that sandbox builds within a venv ignore the venv, e.g. when looking for packages. Correct this by detecting the venv and adding the toolchain path after the venv path. Signed-off-by: Simon Glass <[email protected]>
2024-08-27tools/scripts/Kconfig: Update references from kconfig-language.txt to rstHiago De Franco
The Linux kernel documentation has transitioned from using kconfig-language.txt to kconfig-language.rst. Therefore update all occurrences of kconfig-language.txt. Signed-off-by: Hiago De Franco <[email protected]>
2024-08-26buildman: Make test_process_limit handle time.monotonic()Simon Glass
Newer versions of filelock use time.monotonic() instead of time.time(). Update the test the handle this. It would be better if filelock had support for writing unit tests which use locking. Signed-off-by: Simon Glass <[email protected]>
2024-07-26buildman: Show board list with -xSimon Glass
When -x is used, buildman does not show the list of boards that will be built, since there are no terms which cause boards to be added, only terms which cause them to be removed. Add a special case to fix this. Signed-off-by: Simon Glass <[email protected]> Tested-by: Tom Rini <[email protected]>
2024-07-08Revert "buildman: Always use the full path in CROSS_COMPILE"Tom Rini
There are operations in buildman that result in running the cross-tools (such as performing size checks) and now that we have not modified PATH to know where our tools are, these operations fail. This reverts commit 6c0a3cf75f72370deec3ee516a9dd377397af207. Signed-off-by: Tom Rini <[email protected]>
2024-07-03buildman: Always use the full path in CROSS_COMPILESimon Glass
The feature to set the toolchain path does not seem to be needed. It causes problems with venv (see [1]). Let's remove it. Add some tests while we are here. It does not look like any docs changes are needed for this. [1] https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/ Signed-off-by: Simon Glass <[email protected]> Suggested-by: Tom Rini <[email protected]> Reviewed-by: Tom Rini <[email protected]> Reviewed-by: Andrejs Cainikovs <[email protected]>
2024-07-03buildman: Fix a few typos in toolchain codeSimon Glass
Fix 'Thie' and capitalise 'unicode'. Signed-off-by: Simon Glass <[email protected]> Suggested-by: Heinrich Schuchardt <[email protected]>
2024-07-03buildman: Add python3-pycryptodomeSimon Glass
This is used by some Binman entry types, so add it to allow more tests to pass. Signed-off-by: Simon Glass <[email protected]> Reviewed-by: Tom Rini <[email protected]>
2024-07-03buildman: Add python3-coverageSimon Glass
Add this package so we can run code-coverage tests for Binman. Signed-off-by: Simon Glass <[email protected]> Reviewed-by: Tom Rini <[email protected]>
2024-07-03buildman: Add a way to limit the number of buildmansSimon Glass
Buildman uses all available CPUs by default, so running more than one or two concurrent processes is not normally useful. However in some CI cases we want to be able to run several jobs at once to save time. For example, in a lab situation we may want to run a test on 20 boards at a time, since only the build step actually takes much CPU. Add an option which allows such a limit. When buildman starts up, it waits until the number of running processes goes below the limit, then claims a spot in the list. The list is maintained with a temporary file. Note that the temp file is user-specific, since it is hard to create a locked temporary file which can be accessed by any user. In most cases, only one user is running jobs on a machine, so this should not matter. Signed-off-by: Simon Glass <[email protected]>
2024-07-03buildman: Add a flag to force mrproper on failureSimon Glass
When a file is removed by a commit (e.g. include/common.h yay!) it can cause incremental build failures since one of the dependency files from a previous build may mention the file. Add an option to run 'make mrproper' automatically when a build fails. This can be used to automatically resolve the problem, without always adding the large overhead of 'make mrproper' to every build. Signed-off-by: Simon Glass <[email protected]>
2024-07-03buildman: Avoid rebuilding when --mrproper is usedSimon Glass
When this flag is enabled, 'make mrproper' is always used when reconfiguring, so there is no point in doing it again. Update this. Signed-off-by: Simon Glass <[email protected]>
2024-07-03buildman: Make mrproper an argument to run_commit()Simon Glass
Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass <[email protected]>
2024-07-03buildman: Make mrproper an argument to _config_and_build()Simon Glass
Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass <[email protected]>
2024-07-03buildman: Make mrproper an argument to _reconfigure()Simon Glass
Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass <[email protected]>
2024-07-03tools: binman: fix deprecated Python ConfigParser methodsBrandon Maier
The method `ConfigParser.readfp()` is marked deprecated[1]. In Python 3.12 this method have been removed, so replace it with `ConfigParser.read_file()`. [1] https://docs.python.org/3.11/library/configparser.html#configparser.ConfigParser.readfp Signed-off-by: Brandon Maier <[email protected]> CC: Simon Glass <[email protected]> Reviewed-by: Simon Glass <[email protected]>
2024-07-03tools: binman: fix deprecated Python unittest methodsBrandon Maier
The methods `unittest.assertEquals()` and `unittest.assertRegexpMatches()` are marked deprecated[1]. In Python 3.12 these aliases have been removed, so do a sed to replace them with their new names. [1] https://docs.python.org/3.11/library/unittest.html#deprecated-aliases Signed-off-by: Brandon Maier <[email protected]> CC: Simon Glass <[email protected]> CC: Alper Nebi Yasak <[email protected]> Reviewed-by: Simon Glass <[email protected]>
2024-05-03tools: typo arguemntsHeinrich Schuchardt
%s/arguemnts/arguemnts/ Signed-off-by: Heinrich Schuchardt <[email protected]>
2024-01-29tools: buildman: fix non-existing SafeConfigParser in Python 3.12+Quentin Schulz
SafeConfigParser was renamed back in Python 3.2 (yes, no typo) to ConfigParser[1], but it was still working as an alias until it got removed in 3.12[2]. [1] https://docs.python.org/3.8/whatsnew/3.2.html#configparser [2] https://github.com/python/cpython/pull/92503 Cc: Quentin Schulz <[email protected]> Signed-off-by: Quentin Schulz <[email protected]>
2024-01-18tools: binman: ti_board_cfg: Check for linting problemsNeha Malcom Francis
Use yamllint for checking whether YAML configuration files are adhering to default yamllint rules. Signed-off-by: Neha Malcom Francis <[email protected]> Suggested-by: Nishanth Menon <[email protected]>
2023-12-31buildman: type cotainingHeinrich Schuchardt
%s/cotaining/containing/ Signed-off-by: Heinrich Schuchardt <[email protected]> Reviewed-by: Simon Glass <[email protected]>
2023-12-13tools: Move python tools to version 0.0.6Simon Glass
A new release has been done with this version, so update it. Use the version numbers in dependencies also. Signed-off-by: Simon Glass <[email protected]>
2023-12-13doc: Update documentation URLSimon Glass
Update to use the new docs.u-boot.org URL for documentation. Signed-off-by: Simon Glass <[email protected]>
2023-11-07buildman: Use oldconfig when adjusting the configSimon Glass
We cannot be sure that the new config is consistent, particularly when changing a major item like CONFIG_CMDLINE. Use 'make oldconfig' to check that and avoid any such problems. Signed-off-by: Simon Glass <[email protected]>
2023-11-02buildman: Include symbols in the read-only data sectionSimon Glass
When symbols switch between the inited data section and the read-only data section their visbility changes, at present, with the -B option. This is confusing, since adding 'const' to a variable declaration can make it look like a significant improvement in bloat. But in fact nothing has changed. Add 'r' to the list of symbols types that are recorded, to correct this problem. Add a constant to make it easier to find this code next time. Signed-off-by: Simon Glass <[email protected]> Reported-by: Tom Rini <[email protected]> Reviewed-by: Tom Rini <[email protected]>
2023-10-02Merge branch 'next'Tom Rini
Signed-off-by: Tom Rini <[email protected]>
2023-09-23buildman: Start the clock when the build startsSimon Glass
The Kconfig and maintainer processing can take a while, sometimes 5 seconds or more. This skews the timing printed by buildmand when the build completes. Start the clock when the threads start to avoid this problem. Signed-off-by: Simon Glass <[email protected]> Suggested-by: Tom Rini <[email protected]>
2023-09-23buildman: Show progress when regenerating the board.cfg fileSimon Glass
This can take a while, so show a message when starting. Signed-off-by: Simon Glass <[email protected]> Reported-by Tom Rini <[email protected]>
2023-09-23buildman: Keep all common output filesSimon Glass
Make a list of common output extensions and use it to ensure that the -k option preserves all of these. Signed-off-by: Simon Glass <[email protected]> Suggested-by: Tom Rini <[email protected]> Reviewed-by: Tom Rini <[email protected]>
2023-09-23buildman: Fix full help for Python 3.8Simon Glass
With Python versions older than 3.9 Buildman produces an error on start-up. Fix this with a workaround for importlib. There is already a workaround for v3.6 but I am not sure if that is still functioning. Signed-off-by: Simon Glass <[email protected]>