summaryrefslogtreecommitdiff
path: root/tools/buildman
AgeCommit message (Collapse)Author
2022-10-07buildman: differentiate between riscv32, riscv64Heinrich Schuchardt
riscv32 needs a different toolchain than riscv64 Signed-off-by: Heinrich Schuchardt <[email protected]> Reviewed-by: Simon Glass <[email protected]> Reviewed-by: Rick Chen <[email protected]>
2022-08-05buildman: Drop a TODO that is doneSimon Glass
Buildman now uses worktrees when available, instead of doing a full clone. This was done in this commit: 76de29fc4f buildman: Use git worktrees instead of git clones when possible Drop the TODO. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Drop a Python 2.7 commentSimon Glass
This is well out of date, but it is still reasonable to use a list. Drop the comment. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Allow lines without a symbolSimon Glass
The 'nm' tool can produce lines without a symbol, for example: 00000004 t Silently skip these and anything else without three fields. Drop the warning since there is nothing the user can do about it. Signed-off-by: Simon Glass <[email protected]> Reported-by: Tom Rini <[email protected]>
2022-08-05buildman: Return an error if there are maintainer warningsSimon Glass
Detect warnings about missing maintain info and return result code 2 in that case. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Replace the Options column with config nameSimon Glass
This appears in boards.cfg but we want to remove it. Drop support for generating it and reading it. Detect an old boards.cfg file that has this field and regenerate it, to avoid problems. Instead, add the config name in that place. This fixes a subtle bug in the generation code, since it uses 'target' for the config name and then overwrites the value in scan() by setting params['target'] to the name of the defconfig. The defconfig name is not the same as the SYS_CONFIG_NAME variable. With this change, we still have the config name and it can be searched by buildman, e.g. with: buildman -nv sun5i Signed-off-by: Simon Glass <[email protected]> Reported-by: Tom Rini <[email protected]>
2022-08-05buildman: Tidy up pylint problems in boards moduleSimon Glass
Fix all the pylint warnings. Also tidy up the comments so that they show type information, as required. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Incorporate the genboardscfg.py toolSimon Glass
Bring this tool into buildman, so we don't have to run it separately. The board.cfg file is still produced as part of the build, to save time when doing another build in the same working directory. If it is out of date with respect to the Kconfig, it is updated. Time to regenerate on a recent single-thread machine is 4.6s (1.3s on a 32-thread machine), so we do need some sort of cache if we want buildman to be useful on incremental builds. We could use Python's pickle format but: - it seems useful to allow boards.cfg to be regenerated, at least for a while, in case other tools use it - it is possible to grep the file easily, e.g. to find boards which use a particular SoC (similar to 'buildman -nv <soc>' Signed-off-by: Simon Glass <[email protected]> Suggested-by: Tom Rini <[email protected]>
2022-08-05buildman: Split out Boards into its own fileSimon Glass
Use a separate file for the Boards class so that its name matches the module name. Fix up the function names to match the pylint style and fix some other warnings. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Convert camel case in board.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Drop use of 'board' in board moduleSimon Glass
Use brds instead so that we can reserve 'boards' and 'board' as module names. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Drop use of 'boards' in controlSimon Glass
Use brds instead so that we can reserve 'boards' for a module name. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Fix use of 'boards' in testSimon Glass
We want to create a module called 'boards' so avoid use of this variable name in this module. Change the global to be capitalised, as required by Python style. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Fix use of 'boards' in func_testSimon Glass
We want to create a module called 'boards' so avoid use of this variable name in this module. Change the global to be capitalised, as required by Python style. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Avoid using board as a variableSimon Glass
We have a module called 'board'. Sometimes buildman uses 'brd' as an instance variable but sometimes it uses 'board', which is confusing and can mess with the module handling. Update the code to use 'brd' consistently, making it easier for tools to determine when the module is being referenced. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Support running from an IDESimon Glass
Add a flag to allow buildman to behave properly for use from an IDE. This shows error/warning output on stderr and drops all summary and progress information. This should normally only be used when building a single board. Fix up a confusing comment for GetResultSummary() while we are here, since we want to use the Outcome object to access the unprocessed error lines from the build. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Drop -I optionSimon Glass
This has been deprecated with a notice that it will be removed after April 2021. Drop it now. Signed-off-by: Simon Glass <[email protected]>
2022-06-28patman: test_util: Use unittest text runner to print test resultsAlper Nebi Yasak
The python tools' test utilities handle printing test results, but the output is quite bare compared to an ordinary unittest run. Delegate printing the results to a unittest text runner, which gives us niceties like clear separation between each test's result and how long it took to run the test suite. Unfortunately it does not print info for skipped tests by default, but this can be handled later by a custom test result subclass. It also does not print the tool name; manually print a heading that includes the toolname so that the outputs of each tool's tests are distinguishable in the CI output. Signed-off-by: Alper Nebi Yasak <[email protected]> Reviewed-by: Simon Glass <[email protected]>
2022-04-25nds32: Remove the architectureTom Rini
As removal of nds32 has been ack'd for the Linux kernel, remove support here as well. Cc: Rick Chen <[email protected]> Signed-off-by: Tom Rini <[email protected]> Reviewed-by: Rick Chen <[email protected]>
2022-03-28Merge tag 'v2022.04-rc5' into nextTom Rini
Prepare v2022.04-rc5
2022-03-20tools: buildman: Fix doc path in warning messageJohannes Krottmayer
Fix documentation path in deprecated warning message about device driver. Signed-off-by: Johannes Krottmayer <[email protected]> Reviewed-by: Heinrich Schuchardt <[email protected]>
2022-03-02buildman: Update default config to build for sandboxSimon Glass
At present the default .buildman file written by buildman does not specify a default toolchain. Add an 'other' line so this works correctly and sandbox builds run as expected. Signed-off-by: Simon Glass <[email protected]>
2022-03-02buildman: Correct pylint errorsSimon Glass
Fix pylint errors that can be fixed and mask those that seem to be incorrect. Signed-off-by: Simon Glass <[email protected]>
2022-02-09patman: Convert camel case in terminal.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <[email protected]>
2022-02-09patman: Rename Color() method to build()Simon Glass
This method has the same name as its class which is confusing. It is also annoying when searching the code. It builds a string with a colour, so rename it to build(). Signed-off-by: Simon Glass <[email protected]>
2022-02-09patman: Rename Print() to Tprint()Simon Glass
Rename this function so that when we convert it to snake case it will not conflict with the built-in print() function. Signed-off-by: Simon Glass <[email protected]>
2022-02-09patman: Convert camel case in test_util.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <[email protected]>
2022-02-09patman: Convert camel case in gitutil.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <[email protected]>
2022-02-09patman: Convert camel case in command.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <[email protected]>
2022-02-09patman: Convert camel case in tools.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <[email protected]>
2022-02-08buildman: Allow adjusting board config on the flySimon Glass
Add a -a option to specify changes to the config before the build commences. For example buildman -a ~CONFIG_CMDLINE disables CONFIG_CMDLINE before doing the build. This makes it easier to try things out as well as to write tests without creating a new board or manually manging the .config file. Signed-off-by: Simon Glass <[email protected]>
2022-02-08buildman: Provide a hint on how to debug thread crashesSimon Glass
If a thread crashes it is helpful to try the operation again with threading disabled. Add a hint about that. Signed-off-by: Simon Glass <[email protected]>
2022-02-08buildman: Add helper functions for updating .config filesSimon Glass
At present the only straightforward way to write tests that need a slightly different configuration is to create a new board with its own configuration. This is cumbersome. It would be useful if buildman could adjust the configuration of a build on the fly. In preparation for this, add a utility library which can modify a .config file according to various parameters passed to it. Signed-off-by: Simon Glass <[email protected]>
2022-02-08buildman: Make use of test_utilSimon Glass
Use test_util to run the tests, with the ability to select a single test to run, if desired. Signed-off-by: Simon Glass <[email protected]>
2022-02-08buildman: Add a flag to control the tracebackSimon Glass
At present the full horror of the Python traceback is shown by default. It is normally only useful for debugging. Turn it off by default and add a --debug flag to enable it. Signed-off-by: Simon Glass <[email protected]>
2022-01-25patman: Tidy up the download function a littleSimon Glass
Reverse the order of the return tuple, so that the filename is first. This seems more obvious than putting the temporary directory first. Correct a bug that leaves a space on the final line. Allow the caller to control the name of the temporary directory. Signed-off-by: Simon Glass <[email protected]>
2022-01-25buildman: Move the download function to toolsSimon Glass
This function is handy for binman as well. Move it into the shared 'tools' module. Signed-off-by: Simon Glass <[email protected]>
2022-01-24tools: Add init files for Python toolsSimon Glass
Add some empty __init__ files for binman, buildman and dtoc so that pylint is able to recognise these as Python modules and produce more useful pylint output. Signed-off-by: Simon Glass <[email protected]>
2021-10-21buildman: Add gcc-11.1.0 to the directory listTom Rini
While CI has been using gcc-11.1.0 for a long time, we have not updated buildman to match. Correct this omission. Signed-off-by: Tom Rini <[email protected]>
2021-10-20buildman: Detect Kconfig loopsSimon Glass
Hex and int Kconfig options are supposed to have defaults. This is so we can configure U-Boot without having to enter particular values for the items that don't have specific values in the board's defconfig file. If this rule is not followed, then introducing a new Kconfig can produce a loop like this: Break things (BREAK_ME) [] (NEW) Error in reading or end of file. Break things (BREAK_ME) [] (NEW) Error in reading or end of file. The continues forever since buildman passes /dev/null to 'conf', and the build system just tries again. Eventually there is so much output that buildman runs out of memory. We can detect this situation by looking for a symbol (like 'BREAK_ME') which has no default (the '[]' above) and is marked as new. If this appears multiple times in the output, we know something is wrong. Add a filter function for the output which detects this situation. Allow it to return True to terminate the process. Implement this termination in cros_subprocess. With this we get a nice message: buildman --board sandbox -T0 Building current source for 1 boards (0 threads, 32 jobs per thread) sandbox: w+ sandbox +.config:66:warning: symbol value '' invalid for BREAK_ME + +Error in reading or end of file. +make[3]: *** [scripts/kconfig/Makefile:75: syncconfig] Terminated +make[2]: *** [Makefile:569: syncconfig] Terminated +make: *** [Makefile:177: sub-make] Terminated +(** did you define an int/hex Kconfig with no default? **) Signed-off-by: Simon Glass <[email protected]>
2021-10-20buildman: Write output even on fatal errorSimon Glass
At present buildman does not write any output (to the 'out' and 'err) files if the build terminates with a fatal error. This is to avoid adding lots of spam to the logs. However there are times when this is actually useful, such as when the build fails for an obscure reason such as a Kconfig loop. Update the logic to always write the output, so that the user gets a clue as to what is happening. Signed-off-by: Simon Glass <[email protected]>
2021-09-24tools: Refactor full help printingPaul Barker
Collect the code for printing the full help message of patman, buildman and binman into a single function in patman.tools. Signed-off-by: Paul Barker <[email protected]>
2021-09-04net: Rename SPL_NET_SUPPORT to SPL_NETSimon Glass
Rename this option so that CONFIG_IS_ENABLED can be used with it. Signed-off-by: Simon Glass <[email protected]>
2021-09-04mmc: Rename MMC_SUPPORT to MMCSimon Glass
Rename these options so that CONFIG_IS_ENABLED can be used with them. Signed-off-by: Simon Glass <[email protected]> Reviewed-by: Jaehoon Chung <[email protected]> [trini: Fixup some incorrect renames] Signed-off-by: Tom Rini <[email protected]>
2021-07-28Rename SPL_USB_HOST_SUPPORT to SPL_USB_HOSTSimon Glass
Rename this option so that CONFIG_IS_ENABLED can be used with it. Signed-off-by: Simon Glass <[email protected]>
2021-04-29buildman: Use bytes for the environmentSimon Glass
At present we sometimes see problems in gitlab where the environment has 0x80 characters or sequences which are not valid UTF-8. Avoid this by using bytes for the environment, both internal to buildman and when writing out the 'env' file. Add a test to make sure this works as expected. Reported-by: Marek Vasut <[email protected]> Fixes: e5fc79ea718 ("buildman: Write the environment out to an 'env' file") Signed-off-by: Simon Glass <[email protected]>
2021-04-29buildman: Handle exceptions in threads gracefullySimon Glass
There have been at least a few cases where an exception has occurred in a thread and resulted in buildman hanging: running out of disk space and getting a unicode error. Handle these by collecting a list of exceptions, printing them out and reporting failure if any are found. Add a test for this. Signed-off-by: Simon Glass <[email protected]>
2021-04-29buildman: Use common code to send an resultSimon Glass
At present the code to report a build result is duplicated. Put it in a common function to avoid this. Signed-off-by: Simon Glass <[email protected]>
2021-04-29buildman: Tidy up a few commentsSimon Glass
Add some function comments which are missing, or missing arguments. Signed-off-by: Simon Glass <[email protected]>
2021-04-06buildman: Add an encoding to the out-env fileSimon Glass
The environment may contain some unicode characters. At least that is what seemed to happen on one commit: Building current source for 1 boards (0 threads, 64 jobs per thread) 0 0 0 /1 -1 (starting) Traceback (most recent call last): File ".../tools/buildman/buildman", line 64, in <module> ret_code = control.DoBuildman(options, args) File "tools/buildman/control.py", line 372, in DoBuildman options.keep_outputs, options.verbose) File ".../tools/buildman/builder.py", line 1704, in BuildBoards results = self._single_builder.RunJob(job) File ".../tools/buildman/builderthread.py", line 526, in RunJob self._WriteResult(result, job.keep_outputs, job.work_in_output) File ".../tools//buildman/builderthread.py", line 349, in _WriteResult print('%s="%s"' % (var, env[var]), file=fd) UnicodeEncodeError: 'ascii' codec can't encode characters in position 311-312: ordinal not in range(128) The problem defies repetition with any change at all to buildman. But let's set an encoding in any case. Signed-off-by: Simon Glass <[email protected]>