summaryrefslogtreecommitdiff
path: root/tools/buildman/builderthread.py
AgeCommit message (Collapse)Author
2024-09-26buildman: Allow skipping the dtc buildSimon Glass
For most boards, the device-tree compiler is built in-tree, ignoring the system version. Add a special option to skip this build. This can be useful when the system dtc is up-to-date, as it speeds up the build. Signed-off-by: Simon Glass <[email protected]>
2024-09-26buildman: Retry the build for current sourceSimon Glass
Buildman retries a failed build when processing a branch, but does not do this when building current source. It is useful to do this retry in both cases, so add the logic for it. Signed-off-by: Simon Glass <[email protected]>
2024-07-08Revert "buildman: Always use the full path in CROSS_COMPILE"Tom Rini
There are operations in buildman that result in running the cross-tools (such as performing size checks) and now that we have not modified PATH to know where our tools are, these operations fail. This reverts commit 6c0a3cf75f72370deec3ee516a9dd377397af207. Signed-off-by: Tom Rini <[email protected]>
2024-07-03buildman: Always use the full path in CROSS_COMPILESimon Glass
The feature to set the toolchain path does not seem to be needed. It causes problems with venv (see [1]). Let's remove it. Add some tests while we are here. It does not look like any docs changes are needed for this. [1] https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/ Signed-off-by: Simon Glass <[email protected]> Suggested-by: Tom Rini <[email protected]> Reviewed-by: Tom Rini <[email protected]> Reviewed-by: Andrejs Cainikovs <[email protected]>
2024-07-03buildman: Add a flag to force mrproper on failureSimon Glass
When a file is removed by a commit (e.g. include/common.h yay!) it can cause incremental build failures since one of the dependency files from a previous build may mention the file. Add an option to run 'make mrproper' automatically when a build fails. This can be used to automatically resolve the problem, without always adding the large overhead of 'make mrproper' to every build. Signed-off-by: Simon Glass <[email protected]>
2024-07-03buildman: Avoid rebuilding when --mrproper is usedSimon Glass
When this flag is enabled, 'make mrproper' is always used when reconfiguring, so there is no point in doing it again. Update this. Signed-off-by: Simon Glass <[email protected]>
2024-07-03buildman: Make mrproper an argument to run_commit()Simon Glass
Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass <[email protected]>
2024-07-03buildman: Make mrproper an argument to _config_and_build()Simon Glass
Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass <[email protected]>
2024-07-03buildman: Make mrproper an argument to _reconfigure()Simon Glass
Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass <[email protected]>
2023-11-07buildman: Use oldconfig when adjusting the configSimon Glass
We cannot be sure that the new config is consistent, particularly when changing a major item like CONFIG_CMDLINE. Use 'make oldconfig' to check that and avoid any such problems. Signed-off-by: Simon Glass <[email protected]>
2023-09-23buildman: Keep all common output filesSimon Glass
Make a list of common output extensions and use it to ensure that the -k option preserves all of these. Signed-off-by: Simon Glass <[email protected]> Suggested-by: Tom Rini <[email protected]> Reviewed-by: Tom Rini <[email protected]>
2023-07-24buildman: Move copy_files() out ot BuilderThread classSimon Glass
This does not need to be in the class. Move it out to avoid a pylint warning. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Tidy up some comments in builderthreadSimon Glass
Make sure all functions have full argument documentation. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Tidy up reporting of a toolchain errorSimon Glass
Provide the text of the exception when something goes wrong. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Avoid passing result into _read_done_file()Simon Glass
Move the creating of the result object into the function which sets it up, to simplify the code. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Create a function to handle config and buildSimon Glass
Move this code into a _config_and_build() function, so reduce the size of run_commit(). Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Move checkout code to a separate functionSimon Glass
Put this in its own function to reduce the size of the run_commit() function Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Move code to decide output dirsSimon Glass
Put this in its own function to reduce the size of the run_commit() function. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Move code to remove old outputsSimon Glass
Put this in its own function to reduce the size of the run_commit() function. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Move reading of the done file into a functionSimon Glass
Move this logic into its own function to reduce the size of the run_commt() function. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Move bulid code into its own functionSimon Glass
Split this into its own function so reduce the size of run_commit(). Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Move reconfigure code into its own functionSimon Glass
Split this into its own function so reduce the size of run_commit(). Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Convert config_out to string IOSimon Glass
This is probably a little more efficient and it allows passing the object to another function to write data. Convert config_out to use a string I/O device. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Move more things into _build_args()Simon Glass
Move more of the argument-building code into this function. Fix a missing assignment for out_rel_dir too. Rename the function since it now builds all the arguments. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Move setting of toolchain arguments to _build_args()Simon Glass
Move a few more pieces to this new function. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Start a function to set up the make argumentsSimon Glass
Move some of this code into a new funciion, to help reduce the size of the run_commits() function. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Drop unnecessary assignment of config_outSimon Glass
This is already set up earlier in the function, so drop the extra assignment. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Correct invalid use of out_dir variableSimon Glass
This variable has a different meaning in the outer scope. Use a different name to avoid confusion, or bugs. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Export _get_output_dir() to avoid warningsSimon Glass
Make this a public memory since it is used outside the class. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Correct most pylint warnings in builderthreadSimon Glass
Fix the easy warnings in this file. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Convert camel case in builderthread.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <[email protected]>
2023-07-24buildman: Convert camel case in builder.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <[email protected]>
2023-05-03buildman: Pass -Werror to the host compiler tooSimon Glass
The host compiler is not failing on warnings at present, when the -E flag is used in buildman. Add the required flag to fix this. Signed-off-by: Simon Glass <[email protected]> Tested-by: Tom Rini <[email protected]>
2023-03-08patman: Move library functions into a library directorySimon Glass
The patman directory has a number of modules which are used by other tools in U-Boot. This makes it hard to package the tools using pypi since the common files must be copied along with the tool that uses them. To address this, move these files into a new u_boot_pylib library. This can be packaged separately and listed as a dependency of each tool. Signed-off-by: Simon Glass <[email protected]>
2023-03-08buildman: Add a flag for reproducible buildsSimon Glass
This is quite a useful thing to use when building since it avoids small size changes between commits. Add a -r flag for it. Also undefine CONFIG_LOCALVERSION_AUTO since this appends the git hash to the version string, causing every build to be slightly different. Signed-off-by: Simon Glass <[email protected]>
2023-03-08buildman: Support disabling LTOSimon Glass
This cuts down build performance considerably and is not always needed, when checking for build errors, etc. Add a flag to disable it. Signed-off-by: Simon Glass <[email protected]>
2023-03-08buildman: Write out the build command usedSimon Glass
It is sometimes useful to see the exact 'make' command used by buildman for a commit. Add an output file for this. Signed-off-by: Simon Glass <[email protected]>
2022-11-22buildman: Add --allow-missing flag to allow missing blobsTom Rini
Add a new flag to buildman so that we will in turn pass BINMAN_ALLOW_MISSING=1 to 'make'. Make use of this flag in CI. Allow the settings file to control this. Cc: Rasmus Villemoes <[email protected]> Cc: Simon Glass <[email protected]> Signed-off-by: Tom Rini <[email protected]> Signed-off-by: Simon Glass <[email protected]>
2022-11-22buildman: Detect binman reporting missing blobsSimon Glass
Buildman should consider a build as a success (with warnings) if missing blobs have been dealt with by binman, even though buildman itself returns and error code overall. This is how other warnings are dealt with. We cannot easily access the 103 exit code, so detect the problem in the output. With this change, missing blobs result in an exit code of 101, although they still indicate failure. Signed-off-by: Simon Glass <[email protected]>
2022-08-05buildman: Avoid using board as a variableSimon Glass
We have a module called 'board'. Sometimes buildman uses 'brd' as an instance variable but sometimes it uses 'board', which is confusing and can mess with the module handling. Update the code to use 'brd' consistently, making it easier for tools to determine when the module is being referenced. Signed-off-by: Simon Glass <[email protected]>
2022-02-09patman: Convert camel case in gitutil.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <[email protected]>
2022-02-09patman: Convert camel case in command.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <[email protected]>
2022-02-08buildman: Allow adjusting board config on the flySimon Glass
Add a -a option to specify changes to the config before the build commences. For example buildman -a ~CONFIG_CMDLINE disables CONFIG_CMDLINE before doing the build. This makes it easier to try things out as well as to write tests without creating a new board or manually manging the .config file. Signed-off-by: Simon Glass <[email protected]>
2022-02-08buildman: Provide a hint on how to debug thread crashesSimon Glass
If a thread crashes it is helpful to try the operation again with threading disabled. Add a hint about that. Signed-off-by: Simon Glass <[email protected]>
2021-10-20buildman: Write output even on fatal errorSimon Glass
At present buildman does not write any output (to the 'out' and 'err) files if the build terminates with a fatal error. This is to avoid adding lots of spam to the logs. However there are times when this is actually useful, such as when the build fails for an obscure reason such as a Kconfig loop. Update the logic to always write the output, so that the user gets a clue as to what is happening. Signed-off-by: Simon Glass <[email protected]>
2021-04-29buildman: Use bytes for the environmentSimon Glass
At present we sometimes see problems in gitlab where the environment has 0x80 characters or sequences which are not valid UTF-8. Avoid this by using bytes for the environment, both internal to buildman and when writing out the 'env' file. Add a test to make sure this works as expected. Reported-by: Marek Vasut <[email protected]> Fixes: e5fc79ea718 ("buildman: Write the environment out to an 'env' file") Signed-off-by: Simon Glass <[email protected]>
2021-04-29buildman: Handle exceptions in threads gracefullySimon Glass
There have been at least a few cases where an exception has occurred in a thread and resulted in buildman hanging: running out of disk space and getting a unicode error. Handle these by collecting a list of exceptions, printing them out and reporting failure if any are found. Add a test for this. Signed-off-by: Simon Glass <[email protected]>
2021-04-29buildman: Use common code to send an resultSimon Glass
At present the code to report a build result is duplicated. Put it in a common function to avoid this. Signed-off-by: Simon Glass <[email protected]>
2021-04-29buildman: Tidy up a few commentsSimon Glass
Add some function comments which are missing, or missing arguments. Signed-off-by: Simon Glass <[email protected]>
2021-04-06buildman: Add an encoding to the out-env fileSimon Glass
The environment may contain some unicode characters. At least that is what seemed to happen on one commit: Building current source for 1 boards (0 threads, 64 jobs per thread) 0 0 0 /1 -1 (starting) Traceback (most recent call last): File ".../tools/buildman/buildman", line 64, in <module> ret_code = control.DoBuildman(options, args) File "tools/buildman/control.py", line 372, in DoBuildman options.keep_outputs, options.verbose) File ".../tools/buildman/builder.py", line 1704, in BuildBoards results = self._single_builder.RunJob(job) File ".../tools/buildman/builderthread.py", line 526, in RunJob self._WriteResult(result, job.keep_outputs, job.work_in_output) File ".../tools//buildman/builderthread.py", line 349, in _WriteResult print('%s="%s"' % (var, env[var]), file=fd) UnicodeEncodeError: 'ascii' codec can't encode characters in position 311-312: ordinal not in range(128) The problem defies repetition with any change at all to buildman. But let's set an encoding in any case. Signed-off-by: Simon Glass <[email protected]>