summaryrefslogtreecommitdiff
path: root/.github/workflows/regression_template.yml
AgeCommit message (Collapse)Author
2026-09-30Made the build step timeout a template input (#791)Frédéric Desbiens
The build step is capped at fifteen minutes for every consumer, on the same shape of assumption the install step carried: that a consumer builds once. Measured build steps are 0.5 minutes for LevelX, 1.1 for FileX and 6.4 for USBX, so fifteen is right for them. GUIX builds five configurations in that step, because its suite is staged across the three steps, and reaches 11.8 minutes. That is 79 per cent of the cap, and it grows with every regression test added -- six advisories landed tests this week. The cap is now an input defaulting to fifteen, matching install_timeout_minutes above. No consumer changes unless it asks to. Assisted-by: Claude Code (Opus 5) <[email protected]>
2026-09-30Made the install step timeout a template input (#790)Frédéric Desbiens
The install step is capped at ten minutes for every consumer. That suits a consumer that only installs packages, which is all of them bar one: measured install steps run 0.2 to 0.5 minutes, and NetX Duo's secure interoperability job, which builds OpenSSL and wolfSSL from source, takes 2.1 to 2.8. GUIX does not fit. Its regression suite is staged across the install, build and test steps to keep each inside its limit, so the install step also builds a configuration, and on a slow mirror the packages and the ThreadX checkout leave too little of the ten minutes for it. Three runs on dev were lost that way. The cap is now an input defaulting to ten, so a consumer that needs longer asks for it and every other consumer keeps the timeout it has today. Assisted-by: Claude Code (Opus 5) <[email protected]>
2026-09-10Stated the compiler and coverage requirements directly, and said who the ↵Frédéric Desbiens
pinned toolchain default serves (#718) * Stated the compiler and coverage requirements instead of citing a file no contributor can open Seven comments across five files cited a maintainer-local document as the source for two project requirements: that GCC 14 on Linux is the default compiler, and that the coverage target is 100%. That document is not part of this repository and is not published anywhere, so the citation gave a reader nothing to follow -- it named a source they cannot open, in place of simply stating the requirement. Both requirements are real and both stay. Only the pointer goes: each comment now states the requirement on its own terms, which is what the surrounding prose was already doing everywhere else. cmake/cortex_r52.cmake the pinned reference toolchain scripts/check_gcc.sh why the script exists .github/workflows/gcc_check.yml why the workflow exists, and the GCC_VERSION pin .github/workflows/r52_fvp.yml the GCC_VERSION pin .github/workflows/regression_template.yml the coverage floor, twice Comments only; no behaviour changes. Two paragraphs are rewrapped where the shorter text left a ragged line. Verified that scripts/check_gcc.sh still parses and prints its help from the header range it slices, and that cmake/cortex_r52.cmake still configures the Cortex-R52 build. Assisted-by: Claude Code (Opus 5) <[email protected]> * Said who the pinned toolchain default in the CMake toolchain files serves Both cmake/cortex_r52.cmake and cmake/cortex_m52.cmake default ARM_TOOLCHAIN_PATH to a toolchains directory under the user's home, guarded by an EXISTS check. Nothing said whether CI relies on that, and the natural reading is that it does. It does not. The three workflows that install a toolchain unpack it into the workspace and cache it there, and r52_fvp.yml puts that directory on PATH before configuring; scripts/check_gcc.sh passes -DARM_TOOLCHAIN_PATH at each of its three CMake call sites. On a runner the guarded directory is absent, the EXISTS check falls through, and the compiler comes from PATH. The default only ever fires on a developer machine, where it is what makes a no-flag build work. Both comments now say that, so the default is not mistaken for a CI dependency and not removed as dead code. cortex_r52.cmake carries the explanation and cortex_m52.cmake refers to it, matching the cross-reference already there. The r52 comment also claimed absolute paths mean "the build does not depend on PATH ordering", which is only true where the pinned directory exists -- in CI the build depends on PATH and nothing else. Qualified accordingly. Comments only; no behaviour changes. Verified that both toolchain files still configure, and that the fall-through is real: with HOME pointed at a directory holding no toolchains, cortex_r52.cmake configures against the arm-none-eabi-gcc found on PATH. Assisted-by: Claude Code (Opus 5) <[email protected]>
2026-08-26Enforced a coverage floor on the merged report (#667)Frédéric Desbiens
The coverage summary reported a percentage and could not fail. Coverage could fall from 99.97% to anything at all and every check stayed green, against an AGENTS.md that asks for 100% test coverage -- a stated requirement measured with a gauge that had no failure mode. CodeCoverageSummary already takes thresholds and fail_below_min; neither was set. Both are now, through a new coverage_thresholds input on the template, because the two suites do not sit at the same figure: ThreadX 99, SMP 98. Three things were probed against the pinned action on a runner before picking those numbers, using the real merged.xml files from the dev push run of #666. The floor compares the line rate and nothing else. That mattered because branch coverage is around 78% in both suites while line coverage is 98.8-100%, so a floor aimed at the line figure would have been an immediate red wall had it tested branches or the lower of the two. The ThreadX report at 100.00% lines and 77.67% branches clears a floor of 99. The thresholds are whole numbers. '99.9 100' -- the value this was meant to be -- is rejected with 'System.ArgumentException - Threshold parameter set incorrectly.', and the step fails whether or not fail_below_min is set. So the choice is 99 or 100 with nothing between. 100 would fail on a race. tx_thread_system_resume.c:529 is reached by timing rather than by construction and flaps between runs of the same green tree, which is why #666 left it; 4502/4503 fails a floor of 100 and clears one of 99. A coverage gate that goes red on a coin toss is how coverage gates get switched off. SMP is 5114/5178 lines, 98.76%, with 64 uncovered lines across 11 files of common_smp/src -- #666 closed the equivalent gaps in common/src only. A shared floor of 99 would have failed that job on every run while ThreadX passed. One limit is recorded in the file rather than fixed: an empty report reads as 100%. gcovr writes line-rate="1.0" beside lines-valid="0" when it finds no data, and the action prints 'Line Rate = 100% (0 / 0)' and passes any floor. The check for that is the emptiness assertion #664 put in each suite's coverage.sh, not this one. Also corrected two stale filenames in the deploy job's comment: since #665 each coverage artifact carries merged.xml, not default_build_coverage.xml. Verified on the runner. Assisted-by: Claude Opus 5 <[email protected]>
2026-08-26Instrumented every build configuration and merged their coverage (#665)Frédéric Desbiens
Only default_build_coverage carried -fprofile-arcs, because the gate was the build type and it is the only one of five whose name ends in _coverage. The other four build and run all their tests and their coverage was discarded. That is not redundancy thrown away: each configuration selects a different set of TX_ feature macros, so the code the other four compile is absent from the denominator rather than uncovered in it. TX_COVERAGE instruments a build regardless of its name, defaulting to OFF so a single configuration built by hand behaves as before. coverage.sh gains a --merge mode that unions the per-configuration JSON tracefiles, and cmake_bootstrap.sh runs it after the test loop so a local run produces the same merged report CI reads. The template sets TX_COVERAGE for build and test, and coverage_name moves to the merged report. Measured on the ThreadX suite, all 480 tests passing: default_build_coverage 3827 valid 3827 covered disable_notify_callbacks_build 3767 3766 stack_checking_build 3857 3856 stack_checking_rand_fill_build 3862 3861 trace_build 4123 4108 merged 4503 4487 The denominator grows by 676 lines, 17.7%, and the figure moves from 99.97% to 99.64%. The second one is honest, and the drop is the point rather than a regression: the denominator now includes code the old report never counted. The union also contains a file the old report did not contain at all -- tx_thread_stack_error_handler.c compiles only under TX_ENABLE_STACK_CHECKING, so it was not listed at 0%, it was simply absent. 177 files becomes 178. Coverage collection moved out of test() and now runs after the test loop, one configuration at a time. gcov writes its intermediate gcov files into the directory gcovr is rooted at, and coverage.sh roots every configuration at the repository root so filenames come out repo-relative. Five concurrent gcovr processes therefore share one scratch directory and delete each other's output: the first full run of this change passed all 480 tests and produced no report for three of the five configurations. Measured both ways -- two gcovr rooted at the repository root fail concurrently and succeed in sequence. CI would not have caught it, because test_tx.sh sets CTEST_PARALLEL_LEVEL=1 and takes the serial branch. Per-configuration output moved under coverage_report/per_configuration/ and is excluded from the Pages artifact. The deploy job merges the ThreadX and SMP artifacts into one tree and every configuration directory has the same name in both, so left at the top level one suite's would overwrite the other's on the published site. On the SMP suite, an earlier run of this change saw trace_build fail threadx_smp_time_slice_test and then hang, which raised the question of whether -fprofile-arcs perturbs a timing-sensitive test. It does not. Sixteen runs settle it, and the decisive one is that threadx_smp_time_slice_test failed ERROR #31 -- twice in a row under --repeat until-pass:2 -- on an uninstrumented build, in the exact shape CI runs, while three instrumented runs of that shape passed 5 of 5. In the CI shape, CTEST_PARALLEL_LEVEL=1 run.sh test all: TX_COVERAGE=OFF 3 runs 2 green, one ERROR #31 310 s TX_COVERAGE=ON 3 runs 3 green, 5/5 each 325-329 s So the test is a pre-existing flake on dev and instrumenting all five costs about 5% of the suite's wall clock. Separately, and also in both instrumented and uninstrumented builds, run.sh's parallel branch -- what a developer gets typing run.sh test all with no CTEST_PARALLEL_LEVEL -- hangs under its own load, four times in twelve runs. Several SMP tests create 1024 ThreadX threads by construction and the Linux port backs each with a pthread, so five configurations at once put on the order of 5000 threads on the machine. CI sets CTEST_PARALLEL_LEVEL=1 and does not take that branch. Assisted-by: Claude Opus 5 <[email protected]>
2026-08-25Pinned every action to a commit SHA, and moved them off Node 20 (#660)Frédéric Desbiens
Node 20 is removed from the GitHub runners on 16 September 2026. Every run in this repository currently emits the deprecation warning for it, naming actions/checkout, actions/configure-pages, actions/upload-artifact, LouisBrunner/checks-action and marocchino/sticky-pull-request-comment among others. After that date those actions stop working rather than warning, so this is a deadline and not housekeeping. Every action is now referenced by a 40-character commit SHA with the version in a trailing comment. A tag can be repointed at any commit; a SHA cannot, so this is what makes "which code ran in our CI" answerable from the repository rather than from whatever the tag meant at the time. The versions were behind by as much as four majors -- download-artifact was on v4.3.0 against v8.0.1 -- because nothing in this repository has ever reported that an action moved. Compatibility was checked against each new action.yml rather than assumed, for every input this repository actually passes: checkout submodules is unchanged cache path and key are unchanged upload-artifact name, path and retention-days are unchanged download-artifact pattern, merge-multiple and path are unchanged configure-pages takes no input here, and none became required deploy-pages still exposes page_url, which the job reads upload-pages-art. path is unchanged checks-action token, name, conclusion, output and output_text_description_file all survive v2 to v3 sticky-comment header and path survive v2 to v3, and the new GITHUB_TOKEN input defaults to github.token, which is what v2 used implicitly delete-artifact name survives v5 to v6, and useGlob still defaults to true, so the coverage_report-* glob from #655 still matches CodeCoverageSummary already current at v1.3.0; pinned, not moved The artifact pair moves together, as it must. The round trip was verified on a runner before this commit: upload-artifact v7 to download-artifact v8, through the pattern and merge-multiple selection #655 introduced, filtered 4 artifacts to 2 and produced exactly the tree the deploy expects. Two behaviour changes worth knowing. download-artifact v8 adds a digest-mismatch input defaulting to error, so a corrupted artifact now fails the job instead of passing through -- the right default, but a change. upload-artifact v6 and above require a runner of at least 2.327.1, which the hosted runners satisfy and a self-hosted runner would need checking for. Assisted-by: Claude Opus 5 <[email protected]>
2026-08-24Kept the coverage report from the runs that most need one (#659)Frédéric Desbiens
A failing test threw away coverage that had already been collected, and the run whose behaviour changed is exactly the run whose coverage is worth reading. Measured on the failing run of 2026-08-18: it uploaded test_reports for all three suites and no coverage_report artifact at all. Two causes, and the workflow one is the smaller of them. cmake_bootstrap.sh runs under set -e, so a failing ctest aborted test() before ./coverage.sh was reached. The gcda files exist by that point, so nothing was missing except the step that reads them. ctest's status is now captured and returned at the end, and the summary grep is allowed to fail rather than being the thing that stops the coverage behind it. The serial branch of the test dispatch collected no status either, so under set -e the first failing configuration stopped the remaining four from being tested at all -- and their coverage from being collected. That was cheap while the suites ran in parallel, because the parallel branch already collects exit codes from its background jobs. Moving to serial execution in #643 quietly made one failure cost the other four configurations. The serial branch now collects status the same way the parallel branch does. With those fixed the report exists, so the workflow steps that publish it no longer skip on failure. They are guarded with !cancelled() rather than always(), so a cancelled run still stops promptly, which is the idiom deploy_code_coverage already uses. The ${{ }} wrapping is required and not decoration: a bare ! opens a YAML tag, and the file will not parse without it. Verified locally by replacing one test binary with a stub that exits 1: before the failing run of 2026-08-18 produced no coverage_report artifact after run.sh test default_build_coverage exits 8, and produces coverage_report/default_build_coverage.xml with 177 files and 3804 of 3827 lines after run.sh test all exits 8, and all five configurations run rather than stopping at the first The failure still fails. Only the reporting around it changed. Assisted-by: Claude Opus 5 <[email protected]>
2026-08-24Published the coverage report instead of everything the run produced (#655)Frédéric Desbiens
The download step in deploy_code_coverage asked for the artifact named ${{ steps.artifact.outputs.coverage_report }}. That output is set by the "Coverage Report name" step of run_tests, which is a different job, and the steps context does not cross jobs. So the expression evaluated to the empty string and the action took its documented path for an unspecified name: No input name, artifact-ids or pattern filtered specified, downloading all artifacts Total of 4 artifact(s) downloaded The four are the two coverage reports and the two test_reports bundles of JUnit XML, each extracted into a directory named after the artifact. The next step uploads the lot to Pages, so the published site has carried the test reports alongside the coverage, one directory deeper than intended, under a path containing a run timestamp that changed on every publish. Any link to a coverage report broke the next time one was published. Selecting by pattern with merge-multiple fixes both halves: the pattern excludes the test_reports bundles, and merging puts the contents of the two coverage artifacts directly into coverage_report rather than under a directory named for each. Each artifact holds one directory named for its suite, renamed from default_build_coverage by "Prepare Coverage GitHub Pages", so the result is the two suite directories the deploy expects and the timestamped artifact name no longer appears in the published path. Verified on a runner rather than reasoned about, with an isolated workflow that uploads artifacts shaped like the real ones and downloads them both ways: OLD coverage_report/coverage_report-<epoch>-ThreadX/ThreadX/index.html coverage_report/coverage_report-<epoch>-ThreadX/default_build_coverage.xml coverage_report/coverage_report-<epoch>-SMP/SMP/index.html coverage_report/coverage_report-<epoch>-SMP/default_build_coverage.xml coverage_report/test_reports SMP/results.xml coverage_report/test_reports ThreadX/results.xml NEW coverage_report/ThreadX/index.html coverage_report/SMP/index.html coverage_report/default_build_coverage.xml Both artifacts carry a default_build_coverage.xml and the merge means one overwrites the other, which the run above also shows. That file is consumed by CodeCoverageSummary back in run_tests and is not read here, so it is untidy rather than wrong, and it is called out in a comment. The delete step is fixed in the same place and for a related reason. The artifacts are named coverage_report-<epoch>, useGlob defaults to true in this action, and as a glob "coverage_report" matches only the literal string. It has been deleting nothing, without failing, and retention-days: 1 on the upload is what has actually been clearing these up. Assisted-by: Claude Opus 5 <[email protected]>
2026-08-20Stopped a stalled install from costing a whole regression run (#641)Frédéric Desbiens
The Install softwares step runs apt-get update, an apt-get install and a pip install, and it has stalled twice: 55 minutes on the SMP job of one run, and more than two hours on the ThreadX job of the next, against a normal 27 to 152 seconds across every other run measured. In the second case the tests never started at all. No step in this template had a timeout, so a stall runs until the six hour job limit. That turns a transient apt or PyPI problem into a lost run, and it hides what happened: the job simply sits there, and the failure that eventually gets reported says nothing about which step was stuck. Bound the three steps that do real work. Ten minutes for the install, against a normal worst case of 152 seconds. Fifteen for the build, which has run between 4 and 54 seconds. Sixty for the test step, which is the only one whose length depends on the suites themselves; the longest observed is 37 minutes, and that was with a wait in one test that has since been bounded. A step that trips its timeout fails and names itself, which is the point. None of these numbers is tight enough to trip on work that is merely slow. Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-08-09Added bash A profile update scripts and restored ports_arch as their source ↵Frédéric Desbiens
(#592) The ARMv7-A and ARMv8-A ports are generated by update.ps1, which needs PowerShell, so the cortex-a job in ports_arch_check ran on a Windows image and nobody could reproduce it locally on Linux. Add update.sh beside each update.ps1, with the same cores, compilers, copy sets and patches, and move the job to the same Linux image as everything else. The bash scripts were checked against the PowerShell ones by comparing what each reports as drifted. They agree exactly on the 63 files the Windows job last reported, and differ on 12 more, which turn out to be a defect in update.ps1 rather than in the port. Its two .cproject patterns are written as 'value=`"cortex-a7`"' with backticks that survive into the pattern, so that replacement has never matched, while the neighbouring Cortex-A7.NoFPU pattern has no backticks and always worked. The result is that the AC6 example builds for the A5, A8, A9, A12, A15 and A17 cores name cortex-a7 as their CPU while their FPU string is correct. The bash scripts do what the PowerShell ones intended, so regenerating corrects those twelve files. Restore ports_arch as the source for the rest. The implementation of _tx_thread_smp_time_get from #555 was applied to the twenty four generated SMP ports and never to ports_arch, which still held MOV x0, #0 with a FIXME comment, so regenerating would have replaced a working generic timer read with a stub. That implementation now lives in the source. The remaining differences are cosmetic and resolve in favour of the source: a trailing blank line in 38 copies of tx_thread_schedule.S and comment spacing in one tx_port.h. Note that the Cortex-A VFP fix is already present in ports_arch and was never at risk, contrary to what the description of the port consistency checks change said before this was measured. Extend scripts/check_ports.sh to run the A profile generators too, and make it fail when a generator fails or is missing rather than reporting a clean tree, which would have been a false pass. Pin every workflow to ubuntu-24.04. ubuntu-latest already resolves to that image, so nothing changes today, but a future migration becomes a deliberate commit rather than something that happens underneath the -m32 builds. Assisted-by: Claude Code (Opus 5) <[email protected]>
2026-03-05Updated copyright headers and version number constants (#509)Frédéric Desbiens
* Updated version number constants * Removed revision history from all files * Added Eclipse ThreadX contributors' copyright header
2025-07-29Fixed code coverage report download step in deploy_code_coverage.Frédéric Desbiens
2025-07-29Added condition to "Coverage Report Name". Corrected formatting.Frédéric Desbiens
2025-07-29Fixed code coverage artefacts uploadFrédéric Desbiens
2025-07-17Updated all actions to their latest release.Frédéric Desbiens
Signed-off-by: Frédéric Desbiens <[email protected]>
2025-02-13Upgrading upload-artifact version to 4.6.0.Frédéric Desbiens
2024-09-03Bump actions/download-artifact from 3 to 4.1.7 in /.github/workflowsdependabot[bot]
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 3 to 4.1.7. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](https://github.com/actions/download-artifact/compare/v3...v4.1.7) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]>
2023-12-22Add sudo to move coverage folder created by root user.Wenhui Xie
2023-11-30Correct condition syntax in "Prepare Coverage GitHub Pages" step.l (#329)Ting Zhu
* Correct syntax. * Update regression_template.yml * Update regression_template.yml
2023-11-29Add additional condition for "Prepare Coverage GitHub Pages" step.Ting Zhu
2023-11-27Added input skip_coverage and coverage_name for customiziing. (#327)CQ Xiao
* Update regression_template.yml * Update regression_template.yml * Update regression_template.yml * Update regression_template.yml * Update regression_template.yml * Update regression_template.yml Coverage name of all -> default_build_coverage * Update regression_template.yml * Update regression_template.yml Check inputs.skip_coverage to do steps. * Update regression_template.yml * Update regression_template.yml * Update regression_template.yml * Update regression_template.yml Enable coverage upload when manually triggered. * Update regression_template.yml * Update regression_template.yml * Update regression_template.yml * Update .github/workflows/regression_template.yml Fix comments. Co-authored-by: TiejunZhou <[email protected]> --------- Co-authored-by: TiejunZhou <[email protected]>
2023-11-24Restrict deploy run condition (#325)TiejunZhou
* Restrict deploy run condition * Fail the job for testing purpose * Revert "Fail the job for testing purpose" This reverts commit 6ae18cafe285f07443137b85493f69f95e738c3d.
2023-11-24Split artifacts for multiple jobs (#324)TiejunZhou
* Test multiple code coverage pages * Add affix to artifacts * Test uploading code coverage as artifact * Deploy GitHub pages at last for multiple jobs * Test using unified upload pages * Disable test cases to accelerate experiment * Fix escape character $ * Revert "Test using unified upload pages" This reverts commit 3668d9f67253651c8cea38e9aba639dff081168d. * Set destination for downloaded artifact * Use a different artifact name * Fix escape value * Revert "Disable test cases to accelerate experiment" This reverts commit 8468f17d0298a483692ab613524bc6ab5443a67a. * Override duplicated github-pages in artifact * Revert "Override duplicated github-pages in artifact" This reverts commit 17a83aa97df4233417e20d31d36aa61e976dbebf. * Delete Duplicate Code Coverage Artifact
2023-11-23Convert ADO pipelines to GitHub actions (#321)TiejunZhou
* Convert ADO pipelines to GitHub actions * Remove version in uses as not valid for local workflows * Fix cmake path and add deploy url affix * Add SMP build job * Fix code coverage URL * Add affix to titles of steps * Remove ADO pipelines * Add affix to titles of code coverage * separate PR results for multiple jobs * Revert "separate PR results for multiple jobs" This reverts commit 6da13540fd460b6571c6bfe0196abe52e4b1cccf. * separate PR results for multiple jobs