| Age | Commit message (Collapse) | Author |
|---|
|
The first CI run of this branch failed 13 tests across the five ThreadX
configurations, and every one of them measures time:
threadx_timer_simple_test all five configurations
threadx_timer_information_test four
threadx_timer_multiple_accuracy_test trace_build
threadx_time_get_set_test stack_checking_rand_fill_build
threadx_thread_sleep_for_100ticks_test trace_build
threadx_thread_simple_sleep_non_clear_test default_build_coverage
The suites ran with CTEST_PARALLEL_LEVEL=4, so four of them shared a runner
with fewer cores than that. A test that asserts on tick counts fails when it
does not get the processor, and --repeat until-pass:2 only doubled the cost of
each one: the ThreadX job took twenty minutes against SMP's three.
The same suites pass serially, all 95 tests in all five configurations, and the
coverage evidence on this branch has been measured serially throughout. This is
also what the suites already do upstream, where they moved to serial execution
for this reason.
The cost is wall clock and nothing else, and the whole run is still well inside
the job timeout.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
certification branch
The regression workflows ran on ubuntu-latest with nothing pinned, and the
coverage numbers they produce are certification evidence. Two things follow from
that, and the first is urgent rather than theoretical.
These sources declare cmake_minimum_required(VERSION 3.0.0) and CMake removed
compatibility below 3.5 in 4.0, so this branch stops configuring the day the
hosted image moves to CMake 4.x -- no change here, no warning. Worse, the branch
did not need GitHub's help to get there: scripts/install.sh ran
pip install --upgrade cmake
which installs exactly the CMake version that cannot configure these sources. It
also installed gcovr 4.1, which is far too old for the tracefile and threshold
options the coverage scripts use, and called sudo, which does not exist in a
container.
The job now runs inside debian:trixie, which carries GCC 14, gcc-multilib and
CMake 3.31.6. Being below 4.x, that configures the sources with no policy
override and no edit to CMakeLists.txt. The coverage build is 32-bit, so multilib
is not optional. install.sh installs the distribution toolchain plus gcovr 8.6,
prints all three versions into the job log, and then asserts them: GCC major 14,
CMake major below 4, gcovr exactly 8.6. The image tag is a moving target that
point releases rebuild, and a toolchain that drifts should fail by name and
number rather than quietly produce evidence built with something else.
git is installed before the checkout, because actions/checkout otherwise falls
back to a REST API tarball and leaves no .git behind, and the commit the evidence
was built from is part of the evidence. The Pages preparation step no longer
calls sudo, since the steps run as root in the container.
Nothing gated on the coverage number before: it was published, not enforced.
coverage.sh --merge now ends with a gcovr pass carrying --fail-under-line and
--fail-under-branch, after the reports are written so a failed gate still leaves
the report that explains it.
ThreadX 99.50% line 80.00% branch
SMP 99.40% line 83.00% branch
These are a ratchet, raised as each batch of gaps closes, and deliberately not
100%: a gate nothing can pass gets switched off. They sit below the measured
figures rather than at them, because the same tree does not measure identically
twice -- branches on the resume-from-ISR path are reached or missed depending on
how the host schedules the run, and two clean runs of the ThreadX suite differ by
2 lines and 19 branches. A gate set at the best observation fails on a tree
nobody changed.
Verified by raising each threshold above the measurement: line returns 2, branch
returns 4, the merged report is still written, and run.sh reports the failure
rather than swallowing it -- a run with all 95 tests passing exits 2 when the
line gate is not met.
regression_test.yml now triggers on the certification branch as well as master.
regression_template.yml needed no filter; it is workflow_call only.
Publishing stays on master. The deploy job replaces the project's public coverage
site wholesale and its condition fires on any push, so without a guard the first
push of this branch would have published its numbers as the project's. The
certification branch produces its evidence as workflow artifacts instead.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
Only default_build_coverage carried -fprofile-arcs, because the gate was the
build type and it is the only one of five configurations whose name ends in
_coverage. The other four built and ran every test and their coverage was
discarded. That is not redundancy thrown away: each configuration selects a
different set of TX_ feature macros, so the code the other four compile was
absent from the denominator rather than uncovered in it, and 100% against that
report would not have been a measurement of the kernel.
TX_COVERAGE instruments a build regardless of its name, defaulting to OFF so a
single configuration built by hand behaves as before. coverage.sh gains a
--merge mode that unions the per-configuration JSON tracefiles, and
cmake_bootstrap.sh runs it after the test loop so a local run produces the same
merged report CI reads. The workflow template sets TX_COVERAGE for build and
test, and coverage_name moves to the merged report.
Measured on the ThreadX suite, all 95 tests passing in each configuration:
default_build_coverage 3827 valid 3827 covered
disable_notify_callbacks_build 3767 3767
stack_checking_build 3857 3856
stack_checking_rand_fill_build 3862 3861
trace_build 4123 4108
merged 4220 4204
The denominator grows by 393 lines and the figure moves from 100.00% to 99.62%.
The second one is honest, and the drop is the point rather than a regression:
the denominator now includes code the old report never counted. The union also
contains a file the old report did not contain at all --
tx_thread_stack_error_handler.c compiles only under TX_ENABLE_STACK_CHECKING, so
it was not listed at 0%, it was simply absent. 177 files becomes 178.
The scoping is the positional search path, not --object-directory. Left as
--object-directory while the gcovr root moved to the repository root, all five
configurations reported the union of every configuration's gcda: five identical
reports, and a merge of five copies of the same thing. Nothing about the
resulting percentage looks wrong, which is what makes it worth stating here.
Coverage collection moved out of test(). A failing ctest used to abort the
function before coverage.sh was reached, so the run whose behaviour changed was
the one that produced no report, and the gcda already existed by that point.
Collection now runs after the whole test loop, one configuration at a time,
which is also what --merge needs: every tracefile has to exist before the union
is taken. A coverage failure turns the run red but does not overwrite a test
failure's status.
Per-configuration output moved under coverage_report/per_configuration/ and is
excluded from the deployed artifact. The deploy merges the ThreadX and SMP
artifacts into one tree and every configuration directory has the same name in
both, so left at the top level one suite's would overwrite the other's on the
published site.
coverage.sh now fails when a report contains no files. gcovr treats an empty
report as success: it exits 0 and writes line-rate="1.0" alongside
lines-valid="0", so every downstream consumer reads "no data at all" as "100%
covered", and a coverage threshold cannot catch it because an empty report
passes any threshold.
The tx and smp runners are symlinks to scripts/cmake_bootstrap.sh, so they are
covered by the one change there.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The cmake test runner drove Ninja with its default keep-going of 1, so the
first failing target ended the build. Every target scheduled after it was
simply absent, and ctest reports a missing binary as a failing test. A single
compile error therefore produced a failure count that moved with build
scheduling order rather than with the code.
Measured on this tree with one regression test deliberately broken:
before 0 of 95 test binaries built
after 94 of 95 test binaries built
Passing -k 0 lets Ninja finish everything it can; the build still exits
non-zero when a target fails.
Three related problems in the same paths are fixed with it.
A failing configuration used to abort the loop over configurations, so under
set -e the ones after it went unbuilt or untested. Building all five
configurations with a break confined to the second one attempted two of them
and left three with no binaries at all. The build loop and the serial test
loop now accumulate status and return it at the end, which is what the
parallel test branch already did with wait.
Capturing that status removes the set -e protection inside the functions, so
two latent faults become reachable on the serial path and are closed here. A
failed pushd would have let ctest run in the source tree, where it finds no
tests; the pushd is now guarded. And ctest's status was discarded by the popd
that follows it, so a configuration with failing tests returned 0 and was
reported as a pass. With one test made to fail, the serial branch exited 0
before and exits 8 now. The serial branch is the one CI takes.
The tx and smp runners are symlinks to scripts/cmake_bootstrap.sh, so they are
covered by the one change there.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
CMake records the compiler it detected inside the build directory and keeps
using it on every later configure. Since cmake/linux.cmake began honouring CC,
that made a compiler switch silently ineffective: CC=gcc-14 ./run.sh build <cfg>
against an existing build directory printed "ninja: no work to do", exited 0,
and left the previous compiler in place. Anyone verifying a change against a
second compiler would have been reading stale results while being told the
build had succeeded.
generate() now compares the compiler recorded in the build directory with the
one currently requested, and reconfigures from scratch when they differ. The
comparison uses the path as CMake records it, which is the unresolved path as
given, so /usr/bin/gcc matches command -v gcc rather than the versioned target
its symlink points at. build_libs() gets the same treatment.
Only the C compiler is consulted: both trees that use this script declare
LANGUAGES C, so no CXX compiler is ever detected.
Nothing is reconfigured unless the compiler actually changed, so repeat builds
stay incremental and the default path is unchanged.
Assisted-by: Claude Code (Opus 5)
|
|
|
|
test result.
|
|
|
|
|
|
* Release ARMv7-M and ARMv8-M architecture ports
* Add a pipeline to check ports_arch
|
|
|
|
|
