| Age | Commit message (Collapse) | Author |
|---|
|
Every SMP build configuration is compiled with -DTX_REGRESSION_TEST, which opens
a block in the Linux SMP port header defining ten macros that expand to live code
-- flag tests, state changes and calls -- at eleven sites in common_smp/src. An
application defines none of them and tx_api.h expands them all to nothing, but
gcov attributes a macro expansion to the file that invokes it, so the harness's
own scaffolding was being counted as certified source.
coverage_union.py now drops those sites and prints what it dropped, covered and
total on both axes, so the exclusion can be audited rather than inferred. One
site keeps its line: tx_api.h defines TX_TIMER_INITIALIZE_EXTENSION as a real
statement under TX_MISRA_ENABLE, so that line ships and only its branches come
out. A listed site that no longer matches the report is a hard failure naming the
macro, not a silent skip. The branch gate moves 99.05 to 99.04 because the margin
is ten outcomes rather than a percentage, and a smaller denominator buys nine at
the old value; the line gate is unchanged.
The excluded code measured 10 lines and 36 branch outcomes, every one of them
covered, so the uncovered set is untouched: the same 24 outcomes at the same 22
sites in the same 13 files, over the same 194 files. Eight configurations from a
wiped tree, 136/136 tests in each, 0 failed. The union goes from 5449/5450 lines
and 3588/3612 branch outcomes to 5439/5440 and 3552/3576, re-derived key by key
against the previous measurement. gcovr's merged report is published unchanged.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The comment added in the previous commit said six outcomes in the figure have
no configuration to cover for them. That was the count of outcomes that both
have a single carrier and were seen moving between runs, not the count of
single-carrier outcomes, and as written it is wrong.
Measured over sixteen runs at two commits: thirty-five outcomes are compiled by
two configurations and covered by only one of them in at least one run, almost
all TX_THREAD_STACK_CHECK fence ordinals that stack_checking_build holds alone
because its random-fill sibling short-circuits before them. A further 894 are
compiled by exactly one configuration at all, which is structural rather than
fragile, the racing outcome excepted. The margin's justification is restated on
the figure that actually sizes it: the largest movement observed in any one
configuration is eight outcomes, and the margin is ten.
No threshold changes. The gate stays at 99.79 line and 99.05 branch, and
sixteen runs read 5449/5450 lines and 3588/3612 branch outcomes on the union
basis with an empty unstable set on both axes.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The gate's comment left open whether the ten-line, ten-outcome margin could be
removed once the unstable set closed, and deferred the answer to the coverage
evidence work. It has now been measured.
The thresholds do not move, and the comment records why. Read per configuration
over eight CI runs at one commit, eight branch outcomes and four lines move
between runs, and six outcomes in the published figure are compiled by only one
configuration that covers them: five TX_THREAD_STACK_CHECK arms that
stack_checking_build carries alone in most runs, because
stack_checking_rand_fill_build misses them in seven and four runs of eight, and
the trace race outcome, which only trace_build compiles. The union figure is
stable because the eight configurations cover for each other, not because the
suite is deterministic, so a zero-margin gate would read a runner's timing as a
regression.
Eight runs at the preceding commit read 5449/5450 lines and 3588/3612 branch
outcomes identically on the union basis, unstable set empty on both axes,
136/136 tests in each of the 64 configuration-runs. gcovr's merged report over
those same runs reads 6561, 6563 and 6564 of 6601, which is the same movement
seen through a denominator that does not collapse it.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The coverage ratchet sat at 99.03 branch, set below the 3587 outcomes every run
of the previous sample reached. Covering the user event insert's disabled-buffer
arm moves that floor by one.
Raised to 99.05, still ten outcomes below the set covered in every run, which is
now 3588 of 3612. The line gate is unchanged at 99.79: neither the line total
nor its denominator moved. The comment above the thresholds gains the one thing
about this figure a later reader has to know -- a single outcome in it is
reached by two cores racing rather than by a thread doing something in order,
and it is counted in the always-covered set on a measurement over ten runs on
two machines rather than on one sample.
Seven CI runs at the test head, all green, 136/136 in each of eight
configurations, 5449/5450 and 3588/3612 in every one. Two clean local cycles
read the same two totals.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The coverage ratchet sat at 99.79 line and 98.97 branch against a branch
denominator of 3610. The port hook's new clause adds two covered outcomes to
tx_byte_allocate.c, so the denominator is 3612, and the sweep closed two further
outcomes on top of that.
The branch gate goes to 99.03, still ten outcomes below the 3587 of 3612 covered
in every run of the sample. The line gate is unchanged at 99.79: neither the
line total nor its denominator moved. The comment above the thresholds records
why the denominator moved, and that the unstable set is now empty in two
consecutive samples on two different trees rather than one. The margin stays for
the reason it was kept before -- five earlier samples moved an outcome in every
direction there is -- and whether it can go is the evidence step's question.
Seven CI runs at the sweep head, all green, 135/135 in each of eight
configurations, 5449/5450 and 3587/3612 in every one. Two clean local cycles
read the same two totals.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The coverage ratchet sat at 99.32 line and 97.42 branch, set below the 5423
lines and 3527 branch outcomes every run of the previous sample reached. The
initialize, trace and MISRA shim batch has moved both floors.
Raised to 99.79 and 98.97, still ten lines and ten outcomes below the set
covered in every run of the sample, which is now 5449 lines and 3583 outcomes of
5450 and 3610. The comment above the thresholds is rewritten around the fact
that made this sample different: all seven runs read the same two totals, so the
unstable set is empty for the first time and the always-covered floor and the
worst observation coincide. The margin stays, because five earlier samples moved
an outcome in every direction there is, twice on the pair this batch has just
closed; whether it can go is the evidence step's question and not this one's.
Seven CI runs at the batch head, all green, 134/134 in each of eight
configurations, 5449/5450 and 3583/3610 in every one. Two clean local cycles
read the same two totals.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The byte pool and block pool batch moved the union figure from 5398 lines and
3399 branch outcomes to 5423 and 3531, and the gate was still set for the figure
before it. A gate that trails the work by a whole batch stops being a ratchet.
Set from the set of lines and outcomes covered in every one of seven CI runs --
5423 and 3527 -- ten lines and ten outcomes below it, which is the basis the
comment above the thresholds describes and the same margin the last four raises
used. The always-covered branch total went 3395 to 3527; the line total, 5398 to
5423.
Seven runs at this tree read 5423 lines in every one of them and between 3527 and
3531 outcomes. Four outcomes at two sites remain unstable and both sites belong
to the trace batch that has not run yet; the line axis has now been stable across
thirty-five runs in five samples.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
Closing the trace inserts, the zero-request refusal and the two prioritize
retry arms across the semaphore, mutex and event flags sources took the set of
branch outcomes covered in every run from 3195 to 3395, and the set of lines
from 5396 to 5398, so the gate no longer sits ten below either.
Re-derived on the union basis over a fourth sample of seven CI runs: 5398 of
5450 lines and 3395 of 3610 outcomes are covered in every one, and the gate is
set ten below each, at 98.86 and 93.76. What is left unstable is four outcomes
at two sites, up from two at one, and the comment above the thresholds records
why that is not a regression to chase: the buffer-wrap arm of the ISR enter
insert was unstable over fourteen runs, covered in all seven of the next sample
with nothing touching it, and is missed in one run of seven again here. Four
samples have now moved an outcome in every direction the three columns allow.
Seven CI runs at this commit's parent, all green, 123/123 tests in each of eight
configurations, retries disabled, and two clean workstation runs besides. The
line axis did not move within the sample at all.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
Closing the ready-list walk in the time-slice rebalance took the set of branch
outcomes covered in every run from 3194 to 3195, so the gate no longer sits ten
outcomes below it.
Re-derived on the union basis over a third sample of seven CI runs: 5396 of 5450
lines and 3195 of 3610 outcomes are covered in every one, and the gate is set
ten below each, at 98.82 and 88.22. The line threshold does not move. What is
left unstable is two outcomes at one site, the buffer-wrap arm of the ISR exit
trace insert, down from five at three sites. The comment above the thresholds
records all three samples, including an outcome that came back covered in every
run of the third with nothing touching it.
Twenty-one CI runs across the three samples, all green, 117/117 tests in each of
eight configurations, retries disabled. The margin earned itself immediately: the
first clean workstation run after it was set read 3194, one outcome below the CI
floor, on a stack check covered in all twenty-one CI runs.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The gate's thresholds were derived from one sample of seven CI runs, taken a
commit before the one they ship with. A second sample of seven has since been
taken at the head, and it puts the same 5396 lines and 3194 branch outcomes in
the always-covered column and the same five outcomes in the unstable one.
The comment above the thresholds now states both samples and the fact that they
agree. The thresholds themselves do not move: a floor two independent samples
report identically is the one the gate was already set ten lines and ten
outcomes below.
Fourteen CI runs, all green, 116/116 tests in each of eight configurations,
retries disabled.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The gate held the floor the thread tests left, 98.78 line and 85.95 branch on
the union of the eight configurations. The timer and time batch has since
closed 82 of its 84 branch outcomes and both of its uncovered lines, so the
threshold is well below what the tree now reaches and no longer stops a
regression in this batch from passing.
Re-derived on the union basis over seven CI runs at the head of the batch:
5396 of 5450 lines and 3194 of 3610 branch outcomes are covered in every one of
them, and the gate is set ten lines and ten outcomes below each, at 98.82 and
88.20. The line figure is now identical in all seven runs -- the two lines that
used to move were the wrapped arm of the timer list calculation, and a test now
builds the arrangement instead of waiting for the tick to produce it. Five
branch outcomes still move: the buffer-wrap arms of the two ISR trace inserts,
and the exit arm of the ready-list walk in tx_thread_time_slice.c, which was
covered in all seven runs of the previous sample and so appeared on no list.
The seven runs read 3194, 3195, 3195, 3195, 3197, 3199 and 3199 of 3610, all
green, 116/116 tests in each of eight configurations, retries disabled.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The ratchet stops later work undoing earlier work, so it has to move when a
batch closes. It was set for the figure before the thread batch: 98.70 line and
80.10 branch on the union basis, against 98.97 and 86.26 now measured.
The gate becomes 98.78 line and 85.95 branch, ten lines and ten outcomes below
the floor a run can honestly reach -- the 5394 lines and 3113 branch outcomes
covered in every one of seven CI runs on this tree. The comment above it records
what moved: the branch spread is down from eight outcomes to three and the
unstable set from eleven to five, because the thread trace test closed the two
buffer-wrap ordinals on the thread resume, thread suspend, system resume and
system suspend inserts, and they are now covered in every run rather than in
some. Two outcomes went the other way, from uncovered in seven consecutive runs
to unstable, which is why the margin is not zero.
Seven CI runs read 5394 to 5396 lines and 3115 to 3117 branches; two clean local
runs read 5396 and 3114 to 3116. 113/113 tests in each of eight configurations,
retries disabled.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The gate is the ratchet that stops later work undoing earlier work, so it has to
follow the coverage up. It sat at 98.60 line and 77.30 branch, derived when the
union denominator was still being settled, and the queue trace test has since
closed 103 branch outcomes that no run covered before.
The new floor is 98.70 and 80.10, on the union basis. The worst of nine runs --
seven in CI and two clean local ones -- is 5390 of 5450 lines and 2906 of 3610
branches, so the gate sits ten lines and fourteen outcomes below it. The stale
figures in the comment above it are corrected at the same time: the denominators
are 5450 and 3610, the merged one is 6595, and the eleven branch outcomes that
still move between runs are named.
Fourteen outcomes of headroom rather than the twelve that preceded it, because
an outcome's stability turns out to depend on how many runs have been looked at.
The two ordinals at tx_trace_isr_enter_insert.c:93 were covered in three of seven
runs before this change and in none of seven after it, and nothing here touches
that site. A gate set flush against a worst observation is set against a sample.
Verified: all eight configurations pass 110 of 110 and clear the new gate.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
common_smp/src/tx_thread_smp_debug_entry_insert.c is 222 lines of certified
source sitting entirely under TX_THREAD_SMP_DEBUG_ENABLE. No build configuration
defined that macro, so the file compiled to nothing in all seven and was absent
from the coverage denominator rather than uncovered in it. Eight further files
carry regions behind the same macro, twenty-two regions in nine files.
This is the same argument that added misra_build, on a second whole-file macro,
and it has no monoprocessor counterpart: TX_THREAD_SMP_DEBUG_ENABLE does not
exist in common/src.
One configuration is enough. Every region is a top-level ifdef sitting beside
other guards rather than nested inside one, and the file's own inner guards
resolve without help, so no combination build of the kind misra_trace_build
needed is required here.
It brings in 54 lines and 12 branch outcomes, of which all 54 lines and 10
outcomes arrive covered with no new test written, and the file itself reads 32/32
lines and 8/10 branches. The merged report now covers 194 of the 194 files in
common_smp/src, so the scope statement needs no caveat about which files
contribute code. 108/108 tests pass in the new configuration in every run made.
The two residual outcomes are infinite spin-on-error loops and no terminating
test can cover them: the taken arm never returns, by design.
_tx_thread_smp_system_error is written in one place in the certified source, set
false at initialisation, and read only by that loop.
The gates are re-derived on the union basis: 5386/5450 lines and 2810/3608
branches at best, 5384/5450 and 2805/3608 at worst across three runs.
The suite is not yet reliably green, for reasons this configuration does not
cause and does not share. Three runs produced four hangs on the full ctest
timeout, in four other configurations, on threadx_smp_random_resume_suspend_
exclusion_pt_test, its non-pt twin and threadx_event_flag_isr_set_clear_test;
the last of those timed out on both attempts and failed. Retrying until pass
hides the rest.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The merged report's branch denominator is not the number of branches in
common_smp/src. gcovr keys each branch by the basic-block pair gcov assigned it,
and those numbers shift when a file compiles to a different amount of code, so a
source branch that survives block renumbering between two configurations is
counted twice. Over the seven configurations the denominator reads 6334 where
the source carries 3596, and 1265 of 1343 branch sites carry more branches in
the merged report than any single configuration has. Adding misra_trace_build
moved the merged figure by 994 branches against the dozen the source gained.
It is not harmless. A figure quoted as branch coverage of the source has to have
the source in its denominator, and this one moves when a build configuration is
added for reasons that have nothing to do with the code. It also has to mean the
same thing as the figure the monoprocessor certificate quotes, and until now the
two suites were counting differently.
coverage_union.py reports the union instead: a line is covered if any
configuration executed it, and the Nth branch on a line is covered if any
configuration took it, keyed by position within the line rather than by block
number. That keying is exact here rather than approximate - all 1343 branch
sites carry an identical branch count in every configuration that compiles them,
so no ordinal means two different things. gcovr's merged report is still
produced and published unchanged; the gate reads the union.
The branch gate falls from 83.00 to 77.20 and no coverage was lost doing it.
83.00 was a percentage of 4773 counted branch instances; 77.20 is a percentage
of the 3596 branches that exist.
The gate keeps a margin because the tree has not earned losing one. Over four
clean seven-configuration runs on an unchanged tree, lines did not move at all
and branches read 2792, 2792, 2796 and 2798. All ten outcomes that move are
ordinal 8 or 10 of the twelve-branch TX_TRACE_IN_LINE_INSERT expansion, at five
sites on the thread suspend, thread resume and ISR exit paths. Only 2790 were
covered by every run, and the fifth site first appeared on the fourth run, which
is the argument for the margin rather than against it. The gate sits below that
floor.
Verified: green at the committed values, and failing on either axis when raised
above the measurement, with the reports still written.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
common_smp/src/tx_misra.c is 832 lines of certified source whose whole body sits
under TX_MISRA_ENABLE, with no else branch at all. None of the five build
configurations defined that macro, so the file compiled to nothing in every one
of them: an object and a note file existed, no instrumented code did, and the
merged report covered 186 of the 194 files in common_smp/src while claiming to
cover common_smp/src.
The decisive argument is not the file count. The shim exists to make the
kernel's pointer conversions MISRA compliant, and the MISRA analysis will be run
against the source with the macro defined. Without this configuration the build
that is analysed and the build that is covered are different builds of the same
tree, and two certificates assessed together would be making different claims
about the same shim.
The configuration is instrumented and merged like the other five. All 108 tests
pass in it unchanged.
The port empties six more files without the macro. ports_smp defines
TX_INLINE_INITIALIZATION only when TX_MISRA_ENABLE is absent, so the six
*_initialize.c files carry their bodies here, and the report covers 193 files
rather than 186.
The gates are re-baselined rather than carried forward. The denominator gained
165 lines and 567 branches that were never counted before, so the line
percentage falls while the covered line count rises: 5283/5343 lines and
4528/5340 branches over the six configurations, against 5150/5178 and 3991/4773
over the five re-merged from the same tracefiles.
One file is still absent from the denominator and this configuration does not
reach it. common_smp/src/tx_thread_smp_debug_entry_insert.c sits entirely under
TX_THREAD_SMP_DEBUG_ENABLE, which nothing defines, and eight further files carry
regions behind it. That is the same shape as this commit and has no
monoprocessor counterpart.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
The gates were set from local runs alone and left about three lines of headroom
each. CI has since measured both suites and the margin is too thin to survive an
environment sampled that few times.
observed range gate headroom
ThreadX line 4202-4204 / 4220 99.40 7 lines
ThreadX branch 3310-3329 / 4108 79.50 44 branches
SMP line 5150-5152 / 5178 99.30 8 lines
SMP branch 3994-3999 / 4773 83.00 32 branches
Every gate now sits at two to six times the spread actually observed, where they
previously sat at about one and a half. Nothing real is let through by that: a
regression worth catching moves tens of lines, not two, and these ratchet upward
as each batch of coverage work lands.
The variance is the resume-from-ISR path and the timer information path, which
are reached or missed according to how the host schedules the run rather than by
anything the tests arrange. Covering them by construction is what removes the
need for the margin.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
certification branch
The regression workflows ran on ubuntu-latest with nothing pinned, and the
coverage numbers they produce are certification evidence. Two things follow from
that, and the first is urgent rather than theoretical.
These sources declare cmake_minimum_required(VERSION 3.0.0) and CMake removed
compatibility below 3.5 in 4.0, so this branch stops configuring the day the
hosted image moves to CMake 4.x -- no change here, no warning. Worse, the branch
did not need GitHub's help to get there: scripts/install.sh ran
pip install --upgrade cmake
which installs exactly the CMake version that cannot configure these sources. It
also installed gcovr 4.1, which is far too old for the tracefile and threshold
options the coverage scripts use, and called sudo, which does not exist in a
container.
The job now runs inside debian:trixie, which carries GCC 14, gcc-multilib and
CMake 3.31.6. Being below 4.x, that configures the sources with no policy
override and no edit to CMakeLists.txt. The coverage build is 32-bit, so multilib
is not optional. install.sh installs the distribution toolchain plus gcovr 8.6,
prints all three versions into the job log, and then asserts them: GCC major 14,
CMake major below 4, gcovr exactly 8.6. The image tag is a moving target that
point releases rebuild, and a toolchain that drifts should fail by name and
number rather than quietly produce evidence built with something else.
git is installed before the checkout, because actions/checkout otherwise falls
back to a REST API tarball and leaves no .git behind, and the commit the evidence
was built from is part of the evidence. The Pages preparation step no longer
calls sudo, since the steps run as root in the container.
Nothing gated on the coverage number before: it was published, not enforced.
coverage.sh --merge now ends with a gcovr pass carrying --fail-under-line and
--fail-under-branch, after the reports are written so a failed gate still leaves
the report that explains it.
ThreadX 99.50% line 80.00% branch
SMP 99.40% line 83.00% branch
These are a ratchet, raised as each batch of gaps closes, and deliberately not
100%: a gate nothing can pass gets switched off. They sit below the measured
figures rather than at them, because the same tree does not measure identically
twice -- branches on the resume-from-ISR path are reached or missed depending on
how the host schedules the run, and two clean runs of the ThreadX suite differ by
2 lines and 19 branches. A gate set at the best observation fails on a tree
nobody changed.
Verified by raising each threshold above the measurement: line returns 2, branch
returns 4, the merged report is still written, and run.sh reports the failure
rather than swallowing it -- a run with all 95 tests passing exits 2 when the
line gate is not met.
regression_test.yml now triggers on the certification branch as well as master.
regression_template.yml needed no filter; it is workflow_call only.
Publishing stays on master. The deploy job replaces the project's public coverage
site wholesale and its condition fires on any push, so without a guard the first
push of this branch would have published its numbers as the project's. The
certification branch produces its evidence as workflow artifacts instead.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
Only default_build_coverage carried -fprofile-arcs, because the gate was the
build type and it is the only one of five configurations whose name ends in
_coverage. The other four built and ran every test and their coverage was
discarded. That is not redundancy thrown away: each configuration selects a
different set of TX_ feature macros, so the code the other four compile was
absent from the denominator rather than uncovered in it, and 100% against that
report would not have been a measurement of the kernel.
TX_COVERAGE instruments a build regardless of its name, defaulting to OFF so a
single configuration built by hand behaves as before. coverage.sh gains a
--merge mode that unions the per-configuration JSON tracefiles, and
cmake_bootstrap.sh runs it after the test loop so a local run produces the same
merged report CI reads. The workflow template sets TX_COVERAGE for build and
test, and coverage_name moves to the merged report.
Measured on the ThreadX suite, all 95 tests passing in each configuration:
default_build_coverage 3827 valid 3827 covered
disable_notify_callbacks_build 3767 3767
stack_checking_build 3857 3856
stack_checking_rand_fill_build 3862 3861
trace_build 4123 4108
merged 4220 4204
The denominator grows by 393 lines and the figure moves from 100.00% to 99.62%.
The second one is honest, and the drop is the point rather than a regression:
the denominator now includes code the old report never counted. The union also
contains a file the old report did not contain at all --
tx_thread_stack_error_handler.c compiles only under TX_ENABLE_STACK_CHECKING, so
it was not listed at 0%, it was simply absent. 177 files becomes 178.
The scoping is the positional search path, not --object-directory. Left as
--object-directory while the gcovr root moved to the repository root, all five
configurations reported the union of every configuration's gcda: five identical
reports, and a merge of five copies of the same thing. Nothing about the
resulting percentage looks wrong, which is what makes it worth stating here.
Coverage collection moved out of test(). A failing ctest used to abort the
function before coverage.sh was reached, so the run whose behaviour changed was
the one that produced no report, and the gcda already existed by that point.
Collection now runs after the whole test loop, one configuration at a time,
which is also what --merge needs: every tracefile has to exist before the union
is taken. A coverage failure turns the run red but does not overwrite a test
failure's status.
Per-configuration output moved under coverage_report/per_configuration/ and is
excluded from the deployed artifact. The deploy merges the ThreadX and SMP
artifacts into one tree and every configuration directory has the same name in
both, so left at the top level one suite's would overwrite the other's on the
published site.
coverage.sh now fails when a report contains no files. gcovr treats an empty
report as success: it exits 0 and writes line-rate="1.0" alongside
lines-valid="0", so every downstream consumer reads "no data at all" as "100%
covered", and a coverage threshold cannot catch it because an empty report
passes any threshold.
The tx and smp runners are symlinks to scripts/cmake_bootstrap.sh, so they are
covered by the one change there.
Assisted-by: Claude Code (Opus 5) <[email protected]>
|
|
|
