Skip to content

Verify cached jq before use and preserve fresh coverage proofs - #643

Merged
pithead-reviewer[bot] merged 3 commits into
developfrom
astra/621-rigforge
Nov 7, 2026
Merged

pithead-reviewer[bot] merged 3 commits into
developfrom
astra/621-rigforge

Conversation

@pithead-developer

Copy link
Copy Markdown
Contributor

What's the problem

Issue #621: tests/coverage.sh trusted an executable jq cache hit without checking its pinned
checksum. A corrupt or stale executable could reach the coverage container. Failed tool setup
also left the previous coverage report in place.

This checkout has no shared hosted archive/wheel cache or cached miner/build tree. Docker inputs
already use digest pins. Adding shared caches before measuring setup misses would expand the
rollout without the evidence the issue requires.

How I fixed it

  • Key the existing jq download cache by release, Linux architecture and SHA256.
  • Copy each hit into private per-run storage, verify those exact bytes, then mount that copy
    read-only. Refuse corrupt, symlink and directory entries before Docker starts.
  • Download into private storage and publish only verified bytes through an atomic rename. A
    warm hit works with a read-only cache directory; legacy unqualified entries are ignored.
  • Clear previous reports before tool setup and clean temporary files on success or failure.
  • Stamp each invocation with the real Git commit and a checksum manifest of tracked and
    non-ignored untracked inputs, including file type, executable status and link identity. Escape
    names so newline-bearing paths cannot forge rows; do not follow symlinks outside the checkout.
  • Select Linux/amd64 explicitly for the existing kcov digest and static jq input.
  • Log cache hit/miss, setup seconds and verification seconds. Preserve all existing fresh
    installation/build commands and gates. Add an offline black-box regression to make test-suite
    and document the rollout requirements in tests/README.md.

No existing test assertion, workflow, version, release note, miner source, build tree or coverage threshold
was changed. No shared cache was enabled. No commit, push, GitHub write, remote connection or bench
job was made.

Here is the proof

Base candidate: 81ec49584fa88196ef915435f7a768d0c20b866c; the implementation is the uncommitted
delta in this checkout. The before regression uses git show HEAD:tests/coverage.sh in a fresh
scratch directory, so the working implementation is never swapped out.

Commands and results

from pathlib import Path
import shutil
import subprocess
import tempfile
with tempfile.TemporaryDirectory(prefix='rigforge-621-before-') as scratch:
    checkout = Path(scratch) / 'source'
    subprocess.run(['git', 'clone', '--shared', '--no-checkout', '--quiet', '.', str(checkout)], check=True)
    tests = checkout / 'tests'
    tests.mkdir()
    (tests / 'coverage.sh').write_bytes(subprocess.check_output(['git', 'show', 'HEAD:tests/coverage.sh']))
    shutil.copyfile('tests/test-coverage-cache.sh', tests / 'test-coverage-cache.sh')
    result = subprocess.run(['bash', str(tests / 'test-coverage-cache.sh')])
    print(f'baseline_exit={result.returncode}')
    assert result.returncode == 1

Executed with python3 -: the original script fails the new regression with
FAIL: cold cache run failed, exit 1. The same regression against the implementation,
bash tests/test-coverage-cache.sh, passes on macOS. The harness verifies the actual coverage
script using synthetic curl/Docker boundaries and real checksum/jq tools; it is not a kcov run.

  • bash tests/test-tool-pin-watch.sh: all nine pin-watcher cases pass, including malformed input.
  • shellcheck --severity=warning tests/coverage.sh tests/test-coverage-cache.sh: passes.
  • shfmt -i 4 -d tests/coverage.sh tests/test-coverage-cache.sh: passes.
    Also passes with the checksum-verified CI-pinned shfmt 3.14.1 binary.
  • git diff --check: passes.
  • FILE_BUDGET_REQUIRE_BASE=1 bash scripts/lint-file-budget.sh: passes, including the base ratchet.
  • make test: failed in lint, exit 2 after 1092.676 seconds; the test recipes did not run.
    The output filter retained only the make failure, so it cannot identify the lint diagnostic.
    A separate unfiltered make lint and standalone make test-suite were started for that reason.
  • make test-suite: passes, exit 0 in 1698.900 seconds. The main runner reports 2,431 passed
    and 0 failed; all subsequent credential, readiness, clone-pin, rebuild, pin-watcher and new
    coverage-cache regressions complete successfully. The existing explicit macOS skip remains:
    SKIP: tune bench black-box timing tests run in the Linux CI jobs (#292).
  • make lint with installed shfmt 3.13.1: ShellCheck and topology/file-budget gates pass;
    formatting fails on unchanged tests/run.sh. CI pins shfmt 3.14.1. Reran make lint with
    that version first on PATH, downloaded into temporary storage and verified against its official
    release SHA256 b7c872db63553ccffc7253aba3ed7d4885a27d83f1ba567b1138c6315a5847e5: passes,
    exit 0. Existing tests were not reformatted.
  • make coverage before the change: cold jq acquisition reached Docker, then failed with
    exec format error; elapsed 8.341 seconds. The output-filter pipeline returned zero, so this is
    recorded as a failed coverage attempt, not a successful command.
  • make coverage after the change, cold cache: setup 2 seconds, verification 0 seconds;
    failed at kcov startup with exec format error; command exit 2, elapsed 3.502 seconds.
  • make coverage after the change, warm cache: setup 0 seconds, verification 0 seconds;
    subprocesses failed with Exec format error. Stopped only this session's container, identified
    by its exact checkout mount; command exit 2, elapsed 67.892 seconds. No coverage PASS was obtained.
  • make test-e2e: both deploy and appliance containers attempted; both failed to launch Bash
    with exec format error. Exit 2, elapsed 12.648 seconds. Neither mode is claimed as passing.
  • Requesting the repository's Ubuntu digest as Linux/arm64 also failed: Docker could not overwrite
    the locally registered digest, and a manifest-inspection attempt failed verification. No shared
    image was removed or replaced; the supplementary native test below uses a separately inspected
    immutable Linux/arm64 image already available to the engine.
  • npx --yes markdownlint-cli2@0.23.3 tests/README.md: the default npm cache was not writable.
    Retried with a fresh temporary npm cache; lint found the pre-existing extra blank line in
    untracked ISSUE.md. That operator input was left unchanged.
  • npx --yes markdownlint-cli2@0.23.3 in a temporary directory containing the unchanged repository
    Markdown lint configuration, tests/README.md and PR-BODY.md, using a fresh temporary npm
    cache outside the scanned directory: passes, two files, zero issues. This isolates the authored
    prose from untracked issue input. A final retry with the npm cache inside the scan directory
    also linted dependency READMEs and failed; moving that temporary cache outside restores the
    intended two-file check without changing the lint configuration.

These are individual local observations, not a before/after CI performance comparison. The
whole-second setup counters cannot resolve subsecond verification cost. Failed Docker startup
times do not measure valid cold/warm coverage runs.

Before/after coverage behavior, inputs and platform manifest

Block Before After
jq input jq 1.8.2, Linux/amd64, SHA256 b1c22172dd303f3be49e935aa56aa48a8b7a46e0bc838b4997d3bb451495870f Same release, platform and digest
Cache identity Unqualified executable filename Release, platform and digest in filename
Cold acquisition Download into shared cache path; verify on miss Private download; verify; atomic publication
Warm acquisition Executable hit skips verification Copy into private storage; checksum before any consumer; no cache writes
Stale/corrupt input Executable corrupt hit reaches Docker Legacy identity ignored; corrupt/nonregular entry refused before Docker
Per-run fixture Cached file mounted directly Fresh private directory and read-only verified jq mount
kcov input v42, digest 30c442617f3d8e040bf0ec2cba19cc2ee517b668f3a3d50b2d3de1c435138a8a Same digest; explicit Linux/amd64 selection
Coverage invocation tests/run.sh, kcov --clean, ptrace capability and seccomp exception Unchanged
Coverage scope rigforge.sh and util/proposed-grub.sh Unchanged
Previous output Removed after successful tool preparation Removed before tool preparation, including failures
Candidate changes Every successful invocation runs the suite again, without a source stamp Real Git SHA plus sorted source identities; changed SHA and changed working input both regenerate provenance and rerun the consumer
Report guards Missing report and zero instrumented lines fail Unchanged, regression-tested
Normalization Cobertura paths become repository-relative Unchanged, regression-tested
Total floor Committed floor 80; explicit report-only override Unchanged, both success/failure and override tested
Patch floor diff-cover 10.6.0, 90%, PR base fetched with full history Unchanged workflow
Python/API Existing Python-presence branches and explicit skips inside the measured suite Unchanged; the separate credential regression remains in the hosted Test suite job
Proof reuse No cache of coverage reports or PASS results None added; failed setup cannot retain the previous report

The new regression adds: cold acquisition with an ignored legacy cache; read-only warm hit;
corrupt executable refusal; bad checksum download; interrupted download; symlink and directory
refusal; recovery after manual deletion; changed source producing a new report; changed jq digest
selecting another cache entry; Docker failure; zero-line and missing reports; below-floor failure;
explicit report-only behavior; normalized paths; fixed scope/platform/clean invocation; private
storage permissions and cleanup; newline and leading-dash filenames; link identity, missing
targets and executable-mode changes; and an absent tracked input. The candidate-change check uses an actual disposable Git clone
and switches its reference to an already-existing parent commit without creating any commit.
It verifies the real production source-commit.txt and source-files.sha256 outputs. A separate
negative control removes only the provenance block and fails on the missing commit stamp.

Exact cache-verification negative-control command, executed with python3 -:

from pathlib import Path
import shutil
import subprocess
import tempfile
with tempfile.TemporaryDirectory(prefix='rigforge-621-cache-before-') as scratch:
    checkout = Path(scratch) / 'source'
    subprocess.run(['git', 'clone', '--shared', '--no-checkout', '--quiet', '.', str(checkout)], check=True)
    tests = checkout / 'tests'
    tests.mkdir()
    source = Path('tests/coverage.sh').read_text()
    source = source.replace('if ! echo "${JQ_SHA256}  ${JQ_BIN}" | sha256sum -c - >/dev/null; then', 'if [ "$CACHE_STATE" = miss ] && ! echo "${JQ_SHA256}  ${JQ_BIN}" | sha256sum -c - >/dev/null; then')
    (tests / 'coverage.sh').write_text(source)
    shutil.copyfile('tests/test-coverage-cache.sh', tests / 'test-coverage-cache.sh')
    result = subprocess.run(['bash', str(tests / 'test-coverage-cache.sh')], capture_output=True, text=True)
    print(result.stderr.strip())
    print(f'unverified_hit_exit={result.returncode}')
    assert result.returncode == 1

Result: FAIL: corrupt cache reached Docker, exit 1. This restores only the original warm-hit
verification bypass in a scratch implementation; the shipped regression is unchanged.

Exact provenance negative-control command, executed with python3 -:

from pathlib import Path
import shutil
import subprocess
import tempfile
with tempfile.TemporaryDirectory(prefix='rigforge-621-provenance-before-') as scratch:
    checkout = Path(scratch) / 'source'
    subprocess.run(['git', 'clone', '--shared', '--no-checkout', '--quiet', '.', str(checkout)], check=True)
    tests = checkout / 'tests'
    tests.mkdir()
    source = Path('tests/coverage.sh').read_text()
    start = source.index('git rev-parse --verify HEAD >')
    end = source.index('# --- Run the suite under kcov', start)
    (tests / 'coverage.sh').write_text(source[:start] + source[end:])
    shutil.copyfile('tests/test-coverage-cache.sh', tests / 'test-coverage-cache.sh')
    result = subprocess.run(['bash', str(tests / 'test-coverage-cache.sh')], capture_output=True, text=True)
    assert result.returncode == 1 and 'source commit does not match the tested HEAD' in result.stderr, result.stderr
    print('PASS: removing provenance fails the source-commit regression (exit 1)')

Result: the driver passes after verifying the regression fails with exit 1 and the specific
missing-source-stamp diagnostic. This experiment is separate from the shipped regression script.

Native Linux/arm64 regression command, executed with python3 -:

from pathlib import Path
import shutil
import subprocess
import tempfile
with tempfile.TemporaryDirectory(prefix='rigforge-621-linux-') as scratch:
    checkout = Path(scratch) / 'source'
    subprocess.run(['git', 'clone', '--no-hardlinks', '--no-checkout', '--quiet', '.', str(checkout)], check=True)
    (checkout / 'tests').mkdir()
    for name in ('coverage.sh', 'test-coverage-cache.sh'):
        shutil.copyfile(Path('tests') / name, checkout / 'tests' / name)
    result = subprocess.run(['docker', 'run', '--rm', '--platform', 'linux/arm64', '-e', 'GIT_CONFIG_COUNT=1', '-e', 'GIT_CONFIG_KEY_0=safe.directory', '-e', 'GIT_CONFIG_VALUE_0=/src', '-v', f'{checkout}:/src:ro', 'ubuntu@sha256:24737cef17f4e22b1977df7d9a05e77c4e28b7da5b8911f8d8ccda136bf3573c', 'bash', '-c', 'apt-get update -qq && apt-get install -y -qq jq git ca-certificates >/dev/null && bash /src/tests/test-coverage-cache.sh'])
    raise SystemExit(result.returncode)

Result: passes in a fresh container with real package acquisition. This is supplementary native
Linux regression proof using an already-available immutable image. It does not replace either
existing Linux/amd64 e2e mode or the real kcov gate. The worktree's external Git metadata is copied
into a disposable local clone for this container; only that clone is mounted read-only.

The source manifest distinguishes regular/executable files, symlinks and missing tracked files.
It hashes regular-file bytes through stdin and symlink target text plus the terminating newline
without following links. Names use Bash shell escaping under the C locale; sorted records remain
single lines even for unusual filenames. Ignored/generated files are excluded.

Before/after required-gate and input inventory

Every row below is retained at its existing frequency. No required status was renamed, removed,
made optional or replaced by cached proof. Hosted runners remain Ubuntu 24.04.

Workflow job/status name Before proof/input/platform After
Lint (yamllint) Pinned yamllint; pinned/checksummed actionlint; YAML and workflow checks Identical
Lint (markdownlint) Node 22; package-manager cache disabled; pinned Markdown linter Identical
Lint (shellcheck + shfmt) Fresh pinned/checksummed tools; shell, topology and file-budget gates; full history/base required Identical
Test suite make test-suite; all existing entries below; simulated Linux CPU profiles and deprecated Darwin paths All retained plus the offline cache regression
Stock home setup (historical) Exact historical revision b2d4c3d9ca1da74d0e6d5ccce772d401acc8be03; fresh private home; real packages/build/setup; denied-entrypoint negative control; uninstall Identical
Stock home setup (current) Exact checked-out candidate; separate fresh private home; real packages and pinned compilation; service/API readiness; failed apply, recovery and uninstall Identical
End-to-end (Docker) Digest-pinned Ubuntu, Linux/amd64; two fresh containers; real prerequisite package acquisition and GNU tools; writable disposable fixtures Both mode definitions preserved; local execution failed as recorded above
Coverage (kcov + diff-cover) Entire measured suite, total floor, nonempty report, scope and patch floor Same proofs plus verified tool input

Branch-protection settings are not present in this checkout; no remote required-status audit was
performed. The documented independent adversarial-review requirement remains unchanged. The
local reviewer passes are not a GitHub head-SHA review status.

make test-suite retains tests/run.sh, test-contract-credentials.sh,
test-access-token-readiness.sh, test-docs-clone-pin.sh, test-staged-rebuild.sh, and
test-tool-pin-watch.sh; only test-coverage-cache.sh is added. The measured suite retains
test-suite-liveness.sh, test-control-hardening.sh, test-appliance-pools.sh, test-donation.sh,
test-sister-reconcile.sh, onion-proxy.sh, test-api-refresh-liveness.sh,
test-control-upgrade.sh, test-e2e-apt-prereqs.sh, test-e2e-gate-fail-closed.sh, and
test-control-spool-cap.py, including Python compilation/server checks and their explicit skips.
The separate credential/readiness regressions retain their extracted functions from
e2e-pithead.sh, e2e-pithead-control.sh, e2e-pithead-tokens.sh and e2e-control-replay.sh.

The untouched measured suite contains 188 named blocks, 1,996 static assertion-helper calls
(944 equality, 722 contains, 136 absent, 194 return-code), and 26 lexical Python asserts. These
are source-inventory counts, not runtime pass counts. Every block below is before = after:

  • 1–23: pool passthrough; pool validation; pool arrays/defaults; sanitization; workspace/token
    parsing; pool user/token default; parse errors; first-run prompts; donation validation;
    append-once; line removal; generic Linux; API default; server CPU; desktop CPU; Darwin;
    multiple pools; explicit user; TLS fingerprint; proxy; miner-user/MSR; IPv6/TLS/empty token;
    generation idempotency.
  • 24–37: GRUB hardware math; co-resident headroom; reserved pages; early-close fixture; pool
    ceiling; ceiling propagation; reserve/thread parsing; ceiling parsing; numeric rejection;
    thread cap; keep-existing guard; kernel merge; parameter removal; rewrite escaping.
  • 38–48: commit pin; RAM build cap; error step; running-verb errors; staged rebuild/pruning;
    already-built detection; rebuild flag; upgrade/help/unknown command; stopped state;
    rebuild command surface; upgrade check.
  • 49–60: consistency sweep; command surface; apply/bench; pool reporting; deliberate stop;
    runtime state; apply while stopped; autotune target; autotune reconciliation; sister API
    reconciliation; control reconciliation; worker-root validation.
  • 61–70: autotune timer tri-state; watchdog timer; watchdog rendering failure; watchdog keys;
    watchdog health; autotune tri-state; target ranking; upgrade retune; live-worker wait; bearer handling.
  • 71–79: support-bundle redaction; dry-run; completion drift; curl failures; status summary;
    unavailable API; unavailable API during tuning; HugePages shapes; hashrate parsing.
  • 80–97: unattended dependencies; versioned tools; CPU-power package probing; jq prerequisite;
    package-manager branches; deployment pointer; Darwin processes; Darwin login agent; reboot gate;
    miner-user service; MSR preset/RMW; ownership handoff; miner-user rendering; service override;
    full deployment; rerun idempotency; CLI install flag; noninteractive output.
  • 98–117: doctor health; restricted API; sister-feed implication; hashrate limits; BIOS advisory;
    lockdown; ISA preflight; control receiver; pool connection; NUMA; BIOS flow; inactive service;
    ownership reconciliation; script-directory symlinks; CLI-link guard; MSR verification;
    unverifiable MSR; miner-user/root MSR; exposure advisory; MSR preset table.
  • 118–120: uninstall reversion; uninstall refusal; appliance runtime disable.
  • 121–154: hill climb; bench-crash recovery; target announcement; tune history; median noise;
    minimum delta; variance; large-page knob; power/temperature; temperature fallback; power helpers;
    load power sampling; built-in power; efficiency acceptance; scoring fallback; efficiency winner;
    no-power fallback; live API tuning; aborted-tune restoration; A/B confirmation; frequency rounding;
    thermal rejection; explicit false; MSR tuning; reservation threads; reservation sizing;
    grow-only pages; ceiling bounds; appliance mode; live autotune trial/merge; aborted-autotune
    restoration; grid search; JIT pages; SMT candidates.
  • 155–162: backup/restore; backup abort cleanup; restore abort cleanup; support-bundle abort
    cleanup; SemVer; reference config; minimal config; sister API keys.
  • 163–179: firewall lifecycle; API server/timer; response files; persistent server; dual-stack
    binding; control keys/auth; control units; atomic commit; effective-config redaction; provenance;
    pool-secret preservation; fast-path allowlist; watchdog reuse; fast-apply run state; stopped-rig
    full apply; watchdog-render failure propagation; control rollback.
  • 180–188: nested-sudo git; Bash/Python key parity; control server; real-e2e performance-record
    gate; checkout traversal; dashboard/nonvacuous-offline guard; shared rig lock; wire fixtures;
    tree hygiene.

Docker's unchanged 119 static helper assertions retain appliance configuration immutability,
runtime unit placement/enables, no deployment package writes and repeat idempotency; the deploy
pass retains dry-run, generated defaults, service hardening, ownership, firewall, successive runs,
tuning and uninstall. Actual prerequisite acquisition remains before stubs; compilation in these
two containers remains stubbed as before. Genuine compilation stays in both stock-home jobs and
the existing release/hardware gates.

Identity check: git diff --quiet -- tests/run.sh .github/workflows/ci.yml tests/e2e/linux.sh tests/e2e/in-container.sh tests/test-stock-home.sh passes. Their SHA256 values before and after:

File Identical SHA256
tests/run.sh 7eff04040ea5f36b056f5d58215fa4ffead516cbe5e4161e335412e487c77cf0
.github/workflows/ci.yml d583e7f60df43769b105dd9b1b53d977e087c2d44cbfef75264d633808174cfd
tests/e2e/linux.sh 06962253a6256bba77dfb48e1fec5df94b58c14040cea2061d33438c8eb4ef40
tests/e2e/in-container.sh 5609c2d0a4b364076bab978299cef4e12b6c9ed465942c2495385414ec14c2ff
tests/test-stock-home.sh cbd3be36c99a977184eaba0c2a148206d1d963ccf0d75db797b024a3529b5355

Open acceptance and unavailable proof

Shared-cache rollout remains blocked on complete cold/warm runtime assertion/input/platform
manifests, required-status verification and a week of setup timings, CI median/p90, verification
time, failures and occupied hours. The hand-written source manifest above preserves the old/new
proof set; it does not substitute for missing runtime evidence or the parallel shared-manifest
work. The local source stamps identify inputs but do not assert test success. Missing, truncated or skipped proof
does not count as acceptance; existing explicit skips and row ownership remain unchanged.

Hosted stock-home historical/current jobs, successful native Linux/amd64 coverage and both Docker
modes, 90% patch-coverage enforcement, and the physical release/worker-to-coordinator proofs still
need their existing environments. Donation builds, cold starts, compilation, install/recovery/
uninstall, Python/API gates and both independent bench-ci checks retain their existing owners and
frequencies. No KVM, live bench, hardware or production proof ran in this session; no capacity was
added. Native macOS e2e is still a separate deprecated/manual gate and was not run.

Closes #621

Draft from an operator-run gpt-6-astra session (owner request, 2026-11-01). The operator reviews and gates it before it leaves draft.

🤖 Generated with Claude Code

Part of #621. Implemented by an operator-run gpt-6-astra session at the owner's request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pithead-developer pithead-developer Bot added the operator-action Needs the operator's hands, not an answer: nothing waits on it and no ruling clears it label Nov 2, 2026
@pithead-developer

Copy link
Copy Markdown
Contributor Author

Operator: this stays a held draft until RigForge 1.18.1 is tagged (due Fri 2026-11-06, Sat 11-07 at the latest). 1.18.1 is cut from develop, so merging this first would widen that release. After the tag: rebase on develop, run the agent review, and request the owner's approval.

@pithead-developer pithead-developer Bot added this to the maintenance & gates milestone Nov 2, 2026
@pithead-developer pithead-developer Bot added stage/operator parked for the operator and removed operator-action Needs the operator's hands, not an answer: nothing waits on it and no ruling clears it labels Nov 2, 2026
@pithead-developer

Copy link
Copy Markdown
Contributor Author

Operator: released to the runner (owner 2026-11-06). New work goes to the local Flash-Next model first; Claude only after three local rounds or for sensitive work.

@pithead-developer pithead-developer Bot added stage/needs-work returned by the reviewer, red CI, or conflicts with the base and removed stage/operator parked for the operator labels Nov 6, 2026
@pithead-developer pithead-developer Bot added stage/in-progress stage/needs-check and removed stage/needs-work returned by the reviewer, red CI, or conflicts with the base stage/in-progress labels Nov 7, 2026
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@pithead-developer pithead-developer Bot added stage/needs-review green or pending CI, no reviewer verdict on the head yet stage/in-review a reviewer session is on it now and removed stage/needs-check stage/needs-review green or pending CI, no reviewer verdict on the head yet labels Nov 7, 2026
@pithead-reviewer

Copy link
Copy Markdown
Contributor

PASS

What I investigated

PR #643 (head 8c43f6a) closes the part of #621 that this repository can finish alone: tests/coverage.sh now verifies jq on every use and stamps each run with the commit and a source manifest. The new regression test fails against develop's script and against a script with only the warm-hit bypass restored, and all 10 required checks are green on the head. Shared-cache rollout and the week of measurements are named open in the body, which the operator's 2026-11-02 scope note allows.

  • Claim: Verify cached tools on every use and retain fresh installation proofs #621 asks for jq cache hits in tests/coverage.sh to be revalidated, digest-keyed entries, a changed SHA to regenerate commit-stamped outputs, stale or corrupt input refused before proof, and every existing gate kept. I read the issue and its comments. Only the pithead-developer account commented; the operator scope note allows leaving the shared-manifest and fleet-data criteria open. The prefetched linked issues (bench-ci#2360, pithead#3888, bench-ci#2362, pithead#3891) are all readable and are parents or siblings that this PR does not depend on.

  • gh pr view 643 --json ...: base is develop, draft, head 8c43f6a. gh run list --commit <head> shows the CI and Security runs on that SHA, both success.

  • gh pr checks 643 and the statusCheckRollup list 10 checks, all SUCCESS: Lint (yamllint), Lint (markdownlint), Lint (shellcheck + shfmt), Test suite, Stock home setup (historical), Stock home setup (current), End-to-end (Docker), Coverage (kcov + diff-cover), Secret scan (gitleaks), Workflow audit (zizmor).

  • gh run view 42352125362 --log on the head: the Test suite job prints PASS: coverage cache is digest-keyed, verified, staged, and fail-closed. The Coverage job prints coverage: jq cache=miss setup_seconds=0 verification_seconds=0, then TOTAL: 95.33% (floor: 80%) and COVERAGE: PASS. The SKIP lines in that log are pre-existing python3-absent skips inside the kcov container.

  • git diff --stat origin/develop...HEAD: only Makefile (+1 line), tests/README.md, tests/coverage.sh and tests/test-coverage-cache.sh changed. ci.yml, tests/run.sh and the e2e scripts are untouched, so no existing gate or assertion was weakened.

  • bash tests/test-coverage-cache.sh on this Mac: PASS, exit 0 (23.7 s).

  • Control 1: develop's tests/coverage.sh swapped into a scratch worktree under $TMPDIR, then the new test run: FAIL: cold cache run failed.

  • Control 2: head's coverage.sh with only the warm-hit checksum skip restored ([ "$CACHE_STATE" = miss ] && ! sha256sum -c): FAIL: corrupt cache reached Docker. The test therefore catches the bypass Verify cached tools on every use and retain fresh installation proofs #621 is about.

  • Attack on the main risk, that kcov --clean wipes the provenance stamps written into the bind-mounted /src/coverage: the kcov source (src/reporter.cc:393) uses clean-output only to skip unmarshalling the old database. It does not delete the output directory, so source-commit.txt and source-files.sha256 survive.

  • FILE_BUDGET_REQUIRE_BASE=1 bash scripts/lint-file-budget.sh: file budget OK, exit 0. The new test is registered in Makefile test-suite (line 24), not sourced.

  • Comment discipline: no # comments were added in the coverage.sh hunks. grep on tests/test-coverage-cache.sh finds only heredoc shebangs. The tests/README.md prose is documentation and is exempt.

  • Topology: detect-topology.py over the PR body, the PR comments and the issue comments each printed clean, exit 0.

  • Security: no workflow changes and no shared cache enabled, so there is no untrusted-PR cache write path. The jq copy is verified after the copy and mounted read-only from a mode-700 private directory. Symlink and directory cache entries are refused, and the download is published by atomic rename.

  • Not run: no bench-ci job is required or quoted. The diff touches only tests/, Makefile and docs, which is the tiers 1-3 row, and the GitHub checks cover it. No Docker or make check was run on this Mac.

  • PR body, 'Here is the proof' and 'Base candidate': The body is stale against the head. It says the implementation is 'the uncommitted delta', that no commit or push was made, and that make test failed in lint. All of its local runs predate the head's green CI checks. Refresh it to cite the head SHA and the CI results. This does not block the merge.

  • tests/coverage.sh:79-95: The per-file source manifest (type, SHA256 and shell-escaped name for every tracked and untracked file) goes beyond the commit stamp the issue asks for. It is tested, but a smaller change would stamp only the commit plus a git status --porcelain digest. Over-engineering note only, not a blocker.

  • tests/README.md:13-36: The new README paragraph recounts rollout policy in prose that duplicates the PR body. Keep it, but trim it if a later PR touches that file. Not a blocker.

Reviewed head: 8c43f6ad63069276dcfbb5a9ec26e4dc103834f8.

@pithead-reviewer
pithead-reviewer Bot marked this pull request as ready for review November 7, 2026 08:45
@pithead-developer pithead-developer Bot added stage/awaiting-merge the reviewer's PASS is on the head; merging when checks and the base allow and removed stage/in-review a reviewer session is on it now labels Nov 7, 2026
@pithead-reviewer

Copy link
Copy Markdown
Contributor

Owner brief for 8c43f6ad6306

What it does. The coverage script (tests/coverage.sh) now checks the pinned jq tool's checksum every time it runs, including when it comes from the cache. It copies the cached file into private temporary storage, verifies those exact bytes, and mounts that copy read-only into the coverage container. It also clears the old coverage report before setup, so a failed setup can't leave a stale report. Each run records the Git commit and a per-file checksum list of the sources, and a new offline test in make test-suite covers all of it.

Why. Issue #621 (milestone: maintenance & gates) asks that cached tools be verified on every use and that fresh-build proofs be kept. Before this, a cached jq that was executable was trusted without a checksum check, so a corrupt or stale file could reach the coverage container. A failed setup also left the previous report in place. This PR closes the part of #621 that this repo can do alone. Shared hosted caches and the week of before/after measurements are left open on purpose.

Risk. Low. Only the coverage script, a Makefile test line, a README section and one new test file change. Nothing in the product itself is touched, and no shared cache is enabled. The script now requires a Git checkout and forces Linux/amd64 for Docker, which fails on arm64 Macs without emulation. The author's local runs hit exec-format errors for that reason, and no real kcov coverage pass was obtained locally. Old unkeyed jq cache entries are silently ignored, so the next run downloads jq again. Nothing is irreversible.

Proof. CI: 11 success. Reviewer: PASS at 8c43f6a by pithead-reviewer, round 1 (recorded 2026-11-07T08:45:01Z). Bench: no job cited in the PR body.

What to check.

  • tests/coverage.sh:41-80: the jq cache flow. Check the order of copy, verify, chmod and atomic publish, the exit trap, and that a corrupt cache entry stops the run before Docker starts.
  • tests/coverage.sh:79-95: the per-file source manifest. The reviewer called it more than the issue needs; decide if you want that much code. Check that symlinks aren't followed and that odd file names are escaped.
  • tests/coverage.sh:36-37: rm -rf "$OUT" now runs before the Docker check and tool setup. Confirm that clearing old reports early is what you want, since a failure now leaves no report at all.
  • The PR body is stale. It says the change is uncommitted and that make test failed in lint. The head's CI is green, so treat the body as history.

Verdict. merge

@pithead-reviewer
pithead-reviewer Bot merged commit 1156f1c into develop Nov 7, 2026
11 checks passed
@pithead-reviewer
pithead-reviewer Bot deleted the astra/621-rigforge branch November 7, 2026 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stage/awaiting-merge the reviewer's PASS is on the head; merging when checks and the base allow

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Verify cached tools on every use and retain fresh installation proofs

1 participant