| 2026-07-31 |

Build the D-135 amendment: jammy-backports at source, T10 re-pointed
...
scripts/dc-mirror.sh's generated sync script gains jammy-backports in --dist=,
with a comment at the generator recording why it is not optional: the MAAS
jammy image ships backports in its sources.list and juju's apt-mirror
model-config rewrites every suite to this mirror, so a mirror without it 404s
and every charm install hook fails. That cost 22 units today.
FIXED AT SOURCE, not only on the live rack -- a live-only edit would leave the
generator regenerating the old scope at the next install or at any future DC
standup.
tests/dc-mirror T10 asserted the old dist string verbatim. RE-POINTED to the
new invariant with the reason recorded in-file, never deleted to go green.
Mutation-proven: reverting the generator's dist line turns T10 red (23/1);
restored byte-identical, 24/24.
Gauntlet ALL GREEN (93); repo-lint 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
| 2026-07-27 |

close-out durability sweep: 4 transcript-only items landed; dc1-is-not-a-mirror warning
...
Operator-requested sweep for content that existed only in the session transcript, before
clearing. Precedent: docs/audit/queued-findings-20260726.txt. Capture:
docs/audit/queued-findings-20260727.txt.
THE CONSEQUENTIAL ONE. scripts/dc-mirror.sh's dc1 site row carried NO warning that dc1 is no
longer a mirror site, so `dc-mirror.sh install dc1` would have SILENTLY REBUILT everything the
same-day teardown removed -- units, nginx vhost, sync helper, and an ENABLED daily debmirror
timer that would start a fresh ~950G pull on a rack whose ruled artifact path is the proxy. That
is precisely the command someone would reach for after treating a failing `check dc1` as a
regression. The row now states dc1 is proxy-only per the D-135 amendment, that `check dc1` FAILS
BY DESIGN, and that `install dc1` is a deliberate strategy change and never a repair. The row is
RETAINED so its measured values survive a legitimate rebuild.
QUEUED, not built (hard rule 1): a RUNTIME guard in do_install for a site whose ruled path is
not the mirror. A comment is strictly weaker than a guard, and prose-only prevention has already
failed twice on this project (DoD item 8 missed dc1; D-137's wiring sat as prose and missed BOTH
DC standups). Recording that the interim is an interim.
A WRONG CLAIM CORRECTED. docs/audit/stage4-mirror-gate-20260727.txt said `systemctl reset-failed`
would re-arm the paused dc1 debmirror. It cannot: reset-failed on the SERVICE does not start an
INACTIVE TIMER. The real vector was a REBOOT -- `enabled` starts the timer at boot and
`Persistent=yes` fires it IMMEDIATELY because its window had passed. Correction APPENDED rather
than edited in place: the file is dated evidence, so a wrong analytical claim is superseded
openly, not quietly rewritten.
THE GENERALISABLE LESSON, previously unrecorded, is now platform-traps section 5:
5a "stopped is not dormant across a reboot" -- enabled + inactive + EMPTY NextElapse looks
harmless and is not; plus reset-failed does not re-arm, and Result=signal/status=15 on a
oneshot means it was SIGTERM'd and never wrote a terminal state (so a stale RUNNING is a
corpse, not progress).
5b a `oneshot` with RemainAfterExit=yes does NOT undo its work on stop, so stopping it proves
NOTHING about dependents -- which is exactly why the dc1 teardown test deleted the live
address and route instead of stopping the mirror net unit.
Three rows added to the verbatim-error index. Note dc0-mirror-sync.timer is still enabled with
Persistent=yes, so 5a describes dc0 today -- benign while its syncs succeed, same shape.
HARD RULE 2. dc-cache-proxy.sh stated that the two net units coexist safely on one host as
FACT. It is REASONED (idempotent replace verbs, identical measured values), never MEASURED --
the proxy has only run on dc1, where the mirror unit was then removed, so nobody has run both.
Marked accordingly, with an instruction to verify rather than trust the sentence.
VERIFICATION: dangling-reference sweep over docs/, runbooks/, scripts/ and the skill -- every
path this session introduced or cited RESOLVES; all pre-existing dangles are legitimate
(deleted-as-history, not-yet-built, or the deliberately-absent octavia PKI overlay preflight P4
fails on). ledger-scan reconciled against the machine-derived block. Gauntlet ALL GREEN (81),
repo-lint 0-fail, dc-mirror 24/24, dc-cache-proxy 20/20.
Revert: text + comments only; the capture correction is an append.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Stage 4 close-out: fix dc-mirror.sh check false-greening the mirror gate
...
Operator directed closing Stage 4 before resuming the DC1 Stage-5 chain. Verifying the DoD's
fifth bullet ("per-DC mirror reachable") found that the check the gate would close on COULD
NOT FAIL.
scripts/dc-mirror.sh:271-273 tested `[ -f last-sync.status ]` and printed the file's contents
behind an unconditional "OK" -- the status word was displayed but never asserted on, so the
check exited PASS regardless. MEASURED on both racks:
- dc0 read "FAIL 2026-07-27T00:54:54Z ubuntu=255 uca=0" -> printed OK, PASS
- dc1 read "RUNNING 2026-07-23T21:49:35Z", four days stale,
left by the debmirror the D-135 amendment killed by signal -> printed OK, PASS
GA-R6 requires a stage to close on a NAMED executable check whose captured output the closing
commit cites. A check that cannot distinguish "synced" from "failed" or "died mid-run" is not
that check.
FIXED: the status word is case-analysed. OK* passes. FAIL* misses. An ABSENT file misses (it
was a passing `note`, but nothing has attested the content). An unrecognised word REFUSES
instead of defaulting to success. RUNNING* is deliberately neither pass nor plain fail --
in-flight and died-mid-run are indistinguishable from the file alone, so it reports an explicit
UNKNOWN and cross-checks the unit, naming a RUNNING marker with no active unit as a corpse.
Harness 19 -> 24; T20-T24 lock each status word and T24 locks out the return of the exact
defective shape.
PROOF, re-run against both live racks -- the gate is now honestly NOT MET for the first time:
dc-mirror check (dc0): FAIL / dc-mirror check (dc1): FAIL. Capture
docs/audit/stage4-mirror-gate-20260727.txt.
SUBSTANCE (measured, not inferred):
- dc0's mirror CONTENT is complete: 949G ubuntu + 342M cloud-archive, all three dists plus
pool. Only the overnight INCREMENTAL failed -- "Download of dists/jammy/Release failed: 500
read timeout" aborted the ubuntu leg with 255; the UCA leg succeeded. Transient upstream
fetch failure, not mirror damage. Timer re-arms 2026-07-28 00:14:48.
- dc1's PROXY -- its RULED artifact path per the D-135 amendment -- checks PASS genuinely
(apt-cacher-ng on .4:3142 serving archive + UCA Release 200). Its dormant fallback debmirror
is dormant only ACCIDENTALLY: the timer is `enabled` with an EMPTY next-elapse because the
unit sits in failed/Result=signal, so the ruling is enforced by nothing and a reset-failed
would re-arm a 330G -> ~949G pull.
ALSO RECORDED, not yet actioned: DoD bullet 6 ("NTP from the DC's own OPNsense edge") is STALE
and unsatisfiable -- superseded by D-129(iv) 2026-07-21 ("Keep MAAS hierarchy", no NTP role on
the edge) -- and survives in four surfaces needing a DOCFIX before it can be checked at all.
The node-side half of bullet 5 needs an operator decision (gated rescue-boot check vs its own
gate row per GA-R6 E3); nodes are powered off by the READY-handoff ruling, so no node-side
probe can run as things stand. There is no conditional close.
No live state was altered: both racks were read only. repo-lint 0-fail, dc-mirror 24/24.
Revert: git revert the scripts/dc-mirror.sh + tests/dc-mirror/run-tests.sh changes; the check
returns to passing unconditionally on last-sync.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
| 2026-07-25 |
dc-mirror: note mirror.uoregon.edu as preferred upstream (future)
...
Operator perf note (2026-07-25): mirror.uoregon.edu/ubuntu/ is the closest/fastest
mirror for this deployment -> prefer it over archive.ubuntu.com for the MAIN archive
debmirror in a future mirror-tuning pass. Header comment ONLY (not applied; does not
change the generated sync unit). Scoped to the main archive -- the UCA cloud-archive
host is separate and uoregon does not mirror it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
| 2026-07-23 |
Mirrors installed both racks, syncs RUNNING; dc1 edge pf incident resolved; HOME fix (harness 19)
...
Item 9: fresh dc1 edge passed no LAN traffic -- pf ruleset never
regenerated after v4 addressing (set-interface reloads the interface,
not the filter; dc0 masked by 07-20 plugin work). configctl filter
reload -> egress 0%/301. Appendix-A + standup-DoD + script amendment
queued to stage close. Item 10: installs PASS 15/15 both racks, first
sync HOME-under-systemd failure fixed in the unit (T19), reinstalls +
syncs running (dc0 400M/dc1 343M in the first minute). Ledger in-flight
entry added; CURRENT-STATE same-commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
dc-mirror.sh shipped (D-135 item 1): per-DC apt+UCA mirror, harness 18/18, gauntlet 77 ALL GREEN
...
Site-keyed (dc0/dc1 MEASURED rows), check/install/sync; net layer = D-134
utility .4 alias + default route via the DC edge (egress path MEASURED
working; edge DNS answers). debmirror GPG-verified jammy triple + UCA
caracal, nginx on the utility address, daily timer, niced/idle-IO. Install
fails loud on missing packages with the exact apt line (net-first ordering
resolves the no-egress chicken-and-egg). Gauntlet capture + CURRENT-STATE
same-commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|