P8 host guard: && -> || (live false FAIL on voffice1); dc0 MAAS path restored ...

P8's guard warned only when BOTH terraform.tfstate and opentofu/.terraform were
absent. voffice1 has no state file but does have .terraform -- D-128 has it run the
INNER tofu roots, and `tofu init` leaves a provider cache -- so neither warn branch
fired, P8 ran a real `tofu plan`, and it died on a gitignored tfvar -> exit 1 -> hard
FAIL on the very host the Stage-5 runbook designates for preflight. terraform.tfstate
is the OWNERSHIP marker; .terraform is only a cache. Both must be present to evaluate.

No test caught it because T38 removes the whole opentofu/ directory, so both markers
vanish together; the fixture itself sat in the defective state and passed only because
the && was wrong. Fixture now creates both markers; new T39 covers voffice1's real
shape (cache present, state absent -> WARN). Mutation: reverting || to && turns T39
and only T39 red; script restored sha256-identical.

Also: the dc0 MAAS path was down -- the vr1-dc0-region profile points at an SSH
forward (127.0.0.1:5241) that died with the 05:48 rack reboot. Restored and proven
(plan vr1-dc0 PASS exit 0, was REFUSE exit 3). Not an SEC-010 puncture: an SSH forward
originates on the rack. voffice1 pulled to HEAD. Preflight re-run: 12 [FAIL] lines,
11 = the ruled-accepted P5 set compared by identity, 1 = P8.

tests/preflight 38 -> 39/39; gauntlet ALL GREEN (96); repo-lint 0 fail / 1 warn.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Both DC VMs 416 -> 480 GiB; 128 GiB swap; new preflight gate P8 (substrate drift) ...

POST-BOOKEND: the GA-R4 bookend (4b8ba3c) was committed BEFORE this work, so the
ledger close summary and the 08-01 sweep do not cover it. The ledger line is
AMENDED in this commit rather than left stale.

Operator: "Option 2 look sgood me. I approve the sequencing, process as
autonomously as possible" -- +64 GiB to EACH DC with swap first, over my
recommendation of +48. The deciding arithmetic, put up before the ruling: at +64
each, 994 GiB allocated against a 1007.4 GiB host = 13.4 GiB residual while the
host's OWN measured footprint is 18.0 GiB, with 1 GiB swap free. Swap is what
makes +64 safe, hence the sequencing.

PROBLEM FIXED, MEASURED: dc0's rack ran 402 GiB of inner guests in a 409 GiB host
(98.3%), leaving 7 GiB for the rack OS, mirror, snap proxy and page cache. Each
rack now has ~71 GiB.

SWAP: operator-run -- sudo -n is NOT available on vcloud so that half could not be
automated. /swap2.img 128 GiB, 600 root:root, live + fstab. 135 GiB total.

CHANGED THROUGH TOFU, the only correct place: a virsh setmaxmem would have been
reverted by the next apply. The derivation now lives IN the variable comment.

PLAN ASSERTED ON CONTENT (2026-07-20: an in-place apply silently regenerated 9
node MACs): per DC 1 in-place, 0 create/destroy/replace, 0 MAC changes, memory the
only changed attribute. A THIRD resource appeared and was resolved BEFORE
applying: module.office1_opnsense ~ id = 2 -> 11 sits under "has changed outside
of OpenTofu" (drift OBSERVED), not "will be updated in-place" (action PLANNED) --
libvirt's domain id is a runtime value. It was not touched.

dc1 FIRST AS CANARY, and it answered the open question: the in-place update
BOUNCES the guest (domain id 9 -> 12, uptime 0 min). Guest sees 472.2 GiB.

>>> FINDING: vr1-dc0-maas-01 (MAAS region) and vr1-dc0-juju-01 (juju controller)
have autostart=disable. <<< Only the edge auto-recovers. Checking first is the
only reason the dc0 bounce did not come back looking catastrophic. Both started by
hand and verified. LOGGED NOT FIXED: a host reboot leaves dc0 with no region and
no controller -- a standing exposure independent of this change.

RESULT: host used 809 -> 63 GiB, available 197 -> 944 GiB; the restarts also
released the stranded RSS (vvr1-dc1 354 -> 12 GiB), discharging 08-01 sweep F1 as
a side effect. Recovery verified end to end: MAAS region API 000 -> 502 -> 200 (a
real boot progression), snap proxy listening, mirror 200, juju controller model
"Last connection: just now".

GATE P8 -- substrate drift. Built because answering "does tofu need updates
logged" produced a measurement: preflight.sh and pre-flight-checks.sh contained
ZERO tofu plan checks and the opnsense drift had sat unseen. DESIGN POINT:
-detailed-exitcode returns 2 for BOTH a pending change and a harmless observation,
so P8 asserts on CONTENT -- pending ACTION FAILS, observed-only drift WARNS,
unrecognised shape REFUSES. A gate permanently red on benign drift gets ignored.
It WARNS rather than fails when it cannot look, since the outer root lives on
vcloud and "not the substrate host" is a legitimate state.

Harness 33 -> 38 (T34-T38). Baseline proven by stashing: my first draft's 7
failures were MINE, not pre-existing. Three mutations each killed a NAMED test,
script restored sha256-identical. T38 exercises the absence by removing the
fixture STATE, never the fake tofu from fakebin -- the real tofu is on the system
PATH and a "closed" PATH would fall through to the LIVE substrate. PROVEN LIVE:
tofu exit=0, 0 pending, 0 drift -> [ok] zero diff.

SCOPE: tofu owns the SUBSTRATE only. MAAS carves, rack services and the v6 node
carve are script-driven by design (Model B / D-123); P8 does not check them.

gauntlet ALL GREEN (96); repo-lint 0 fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

P7: the Octavia amphora PKI becomes an actual pre-deploy gate ...

Operator: 'Yes, I accept the recommendation. Convert to a gate.'

THE GAP: octavia-pki.sh was invoked by NOTHING executable. P4's CHECK 0 asserts
only that the overlay FILE EXISTS, and the harness runs against throwaway
fixtures -- so 'gauntlet ALL GREEN' said nothing about the live PKI and the only
real check ran when a human remembered to type it. That is how F9 survived.

Three problems solved, each from a measured precedent:

1. rc 3 must not reach note()'s *) arm. octavia-pki.sh uses exit 3 for REFUSE,
   but note() calls any rc>=3 'UNEXPECTED exit code'. P7 maps rc explicitly, so
   a REFUSE on the headend FAILS and is NAMED as an unevaluated trust domain.
2. HOST AUTHORITY. The PKI lives on the headend (D-109 (b)); elsewhere P7 says
   NOT EVALUATED (WARN) -- never a silent pass, never a hard FAIL for being on
   the jumphost. The binding is read from the SAME host-identity file
   octavia-pki.sh reads, so they cannot drift. P5's F6 defect designed out.
3. A PASS VERDICT IS NOT SUFFICIENT. While os-public-hostname is unset, A12/A13
   RECORD a wrong-zone cert rather than failing it, so verify exits 0 on exactly
   the defect this gate exists to catch. P7 also requires the literal line
   'DNS SANs are all in this DC.s expected zone'. Satisfiable by construction
   now that GEN.c derives the zone and both DCs were reissued.

Also skips cleanly for vr0-dc0 (a gate that fires on a target it does not apply
to is how gates get bypassed) and FAILS CLOSED if the checker is absent.

HARNESS 26/26 -> 33/33. Ten existing cases went RED when P7 landed -- correctly,
the fixtures had no octavia-pki.sh. Fixed by making mkfix model a healthy headend,
NOT by patching cases one by one: that is the moment a gate gets demoted to a
warning to go green. T27 is the case that matters -- a wrong-zone cert on which
verify exits 0 must still FAIL P7.

Two harness bugs of my own, caught by running not reading: a stub emitted DC.s
where the real line has an apostrophe, then \x27\x27\x27 produced three of them
because \x27 in a printf FORMAT already yields one. Fixed the stub, not the gate.

Preflight's overall verdict is still FAIL -- that is P5 working as designed on
the pre-existing red register. P7 itself passes on the headend.

gauntlet ALL GREEN (89); repo-lint 0 fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

P4 is DC-aware, 3.7 closed, and lib-net's dc1 arm populated ...

DC references in pre-flight-checks.sh went from 1 (a comment) to 45. CHECK 2/4 now iterate the
selected DC's own HOSTS and resolve system_ids by pinned boot MAC on VR1 -- the cause of 8 of
P4's 11 fatals, which were the gate probing VR0's openstack0..3. Role-node scoping reuses
maas-role-tags.sh's existing D-134 utility-band rule rather than a second copy; the D-104
controller at .5 is named and excluded with its reason, its MAAS record still asserted, and
deliberately not asserted Ready since it is legitimately Deployed after bootstrap.

The two VR0-inherited expectations were RETIRED, NOT DELETED: VID 103 (D-133) and the
metal-admin gateway pin (D-134 -- measured, 10.12.8.1/10.12.68.1 are held by nothing, so the
pin was the defect). A spurious gateway still warns and a missing provider-public gateway still
fails, so both checks kept their teeth.

Item 3.7's guards are not skips: each guarded branch asserts D-133's own invariant instead --
VID must be 0 on VR1, and the metal-internal link must be MAAS type == physical, which asserts
the flat carve and is falsifiable. An UNREADABLE VLAN now REFUSES rather than satisfying the
untagged branch: `.vlan.vid // 0` had rendered "missing" and "untagged" identically.

F5: DC is validated against the D-119 token set and refuses before any gate runs, is exported,
and appears in the header and verdict -- keeping the PREFLIGHT: <verdict> token contiguous,
which ten existing cases grep.

R9 residue: dc1 arm populated (KEYSTONE_VIP_DEFAULT=10.12.64.50). The D-133 guard HELD --
METAL_INTERNAL_VID/IFACE stay unset on both VR1 arms, verified independently here, not from
reported output. Two dc-selector assertions were replaced with the new invariant and explained
in-file, never deleted to go green. The arm is hand-authored with a drift check; R9's generator
half is still owed and is recorded as such.

Corrected from prediction to measurement, and it changes the remaining work: all four other
DC-dependent consumers carry set -u, so adding the selector ALONE to
phase-04-network-{create,verify}.sh makes them ABORT with PLANE_GW[...]: unbound variable on
dc1 (reproduced). The hardcoded PROVIDER_CIDR must be derived in the same edit.

Harnesses: pre-flight-checks 31/31 (new), preflight 26/26 (was 16), dc-selector 69 (was 45).
repo-lint 0 fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

R15(3): preflight no longer fails open on an unexpected sub-gate exit ...

Reproduced first, in isolation with the exact note() from preflight.sh:27 --
sub-gate exits 127 (command not found), 126 (not executable), 130 (interrupted)
and 3 (undefined) ALL left "PREFLIGHT: PASS -- clear to add-model / deploy".
This is the gate that authorises the deploy.

rc=2 handled PER GATE rather than blanket-remapped, because R15's stated
rationale is only partly right on measurement: repo-lint and pre-flight-checks
both DOCUMENT rc2 as a legitimate warning, while provider-bundle-check and
creds-matrix use it only for could-not-evaluate. A blanket 2->FAIL would have
turned the standing L1 legacy-ASCII warn into a deploy blocker. channel_assert
uses 2 for BOTH meanings, so it is ambiguous in the checker and kept WARN; its
missing-PyYAML root cause is caught by P2 anyway, which shares it and is strict.
Flagged rather than guessed -- fixing that belongs in channel_assert.py.

Harness 16/16 (was 10): T11-T13 lock the fail-open cases, T14/T15 lock
could-not-evaluate -> FAIL, and T16 locks the non-over-correction (P1 rc=2 stays
WARN). Gauntlet ALL GREEN (82). Real-repo preflight still exits 1 for the same
known reasons with no new failure classes fired.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

D-137 ruling 1 EXECUTED: tier 1 wired as the blocking preflight P5 ...

Operator answer, exact utterance: "wire tier 1 as the blocking preflight Pn"
(2026-07-26), resolving the open sequencing question. Recorded on D-137.

preflight.sh gains P5: credential matrix (D-137 tier 1), blocking, with
worst-exit aggregation, placed AHEAD of the stage-2 reminders block (now P6)
so its verdict participates in the deploy decision. Tier 1 only -- it is
offline and its findings ARE the three ruled failure classes. Tier 2 needs
--remote/--privileged and a caller-supplied --pending-stage, which do not
belong in an unattended gate; that stays an open decision.

FAIL-OPEN in my own wiring, caught by the new test: a missing creds-matrix.py
makes python3 exit 2, which `note` maps to WARN -- so deleting the gate would
have downgraded it to a warning rather than stopping the deploy. P4 already
guarded this class; P5 did not until T9 failed. Now guarded explicitly.

tests/preflight 7 -> 10: T8 a red register FAILS the deploy (blocking, not
advisory); T9 fails closed when the checker is absent; T10 the gate runs
before the reminders block.

CONSEQUENCE, not overclaimed: preflight exits 1 with P5 among the reasons,
but preflight was ALREADY exiting 1 before this change (P4: octavia-pki.yaml
absent, MAAS unreachable from the jumphost). P5 adds a fifth reason to an
already-red gate; it flipped nothing and closes no open deploy path.
Verified by stashing the change and re-running.

Gauntlet ALL GREEN (80), repo-lint 0-fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Test patches

Patches