# Session changelog 2026-08-01 -- Stage 5 deploy prep (dc0) Session opened after `/clear` with the GA-R1 bookend: `git pull` (already current), `repo-lint` 0 fail / 1 warn, `ledger-scan` reconciled against the 08-01 close summary (3 open decisions, 26 open SEC, next-free D-140 / DOCFIX-207 / BUNDLEFIX-053 -- all matching). `run-tests-all` deliberately SKIPPED at open (no tooling change yet; the 08-01 close recorded 96 GREEN) and run in full at item 3 once a script changed. Status claims live in `docs/CURRENT-STATE.md` ONLY. Nothing here is a status claim. --- ## Item 1 -- Four stale processes cleared from the jumphost (operator-approved) **What.** Three orphaned background polling loops (PPID 1, ~34h old, from before the `/clear`) and one suspended Claude Code session (7 days, state `T`, 417 MiB) were terminated on vcloud. **Why.** Two of the three loops were structurally incapable of ever exiting: `until ! pgrep -f run-tests-all` matches **its own `bash -c` command line**, so the guard can never be true. Proven by measurement rather than argued -- `pgrep -f run-tests-all` returned both loops AND the probe asking the question, while `pgrep -af 'scripts/run-tests-all\.sh'` returned no real script. The gauntlet they waited to announce had finished 34 hours earlier, and its payload (`ALL GREEN (92)`) was itself superseded by the current 96. The third loop awaited a `maas=(Ready|Failed)` pattern in a file with 0 matches, unmodified since 07-30 and with no open writers. **How.** Identity asserted immediately before each signal (PPID==1 + cmdline substring), with a hard PROTECT list (live session, its shell, tmux server, the live daemon) that REFUSES rather than signals. TERM for the loops; the suspended session needed CONT+TERM and then SIGKILL, because its TERM handler resumed, touched a terminal it does not own (`pgid` != `tpgid`), took SIGTTIN and re-stopped. It had no children to orphan. Read-back: all four gone, all four protected alive, zero zombies, zero `T`-state processes. **NOT a leak, checked rather than assumed:** the `claude daemon` at PPID 1 with a dead `--spawned-by` pid is the LIVE supervisor (5 unix sockets, 23 fds, `auth: proactive refresh succeeded` 4h earlier). Outliving its spawner is what `--origin transient` does. Reporting it as stale would have been a false finding. **Two defects in the sweep's own instruments, recorded because they are reusable:** (i) `grep -v grep` **eats `pgrep`** -- the substring matches -- so the first shell inventory read clean while filtering out exactly the processes being hunted; (ii) the `pgrep -f run-tests-all` probe matched itself, reproducing the bug live. Guards: match on a PATH (`pgrep -f 'scripts/run-tests-all\.sh'`) or `pgrep -x`, and bound waiters with `timeout` so a wrong predicate expires instead of spinning. **Revert.** None possible or wanted (processes). Nothing in the repo changed. --- ## Item 2 -- The dc0 MAAS path was down; DIAGNOSED, then restored (operator-gated) **What.** Every dc0 MAAS call from voffice1 failed. `maas-profile-assert.sh vr1-dc0-region hot-kid` REFUSED; `dc-node-v6-verify.sh plan vr1-dc0 --profile vr1-dc0-region` -- the exact command captured PASS/exit 0 at 01:53Z -- returned **REFUSE, exit 3**. **Root cause, measured.** The `vr1-dc0-region` profile is registered against `http://127.0.0.1:5241/MAAS/api/2.0/` -- an SSH port-forward on voffice1, not the region's real address. That forward **died when the dc0 rack rebooted at 05:48:34** during the 08-01 containment-VM resize. Nothing listens on `127.0.0.1:524x`; no `ssh -L` process survives. Documented at `changelog-20260730-dc0-region-migration.md:408` and `CURRENT-STATE.md:452`. **This settles an SEC-010 question raised during the diagnosis and answers it NO:** the working state at 01:53Z was **not** a puncture. An SSH forward terminates on the rack and originates the API call there, which is precisely the rack-originated traffic SEC-010 permits. No kernel forwarding was ever involved and nothing ruled was violated. **How restored.** The documented command, with `-o ExitOnForwardFailure=yes` added so a failed forward cannot exit 0 silently (declared deviation, not verbatim): ssh voffice1 "ssh -f -N -o BatchMode=yes -o ExitOnForwardFailure=yes \ -L 127.0.0.1:5241:10.12.8.6:5240 jessea123@172.31.0.2" Pre-checked that nothing was already listening, so tunnels could not stack. **Proven, not assumed.** Endpoint answers 200 with real capability JSON; `plan vr1-dc0` PASS exit 0 (was REFUSE exit 3); `maas-profile-assert vr1-dc0-region hot-kid` OK exit 0 with the control `admin`-asserted-as-`hot-kid` still FAILing exit 1, so the gate retains its failing direction. **FINDING, logged not fixed (hard rule 1).** The entire dc0 MAAS toolchain -- preflight P4, `maas-profile-assert`, `maas-role-tags`, `dc-plane-ipam`, `dc-node-v6-verify`, `dc-node-v6-carve` -- depends on a **session-lifetime tunnel with no unit, no restart and no dependency on the rack it traverses**. Any rack reboot silently disarms all of them, and they fail in a shape that reads as "the region is unreachable" rather than "your transport is down". Same class as the D-126/D-128 base-leg problem `site-baseleg.sh` exists to solve, except this reach is APPLICATION-layer, so `site-baseleg.sh` does not cover it and D-138 ruled out a host-side L3 leg. Whether it gets a durable unit is an unruled operator decision. **Also logged not fixed:** `dc-rack-net.sh`'s `DNS_UPSTREAM="10.10.0.20"` still points node DNS at voffice1's BIND across the fiber, while the 07-30 record says node DNS now uses the DC-local `10.12.8.6` and lists "rack DNS forwarder upstream" as OWED. The gate PASSes because it checks units and legs, not the upstream's correctness. **Revert.** `ssh voffice1 "pkill -f '5241:10.12.8.6'"`. Nothing in the cloud was touched. --- ## Item 3 -- preflight P8: host guard used `&&` where both markers are required (FIX) **What.** `scripts/preflight.sh:284`, one operator character: - elif [ ! -f opentofu/terraform.tfstate ] && [ ! -d opentofu/.terraform ]; then + elif [ ! -f opentofu/terraform.tfstate ] || [ ! -d opentofu/.terraform ]; then plus a comment block stating why the two markers are not equivalent evidence. **Why -- a LIVE false FAIL, not a review finding.** P8 hard-FAILED the dc0 preflight on voffice1 with `tofu plan exited 1 ... variable "vr1_dc0_rack_transit_peer_ip" is not set`. voffice1 has NO `terraform.tfstate` but DOES have `opentofu/.terraform`, because D-128 has it run the INNER tofu roots and `tofu init` leaves a provider cache. Under `&&` neither warn branch fired, so P8 ran a real `tofu plan` that died on a gitignored tfvar. **`terraform.tfstate` is the OWNERSHIP marker; `.terraform` is a cache present on any host that ever initialised any root.** The gate therefore false-FAILED on the exact host the Stage-5 runbook designates for preflight, contradicting its own rule at `preflight.sh:274` ("'not the substrate host' is a legitimate state, not a missing gate") and its own rationale that a permanently-red gate gets ignored. **Why no test caught it.** T38 removes the WHOLE `opentofu/` directory, so both markers vanish together and it can never distinguish them. **The fixture itself sat in the defective state** -- it created `.terraform` and no state file, and worked only BECAUSE the `&&` was wrong. Fixed: `mkfix` now creates both markers, which is what a real substrate host looks like. **New harness case T39** -- provider cache present, state absent (voffice1's actual shape) -> must WARN, not FAIL. **Proven able to fail:** reverting `||` to `&&` turns **T39 and only T39** red (`rc=0 want=2`); script restored from a pristine snapshot and verified sha256-identical afterwards. **Stated limit, because the mutant's failure mode is not the live one:** the harness's fake `tofu` returns zero-diff, so the mutant yields `rc=0` rather than reproducing the live `exit 1`. T39 pins the BRANCH SELECTION, which is the defect; it does not reproduce the downstream tofu error. **Deliberately NOT changed (scope):** the `*)` catch-all still FAILs on a non-0/2 tofu exit. On the substrate host a `tofu plan` error IS a real failure, and demoting it would weaken the gate. **Verification.** `tests/preflight` 38 -> **39/39 PASS**; gauntlet **ALL GREEN (96)**; repo-lint **0 fail, 1 warn** (standing legacy-ASCII carve-out). **Revert.** `git revert `, or restore `&&` at `preflight.sh:284` and drop T39 plus the `mkfix` state-file line. --- ## Item 4 -- voffice1 clone pulled to HEAD **What.** `git pull --ff-only` on voffice1: `844b2e4` -> `f79c9e8`, tree clean, same branch. Both clones now report identical HEAD hashes. **Why.** The 08-01 close recorded voffice1 36 commits behind (F2 of that sweep); it had since closed to 3, but those 3 carried gate **P8** -- so running preflight there would have silently used yesterday's gate set. This is the 07-27 stale-clone hazard on the D-128 Plane-2 host. **Revert.** `git -C ~/openstack-caracal-dc-dc reset --hard 844b2e4` on voffice1 (not wanted; the remote is authoritative). --- ## Not done, and why - **Step 3.5** (`add-model vr1-dc0` + spaces gate + `apt-mirror`) -- gated mutation, not yet approved this session. - **D-139 step 1, the apex CREATE-only push** -- the recorded NEXT, but its ordering against the bundle deploy is UNRULED and is being put to the operator as its own GA-R5 exchange. Deploying on the current ULA carve means D-139 steps 2-6 later re-carve node addresses under a live cloud, which the entry warns is the expensive path since addresses bake into `agent.conf` at deploy time. - **A durable unit for the 5241 tunnel** -- item 2's finding; unruled, logged only. --- ## Item 5 -- D-139 step 1 EXECUTED (apex GUA carve, vr1-dc0) See `docs/audit/d139-step1-apex-push-dc0-20260801.txt` and the CURRENT-STATE block. `--commit` exit 0, 13/13 read-back, apex 139 -> 152 verified by an INDEPENDENT query. Nothing deleted: 9 ULA prefixes and all 26 ULA VIP legs still resolve. **Revert.** Delete the 13 created prefixes (they are new and carry no dependents). --- ## Item 6 -- the VIP overlay re-rendered onto GUA, and the two apex readers taught FAMILY **What.** `overlays/vr1-dc0-vips.yaml` now carries GUA on all 39 v6 VIP legs (was 13 GUA + 26 ULA). `render/values/vr1-dc0-vips.yaml` gains the two GUA prefixes, a refreshed `apex_record`, and a new `v6_family: gua` provenance key. `scripts/render-dc-overlays.py` gains `v6_family()` + `--v6-family`; `scripts/provider-bundle-check.py` gains per-leg, per-application family resolution. **Why it was needed.** Step 1 creates GUA ALONGSIDE the ULA rows and step 6 retires the ULA rows, so for the WHOLE transition every v6-only plane has two `/64`s under one (role, kind) key. Both apex readers keyed without a family dimension, so both could only REFUSE. Measured live minutes after the push: `derive --dual-family` -> rc 2, and the gate -> "cannot evaluate barbican's dual-family vip". The D-136 pipeline was unrunnable for exactly the migration it exists to serve. **A LOSSY PATH FOUND AND NOT TAKEN.** `derive` does not reproduce the per-app `comment:` fields, so a full re-derive silently dropped 13 comments and shrank the overlay 4368 -> 3593 bytes. Reverted; the values file was edited surgically instead (3 lines), giving exactly 26 changed overlay lines -- 13 vip lines x 2 -- with every comment intact. **`derive` being lossy against its own committed values file is a real gap, LOGGED NOT FIXED.** **TWO BUGS I INTRODUCED, BOTH CAUGHT BY THE HARNESS AND NOT BY REVIEW.** (i) I inferred the family ONCE from `parts[3]`, the provider leg -- but provider-public's VIP `/64` is GUA in BOTH worlds (`:11::`), so every dc1 fixture asked for GUA bands on a still-ULA DC: 7 cases red. (ii) The corrected version still resolved bands ONCE for the whole bundle and reused them across all 13 applications, grading every later app against the first app's families. Fixed by resolving per application AND per leg, cached on the family triple. **>>> A PROPERTY WAS TRADED AWAY, AND IT IS NOT A WIN: THE GATE NO LONGER CATCHES A WRONG-FAMILY LEG. <<<** Measured: substituting a ULA admin leg into the GUA dc0 overlay now PASSES, because per-leg resolution grades it against the ULA band -- which still exists until step 6. This is NECESSARY (dc1 is legitimately ULA, so no gate can hardcode a family) but it means D-139 family conformance is now checked by NOTHING. That was already recorded as a gap (`docs/audit/d139-conformance-dc0-20260801.txt` D.2); this change makes it the ONLY thing that would catch a DC deployed on the wrong family. A ruled-table conformance gate is OWED. **Verification.** `tests/render-dc-overlays` 18 -> **23/23** (T17/T17b/T18/T19/T20), three mutations each killing named cases, script restored sha256-identical. `tests/provider-bundle-check` **55/55**. Gauntlet **ALL GREEN (96)**; repo-lint 0 fail / 1 warn. Renderer `check` reproduces the new overlay byte-for-byte. The real deploy input gates PASS with `13 clustered VIP(s) ... (13 dual-family)`, and a corrupted GUA leg still FAILS. **vr1-dc1 is UNTOUCHED** -- its apex has no GUA, so its overlay stays ULA and now gates correctly through the same per-leg path. **Revert.** `git revert `; the overlay's prior bytes are recoverable from the values file by restoring the three prefix lines and re-rendering.