# Session changelog 2026-08-01 -- Stage 5 deploy prep (dc0)

Session opened after `/clear` with the GA-R1 bookend: `git pull` (already current),
`repo-lint` 0 fail / 1 warn, `ledger-scan` reconciled against the 08-01 close summary
(3 open decisions, 26 open SEC, next-free D-140 / DOCFIX-207 / BUNDLEFIX-053 -- all
matching). `run-tests-all` deliberately SKIPPED at open (no tooling change yet; the
08-01 close recorded 96 GREEN) and run in full at item 3 once a script changed.

Status claims live in `docs/CURRENT-STATE.md` ONLY. Nothing here is a status claim.

---

## Item 1 -- Four stale processes cleared from the jumphost (operator-approved)

**What.** Three orphaned background polling loops (PPID 1, ~34h old, from before the
`/clear`) and one suspended Claude Code session (7 days, state `T`, 417 MiB) were
terminated on vcloud.

**Why.** Two of the three loops were structurally incapable of ever exiting:
`until ! pgrep -f run-tests-all` matches **its own `bash -c` command line**, so the
guard can never be true. Proven by measurement rather than argued -- `pgrep -f
run-tests-all` returned both loops AND the probe asking the question, while
`pgrep -af 'scripts/run-tests-all\.sh'` returned no real script. The gauntlet they
waited to announce had finished 34 hours earlier, and its payload (`ALL GREEN (92)`)
was itself superseded by the current 96. The third loop awaited a
`maas=(Ready|Failed)` pattern in a file with 0 matches, unmodified since 07-30 and
with no open writers.

**How.** Identity asserted immediately before each signal (PPID==1 + cmdline
substring), with a hard PROTECT list (live session, its shell, tmux server, the
live daemon) that REFUSES rather than signals. TERM for the loops; the suspended
session needed CONT+TERM and then SIGKILL, because its TERM handler resumed, touched
a terminal it does not own (`pgid` != `tpgid`), took SIGTTIN and re-stopped. It had
no children to orphan. Read-back: all four gone, all four protected alive, zero
zombies, zero `T`-state processes.

**NOT a leak, checked rather than assumed:** the `claude daemon` at PPID 1 with a
dead `--spawned-by` pid is the LIVE supervisor (5 unix sockets, 23 fds,
`auth: proactive refresh succeeded` 4h earlier). Outliving its spawner is what
`--origin transient` does. Reporting it as stale would have been a false finding.

**Two defects in the sweep's own instruments, recorded because they are reusable:**
(i) `grep -v grep` **eats `pgrep`** -- the substring matches -- so the first shell
inventory read clean while filtering out exactly the processes being hunted;
(ii) the `pgrep -f run-tests-all` probe matched itself, reproducing the bug live.
Guards: match on a PATH (`pgrep -f 'scripts/run-tests-all\.sh'`) or `pgrep -x`, and
bound waiters with `timeout` so a wrong predicate expires instead of spinning.

**Revert.** None possible or wanted (processes). Nothing in the repo changed.

---

## Item 2 -- The dc0 MAAS path was down; DIAGNOSED, then restored (operator-gated)

**What.** Every dc0 MAAS call from voffice1 failed. `maas-profile-assert.sh
vr1-dc0-region hot-kid` REFUSED; `dc-node-v6-verify.sh plan vr1-dc0 --profile
vr1-dc0-region` -- the exact command captured PASS/exit 0 at 01:53Z -- returned
**REFUSE, exit 3**.

**Root cause, measured.** The `vr1-dc0-region` profile is registered against
`http://127.0.0.1:5241/MAAS/api/2.0/` -- an SSH port-forward on voffice1, not the
region's real address. That forward **died when the dc0 rack rebooted at 05:48:34**
during the 08-01 containment-VM resize. Nothing listens on `127.0.0.1:524x`; no
`ssh -L` process survives. Documented at `changelog-20260730-dc0-region-migration.md:408`
and `CURRENT-STATE.md:452`.

**This settles an SEC-010 question raised during the diagnosis and answers it NO:**
the working state at 01:53Z was **not** a puncture. An SSH forward terminates on the
rack and originates the API call there, which is precisely the rack-originated traffic
SEC-010 permits. No kernel forwarding was ever involved and nothing ruled was violated.

**How restored.** The documented command, with `-o ExitOnForwardFailure=yes` added so
a failed forward cannot exit 0 silently (declared deviation, not verbatim):

    ssh voffice1 "ssh -f -N -o BatchMode=yes -o ExitOnForwardFailure=yes \
      -L 127.0.0.1:5241:10.12.8.6:5240 jessea123@172.31.0.2"

Pre-checked that nothing was already listening, so tunnels could not stack.

**Proven, not assumed.** Endpoint answers 200 with real capability JSON;
`plan vr1-dc0` PASS exit 0 (was REFUSE exit 3); `maas-profile-assert vr1-dc0-region
hot-kid` OK exit 0 with the control `admin`-asserted-as-`hot-kid` still FAILing exit 1,
so the gate retains its failing direction.

**FINDING, logged not fixed (hard rule 1).** The entire dc0 MAAS toolchain --
preflight P4, `maas-profile-assert`, `maas-role-tags`, `dc-plane-ipam`,
`dc-node-v6-verify`, `dc-node-v6-carve` -- depends on a **session-lifetime tunnel with
no unit, no restart and no dependency on the rack it traverses**. Any rack reboot
silently disarms all of them, and they fail in a shape that reads as "the region is
unreachable" rather than "your transport is down". Same class as the D-126/D-128
base-leg problem `site-baseleg.sh` exists to solve, except this reach is
APPLICATION-layer, so `site-baseleg.sh` does not cover it and D-138 ruled out a
host-side L3 leg. Whether it gets a durable unit is an unruled operator decision.

**Also logged not fixed:** `dc-rack-net.sh`'s `DNS_UPSTREAM="10.10.0.20"` still points
node DNS at voffice1's BIND across the fiber, while the 07-30 record says node DNS now
uses the DC-local `10.12.8.6` and lists "rack DNS forwarder upstream" as OWED. The gate
PASSes because it checks units and legs, not the upstream's correctness.

**Revert.** `ssh voffice1 "pkill -f '5241:10.12.8.6'"`. Nothing in the cloud was touched.

---

## Item 3 -- preflight P8: host guard used `&&` where both markers are required (FIX)

**What.** `scripts/preflight.sh:284`, one operator character:

    - elif [ ! -f opentofu/terraform.tfstate ] && [ ! -d opentofu/.terraform ]; then
    + elif [ ! -f opentofu/terraform.tfstate ] || [ ! -d opentofu/.terraform ]; then

plus a comment block stating why the two markers are not equivalent evidence.

**Why -- a LIVE false FAIL, not a review finding.** P8 hard-FAILED the dc0 preflight on
voffice1 with `tofu plan exited 1 ... variable "vr1_dc0_rack_transit_peer_ip" is not set`.
voffice1 has NO `terraform.tfstate` but DOES have `opentofu/.terraform`, because D-128
has it run the INNER tofu roots and `tofu init` leaves a provider cache. Under `&&`
neither warn branch fired, so P8 ran a real `tofu plan` that died on a gitignored tfvar.
**`terraform.tfstate` is the OWNERSHIP marker; `.terraform` is a cache present on any host
that ever initialised any root.** The gate therefore false-FAILED on the exact host the
Stage-5 runbook designates for preflight, contradicting its own rule at `preflight.sh:274`
("'not the substrate host' is a legitimate state, not a missing gate") and its own
rationale that a permanently-red gate gets ignored.

**Why no test caught it.** T38 removes the WHOLE `opentofu/` directory, so both markers
vanish together and it can never distinguish them. **The fixture itself sat in the
defective state** -- it created `.terraform` and no state file, and worked only BECAUSE
the `&&` was wrong. Fixed: `mkfix` now creates both markers, which is what a real
substrate host looks like.

**New harness case T39** -- provider cache present, state absent (voffice1's actual
shape) -> must WARN, not FAIL. **Proven able to fail:** reverting `||` to `&&` turns
**T39 and only T39** red (`rc=0 want=2`); script restored from a pristine snapshot and
verified sha256-identical afterwards.

**Stated limit, because the mutant's failure mode is not the live one:** the harness's
fake `tofu` returns zero-diff, so the mutant yields `rc=0` rather than reproducing the
live `exit 1`. T39 pins the BRANCH SELECTION, which is the defect; it does not reproduce
the downstream tofu error.

**Deliberately NOT changed (scope):** the `*)` catch-all still FAILs on a non-0/2 tofu
exit. On the substrate host a `tofu plan` error IS a real failure, and demoting it would
weaken the gate.

**Verification.** `tests/preflight` 38 -> **39/39 PASS**; gauntlet **ALL GREEN (96)**;
repo-lint **0 fail, 1 warn** (standing legacy-ASCII carve-out).

**Revert.** `git revert <this commit>`, or restore `&&` at `preflight.sh:284` and drop
T39 plus the `mkfix` state-file line.

---

## Item 4 -- voffice1 clone pulled to HEAD

**What.** `git pull --ff-only` on voffice1: `844b2e4` -> `f79c9e8`, tree clean, same
branch. Both clones now report identical HEAD hashes.

**Why.** The 08-01 close recorded voffice1 36 commits behind (F2 of that sweep); it had
since closed to 3, but those 3 carried gate **P8** -- so running preflight there would
have silently used yesterday's gate set. This is the 07-27 stale-clone hazard on the
D-128 Plane-2 host.

**Revert.** `git -C ~/openstack-caracal-dc-dc reset --hard 844b2e4` on voffice1 (not
wanted; the remote is authoritative).

---

## Not done, and why

- **Step 3.5** (`add-model vr1-dc0` + spaces gate + `apt-mirror`) -- gated mutation,
  not yet approved this session.
- **D-139 step 1, the apex CREATE-only push** -- the recorded NEXT, but its ordering
  against the bundle deploy is UNRULED and is being put to the operator as its own
  GA-R5 exchange. Deploying on the current ULA carve means D-139 steps 2-6 later
  re-carve node addresses under a live cloud, which the entry warns is the expensive
  path since addresses bake into `agent.conf` at deploy time.
- **A durable unit for the 5241 tunnel** -- item 2's finding; unruled, logged only.
