diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index da0fa5f..2333676 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2237,6 +2237,47 @@ citing it would be the inverted-citation class), and states plainly that **this bug is not what blocks a v6-only Ceph plane locally**, since setting `ceph-public-network` / `ceph-cluster-network` to the v6 CIDRs bypasses the buggy call entirely. + **>>> SESSION CLOSE 2026-08-01 (GA-R4 bookend). DURABILITY TRIAD, MEASURED. <<<** + vcloud: **0 uncommitted, 0 unpushed**, HEAD `844b2e4` on `dc-dc-stage5-preconditions`. + **voffice1's clone is 36 COMMITS BEHIND upstream** (HEAD `fbe7b31`) -- NOT a loss, the + remote holds everything, but a live instance of the 2026-07-27 stale-clone hazard sitting on + the D-128 Plane-2 host; it is why `maas-profile-assert.sh` was unavailable there this session + and the wrong-region property had to be asserted inline instead. **A `git pull` there is the + next session's first action.** The dc0 rack's `~/repo-stage` (D-138 client input, NO git) + was DIGEST-COMPARED rather than assumed: `bundle.yaml`, `vr1-dc0-machines.yaml` and + `vr1-dc0-vips.yaml` all sha256-MATCH the repo, and `vr1-dc0-octavia-pki.yaml` is correctly + absent (gitignored PKI). **So the provenance gap is real and the drift is not.** + **GATES AT CLOSE, quoted:** `repo-lint` **0 fail, 1 warn** (the legacy D-001..018 non-ASCII + carve-out), 649 files; `run-tests-all` **GAUNTLET: ALL GREEN (96 harnesses)**; `ledger-scan` + 3 open decisions, **26 open SEC rows** (none opened this session), next-free **D-140** / + DOCFIX-207 / BUNDLEFIX-053. **Reconciled:** D next-free moved 139 -> 140, matching the one + D-number this session assigned; the SEC count is unchanged, matching the zero rows opened. + **CLOSE SWEEP: `docs/audit/queued-findings-20260801-stage5-ipv6-d139.txt` -- SEVEN FIRST + SURFACE items**, each grep-proven absent from every repo surface before being written. + The highest-consequence is **F1: the capacity headroom the next deploy needs is currently + HELD BY TWO IDLE VMs** -- vvr1-dc0 434 GB RSS and vvr1-dc1 371 GB, ~805 GB of 1007 GB, with + dc0's nodes powered off and dc1 carrying no model; qemu does not return guest-freed pages. + This document's own capacity record (85%, "154 GiB headroom") was computed against + ALLOCATION, not resident usage. Also FIRST SURFACE: F2 the voffice1 lag; F3 the host-lag + investigation and its NEGATIVE result (the host was idle -- recorded so nobody re-runs it, + with the 5-second sampling limit stated); F4 the rack digest MATCH; F5 that no gitignored + permission rules were added this session (allow=287/ask=11/deny=0, unchanged from open, so + the 2026-07-30 verbatim record still stands as the recovery copy); F6 a transient + `_fmtprobe.tf` during repo-lint; F7 the PreToolUse guard firing on PROSE containing a + guarded command's name. + **GA-R7 MEMORY REVIEW: CLEAN -- zero entries claiming operator policy, priority or posture** + (the 2026-07-31 violation stays corrected). One update: the instrument-currency memory gains + its NINTH instance, in a new shape -- a "closed" `PATH` that still contained the binaries it + was meant to hide, so absent-binary REFUSE cases ran the real tools and were red for the + WRONG REASON. Generalised there as: **when a negative depends on an ABSENCE, prove it rather + than arranging it.** Index line updated. + **LEDGER ROTATED (GA-R4 rule 3):** the file stood at **296** lines and this close would have + breached the 300 cap. The two oldest closed-session summaries (2026-07-29 arity/renderer; + 2026-07-30 Stage-5 preconditions) moved VERBATIM to + `docs/archive/session-ledger-rotated-20260801.md`; ledger now **282**. **No orphaned + session** -- the ledger holds no in-flight section without a close bookend, so GA-R4 rule 7 + is a no-op this close. **NO STAGE OPENED OR CLOSED**; Stage 5 remains OPEN and this is a + session bookend, not a GA-R6 stage close. - Project: Omega Cloud, VR1 DC-DC rehearsal -- a two-DC + Office1-headend virtual rehearsal on KVM (vcloud host), rehearsing the future bare-metal Roosevelt deployment (D-100, `docs/design-decisions.md:1946`). diff --git a/docs/archive/session-ledger-rotated-20260801.md b/docs/archive/session-ledger-rotated-20260801.md new file mode 100644 index 0000000..a81d330 --- /dev/null +++ b/docs/archive/session-ledger-rotated-20260801.md @@ -0,0 +1,40 @@ +# Rotated session-ledger summaries -- moved 2026-08-01 (GA-R4 rule 3 / F1) + +Moved VERBATIM from docs/session-ledger.md to restore the 300-line cap at this close. +These are HISTORY: they carry no status weight (status lives in docs/CURRENT-STATE.md). + +## SESSION CLOSE 2026-07-29 -- arity gate, renderer, ruling 3, chain audit, OSD carve LIVE (bounded, GA-R4) + +- Branch `dc-dc-stage5-preconditions`, 22 commits pushed. NO stage opened/closed. Scan unchanged: 3 decisions, SEC 21, D 138 / DOCFIX 205 / BUNDLEFIX 053. +- **VIP arity gap CLOSED + R11's three ruled gate changes.** `prefer-ipv6` is coupled to arity BOTH ways, closing the L3-9 merge order in which dropping the v6 legs exited 0. +- **RENDERER SHIPPED** (D-136 (D)), reproducing a reviewed artifact BYTE-FOR-BYTE. **RULING 3, both commits:** `bundle.yaml` is VIP-FREE; both DCs dual-family, vault `.61` / designate `.62` BUILT. +- **CHAIN AUDIT (step 6):** the adversarial lens made the chain go wrong-but-GREEN three times -- R9's ruled drift gate had never been built and dc0's overlay was gated by nothing. + Built `tests/render-drift`, added 3 refusals, fixed 2 harness cases that could not fail. +- **LIVE, each gated and independently verified:** 8 OSD volumes + 2 controller VMs applied both DCs; 9 re-commissions with `skip_networking=1`, every diff EXACTLY the new device; + MAC adoption applied (drift 0 across 20 nodes, both roots ZERO DIFF); both controllers commissioned; all ruled MAAS tags created. **THE STAGE-5 ALLOCATION BLOCKER IS CLOSED.** +- **R1 precondition 3 CLOSED BY MEASUREMENT** -- `skip_networking=1` is the vendor control, and the MAC check sits BETWEEN apply and re-commission, which is what makes 2026-07-20 non-repeatable. +- **RULED (GA-R5):** v6 IP SANs on the Octavia controller cert (D-109 note); juju controller at utility `.5` AND the octet map is a STANDING CROSS-DC STANDARD (D-134 amendment) -- + divergence between DCs at the same octet is now a DEFECT, not a local choice. +- **Octavia: clean negative.** juju 3.6 DEEP-MERGES overlay `options` (charm v12.1.1 source), so octavia-pki cannot clobber the VIP; and nothing inside Octavia requires IPv4. +- **OWNED:** three strict-bash quoting traps, two false-positive greps, one fabricated flag, one red-lint push. All corrected on-surface; traps in `docs/audit/queued-findings-20260729.txt`. +- **NEXT:** octavia-pki generation -- the last item hard-failing preflight, and the generator is now per-DC. Gauntlet ALL GREEN (87) vcloud; repo-lint 0 fail. +- Body: `docs/changelog-20260728-vip-arity-gate.md`. Status ONLY in CURRENT-STATE.md. + +## SESSION CLOSE 2026-07-30 -- Stage-5 preconditions: 19 Phase-3 items, gate integrity, per-DC Octavia PKI LIVE (bounded, GA-R4) + +- Branch `dc-dc-stage5-preconditions`, **25 commits pushed** (`ab4c842..`). NO stage opened/closed. Scan unchanged: 3 decisions, SEC 21, D 138 / DOCFIX 205 / BUNDLEFIX 053. +- **6 agents** (3 read-only recon, 4 delivery). Phase-3 re-measured **2 FIXED / 19 REMAIN / 5 NEW**, not the 21 the 07-27 doc implied; **13 items + 3.9/3.10 + R5 delivered**, phase-4 runbook 434 -> 929 lines. +- **F1 (HIGH): R7's per-DC Octavia PKI was HALF BUILT** -- subject and VIP gate per-DC, `$DC` in NO path, so generating dc1 would have DESTROYED dc0's CA and left one fixed-name overlay applying dc1's CA to dc0. Fixed end to end + REFUSE-IF-PRESENT. +- **F4:** the per-DC rename would have UN-IGNORED a CA private key in a repo SEC-004 calls PUBLIC. Glob + a `check-ignore` self-assert, proven both ways. **F2:** register was blind to a missing 2nd CA set (was RULED work, R13 Part 1). **F6:** P5 was probing the WRONG HOST'S filesystem and reporting it as fact -- 34 findings vs the true 7. +- **PKI GENERATED (operator-executed; guard blocks the mint by design, not worked around), both DCs verified PASS 29/0**, independence proven by sha256, backups **proven by restore** (PKI 12/12 vs live; tfstate byte-identical, correct serials). **E2 + E3 fully closed.** +- **`scripts/octavia-pki.sh verify` NEW** (harness 21/21): asserts the SAN set nothing asserted before (F8's SAN-less cert), and **A12 ARMS ITSELF** from `os-public-hostname` so F9 cannot be missed. +- **Gate integrity:** decorative HA, machines-overlay no-op, cloud-assert arity, G17 node check, gauntlet locale. P4 DC-aware (DC refs 1 -> 45); 3.7 closed; lib-net dc1 arm. +- **NEW ruling-shaped, now BLOCKING:** D-008 + D-106:2563 do not say whether substrate `vr1-dc1` is `dc1` by TOKEN or `dc2` by POSITION -- item 3.1's ambiguity inside a ruling, deciding cert identity. A12 REFUSES rather than picking. + **^ WITHDRAWN 2026-07-30 (DOCFIX-205): NOT a ruling gap.** D-117 ruled it 2026-07-13 (`dc1`/`dc2` retired for `dc0`/`dc1`) and says so in its own Status line; D-106 read as open only because D-117's annotation half was never executed (zero of four) while its Status line claimed "FULLY EXECUTED". Also not blocking -- A12 measures INERT and passes. Now an assertion on the derived zone; status in CURRENT-STATE.md. +- **Headend reconcile found a gate RED ON THE ONLY HOST THAT DEPLOYS** (`opentofu-validate` fmt walking gitignored tfvars). Fixed + scoped; **both hosts now ALL GREEN (89)**, repo-lint 0 fail. +- **Three wrong records found, none catchable by any gate:** D-124 told a reader to source a provider MAC from a file holding only boot MACs (would have bridged `br-ex` onto the PXE NIC); the skill cites D-107 for a tailnet path it does not rule; `office1-tailscale` is labelled a subnet router and advertises none. +- **A "backup set" asserted on three surfaces did not exist.** Built for the PKI (1.0-GEN.e) and the inner tfstates (phase-2 step 13), declared, restores proven. Forward requirement recorded: the pinned secrets-storage solution MUST absorb both. +- OWNED: the operator's own working commands were NOT in the repo until the pre-bookend sweep asked; I waved through 664 CA certs as "harmless" (integrity, not secrecy); two silenced errors (`| grep -q` under pipefail, `git pull -q &&`); four consecutive cwd mistakes. All corrected on-surface. +- **NEXT:** F8 residue (heredoc -> printf when a real generation can be run); D-137 fork-1 guard ruling (blocked correctly once, misfired 5x); preflight still has NO verified-green reading; G17 fixture-green only; the two `phase-04-network-*` scripts. +- Sweep capture `docs/audit/queued-findings-20260730.txt` (F10-F13 + 2 ruling questions). Body: `docs/changelog-20260729-stage5-preconditions.md`. Status ONLY in CURRENT-STATE.md. + diff --git a/docs/audit/queued-findings-20260801-stage5-ipv6-d139.txt b/docs/audit/queued-findings-20260801-stage5-ipv6-d139.txt new file mode 100644 index 0000000..399c3d0 --- /dev/null +++ b/docs/audit/queued-findings-20260801-stage5-ipv6-d139.txt @@ -0,0 +1,187 @@ +queued-findings-20260801-stage5-ipv6-d139.txt +============================================= +Close sweep for the 2026-07-31/08-01 session (snap proxy apply + D-139 IPv6). +Method (ruled 2026-07-31): read back over the whole session, enumerate every +finding/decision/measurement/mistake, then GREP each candidate against repo +surfaces. An item with a hit is ALREADY ON SURFACE and says where. An item with +NO hit is FIRST SURFACE and would have been lost on a context clear. + +Session body: docs/changelog-20260731-snap-proxy-apply-ipv6.md (20 items). +Status claims: docs/CURRENT-STATE.md ONLY. Nothing here is a status claim. + +-------------------------------------------------------------------------------------- +FIRST SURFACE -- these existed ONLY in the transcript. Listed first, by consequence. +-------------------------------------------------------------------------------------- + +F1. >>> THE CAPACITY HEADROOM THE NEXT DEPLOY NEEDS IS CURRENTLY HELD BY TWO IDLE + VMs, AND NOTHING SAYS SO. <<< greps: "434 GB" 0 hits, "idle VMs" 0, "balloon" + 0, "free-page" 0, "host memory" 0. + MEASURED on vcloud 2026-08-01: vvr1-dc0 RSS 434 GB, vvr1-dc1 RSS 371 GB -- + ~805 GB of the host's 1007 GB -- while dc0's nine nodes were POWERED OFF and + dc1 has no deployed model at all. qemu does not return guest-freed pages to the + host without free-page-reporting or a balloon deflate. Swap is 6.8 GB of 8 GB + used and 6.887 GB of that is ONE process, vvr1-dc1; si/so measured 0, so it is + parked, not thrashing. + WHY IT MATTERS AND WHY IT IS NOT MERELY TRIVIA: CURRENT-STATE:411 records the + capacity gate passing at "RAM 870/1024 = 85%, FIT, 154 GiB headroom". THAT + HEADROOM IS THE THING CURRENTLY OCCUPIED. A future session re-running + dc-dc-whole-host-budget.py before a deploy will read a pass that was computed + against allocation, not against resident usage. + LOGGED, NOT EXECUTED (hard rule 1): reclaiming it is a live mutation on running + DC VMs and is an operator call. + +F2. voffice1's CLONE IS 36 COMMITS BEHIND UPSTREAM. greps: "36 commits behind" 0 + hits, "behind upstream" 0 hits. + MEASURED at close: voffice1 HEAD = fbe7b31 ("Build the 2026-07-31 ruling: + renderer + invariant 9 replaced, overlays re-rendered"); upstream is 36 ahead. + Surfaced during the session as a symptom rather than a cause -- scripts/maas- + profile-assert.sh was ABSENT there, so the repo's own wrong-region gate could + not be run and the property had to be asserted inline by rack-controller + identity instead. That inline assertion PASSED, so nothing was done against the + wrong region; but the gate being unavailable on the Plane-2 host is the point. + This is the 2026-07-27 stale-clone hazard, live, on the host D-128 designates + for MAAS/NetBox/inner-tofu work. NOT A LOSS -- the remote holds everything -- + but a hazard, and the fix is one `git pull` the next session must not skip. + +F3. THE HOST-LAG INVESTIGATION AND ITS NEGATIVE RESULT. grep: "tmux" 0 hits. + Operator reported the tmux session lagging badly and asked whether anything + local caused it. MEASURED: load 1.74 on a 108-vCPU host; vmstat us/sy 0/0, id + 99-100%, wa 0, si/so 0/0 across a live 5s sample; 198 GB available + 205 GB + cache; disk 19% used with 7.7 TB free; tmux panes ~3 MB against a 50000-line + limit and 0.0% CPU; all Claude processes ~1.7 GB RSS combined. + VERDICT: nothing local was causing it -- the host was essentially idle, and the + lag is on the path or the client. RECORDED BECAUSE A NEGATIVE RESULT IS STILL A + RESULT: without this, the next person to see lag re-runs the same investigation. + STATED LIMIT AT THE TIME AND REPEATED HERE: the sample was 5 seconds, so an + INTERMITTENT burst would look exactly like this. If the lag recurs, sample long. + +F4. THE RACK'S STAGED DEPLOY INPUT IS DIGEST-IDENTICAL TO THE REPO. (Partial -- + "repo-stage" itself has 9 hits, but the MATCH is a new measurement.) + G2 (2026-08-01) recorded that ~/repo-stage on the dc0 rack is a hand-staged + copy with NO .git, i.e. no provenance. MEASURED AT CLOSE, which is the half + that was missing: bundle.yaml sha256 4c8a7852... MATCHES the repo exactly, and + so do overlays/vr1-dc0-machines.yaml (e3be85e4...) and overlays/vr1-dc0-vips + .yaml (ed19d989...). vr1-dc0-octavia-pki.yaml is absent from the repo, which is + CORRECT -- it is gitignored PKI material. + So the D-138 client host's deploy input is CURRENT, not drifted. The provenance + gap remains real; the drift did not happen. Both halves belong on the record -- + G2 alone reads worse than the measurement warrants. + +F5. NO GITIGNORED PERMISSION RULES WERE ADDED THIS SESSION. + MEASURED at close: .claude/settings.local.json holds allow=287, ask=11, deny=0 + -- byte-for-byte the same counts read at session OPEN. The verbatim record made + by the 2026-07-30 sweep (F1 there) and carried by the 2026-07-31 sweep therefore + STANDS UNCHANGED and is still the recovery copy for this gitignored file. + Recorded because "no change" is only knowable if someone checks; an unchecked + session leaves the next one unable to tell. + +F6. A TRANSIENT UNTRACKED FILE APPEARS DURING repo-lint. grep: "_fmtprobe" 0 hits. + Observed mid-session: opentofu/modules/base-image/_fmtprobe.tf appeared in + `git status` and had vanished by the time it was inspected -- almost certainly + a `tofu fmt` probe repo-lint writes and removes. Harmless in itself. RECORDED + because a stray untracked .tf appearing in a window where a concurrent session + might run `git add -A` is exactly the shape of the e57ad09 contamination + incident. LOW consequence; noted, not chased. + +F7. THE PreToolUse GUARD FIRES ON PROSE, NOT ONLY ON COMMANDS. + At close, a `python3 - <<'PY'` heredoc whose TEXT contained the literal string + "maas list" -- inside a memory-file paragraph being appended, not as a command + -- was BLOCKED by .claude/hooks/guard-destructive.py with "prints the MAAS API + key (DOCFIX-016); use 'maas admin ...' directly". The guard matched the shell + invocation's full text, which included the quoted prose. + NOT A DEFECT IN THE GUARD'S INTENT -- matching broadly is the safe direction, + and the guard has fired correctly twice this session. Recorded because the + RESPONSE matters and will recur: the correct move was to write the file with + the Write/Edit tools, changing the CONTENT and avoiding the shell, rather than + re-issuing the same shell command in a disguised shape. Re-shaping a command to + slip past a guard is the behaviour the rule against retrying refused commands + exists to prevent, and "the guard was wrong" is exactly the reasoning that makes + it feel justified. + CONSEQUENCE FOR FUTURE SESSIONS: writing documentation ABOUT a guarded command + can be blocked by the guard for that command. Use the file tools for prose. + +-------------------------------------------------------------------------------------- +ALREADY ON SURFACE (grep-verified; recorded for completeness) +-------------------------------------------------------------------------------------- +A1. Snap proxy installed, check dc0 PASS exit 0, real snap payload (hsqs) fetched + -- CURRENT-STATE + changelog items 6/7, docs/audit/dc0-snap-proxy-install-*. +A2. BUG-4 fixed at BOTH sites; BUG-3 confirmed; BUG-1's fix WRONG (a secondary + IPv4 alias is never the kernel's chosen source) -- items 2/3/7/8, T23b in-file. +A3. D-139 ADOPTED (rulings A and B, separate exchanges, exact utterances) -- + design-decisions.md, CURRENT-STATE, item 15. +A4. RFC 6724 refutation + ruling B reconsidered and CONFIRMED; struck rationale + kept in place -- design-decisions.md strike block + RULING NOTE 2026-08-01, + docs/audit/gai-conf-rfc6724-verification-20260801.txt, items 10/11. +A5. Juju/container single-address mechanism, no knob, LP #1723240 -- CURRENT-STATE, + docs/audit/v6-only-charm-viability-20260801.txt. +A6. D-139 ruling A not achievable at current charm revisions (ceph-osd both + directions, mysql URIs, hacluster ip_version, OVN encap) -- item 14 + capture. +A7. ceph couples storage+replication via ms_bind_ipv4=False; ceph-*-network CIDRs + bypass the buggy call -- item 16, CURRENT-STATE. +A8. IPv6 PROVEN on the node planes; NTP already running over IPv6; jammy-backports + serving; F7 (redeploy applies hostname) closed -- item 18, g19-ipv6-node-plane- + verify-20260801.txt. +A9. G19 built and PASSED LIVE; dc-node-v6-carve.py's v4 pivot; plan's peer + selection deployment-blind; the ninth instrument-currency harness bug -- + item 19, D-139 CORRECTION NOTE, g19-ipv6-plane-gate-build-20260801.txt. +A10. multicast_snooping=1/querier=0 is ESTATE-WIDE (both racks, WAN bridge, jumphost + uplinks) and cold-start multicast ND works across it anyway -- CURRENT-STATE, + both g19 captures. +A11. Apex IPv6 plan pulled live; zero v6 ip-ranges; v6 records are VIP-only; VR1 + diverges from VR0/Willamette -- docs/audit/apex-ipv6-plan-20260731.txt. +A12. The carve tool + its independent review (D1 "a refusal that does not refuse", + R1 Authorization header across a cross-host redirect) -- items 13/17, d139- + carve-review-20260801.txt. +A13. The juju "blocker" is D-138 working correctly; G1-G4 -- item 12 + capture. +A14. Nodes released to Ready/owner=None; D-139 execution list replaced; LP draft + written not filed -- item 20, lp-draft-20260801-ceph-osd-ipv6-static.md. +A15. Operator approval of the agent's rack permissions, with its scope limits -- + item 5. + +-------------------------------------------------------------------------------------- +CONTRADICTION DETECTOR -- measured this session vs standing documents +-------------------------------------------------------------------------------------- +C1. CURRENT-STATE's recorded root cause of the v4-only containers ("MAAS has + nothing to give for v6", "MAAS answers v4 from the dynamic range") is REFUTED by + measurement (100% available; the v4 container address sat OUTSIDE the dynamic + range). CORRECTED in the same document per GA-R1 C2. RESOLVED. +C2. D-139 ruling B's stated deciding reason (RFC 6724 precedence) is REFUTED for + glibc 2.35, which implements RFC 3484. Ruling reconsidered and CONFIRMED on its + other grounds; rationale STRUCK IN PLACE. RESOLVED. +C3. D-139's own EXECUTION list contained a silent-failure ordering. REPLACED by a + correction note. RESOLVED. +C4. scripts/dc-plane-ipam.sh:368-372 has carried the CORRECT MAAS reserved-range + behaviour since 2026-07-27 while CURRENT-STATE carried the wrong one. The doc is + now fixed; the script was right all along. RESOLVED, recorded as a reminder that + a script comment can outrank the status document. +C5. STILL OPEN, unchanged from the 2026-07-31 sweep: preflight P6 quotes "50 apps / + 97 relations" against a MEASURED 56/108; G17's text names `chronyc`, absent from + the MAAS jammy image (RE-CONFIRMED on a live node this session); the phase-4 + runbook omits the machines overlay from the dc0 deploy block and still labels + add-model/spaces "voffice1" against D-138. All DOCFIX material, none numbered. + +-------------------------------------------------------------------------------------- +DELIBERATELY NOT DONE +-------------------------------------------------------------------------------------- +N1. The dc-snap-proxy simplification (S-plan) was NOT re-derived -- the plan was lost + at a session boundary and re-deriving it is the GA-F06 record-churn case. + Consequence: the prior 53/53 mutation evidence STANDS for untouched assertions. +N2. BUG-1's real fix (probe ACL -> the rack's PRIMARY metal-admin leg) NOT applied: + editing gen_squid_conf turns the LIVE dc0 gate red until install re-runs. To land + WITH the dc1 install so both DCs move together. T23b is annotated so nobody + "fixes" the test to a wrong value. +N3. BUG-2 (`| grep -q` under pipefail) NOT fixed -- a three-script sweep, out of scope. +N4. The DOCFIX backlog (C5 above) NOT swept -- handing an agent a record-cleanup pass + mid-stage is the exact GA-F06 shape. Should ride the next surface touched for a + real reason. +N5. dc1's snap proxy NOT installed; the operator's rack-permission approval was scoped + to dc0 and does not cover the dc1 rack. +N6. The metal-admin container-family DETECTION gate (the only available mitigation for + juju's non-deterministic single-address behaviour on a dual-stack container plane) + is OWED, NOT BUILT. +N7. R1 (Authorization header across a cross-host redirect) logged not fixed -- a + redirect handler is untested code on a control path. +N8. The apex push has NOT been run. The tool is dry-run-only and nothing has been + written to NetBox. +N9. `network-get` on a v6-only bound space -- the prerequisite for D-139's step 7 -- + is STILL NOT MEASURED. It needs a deployed unit and must ride the next deploy. diff --git a/docs/session-ledger.md b/docs/session-ledger.md index 253c711..e60612e 100644 --- a/docs/session-ledger.md +++ b/docs/session-ledger.md @@ -164,41 +164,6 @@ owed, because the next session is directed straight at the juju deployment. Sessions from the 2026-07-27 Phase-0 close onward remain live below. -## SESSION CLOSE 2026-07-29 -- arity gate, renderer, ruling 3, chain audit, OSD carve LIVE (bounded, GA-R4) - -- Branch `dc-dc-stage5-preconditions`, 22 commits pushed. NO stage opened/closed. Scan unchanged: 3 decisions, SEC 21, D 138 / DOCFIX 205 / BUNDLEFIX 053. -- **VIP arity gap CLOSED + R11's three ruled gate changes.** `prefer-ipv6` is coupled to arity BOTH ways, closing the L3-9 merge order in which dropping the v6 legs exited 0. -- **RENDERER SHIPPED** (D-136 (D)), reproducing a reviewed artifact BYTE-FOR-BYTE. **RULING 3, both commits:** `bundle.yaml` is VIP-FREE; both DCs dual-family, vault `.61` / designate `.62` BUILT. -- **CHAIN AUDIT (step 6):** the adversarial lens made the chain go wrong-but-GREEN three times -- R9's ruled drift gate had never been built and dc0's overlay was gated by nothing. - Built `tests/render-drift`, added 3 refusals, fixed 2 harness cases that could not fail. -- **LIVE, each gated and independently verified:** 8 OSD volumes + 2 controller VMs applied both DCs; 9 re-commissions with `skip_networking=1`, every diff EXACTLY the new device; - MAC adoption applied (drift 0 across 20 nodes, both roots ZERO DIFF); both controllers commissioned; all ruled MAAS tags created. **THE STAGE-5 ALLOCATION BLOCKER IS CLOSED.** -- **R1 precondition 3 CLOSED BY MEASUREMENT** -- `skip_networking=1` is the vendor control, and the MAC check sits BETWEEN apply and re-commission, which is what makes 2026-07-20 non-repeatable. -- **RULED (GA-R5):** v6 IP SANs on the Octavia controller cert (D-109 note); juju controller at utility `.5` AND the octet map is a STANDING CROSS-DC STANDARD (D-134 amendment) -- - divergence between DCs at the same octet is now a DEFECT, not a local choice. -- **Octavia: clean negative.** juju 3.6 DEEP-MERGES overlay `options` (charm v12.1.1 source), so octavia-pki cannot clobber the VIP; and nothing inside Octavia requires IPv4. -- **OWNED:** three strict-bash quoting traps, two false-positive greps, one fabricated flag, one red-lint push. All corrected on-surface; traps in `docs/audit/queued-findings-20260729.txt`. -- **NEXT:** octavia-pki generation -- the last item hard-failing preflight, and the generator is now per-DC. Gauntlet ALL GREEN (87) vcloud; repo-lint 0 fail. -- Body: `docs/changelog-20260728-vip-arity-gate.md`. Status ONLY in CURRENT-STATE.md. - -## SESSION CLOSE 2026-07-30 -- Stage-5 preconditions: 19 Phase-3 items, gate integrity, per-DC Octavia PKI LIVE (bounded, GA-R4) - -- Branch `dc-dc-stage5-preconditions`, **25 commits pushed** (`ab4c842..`). NO stage opened/closed. Scan unchanged: 3 decisions, SEC 21, D 138 / DOCFIX 205 / BUNDLEFIX 053. -- **6 agents** (3 read-only recon, 4 delivery). Phase-3 re-measured **2 FIXED / 19 REMAIN / 5 NEW**, not the 21 the 07-27 doc implied; **13 items + 3.9/3.10 + R5 delivered**, phase-4 runbook 434 -> 929 lines. -- **F1 (HIGH): R7's per-DC Octavia PKI was HALF BUILT** -- subject and VIP gate per-DC, `$DC` in NO path, so generating dc1 would have DESTROYED dc0's CA and left one fixed-name overlay applying dc1's CA to dc0. Fixed end to end + REFUSE-IF-PRESENT. -- **F4:** the per-DC rename would have UN-IGNORED a CA private key in a repo SEC-004 calls PUBLIC. Glob + a `check-ignore` self-assert, proven both ways. **F2:** register was blind to a missing 2nd CA set (was RULED work, R13 Part 1). **F6:** P5 was probing the WRONG HOST'S filesystem and reporting it as fact -- 34 findings vs the true 7. -- **PKI GENERATED (operator-executed; guard blocks the mint by design, not worked around), both DCs verified PASS 29/0**, independence proven by sha256, backups **proven by restore** (PKI 12/12 vs live; tfstate byte-identical, correct serials). **E2 + E3 fully closed.** -- **`scripts/octavia-pki.sh verify` NEW** (harness 21/21): asserts the SAN set nothing asserted before (F8's SAN-less cert), and **A12 ARMS ITSELF** from `os-public-hostname` so F9 cannot be missed. -- **Gate integrity:** decorative HA, machines-overlay no-op, cloud-assert arity, G17 node check, gauntlet locale. P4 DC-aware (DC refs 1 -> 45); 3.7 closed; lib-net dc1 arm. -- **NEW ruling-shaped, now BLOCKING:** D-008 + D-106:2563 do not say whether substrate `vr1-dc1` is `dc1` by TOKEN or `dc2` by POSITION -- item 3.1's ambiguity inside a ruling, deciding cert identity. A12 REFUSES rather than picking. - **^ WITHDRAWN 2026-07-30 (DOCFIX-205): NOT a ruling gap.** D-117 ruled it 2026-07-13 (`dc1`/`dc2` retired for `dc0`/`dc1`) and says so in its own Status line; D-106 read as open only because D-117's annotation half was never executed (zero of four) while its Status line claimed "FULLY EXECUTED". Also not blocking -- A12 measures INERT and passes. Now an assertion on the derived zone; status in CURRENT-STATE.md. -- **Headend reconcile found a gate RED ON THE ONLY HOST THAT DEPLOYS** (`opentofu-validate` fmt walking gitignored tfvars). Fixed + scoped; **both hosts now ALL GREEN (89)**, repo-lint 0 fail. -- **Three wrong records found, none catchable by any gate:** D-124 told a reader to source a provider MAC from a file holding only boot MACs (would have bridged `br-ex` onto the PXE NIC); the skill cites D-107 for a tailnet path it does not rule; `office1-tailscale` is labelled a subnet router and advertises none. -- **A "backup set" asserted on three surfaces did not exist.** Built for the PKI (1.0-GEN.e) and the inner tfstates (phase-2 step 13), declared, restores proven. Forward requirement recorded: the pinned secrets-storage solution MUST absorb both. -- OWNED: the operator's own working commands were NOT in the repo until the pre-bookend sweep asked; I waved through 664 CA certs as "harmless" (integrity, not secrecy); two silenced errors (`| grep -q` under pipefail, `git pull -q &&`); four consecutive cwd mistakes. All corrected on-surface. -- **NEXT:** F8 residue (heredoc -> printf when a real generation can be run); D-137 fork-1 guard ruling (blocked correctly once, misfired 5x); preflight still has NO verified-green reading; G17 fixture-green only; the two `phase-04-network-*` scripts. -- Sweep capture `docs/audit/queued-findings-20260730.txt` (F10-F13 + 2 ruling questions). Body: `docs/changelog-20260729-stage5-preconditions.md`. Status ONLY in CURRENT-STATE.md. - ## SESSION CLOSE 2026-07-30 (part 2) -- F9 CLOSED live, the reissue tool, P7, lib-identity (bounded, GA-R4) - Branch `dc-dc-stage5-preconditions`, **19 commits** pushed (`1ddb078..`). NO stage opened/closed. Scan unchanged: 3 decisions, SEC 21, D 138 / **DOCFIX 206** / BUNDLEFIX 053. @@ -294,3 +259,24 @@ - **SNAP PROXY BUILT AND INDEPENDENTLY REVIEWED, NOT APPLIED.** squid at the utility `.4:3129`, allowlisted, 53/53 mutations killed; review says the ruled design is HONOURED on all four counts and logs FOUR real bugs. Nothing proven end to end. - **OWNED:** a red-lint push (`| tail` masked the exit code); a `git add -A` that swept an agent's files into an unrelated commit; two instrument misreads (a `juju models` summary read as progress, and stale unit logs read as current); and a wrong "immutable system_id" claim the operator corrected. - **NEXT:** apply the review cuts, re-run the mutation pass (S8 invalidates it), confirm BUG-3's deny shape once, install the proxy, then `add-model` + spaces + `deploy`. Sweep: `docs/audit/queued-findings-20260731-stage5-deploy.txt` (11 FIRST SURFACE). Bodies: `docs/changelog-20260731-prefer-ipv6-research.md`. Status ONLY in CURRENT-STATE.md. + +## ROTATED 2026-08-01 (GA-R4 rule 3 / F1 -- cap restored at this close) + +The oldest closed-session summary/summaries moved VERBATIM to +`docs/archive/session-ledger-rotated-20260801.md`. The live ledger stood at 296 lines +and this close's summary would have breached the 300-line cap. + +## SESSION CLOSE 2026-08-01 -- snap proxy LIVE, D-139 adopted, IPv6 PROVEN on the node planes (bounded, GA-R4) + +- Branch `dc-dc-stage5-preconditions`, **8 commits** pushed (`d555b90..844b2e4`). NO stage opened or closed. Scan: 3 open decisions, SEC **26** (none opened this session), D **140** / DOCFIX 207 / BUNDLEFIX 053. +- **THE dc0 SNAP PROXY IS LIVE.** `check dc0` PASS exit 0, 16 assertions, re-verified independently of the agent that installed it -- and **a real snap payload fetched through it** (HTTP 206, first bytes `hsqs`). No longer fixture-green. BUG-3 confirmed by measurement (no assertion changed); BUG-4 fixed at BOTH sites where the review had named one. +- **D-139 ADOPTED** -- two rulings taken in separate exchanges: IPv6-only on the east-west planes, and the whole plane carve moved to GUA on the octet map VR0 and Willamette already use. +- **THEN ITS DECIDING REASON WAS REFUTED BY MY OWN VERIFICATION** -- glibc 2.35 implements RFC **3484**, not 6724, so ULA and GUA are EQUAL at precedence 40. Put back to the operator rather than quietly kept; ruling CONFIRMED on its other grounds and the dead rationale STRUCK IN PLACE, not deleted. +- **RULING A IS NOT ACHIEVABLE at current charm revisions** -- `ceph-osd` fails a v6-only plane in BOTH directions, `hacluster` still ships `ip_version: ipv4`, OVN documents encap as IPv4-only. Narrowed by ruling ("B plus C") to a storage+replication experiment plus the upstream fix; LP draft written, operator files it. +- **>>> IPv6 IS PROVEN ON THE dc0 NODE PLANES <<<** 6/6 addresses live on the NICs, 0% loss node-to-node on all six planes, ND resolving from a COLD neighbour table. Gate **G19** built AND run live. **IPv6 is also already load-bearing: node time sync runs over `fd50:840e:74e2:220::6`, stratum 3.** +- **THE RECORDED ROOT CAUSE OF THE v4-ONLY CONTAINERS WAS WRONG.** MAAS has 100% of every v6 `/64` available; the real limit is juju taking `addrs[0]` from an unsorted query, family-blind, with no knob (LP #1723240, open since 2017). Corrected per GA-R1 C2. +- **THE "JUJU CLIENT BLOCKER" WAS NOT ONE** -- the client is live and registered on the rack, exactly where D-138 puts it. Corrects this repo's own 2026-07-31 finding 9d. +- **D-139's OWN EXECUTION LIST WAS DEFECTIVE AND IS REPLACED.** `dc-node-v6-carve.py` pivots on IPv4 existing, so run after v4 removal it would carve four fewer planes per node **and exit clean**. +- **OWNED:** my BUG-1 fix was wrong (a secondary alias is never the kernel's chosen source); I scoped the v6 experiment wrong (ceph couples storage+replication); I stated an agent's input source wrongly; I scoped a research agent with no repo path, so 355 lines landed in `/tmp` and needed rescuing; and one commit went red on ASCII-only em-dashes. +- Gauntlet **ALL GREEN (96)**, repo-lint 0 fail, `d139-gua-carve` 71/71, `dc-node-v6-verify` 55/55. **voffice1's clone is 36 commits BEHIND** -- no loss, but a live hazard on the Plane-2 host. +- **NEXT:** apex CREATE-only push (tool built, independently reviewed, dry-run byte-identical), then the bundle deploy -- its blockers are cleared. `network-get` on a v6-only bound space is still unmeasured and gates the v4-removal experiment. Sweep: `docs/audit/queued-findings-20260801-stage5-ipv6-d139.txt` (**6 FIRST SURFACE**). Body: `docs/changelog-20260731-snap-proxy-apply-ipv6.md`. Status ONLY in CURRENT-STATE.md.