# Session changelog 2026-07-28 -- the VIP arity gap + R11's ruled gate changes ONE changelog per session (GA-R2/D1). Under blanket approval this is the review surface: every item states WHAT, WHY (evidence), and HOW TO REVERT. **No live mutation in this session.** Everything below is a repo change; the only live access was READ-ONLY (a NetBox apex poll and a `pre-flight-checks` run). No logged window was opened because nothing was mutated. Status authority for everything here is `docs/CURRENT-STATE.md` (GA-R1). This file is session-scoped scratch and is NOT citable as status or decision authority. --- ## 1. `provider-bundle-check.py`: VIP arity, family coupling, and the apex-derived v6 bands **What.** The `vip` validator now accepts a v4 TRIPLE (3 addresses) or a dual-family SEXTET (6), validates the three v6 legs against the per-DC v6 /64s, requires the v6 host part to MIRROR the v4 octet textually, and couples `prefer-ipv6` to the arity in both directions. The v6 bands are READ FROM THE NETBOX APEX RECORD, not hardcoded. The standalone octet-uniqueness block (old item 6) folded into the same loop. **Why.** Measured at `:137`: `if len(parts) != 3` fails EVERY application once R2's ruled dual-stack lands, and `:149`'s `p.split(".")[-1]` returns the whole address on a v6 literal, so every v6 leg was trivially "unique" and unchecked. The apex now holds the dual-family set a renderer will emit (verified live this session: 160 ip-addresses, 78 IPv6, 156 VIP objects), so the only gate that validates overlays could not validate what is about to be produced. The `prefer-ipv6` coupling is the substance, not a bonus. `prefer-ipv6` makes HAProxy bind `:::port` in ADDITION to `*:port`; the measured L3-9 finding is that the overlay merge order which keeps `prefer-ipv6` while dropping the v6 VIP legs is the one that EXITS 0 -- the dangerous order is the green one. Both directions now fail. **Authority.** The arity gap was carried in `docs/CURRENT-STATE.md` as an open question, explicitly NOT covered by R11. Operator direction 2026-07-28, exact utterance: **"Fix the arity gap first, then start the renderer"**, with the scope fork (arity alone vs arity + R11's three ruled changes) put separately and answered **"Arity gap + R11's three ruled changes (Recommended)"**. OPS under GA-R3 -- a script change making an already-RULED end state expressible; doubt resolves DOWN, no D-number assigned. **Prior art followed, not reinvented.** The `(role, kind)` apex keying and the "provider leg lives in the DEDICATED GUA VIP /64, admin/internal in their plane /64s" rule are copied from `scripts/dc-plane-ipam.sh:102-131` and `netbox/dc-plane-apex-import.py`. Both live-API and repo-dump shapes are handled (nested `scope.slug` vs flat `scope_site`) -- writing against one and matching zero objects on the other is a trap this repo already hit once. **Refuses rather than passes.** A dual-family vip with an unreadable or absent apex record exits **2**, never 0: "could not look" is never "nothing there". A v4-only bundle needs no apex at all, so this introduces no new dependency for the current tree (harness T25/T26 lock both halves). **Revert.** `git revert` this commit. The checker returns to triple-only; nothing live depends on it. ## 2. R11's three ruled gate changes (D-020 AMENDMENT, RULED 2026-07-27) **What.** - `provider-bundle-check.py`: `OCTET_LO, OCTET_HI` 50-60 -> **50-99**. - `scripts/lib-net.sh`: `VIP_OCTET_MAX` 60 -> **99**, `VIP_COUNT_EXPECT` 11 -> **13**. - `provider-bundle-check.py`: NEW invariant 8 -- an hacluster principal with no `vip` is a FAIL. - `EXPECT_PUBLIC_VIP` deliberately **STAYS 11**. **Why.** R11 ruled vault `.61` and designate `.62` as full dual-family triples and ruled the gate hardening **IN SCOPE, not deferred**. `cluster_count` is asserted NOWHERE in `scripts/` or `tests/`, so a 3->1 rewrite of all 20 values produces a byte-identical PASS -- decorative HA was found by audit rather than by gate. `EXPECT_PUBLIC_VIP` stays 11 because -- measured -- NEITHER vault nor designate carries a `public` binding, so they do not join that count. Bumping both constants together breaks the gate. The two band constants are SEPARATELY NAMED in two files; this is a two-file change by construction, and both sites carry a comment saying so. **MEASURED CONSEQUENCE, stated precisely.** Two sub-checks flip PASS -> FAIL: - `provider-bundle-check` on the base bundle now FAILS with `hacluster relation but no vip: designate`. - `pre-flight-checks` CHECK 1 now reports `VIP validation OK=11 bad=0 (want OK=13)`. **`preflight.sh` was ALREADY exit 1 before this change** (octavia-pki overlay absent, MAAS unreachable from the jumphost, P5's 7 credential findings). Measured after: still exit 1, now `3 fatal, 2 warning`. This did NOT flip preflight from pass to fail and **no deploy path that was open is closed by it** -- the same precedent this document records for P5. The reds are the ruled gate reporting real work owed: R11's `.61`/`.62` VIPs are RULED-BUT-NOT-BUILT, and per R6 they land BEFORE the HA overlay. **Revert.** Revert this commit; the constants and the invariant-8 block are contiguous and independent of item 1. ## 3. Harness: 15 -> 30 cases, with three cases RE-POINTED rather than deleted **What.** `tests/provider-bundle-check/run-tests.sh` gains 15 cases. The fixture base splits: `pristine.yaml` is the repo bundle untouched; `good.yaml` is that bundle PLUS designate's ruled `.62` VIP. **Why the split.** Invariant 8 legitimately fails the real repo bundle. Left alone, every rc=0 case (T1/T9/T14) would have been asserting "fails for the reason we already know" instead of its own invariant. Per the standing rule -- *when remediating a finding turns the harness red, REPLACE the assertion with the new invariant and say so, never delete it to go green* -- the real tree's state is kept honest by a NEW case (T16) that asserts `pristine.yaml` DOES trip it. **When `.62` lands in `bundle.yaml`, T16 must be re-pointed, not deleted.** **Proven able to fail AND to pass** -- the complement this project recorded on 2026-07-27 after building three gates whose every live run failed: - T16 the pristine bundle FAILS on designate; T18 the same check goes GREEN once the ruled VIP lands. - T17 vault FAILS under `dc-ha-scaleup.yaml` -- R6's "VIPs before the overlay" ordering, made executable. - T19 a valid dual-family VIP PASSES and is counted as dual; T20/T21 both directions of the `prefer-ipv6` coupling FAIL; T22 a v6 leg in the node `/64` instead of the VIP `/64` FAILS; T23 the `::32` hex trap FAILS; T24 a 2-address vip FAILS; T25 an unreadable apex REFUSES at rc=2; T26 a v4-only bundle needs no apex; T27/T28 the band is widened to 99 but still bounded. T29/T30 exercise the **dc1** arm of the apex lookup, which T19-T23 never reached (they all run at the default `--dc vr1-dc0`): T29 resolves dc1's `f03:11::/64` / `:320::/64` / `:321::/64` and passes, T30 proves a dc0 v6 leg under `--dc vr1-dc1` FAILS, so a cross-DC copy-paste of a rendered overlay cannot pass. The renderer's first real output is per-DC overlays, and a gate observed in only one DC mode is the same untrustworthiness T16/T18 exist to rule out. T8's assertion string moved `outside 50-60` -> `outside 50-99` (the same re-pointing rule; the case still guards the band, at its ruled bounds). **Evidence.** Harness **30/30** (was 15). Gauntlet **ALL GREEN (84 harnesses) on vcloud** -- the host is named because the gauntlet is measured HOST-DEPENDENT (2/81 fail on voffice1 on an identical commit). `repo-lint` 0 fail / 1 standing warn / 605 files scanned. No harness was added or renamed, so `tests/HARNESS-MANIFEST` is unchanged and needed no re-record. **Revert.** Revert this commit; the harness returns to 15 cases. --- ## Findings LOGGED, NOT ACTIONED (hard rule 1) 1. **`pre-flight-checks.sh` CHECK 1 is triple-shaped.** Its awk parse assumes three columns, so it will mis-read a dual-family sextet when those artifacts land. Same class as the arity gap just fixed, one gate over. Not touched -- it interlocks with BLOCKER-1 (the ruling-3 VIP extraction makes CHECK 1 see 0 VIPs). 2. **`cluster_count` coherence is still unasserted.** Invariant 8 catches an hacluster principal with NO vip, which is what R11 ruled. It does NOT check that `cluster_count` matches the principal's `num_units` -- a 3->1 rewrite of all 20 values still produces a PASS. Out of ruled scope; worth its own exchange. 3. **NetBox instance/target drift (raised by the operator this session).** Four `netbox/*.py` tools carry in-file usage examples pointing at `https://netbox.baldurkeep.com` -- the FROZEN v1 reference. Two of them (`ipv6-mark-reserved.py`, `ipv4-prefixes-import.py`) have **NO `SANDBOX_HOSTS` write guard at all** while carrying write paths (`update_status`, `create_or_report_prefix`, `create_or_report_iprange`). The guarded tools also expose a deliberate `--yes-write-upstream` escape hatch, so "writes only reach 10.10.1.10" is a convention with an override, not a hard invariant. 4. **Credential env-file naming is inverted.** `~/vr1-office1-creds/vr1-netbox.env` points at `netbox.baldurkeep.com` (the read-only v1 reference) while `vr1-netbox-sandbox.env` points at the LIVE VR1 apex `http://10.10.1.10:8000`. The filename says the opposite of what each is. 5. **The NetBox `?site=` API filter silently does not filter.** Measured this session: `/api/ipam/ip-addresses/?site=vr1-dc0` and `?site=vr1-dc1` each returned the full 160. A per-site apex count taken that way would be wrong and look right. --- # 2026-07-29 -- render-pipeline STEP 4: the renderer Same branch, continued. Operator direction: **"You can order the steps in whatever order you want. Deploy agents to help if needed."** ## 4. Ordering decision, and why it INVERTED after measurement I first proposed doing ruling-3's VIP extraction before the renderer, then reversed it once three read-only agents reported. Recording the reversal rather than the conclusion, because the reason is the reusable part: - The byte-for-byte reproduction WINDOW IS STILL OPEN and closes on its own. `overlays/vr1-dc1-vips.yaml` is still byte-identical to `tests/render-baseline/fixtures/vr1-dc1-vips.v4-11app.yaml` (`3f93ecb3...`) -- verified this session. The reconciliation (11->13 apps, dual-family) destroys it. - The extraction's blast radius is FAR larger than ruling 3's "2 commits". Measured: it breaks a DEPLOY GATE (`runbooks/phase-01-bundle-deploy.md:170-176`, where `juju deploy` runs only on `11/11/0`), the Octavia SAN derivation (`:371`), preflight in two places, and 8 of 30 harness cases. - So the extraction is better done BY the renderer, once the renderer is proven -- which makes dc0's overlay a validated output rather than a hand edit. ## 5. `scripts/render-dc-overlays.py` -- two stages, split where F2 froze the interface **What.** `derive` (apex + `lib-net.sh` -> a per-DC VALUES file) and `render` (VALUES -> overlay TEXT). Plus `check`, which renders and byte-compares. **Why split.** Forward item F2 froze the interface: *"the renderer is a deterministic command-line tool whose inputs are the per-DC values file and whose outputs are committed, reviewable artifacts. Nothing about automating it should require changing the renderer."* `render` is therefore PURE -- no apex, no network, no clock -- so it is offline-testable and a future CI job only re-runs `derive`. **Why a text emitter, not `yaml.dump`.** The target is byte-compared. A round-tripper destroys the comment header, normalises `vip: "a b c"` to a plain scalar, and imposes its own key order -- all three are load-bearing. PyYAML reads the values file and never writes an overlay. **Two things that are INPUT, not output, and would otherwise be invented:** - The 12-line comment header cites one-off measurements and a design-doc fork; none of it is computable from the apex. `derive --from-overlay` lifts it VERBATIM from the artifact being reproduced, so a reproduction run is honest about which bytes the tool actually generates. - Emission order is ASCENDING OCTET, not alphabetical (keystone .50 before barbican .51). Sorting by name reproduces nothing. **The ruled app->octet map is READ, not restated** -- parsed with `ast` out of `netbox/dc-plane-apex-import.py`'s `APP_OCTET`, the tool that wrote those values into the apex. A second hand-maintained copy is a second thing to drift. **Revert.** Delete `scripts/render-dc-overlays.py`, `tests/render-dc-overlays/`, `render/values/`, and revert the `render-baseline` T6 hunk + the manifest line. Nothing else consumes any of it; no committed artifact was regenerated by it yet. ## 6. RESULT: byte-for-byte reproduction achieved, and PROVEN ABLE TO FAIL `render` output hashes to `3f93ecb3205cea8b303f5b2cbbbde343a2762156575277ca84684369da8bf896` -- identical to both the committed overlay (1634 bytes) and the frozen fixture. A byte-compare only ever observed PASSING is as untrustworthy as a gate only ever observed failing, so three faults were seeded and all three fail it: a wrong octet, a dropped header, and dc0 prefixes against the dc1 artifact. `render-baseline`'s new T6 was separately proven by making the renderer sort alphabetically -- hash changed, case went red. ## 7. THE ARITY WORK PAID FOR ITSELF WITHIN THE HOUR -- a real renderer bug, caught by the gate The renderer's FIRST dual-family output was malformed. The values file stores each v6 `/64` base with trailing colons stripped (`2602:f3e2:f03:11`), and the emitter joined with a SINGLE colon -- yielding `2602:f3e2:f03:11:50`. That is not a valid address, and it still looks like one. `provider-bundle-check.py` -- fixed yesterday for exactly this class -- rejected all 13 applications with `bad vip ip`. Fixed to `::%d` and locked by harness cases T13/T13b, which assert both that the correct form is present AND that no single-colon leg is emitted. **This is the acceptance criterion the whole arity fix existed to provide, working on its first real use.** ## 8. End-to-end chain now demonstrated apex -> values -> renderer -> overlay -> gate. The dual-family dc1 render with R11's ruled-but-unbuilt apps included produces **13 clustered VIPs, all dual-family**, and `provider-bundle-check --dc vr1-dc1` reports **PASS** including the new hacluster invariant (12 principals, all carrying a VIP). **Evidence.** New harness `tests/render-dc-overlays` **17/17**; `render-baseline` **10/10** (was 9, T6 added -- ADDED per its README, existing assertions NOT repointed). Gauntlet **ALL GREEN (85 harnesses) on vcloud** (was 84); `tests/HARNESS-MANIFEST` re-recorded deliberately after the drift gate caught the addition -- which is the gate working. repo-lint 0 fail / 608 files scanned. --- ## Findings from the three read-only agents -- LOGGED, NOT ACTIONED (hard rule 1) 6. **F3's MAC sourcing claim is FALSIFIED.** `docs/dc-dc-deployment-workflow.md:361` says `ovn-chassis` MACs "stay sourced from `scripts/lib-hosts.sh`". Measured: lib-hosts carries only `HOST_BOOT_MAC` (the PXE NIC, `...:NN:01`); `ovn-chassis` needs the provider-public NIC (`...:NN:02`), which is **not in lib-hosts at all**. The real source is `opentofu/vr1-dcN-substrate/main.tf`, `macs[1]` of each compute node -- and it exists for BOTH DCs. This also narrows CURRENT-STATE's claim that dc0's MACs "cannot be derived": not from a PATTERN, but they are pinned literals in tofu and are perfectly sourceable. F3's conclusion (MACs are renderer output, not apex input) still stands; only its sourcing sentence is wrong. 7. **`scripts/dc-plane-ipam.sh:120` has a latent scope-matcher bug.** It matches `scope.name` or `scope_site` but NOT `scope.slug`, so it would match ZERO prefixes against a live-shaped record. Latent only because it defaults to a repo dump. The other two tools use the three-way matcher; this one is the odd one out. 8. **`pre-flight-checks.sh` CHECK 1 is triple-only AND dc0-only.** It hardcodes `if(n!=3)` (`:86`), reads `bundle.yaml` as RAW TEXT (never a merged input), and never calls `lib_net_select_dc` -- and `lib-net.sh` deliberately UNSETS `VIP_PREFIX_*` for dc1, so CHECK 1 cannot evaluate dc1 at all. The renderer's dual-family output will pass `provider-bundle-check` and FAIL preflight P4. 9. **CURRENT-STATE is stale on dc1's FIP pool** -- an early paragraph still says it "needs a ruling"; a later entry records it RULED and applied, and `lib-net.sh` carries the values. Same in-document supersession class this doc keeps finding. 10. **`render-baseline` T3 compares COUNTS, not SETS.** A deleted pinned fixture plus an added unpinned one keeps the count equal; a `#` comment line in `SHA256SUMS` inflates the pin count and masks one unpinned file; and `-maxdepth 1` cannot see a subdirectory. It catches the fault it was seeded against; a set-difference would close all three. 11. **The B1/B5 comment tokens are defined in `bundle.yaml`'s own header** (`:11-13`, `:19-24`). Ten of the eleven inline VIP comments are the bare token `# B1`. Migrating them into an overlay STRANDS them from their definitions -- `appendix-A:652-661` records the repo's rule for exactly this identifier class. This must be handled when the extraction lands, not discovered then. --- # 2026-07-29 -- RULING 3, COMMIT 1: the neutral VIP MOVE (+ blast-radius repair) Operator: **"Approved, continue"**. This is commit 1 of the two ruling 3 (2026-07-25) prescribes. **The dual-stack ADD is commit 2 and is deliberately NOT in here.** ## 9. `bundle.yaml` is now VIP-FREE; `overlays/vr1-dc0-vips.yaml` is GENERATED **What.** All 11 inline `vip:` lines removed from `bundle.yaml`; `placement`'s then-empty `options:` block dropped rather than left parsing as `null`. The values moved to `overlays/vr1-dc0-vips.yaml`, **rendered by `scripts/render-dc-overlays.py`** from `render/values/vr1-dc0-vips.yaml` -- the renderer's first real output. **The 11 inline comments were carried, not dropped.** Ten are the bare token `# B1` and three carry real operational content (keystone's catalog-endpoint note, ceph-radosgw's un-defer note, openstack-dashboard's ALLOWED_HOSTS warning). They were extracted VERBATIM from `bundle.yaml` by a parser, not retyped, and the renderer grew optional per-app comment support to carry them. An app with no comment emits none, so the dc1 artifact still reproduces byte-for-byte (re-verified: 1634 bytes, unchanged). **The `B1`/`B5` tokens stay DEFINED in `bundle.yaml`'s header**, where they always were; the overlay header names that explicitly so the references do not dangle (`appendix-A:652-661` is the repo's rule for this identifier class). **PROVEN NEUTRAL by the ruling's own named check.** `provider-bundle-check` on the pre-extraction bundle and on `bundle.yaml --overlay overlays/vr1-dc0-vips.yaml --dc vr1-dc0` produce **line-for-line identical output**, including the same known `hacluster relation but no vip: designate` failure. Separately, a structural diff confirmed only `vip` changed: applications 56 -> 56, relations 108 -> 108, every non-VIP option and every non-option key byte-identical. **Revert.** `git revert`; the values file and overlay are additive and the bundle edit is a pure line removal. ## 10. Blast-radius repair -- the part that made this bigger than "2 commits" Every item below would have been BROKEN by the move. All were found by the read-only agent sweep BEFORE the edit, not after. - **`scripts/preflight.sh` P2** validated the BARE base, which post-move reports 11 phantom "no vip" failures. Now validates the MERGED dc0 input; the overlay is passed only when present so a tree without it still runs. - **`scripts/pre-flight-checks.sh` CHECK 1** read `bundle.yaml` as raw text and would have seen ZERO VIPs -> hard-fail. Now reads the merged source. **This is BLOCKER-1, fixed the ruled way (point it at the merged bundle) and explicitly NOT by bumping the expected count.** A measured trap on the way: the file sets `IFS=$'\n\t'`, so a space-joined `"$VIP_SRC"` reached `grep` as ONE filename, `grep` exited 2, `pipefail` propagated and the gate **died silently mid-check**. Fixed with a bash ARRAY. - **`runbooks/phase-01-bundle-deploy.md`** -- the real hazard. Its RUN block gates `juju deploy` on `TOT/HI/LO == 11/11/0`; post-move that reads `0/0/0` and **aborts the deploy**. Both guard blocks now span base + overlay. A second bug had to be fixed in the same edit: `grep -c` over TWO files prints one count PER FILE, so the bare `$( )` would have captured a two-line string and every numeric test would have failed anyway -- the counts are now awk-summed. Verified end to end: the gate reads **11/11/0 -> DEPLOY**. The deploy line itself now carries `--overlay overlays/vr1-dc0-vips.yaml`. - **`phase-01`'s Octavia SAN derivation** (DOCFIX-067) read `bundle['applications']['octavia']['options']['vip']`, which post-move raises KeyError -- swallowed by `|| true` into an EMPTY VIP, then a "implausible VIP" abort. Now reads the overlay. Verified: derives `10.12.4.57`. - **`runbooks/phase-00-teardown-maas-reset.md:233`** bare checker call -> merged input. - **`runbooks/dc-dc-phase4-juju-bundle-per-dc.md`** said "**DC1 needs NO VIP/CIDR changes at all**" and "deploy the EXISTING `bundle.yaml` with **NO VIP edits**". Followed verbatim post-move that deploys a VIP-free DC1. Superseded in place (the D-101 plane reasoning is kept, because it is still correct); the DC2 instruction to "edit `bundle.yaml`'s VIP lines" now points at rendering a new overlay instead. - **`runbooks/appendix-A-troubleshooting.md` L3** told an operator to grep the bundle for the metal-side VIP token -- which now returns zero, and the block's `|| true` makes zero look like a clean result. Note added. - **`tests/render-baseline/README.md`** records a capture-time `bundle.yaml` sha256 that no longer re-verifies. Marked HISTORY with what changed and why, rather than re-pinned -- the fixture hashes themselves are untouched. ## 11. Harness: 30 -> 31, with the fixture BASE re-pointed `tests/provider-bundle-check` now builds its fixture base by MERGING `overlays/vr1-dc0-vips.yaml` into `bundle.yaml`, because that pair is the deploy input. T16's label changed from "pristine repo bundle" to "the real dc0 deploy input" -- same assertion, honest name. **NEW T31 asserts ruling 3's new invariant directly: the BARE base fails for every clustered principal**, so "deployed without its overlay" is a tested state rather than an assumption. 31/31. **Evidence.** Gauntlet **ALL GREEN (85) on vcloud**; repo-lint 0 fail / 610 files; `render-dc-overlays` 17/17; `render-baseline` 10/10 (dc1 reproduction still byte-exact after the comment-support change); `preflight` 16/16. **Still red, unchanged and expected:** `pre-flight-checks` CHECK 1 reports `OK=11 want 13` and `provider-bundle-check` reports designate -- both are R11's `.61`/`.62` VIPs being RULED-BUT-NOT-BUILT, which is commit 2's job, not this one's.