diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index d8a1a28..2ce1569 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -603,7 +603,22 @@ operator-terminal rows, contradicting lens 7's 30. The file is SPACE-ALIGNED, not tab-separated; re-measured correctly it is 30 rows / 16 ids and lens 7 was right. A disagreement with a prior finding was treated as a reason to re-check the instrument. - Remaining: R6-R11 blocking, R12-R15 standing -- all now measured. + **R6 RULED 2026-07-27 -- exact utterance "Close the two VIP gaps first, then apply the + overlay whole (Recommended)"**, recorded as a **D-121 RULING NOTE (2026-07-27)**. It + resolves a real tension: D-121 is titled "VR1 makes HA real", so DEFERRING the overlay + would deploy VR1 in exactly the shape D-121 was written to retire -- while applying it + AS-IS would ship, for vault, precisely the defect D-121 exists to remove. The ruled + sequence satisfies the decision rather than half of it. **CONSEQUENCE FOR SEQUENCING: R11 + is now a HARD Stage-5 precondition ordered BEFORE the overlay**, not a parallel item. + Scope is small and the pattern already exists: eleven of twelve hacluster principals carry + correct VIP triples and `octavia`'s (`10.12.4.57 10.12.8.57 10.12.12.57`) is the shape to + copy. Tracked separately and NOT resolved here: the Stage-6 radosgw multisite path is + single-unit-shaped while this scales `ceph-radosgw` to 3, and `provider-bundle-check.py` + checks `cluster_count` NOWHERE -- which is why decorative HA was found by audit rather + than by gate. R4's band ruling already covers the overlay's growth from 27 to ~55 LXD + units, so address demand is not an argument against it. + Remaining: R7-R11 blocking (R11 next by the R6 sequencing), R12-R15 standing -- all + measured. - Position inside Stage 3: deploy step A EXECUTED 2026-07-19 (6/0/6 exact; convergence zero -- `docs/audit/outer-plan-20260719-postA-converged.txt`). **Deploy step B diff --git a/docs/audit/queued-rulings-20260727.md b/docs/audit/queued-rulings-20260727.md index 3301123..9dc2f45 100644 --- a/docs/audit/queued-rulings-20260727.md +++ b/docs/audit/queued-rulings-20260727.md @@ -311,7 +311,19 @@ - **(b)** Deploy single-unit at Stage 5 and scale up after Stage 6's multisite work, accepting a live-cloud scale-out later. -OPERATOR UTTERANCE: +OPERATOR UTTERANCE: **"Close the two VIP gaps first, then apply the overlay whole +(Recommended)"** -- RULED 2026-07-27. **R6 IS CLOSED.** Recorded as a **D-121 RULING +NOTE (2026-07-27)**. Measured before presenting, which reframed the question: the +overlay scales 14 apps and moves all 12 base hacluster subordinates to +`cluster_count: 3`, but **eleven of twelve principals already carry proper VIPs** -- +the gap is exactly TWO apps, and they differ in kind. designate is in the base bundle +with an hacluster and no vip; **vault has no hacluster in base AT ALL** -- the overlay +INTRODUCES the `vault-hacluster` application, its `cluster_count: 3` and the `vault:ha` +relation, and still no vip. **23 relations consume `vault:certificates`**, so +remediating on a live 3-unit vault would mean re-pointing all of them and re-issuing +SANs on a running cloud. **Consequence for sequencing: R11 is now a hard Stage-5 +precondition ordered BEFORE the overlay**, not a parallel item. The Stage-6 radosgw +single-unit path and the absence of any `cluster_count` check remain tracked separately. --- diff --git a/docs/design-decisions.md b/docs/design-decisions.md index 12eb59a..bba81e6 100644 --- a/docs/design-decisions.md +++ b/docs/design-decisions.md @@ -3874,6 +3874,70 @@ 4. `modules/node-vm` is SHARED by both DCs. Gate the change so it is additive and opt-in per node role, or it will plan a change against all 18 domains. +### RULING NOTE 2026-07-27 -- D-121: the HA scale-up overlay applies only AFTER the two VIP gaps close + +**Status:** RULED 2026-07-27 (operator, GA-R5). Records a ruling; amends nothing in D-121's +node layout or its 3-unit intent. + +Question as presented, verbatim: "R6 -- HA scale-up overlay at Stage 5. D-121's premise is +'VR1 makes HA real', but applying the overlay as it stands gives vault a 3-node pacemaker +cluster with no VIP to manage -- decorative HA for the CA that 23 relations depend on. +Eleven of twelve principals are fine; only designate and vault lack VIPs. How should this +sequence?" Options presented: (a) close the two VIP gaps first, then apply the overlay +whole; (b) apply as-is and remediate vault HA afterwards; (c) defer the overlay and scale +after Stage 6. Operator selection, exact utterance: **"Close the two VIP gaps first, then +apply the overlay whole (Recommended)"**. + +**THE TENSION THIS RESOLVES.** D-121 is titled "VR1 makes HA real -- scale the decorative +single-unit control plane to 3". Option (c) would have deployed VR1 in precisely the shape +D-121 was written to retire. But option (b) would have shipped, for vault, exactly the +defect D-121 exists to remove -- a pacemaker cluster with nothing to manage. The ruled +sequence is the only one that satisfies the decision rather than half of it. + +**MEASURED STATE (capture `docs/audit/r6-r15-measurements-20260727.txt`).** +`overlays/dc-ha-scaleup.yaml` scales FOURTEEN applications 1 -> 3 and moves all TWELVE base +hacluster subordinates from `cluster_count: 1` to `3`. Eleven of those twelve principals +carry proper VIP triples. The gaps are exactly two, and they are NOT the same shape: +- **designate** is in the base bundle WITH an hacluster subordinate (`cluster_count: 1`) and + an `ha: metal-internal` binding, but no `vip`. Consistent at base scale; decorative at 3. +- **vault** has NO hacluster in the base bundle AT ALL and an EMPTY options block. Its + entire HA apparatus -- the `vault-hacluster` APPLICATION (which the base does not + contain), its `cluster_count: 3`, and the `vault:ha <-> vault-hacluster:ha` relation -- + exists ONLY in the overlay. And still no `vip`. + +**WHY VAULT IS THE DECIDING CASE.** Twenty-three relations consume `vault:certificates` +(mysql-innodb-cluster, keystone, glance, glance-simplestreams-sync, nova-cloud-controller, +placement, neutron-api, neutron-api-plugin-ovn, ovn-central, ovn-chassis, cinder, +ceph-radosgw, openstack-dashboard, octavia, ovn-chassis-octavia, barbican, barbican-vault, +magnum, designate), plus `barbican-vault:secrets-storage <-> vault:secrets`. Vault is the +certificate authority for the entire cloud. With no VIP every consumer binds a UNIT +address, so there is nothing for a failover to move. Remediating that on a LIVE 3-unit +vault means re-pointing 23 certificate relations and re-issuing SANs on a running cloud -- +which is why (b) was refused despite being faster. + +**SEQUENCE RULED:** answer R11 (the VIP shape for vault and designate), land those VIPs, +THEN apply `dc-ha-scaleup.yaml` whole at Stage 5. **R11 is therefore a hard Stage-5 +precondition, sequenced BEFORE the overlay**, not a parallel item. + +**SCOPE NOTE that makes this tractable:** it is a TWO-APPLICATION change, not a rework. +`octavia` already carries a correct VIP triple (`10.12.4.57 10.12.8.57 10.12.12.57` in the +dc0 band), so the pattern is established in-repo and there is a shape to copy rather than +design. + +**NOT RESOLVED BY THIS RULING, and tracked separately:** +- The Stage-6 radosgw multisite path is single-unit-shaped (`--unit U`, one restart, per + `scripts/dc-dc-radosgw-multisite.sh --help`) and this ruling scales `ceph-radosgw` to 3. + That coupling applies to any option that scales radosgw and needs its own DOCFIX. +- Whether `provider-bundle-check.py` should FAIL on an hacluster relation without a VIP, or + on a `cluster_count` that does not match its principal's `num_units`. Today it checks + `cluster_count` NOWHERE, which is why this survived to be found by audit rather than by + gate. Queued under R11/R15. + +**Address demand is NOT an argument against this** -- R4 already ruled the reserved bands +get created, which covers the overlay's growth from 27 to ~55 LXD units. + +**Execution is a SEPARATE gated step, not authorised here.** + ## D-122: VR1 site shape -- nested-per-site containment, dark fiber + dedicated per-site L3 ISP, DC edge follows the Office1 pattern [ARCH] **Status:** ADOPTED (2026-07-15, operator ruling this session -- answers Stage-3 Ruling 2). Records