diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 3d2483e..514bde9 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2341,13 +2341,25 @@ apt-cacher-ng proxy hangs ~2 of 6 containers provisioned at once (today's CLOSE-WAIT class) -- remediated per appendix-A + switched to ONE app / 2 containers at a time (clean thereafter); (ii) hacluster scale-up briefly STOPS the leader VIP until cluster_count=3 reforms the cluster (~1-2 min) - -- harmless for stateless API apps, sequence keystone/vault deliberately. **12 of 14 HA apps DONE at 3-unit HA (2026-08-05):** the 8 Wave-1 apps + -- harmless for stateless API apps, sequence keystone/vault deliberately. **12 of 14 HA apps DONE at 3-unit HA (2026-08-05, Wave-1 session):** the 8 Wave-1 apps + placement (pilot) + nova-cloud-controller + rabbitmq-server (native erlang 3-node cluster, no hacluster) + barbican. New units on nova-cc/barbican settle DB/messaging relations transiently. - **REMAINING 2, BOTH OPERATOR-GATED: keystone** (cloud-wide auth VIP blip -- do when ready) **and - VAULT LAST** -- operator unseals each new unit (SEC-003, guard-hook-blocked for the agent) + - cert-SAN check before the VIP flip; vault HA also RESOLVES the barbican-vault `secrets-storage` - dependency. OWED: D-121 execution runbook (dc1 reuse); post-wave bundle/overlay review vs the + **THEN 14 of 14 (2026-08-05, part-2 session): keystone + vault scaled 1->3, completing the 3-unit + control plane.** keystone (cloud-wide auth VIP blip taken deliberately) and VAULT LAST -- operator + unsealed each new vault unit (SEC-003, guard-hook-blocked for the agent) + cert-SAN check before the + VIP flip; vault HA RESOLVED the barbican-vault `secrets-storage` dependency (barbican-vault now + publishes `vault_url`). **Vault landed METAL-ONLY per the D-020 amendment (RULED 2026-08-05, GA-R5):** + its provider VIP `.61` findif-failed (vault's containers have NO provider-public leg), so vault's VIP + is the metal PAIR `10.12.8.61 10.12.12.61` (dc1: `10.12.68.61 10.12.72.61`), not the provider/admin/ + internal triple; vault-hacluster unblocked. vault is the single non-triple clustered VIP and + `provider-bundle-check.py` learned the per-DC exception (band + octet-uniqueness still enforced). + **EVIDENCE GRADE / GAP (GA-R1 rule 2): this 14/14 count is OPERATOR-ATTESTED (2026-08-05 standing + directive) + the part-2 session-ledger close and `changelog-20260805-d121-ha-scaleup.md` -- NOT yet + measurement-backed. A committed `juju status -m vr1-dc0` capture proving all 14 apps at 3 active/idle + units is OWED; it rides with the F4 vault `ha_enabled` live-verify sweep + (`docs/audit/queued-findings-20260805-d121-ha-vault.txt`), the next live-cloud step. Correct this + line from that capture when it lands (C2: measurement wins).** + OWED: D-121 execution runbook (dc1 reuse); post-wave bundle/overlay review vs the live HA installs (pinned task). Body: `docs/changelog-20260805-d121-ha-scaleup.md`. **NAMED-GATE DEFECT found by measurement -- `phase-03-core-verify.md` Step 3.1 asserts expected non-active/idle = 1 (octavia only); the VR1 roster yields 4 deferred-by-design + gss.** That gate diff --git a/docs/changelog-20260805-task2-vault-metal-only-commit.md b/docs/changelog-20260805-task2-vault-metal-only-commit.md new file mode 100644 index 0000000..64b530a --- /dev/null +++ b/docs/changelog-20260805-task2-vault-metal-only-commit.md @@ -0,0 +1,144 @@ +# Changelog 2026-08-05 -- Task #2: D-020 vault-metal-only amendment committed + provider-bundle-check harness reconciled + +Session scope (GA-R2, one changelog per session). Picks up the durability debt the +2026-08-05 (part 2) close left explicitly open: three uncommitted files on the vcloud +working tree (the RULED-but-uncommitted D-020 vault-metal-only amendment) plus a RED +provider-bundle-check harness (6 fail) and repo-lint L10. All offline; no live-cloud +mutation this session. Advisor-reviewed before commit. + +Branch: `dc-dc-stage5-preconditions`. Stage 5 dc0 remains OPEN. + +## Context (measured this session, not carried from prose) + +- HEAD `a5cf01a` did NOT contain the D-020 2026-08-05 amendment despite its commit + message naming "vault metal-only" -- the work was uncommitted on disk. Verified: + `git show HEAD:docs/design-decisions.md | grep -c "D-020 -- AMENDMENT (2026-08-05)"` = 0. +- BUNDLEFIX-053 (committed `ae06c72`) folded the FULL 13-subordinate HA chain + (`vault-hacluster` included) and `num_units: 3` into `bundle.yaml`. Confirmed: + 13 `:ha` relations + 13 hacluster subordinates in `bundle.yaml`; `num_units: 3` on + mysql/ovn-central/ceph-mon (so the harness rolesep builder must be num_units-aware). +- Real checker output MEASURED on the merged dc0 deploy input: `12 clustered VIP(s) + ... (0 dual-family)`, `13 hacluster principal(s) all carry a VIP`, vault excluded + from the triple count via its own metal-only [ok] line. Dual fixture: `12 ... (12 + dual-family)`. These are the numbers the harness assertions were re-pointed to. + +## Items + +### 1. `scripts/provider-bundle-check.py` -- vault metal-only exception, HARDENED (DOCFIX-210) +- WHAT: the vault VIP branch (invariant 9c region) now asserts vault is the metal PAIR + (metal-admin + metal-internal), NO provider-public leg, **per-DC and family-aware** + against the `--dc` PROVIDER/ADMIN/INTERNAL bands -- replacing the first draft's + hardcoded `10.12.4.` literal (which was blind to dc1's `10.12.64.x` provider). The + metal pair is STILL band-checked (50-99) and STILL registers in `octet_owner` for + uniqueness. Docstring invariant 2 records the exception. +- WHY: the ruled D-020 amendment (2026-08-05) makes vault the single non-triple + clustered VIP. **Advisor-caught defect in the first draft:** its `continue` ran + BEFORE the band check and `octet_owner[h] = n`, silently disarming octet-uniqueness + for `.61` -- a second app given `.61` would collide with nothing and pass (a gate + that cannot fail; this repo's named failure mode). PROVEN: the reconstructed draft + returns rc=0 on a glance-takes-vault's-`.61` input (gap); the fixed checker returns + rc=1 `VIP last octet .61 shared by glance and vault` (caught). +- REVERT: `git checkout HEAD -- scripts/provider-bundle-check.py` restores the + no-vault-exception version (vault treated as a normal triple; the live metal-only + overlay would then FAIL the check -- so revert the overlays in item 3 together). + +### 2. `tests/provider-bundle-check/run-tests.sh` -- harness reconciled to 58/0 +- WHAT: (a) rolesep fixture builder is now num_units-aware -- 3-unit HA apps land on + lxd:0/1/2, single-unit on lxd:0, guarded with a `SystemExit` if any `to:` app has + num_units>3 (only 3 control machines exist). Fixes T9, broken by BUNDLEFIX-053's + `num_units: 3` against the old `to:[lxd:0]` pin. (b) Count assertions re-pointed to + the measured values: T16 `12 -> 13` hacluster principals (vault-hacluster now in the + BASE bundle, dissolving the old T16/T17 pre/post-overlay ordering distinction -- + comment re-pointed; the dc-ha-scaleup.yaml-now-redundant question LOGGED as a Task #1 + finding, not fixed here per hard rule 1); T19 `13 -> 12` clustered VIPs; T21/T45 + `13 -> 12` clustered + `13 -> 12` dual-family (vault leaves the triple count). (c) + NEW failing-direction fixtures T54 (provider leg on vault FAILS), T55 (triple FAILS), + T56 (another app taking vault's `.61` octet FAILS -- guards the item-1 defect). Result + 49/6 -> 58/0 ALL PASS. +- WHY: a bundle change breaks the check's harness even when the script is untouched; + every new assertion must be proven able to FAIL (T54-56 have failing fixtures). +- REVERT: `git checkout HEAD -- tests/provider-bundle-check/run-tests.sh`. + +### 3. `overlays/vr1-dc0-vips.yaml` + `overlays/vr1-dc1-vips.yaml` -- vault metal pair, RE-RENDERED +- WHAT: dc0 vault vip -> `"10.12.8.61 10.12.12.61"`; dc1 vault vip the dual-family sextet + -> the dc1 metal pair `"10.12.68.61 10.12.72.61"` (provider leg + the D-141-deprecated + `fd50:` ULA metal v6 legs dropped). Both stay a v4 metal pair THROUGH a future D-141 + rule-3 promotion until a new amendment. **These overlays are GENERATED by + `render-dc-overlays.py` from `render/values/*.yaml` (D-136), NOT hand-editable** -- see + the FINDING below. Both files were re-rendered from the values files (items 6-7); the diff + vs HEAD is EXACTLY the one vault line in each, no other drift. +- WHY: implements the D-020 amendment. +- REVERT: revert items 6-7 (values) and re-render, or `git checkout HEAD -- overlays/vr1-dc*-vips.yaml`. + +### 6. `scripts/render-dc-overlays.py` -- vault metal-only render exception (DOCFIX-210) +- WHAT: `render()` gains a `name == "vault"` branch that drops the provider-public leg AND + forces v4 (no v6 append) so vault renders exactly the 2-leg metal pair in BOTH v4 and + dual-family DCs. Named exception, mirroring provider-bundle-check.py + pre-flight-checks.sh + -- fires for every DC's values file (the ruled Roosevelt delta), not a per-DC flag a new + DC could omit. +- WHY: the overlays are rendered artifacts; the vault metal-only change is only lawful + through the renderer (render-drift forbids hand-editing). +- REVERT: `git checkout HEAD -- scripts/render-dc-overlays.py` (then vault re-renders as a triple). + +### 7. `render/values/vr1-dc0-vips.yaml` + `render/values/vr1-dc1-vips.yaml` -- vault comment +- WHAT: vault's per-app `comment:` field updated (dc0) / added (dc1) to the D-020 metal-only + note. The comment is part of the rendered bytes, so it belongs in the values file, not the + overlay. Octet 61 unchanged; vault stays in the apps list (T12 count still 13). +- WHY: carries the amendment provenance into the rendered overlay. +- REVERT: `git checkout HEAD -- render/values/vr1-dc0-vips.yaml render/values/vr1-dc1-vips.yaml`. + +### 8. `scripts/pre-flight-checks.sh` -- CHECK 1 vault metal-only exception +- WHAT: the awk VIP validator gains app-name tracking (2-space header lines) and a + `name == "vault"` branch accepting exactly the metal PAIR (metal-admin + metal-internal, + octet-aligned, in-band, no provider column) as OK. `VIP_COUNT_EXPECT` stays 13 (vault still + joins the OK count). Without this, vault's 2-leg pair was reported MALFORMED (not 3/6 IPs). +- WHY: this is P4's live VIP validator; it is a fourth consumer of the vault VIP shape. +- REVERT: `git checkout HEAD -- scripts/pre-flight-checks.sh`. + +### FINDING (report to operator) -- overlays are RENDERED; render-drift was ALREADY red pre-session +- The overlays are generated from `render/values/*.yaml`; `tests/render-drift` byte-compares + every values file against its overlay and FAILS with "re-render, do not hand-edit the overlay." + Task #2's guidance ("edit overlays/vr1-dc1-vips.yaml vault->metal pair") and the part-2 + session's dc0 hand-edit BOTH violated this. **render-drift + render-dc-overlays + pre-flight-checks + were red at the part-2 close but NOT in its "RED AT CLOSE" list** (which named only repo-lint L10 + + provider-bundle-check). The part-2 dc0 overlay edit was non-reproducible from the moment it was + made. Resolved here by teaching the renderer (item 6) + values (item 7) and re-rendering, per the + advisor -- the amendment is RULED, so this is OPS implementation, not a new decision. + +### Harness updates for items 6-8 +- `tests/render-dc-overlays`: T15 count 13->12 dual-family (vault excluded) + vault metal-only + [ok]; NEW T15b proof-of-teeth (renaming vault restores the 6-leg sextet -- the exception is + name-keyed). T3b/T14 heal on re-render. 20/3 -> 24/0. +- `tests/render-drift`: heals on re-render (2/2 -> 4/0). +- `tests/pre-flight-checks`: T27/T28 heal; NEW T28b proof-of-teeth (vault as a triple FAILS + CHECK 1). 29/2 -> 32/0. + +### 4. `docs/design-decisions.md` -- D-020 amendment text finalized +- WHAT: the RULED 2026-08-05 D-020 amendment (vault metal-only) CONSEQUENCE section + augmented -- dc1 overlay now done (not "when it exists"), the per-DC/family-aware + checker + retained band/octet gates recorded, and the "stays a v4 metal pair through + D-141 rule-3" durability note added so a later dual-stack session does not "restore" + the dropped v6 legs as an oversight. +- WHY: the amendment was operator-ratified ("Ratify as written") but uncommitted. +- REVERT: `git checkout HEAD -- docs/design-decisions.md`. + +### 5. `docs/CURRENT-STATE.md` -- D-121 12/14 -> 14/14 + D-020 amendment; L10 satisfied +- WHAT: the D-121 HA scale-up status updated to 14/14 (keystone + vault added), with the + D-020 vault-metal-only landing recorded. **EVIDENCE GAP DECLARED (GA-R1 rule 2):** the + 14/14 count is OPERATOR-ATTESTED + part-2 records, NOT yet a committed `juju status` + capture -- flagged OWED, riding the F4 live-verify sweep. Not papered over. +- WHY: L10 requires the design-decisions Status-line change to touch CURRENT-STATE in the + same commit; and the committed status still read 12/14. +- REVERT: `git checkout HEAD -- docs/CURRENT-STATE.md`. + +## Verification (this session) +- `provider-bundle-check` 58/0; `render-dc-overlays` 24/0; `render-drift` 4/0; + `pre-flight-checks` 32/0. Full gauntlet re-run after the renderer work. +- Draft-vs-fixed collision proof captured (rc=0 draft / rc=1 fixed). +- Re-rendered overlays diff vs HEAD = exactly the one vault line each (no other drift). +- `scripts/repo-lint.sh` -> 0 fail (1 pre-existing L1 non-ASCII warn, unchanged). + +## Owed / NOT done this session (out of Task #2 scope) +- Live `juju status -m vr1-dc0` capture backing 14/14 (F4 sweep, next live step). +- F8 ceph-radosgw "Services not running: radosgw" verify; F9 dc0 rack repo-stage staleness. +- Task #1 post-wave bundle/overlay review, incl. whether dc-ha-scaleup.yaml is now redundant. diff --git a/docs/design-decisions.md b/docs/design-decisions.md index 7dbe629..b1e6e01 100644 --- a/docs/design-decisions.md +++ b/docs/design-decisions.md @@ -442,6 +442,48 @@ `provider-bundle-check.py`. The conflict is recorded here so the expansion review's proposal is not later mistaken for the ruled position. +**D-020 -- AMENDMENT (2026-08-05): vault is METAL-ONLY; the provider VIP is DROPPED** +(ADOPTS the 2026-07-25 expansion-review proposal recorded as "refused" immediately above -- +build-time proved it is the only buildable and the security-correct shape). + +**Status:** RULED 2026-08-05 (operator, GA-R5). **Question as presented:** at the D-121 +vault-HA build (2026-08-05), vault's provider-public VIP `10.12.4.61` FAILED to bind -- +hacluster `res_vault_..._vip start ... '[findif] failed'` on all 3 nodes -- because vault's +containers have NO provider-public interface (MEASURED: metal-admin `10.12.8.x` + metal-internal +`10.12.12.x` only; the metal-admin `.8.61` and metal-internal `.12.61` VIPs bound fine). +Verified LIVE that EVERY vault consumer is an internal OpenStack charm (`vault:certificates` +-> ~17 control-plane charms including magnum's OWN service cert; `vault:secrets` -> +barbican-vault only); nothing tenant-facing reaches vault -- the tenant-secrets path is +tenant -> Barbican provider VIP `.51` -> barbican-vault -> vault-internal, and tenant k8s PKI +is CAPI/kubeadm-generated, not from the cloud vault. Options: (a) vault METAL-ONLY -- drop the +provider VIP, keep metal-admin `.8.61` + metal-internal `.12.61`; (b) give vault a +provider-public leg so the VIP binds -- which would expose the PKI/CA ROOT on the tenant-facing +plane (security regression). Operator selection, exact utterance: **"(a) draft the amendment"**, +then on the drafted text: **"Ratify as written"**. + +**RULED SHAPE.** vault's VIP becomes the metal PAIR `"10.12.8.61 10.12.12.61"` (admin + +internal), NOT the provider/admin/internal triple. vault is the internal PKI/CA root and MUST +NOT be tenant-reachable, and has no provider leg to host `10.12.4.61` regardless. vault becomes +the single non-triple clustered VIP in the bundle; `provider-bundle-check.py`'s +"clustered VIPs are provider/admin/internal" invariant learns the vault metal-only exception. + +**CONSEQUENCE / BUILD.** (1) `overlays/vr1-dc0-vips.yaml` vault vip +`"10.12.4.61 10.12.8.61 10.12.12.61"` -> `"10.12.8.61 10.12.12.61"`; `overlays/vr1-dc1-vips.yaml` +vault vip the dc1 dual-family sextet -> the dc1 metal pair `"10.12.68.61 10.12.72.61"` (the +provider leg AND the D-141-deprecated `fd50:` ULA metal v6 legs dropped). Both DCs keep vault a +**v4 metal pair, and it STAYS a v4 metal pair through a future D-141 rule-3 dual-stack promotion** +until a new amendment rules on vault's v6 metal legs -- so a later dual-stack session does not +read the missing v6 legs as an oversight and "restore" them. (2) LIVE: `juju config vault +vip="10.12.8.61 10.12.12.61"` drops the dead provider VIP resource -> vault-hacluster unblocks. +(3) `provider-bundle-check.py` gains a vault metal-only exception (DOCFIX): the metal pair is +asserted **per-DC and family-aware** against the `--dc` PROVIDER/ADMIN/INTERNAL bands (NOT a +hardcoded `10.12.4.` prefix, which was blind to dc1's `10.12.64.x` provider), and is STILL +band-checked and octet-uniqueness-checked -- vault leaves the triple count but does not leave +those gates. (4) `.61` freed on provider-public. **Roosevelt delta:** vault is metal-only at +every DC standup; the PKI/CA root is never on a tenant-facing plane. Supersedes the "full triple +for vault" half of the 2026-07-27 amendment; the designate `.62` triple and the band-widening +below are UNCHANGED. + **BAND WIDENING (mechanical, two files, no choice in it).** `.61` and `.62` are legal under the D-134 AMENDMENT's `.50-.99` VIP band but are REJECTED by the gate as it stands: `scripts/provider-bundle-check.py:41` holds `OCTET_LO, OCTET_HI = 50, 60` and diff --git a/overlays/vr1-dc0-vips.yaml b/overlays/vr1-dc0-vips.yaml index 6237fd7..1fb058c 100644 --- a/overlays/vr1-dc0-vips.yaml +++ b/overlays/vr1-dc0-vips.yaml @@ -66,7 +66,7 @@ vip: "10.12.4.60 10.12.8.60 10.12.12.60" # B1 -- radosgw HA un-deferred for Roosevelt fidelity (decorative HA on testcloud) vault: options: - vip: "10.12.4.61 10.12.8.61 10.12.12.61" # R11 (D-020 amendment) -- was ruled in D-020's own enumeration and never built + vip: "10.12.8.61 10.12.12.61" # D-020 amendment 2026-08-05: vault METAL-ONLY (metal-admin + metal-internal; PKI/CA root, not tenant-facing; no provider leg -- findif). v4 metal pair, kept through D-141 rule-3. designate: options: vip: "10.12.4.62 10.12.8.62 10.12.12.62" # R11 (D-020 amendment) -- NEW to D-020's enumeration; dnsaas is already provider-bound diff --git a/overlays/vr1-dc1-vips.yaml b/overlays/vr1-dc1-vips.yaml index 4fdb204..6c78b98 100644 --- a/overlays/vr1-dc1-vips.yaml +++ b/overlays/vr1-dc1-vips.yaml @@ -66,7 +66,7 @@ vip: "10.12.64.60 10.12.68.60 10.12.72.60 2602:f3e2:f03:11::60 fd50:840e:74e2:320::60 fd50:840e:74e2:321::60" vault: options: - vip: "10.12.64.61 10.12.68.61 10.12.72.61 2602:f3e2:f03:11::61 fd50:840e:74e2:320::61 fd50:840e:74e2:321::61" + vip: "10.12.68.61 10.12.72.61" # D-020 amendment 2026-08-05: vault METAL-ONLY (dc1 metal-admin + metal-internal; no provider leg, no v6 -- PKI/CA root not tenant-facing). v4 metal pair, kept through D-141 rule-3. designate: options: vip: "10.12.64.62 10.12.68.62 10.12.72.62 2602:f3e2:f03:11::62 fd50:840e:74e2:320::62 fd50:840e:74e2:321::62" diff --git a/render/values/vr1-dc0-vips.yaml b/render/values/vr1-dc0-vips.yaml index 9751ae3..a26f6e3 100644 --- a/render/values/vr1-dc0-vips.yaml +++ b/render/values/vr1-dc0-vips.yaml @@ -61,7 +61,7 @@ comment: '# B1 -- radosgw HA un-deferred for Roosevelt fidelity (decorative HA on testcloud)' - name: vault octet: 61 - comment: '# R11 (D-020 amendment) -- was ruled in D-020''s own enumeration and never built' + comment: '# D-020 amendment 2026-08-05: vault METAL-ONLY (metal-admin + metal-internal; PKI/CA root, not tenant-facing; no provider leg -- findif). v4 metal pair, kept through D-141 rule-3.' - name: designate octet: 62 comment: '# R11 (D-020 amendment) -- NEW to D-020''s enumeration; dnsaas is already provider-bound' diff --git a/render/values/vr1-dc1-vips.yaml b/render/values/vr1-dc1-vips.yaml index d4b4d72..a91a7e9 100644 --- a/render/values/vr1-dc1-vips.yaml +++ b/render/values/vr1-dc1-vips.yaml @@ -40,6 +40,7 @@ octet: 60 - name: vault octet: 61 + comment: '# D-020 amendment 2026-08-05: vault METAL-ONLY (dc1 metal-admin + metal-internal; no provider leg, no v6 -- PKI/CA root not tenant-facing). v4 metal pair, kept through D-141 rule-3.' - name: designate octet: 62 header: '# overlays/vr1-dc1-vips.yaml diff --git a/scripts/pre-flight-checks.sh b/scripts/pre-flight-checks.sh index b9a548d..6155242 100644 --- a/scripts/pre-flight-checks.sh +++ b/scripts/pre-flight-checks.sh @@ -198,9 +198,24 @@ if [ "$VIPLINES" -eq "$VIP_COUNT_EXPECT" ]; then pass "vip: line count = $VIPLINES (from $VIP_OVL)"; else warn "vip: line count = $VIPLINES (want $VIP_COUNT_EXPECT, from $VIP_OVL)"; fi VIPOUT="$(awk -v pp="$VIP_PREFIX_PROVIDER" -v pa="$VIP_PREFIX_ADMIN" -v pi="$VIP_PREFIX_INTERNAL" \ -v lo="$VIP_OCTET_MIN" -v hi="$VIP_OCTET_MAX" ' - BEGIN{ok=0;bad=0} + BEGIN{ok=0;bad=0;name=""} + /^ [A-Za-z0-9_-]+:[[:space:]]*$/{name=$1; sub(/:.*/,"",name)} /^[[:space:]]+vip:/{ v=$0; sub(/^[^"]*"/,"",v); sub(/".*/,"",v); n=split(v,a," "); + # D-020 amendment 2026-08-05: vault is METAL-ONLY -- the metal PAIR (metal-admin + + # metal-internal), NO provider-public leg and NO v6 (v4 pair, kept through D-141 + # rule-3). Named exception keyed on the app header two lines up -- matches the + # exceptions in provider-bundle-check.py and render-dc-overlays.py. Still octet- + # aligned and in-band; it simply has no provider column and joins the OK count. + if(name=="vault"){ + if(n!=2){print " vault NOT the metal PAIR (want 2 IPs metal-admin+metal-internal): " $0; bad++; next} + if(index(a[1],pa".")!=1){print " vault leg 1 not metal-admin: " $0; bad++; next} + if(index(a[2],pi".")!=1){print " vault leg 2 not metal-internal: " $0; bad++; next} + split(a[1],x,"."); split(a[2],y,"."); + if(x[4]!=y[4]){print " vault UNALIGNED last octet: " $0; bad++; next} + if(x[4]+0hi){print " vault OUT-OF-RANGE octet: " $0; bad++; next} + ok++; next + } # R2 (RULED 2026-07-27): a vip is the v4 TRIPLE, or a dual-family SEXTET # appending the three v6 legs. Anything else is malformed. if(n!=3 && n!=6){print " MALFORMED(not 3 or 6 IPs): " $0; bad++; next} diff --git a/scripts/provider-bundle-check.py b/scripts/provider-bundle-check.py index 28da39e..67ae506 100644 --- a/scripts/provider-bundle-check.py +++ b/scripts/provider-bundle-check.py @@ -11,6 +11,11 @@ share ONE host part, in 50-99 (D-020 amendment / R11, 2026-07-27), and the v6 host part MIRRORS the v4 octet textually (D-136 ruling, same date). prefer-ipv6 and the v6 legs must land TOGETHER (see invariant 9). + EXCEPTION -- vault (D-020 amendment 2026-08-05): vault is METAL-ONLY, the metal + PAIR (metal-admin + metal-internal) with NO provider-public leg -- the internal + PKI/CA root must not be tenant-facing and has no provider leg to host a VIP. It + is the single sanctioned non-triple clustered VIP; still band- and + octet-uniqueness-checked, just against the metal pair. 3. ovn-chassis bridge-interface-mappings carries ALL FOUR chassis MACs, INCLUDING openstack0's -- the Pattern A revert re-adds the openstack0 MAC that D-057 trimmed for the now-dead provider-vip plane. @@ -420,6 +425,48 @@ continue prefer6 = bool(opts.get("prefer-ipv6")) parts = str(vip).split() + # D-020 amendment 2026-08-05: vault is METAL-ONLY -- exactly the metal PAIR + # (metal-admin + metal-internal), NO provider-public leg. vault is the internal + # PKI/CA root: it must not be tenant-facing, and has no provider leg to host a VIP + # (a provider VIP findif-fails on its containers). Family/DC-aware via the --dc + # PROVIDER/ADMIN/INTERNAL bands, NOT a hardcoded dc0 prefix (a `10.12.4.` literal + # was blind to dc1's 10.12.64.x provider leg). It stays the v4 metal pair THROUGH a + # future D-141 rule-3 dual-stack promotion until a new amendment rules on vault's v6 + # metal legs. The metal pair is STILL subject to the band + octet-uniqueness gates + # (an early `continue` here must NOT disarm them for vault's octet -- if it did, .61 + # would go un-band-checked AND never register in octet_owner, so a collision on .61 + # with any other app would pass silently: a gate that cannot fail). + if n == "vault": + if len(parts) != 2: + fails.append("vault vip must be the metal PAIR (metal-admin + metal-internal); " + "got %d address(es): %r (D-020 amendment 2026-08-05)" % (len(parts), vip)) + continue + try: + a_ip, i_ip = (ipaddress.ip_address(p) for p in parts) + except ValueError as e: + fails.append("vault bad vip ip: %s" % e); continue + if a_ip in PROVIDER or i_ip in PROVIDER: + fails.append("vault vip carries a provider-public leg: %r -- vault is METAL-ONLY, " + "the PKI/CA root must not be tenant-facing (D-020 amendment " + "2026-08-05)" % vip); continue + if not (a_ip in ADMIN and i_ip in INTERNAL): + fails.append("vault vip legs are not [metal-admin, metal-internal] for --dc %s: " + "%r (D-020 amendment 2026-08-05)" % (args.dc, vip)); continue + hosts = {_host_part(p) for p in parts} + if len(hosts) != 1: + fails.append("vault vip host parts differ across legs: %r" % vip); continue + h = hosts.pop() + if not h.isdigit(): + fails.append("vault vip host part %r is not the decimal v4-octet mirror" % h); continue + o = int(h) + if not (OCTET_LO <= o <= OCTET_HI): + fails.append("vault vip octet .%d outside %d-%d" % (o, OCTET_LO, OCTET_HI)); continue + if h in octet_owner: + fails.append("VIP last octet .%s shared by %s and %s" % (h, octet_owner[h], n)); continue + octet_owner[h] = n + oks.append("vault vip is metal-only (metal-admin + metal-internal, no provider), " + "octet .%d in %d-%d -- D-020 amendment 2026-08-05" % (o, OCTET_LO, OCTET_HI)) + continue if len(parts) not in (3, 6): fails.append("%s vip is neither a v4 triple (3) nor a dual-family sextet (6) " "-- got %d address(es): %r" % (n, len(parts), vip)); continue diff --git a/scripts/render-dc-overlays.py b/scripts/render-dc-overlays.py index 4710f83..a0c372b 100755 --- a/scripts/render-dc-overlays.py +++ b/scripts/render-dc-overlays.py @@ -322,15 +322,26 @@ # artifacts use and sorting by name reproduces nothing. for app in sorted(vals["apps"], key=lambda x: int(x["octet"])): o = int(app["octet"]) - addrs = ["%s.%d" % (pre4[l], o) for l in legs] - if fam == "dual": + # D-020 amendment 2026-08-05: vault is METAL-ONLY -- metal-admin + metal-internal, + # NO provider-public leg (the internal PKI/CA root must not be tenant-facing, and + # its containers carry no provider leg to host the VIP -- a provider VIP findif-fails + # live). And NO v6 legs even in a dual-family DC: vault stays the v4 metal PAIR + # through a future D-141 rule-3 promotion until a new amendment rules on its v6 metal + # legs. Named exception, mirroring provider-bundle-check.py and pre-flight-checks.sh + # CHECK 1 -- fires for EVERY DC's values file (the ruled Roosevelt delta) rather than + # relying on a per-DC flag that a new DC's file could silently omit. + app_legs = [l for l in legs if l != "provider-public"] if app["name"] == "vault" else legs + app_fam = "v4" if app["name"] == "vault" else fam + addrs = ["%s.%d" % (pre4[l], o) for l in app_legs] + if app_fam == "dual": + # (vault never reaches here -- app_fam is forced v4 above.) # `::%d`, not `:%d`. The values file stores the /64 base with its # trailing colons stripped (2602:f3e2:f03:11), so the separator has to # put BOTH back -- a single colon yields 2602:f3e2:f03:11:50, which is # a malformed address that still LOOKS like one. This was the # renderer's first dual-family output and provider-bundle-check caught # it; the octet mirror is textual, so the digits stay decimal. - addrs += ["%s::%d" % (pre6[l], o) for l in legs] + addrs += ["%s::%d" % (pre6[l], o) for l in app_legs] out.append(" %s:\n options:\n" % app["name"]) if False and fam == "dual" and app["name"] in prefer6: # RULED OFF -- see below # >>> RULED OFF ENTIRELY 2026-07-31 (D-101 RULING NOTE (b), GA-R5: "Set it diff --git a/tests/pre-flight-checks/run-tests.sh b/tests/pre-flight-checks/run-tests.sh index e315700..c0ce386 100644 --- a/tests/pre-flight-checks/run-tests.sh +++ b/tests/pre-flight-checks/run-tests.sh @@ -278,6 +278,14 @@ cp "$REPO/overlays/vr1-dc0-vips.yaml" "$DR0/overlays/vr1-dc0-vips.yaml" run "$DR0" vr1-dc0 0 'Summary: 0 fatal, 0 warning' "T28 the REAL committed vr1-dc0 VIP overlay passes CHECK 1 (default DC)" +# T28b PROOF-OF-TEETH for the D-020 vault metal-only branch: restore vault's OLD +# provider/admin/internal triple and require CHECK 1 to reject it. Without this the +# vault exception could accept anything named vault -- a gate that cannot fail. +DVT="$(mkfix_dc vault_triple vr1-dc0)"; mkmaas "$DVT" +cp "$REPO/overlays/vr1-dc0-vips.yaml" "$DVT/overlays/vr1-dc0-vips.yaml" +sed -i 's/vip: "10\.12\.8\.61 10\.12\.12\.61"/vip: "10.12.4.61 10.12.8.61 10.12.12.61"/' "$DVT/overlays/vr1-dc0-vips.yaml" +run "$DVT" vr1-dc0 1 'vault NOT the metal PAIR' "T28b vault as a provider/admin/internal triple FAILS CHECK 1 (metal-only)" + echo "== an octet in NO ruled D-134 band REFUSES, once ==" # The classification pass runs ONCE and stores its verdict, because CHECK 2 and CHECK 4 # both need it: deriving it per-check emitted the refusal twice and then printed a diff --git a/tests/provider-bundle-check/run-tests.sh b/tests/provider-bundle-check/run-tests.sh index af112f4..7bf6ebb 100644 --- a/tests/provider-bundle-check/run-tests.sh +++ b/tests/provider-bundle-check/run-tests.sh @@ -159,8 +159,19 @@ continue if n == "ceph-osd": s["num_units"] = 4; s["to"] = ["5","6","7","8"] elif n == "nova-compute": s["num_units"] = 2; s["to"] = ["3","4"] - elif n in ("mysql-innodb-cluster","ovn-central","ceph-mon"): s["to"] = ["lxd:0","lxd:1","lxd:2"] - else: s["to"] = ["lxd:0"] + else: + # num_units-aware since BUNDLEFIX-053 scaled the control plane 1->3 (D-121): + # a 3-unit HA app lands on the 3 distinct control machines (lxd:0/1/2), a + # single-unit app on lxd:0. Before this, control apps were pinned to lxd:0 + # while num_units was 3 -> the anti-affinity/count check FAILED the whole + # fixture (T9). Guarded: only 3 control machines (0,1,2) exist, so an app with + # to: and num_units>3 that is not ceph-osd/nova-compute has no home here and + # the builder must be revisited rather than silently emitting off-role lxd:3. + nu = int(s.get("num_units", 1) or 1) + if nu > 3: + raise SystemExit("rolesep fixture builder: %s num_units=%d exceeds the 3 " + "control machines (0,1,2); builder needs update" % (n, nu)) + s["to"] = ["lxd:%d" % j for j in range(nu)] yaml.safe_dump(b, open(sys.argv[2], "w")) PY @@ -218,8 +229,15 @@ # RE-POINTED 2026-07-29: this asserted designate's VIP was MISSING. R11 built it, so # the assertion is inverted to the surviving invariant -- the real deploy input is now # CLEAN -- and T16b keeps the check's teeth by removing a VIP and demanding it fire. -run 0 '12 hacluster principal\(s\) all carry a VIP' \ - "T16 the real dc0 deploy input is CLEAN (R11 built)" "$TMP/pristine.yaml" +# RE-POINTED 2026-08-05 (BUNDLEFIX-053): bundle.yaml now carries the FULL HA chain, +# vault-hacluster INCLUDED, so pristine (no overlay) already has 13 hacluster principals +# -- was 12 before vault-hacluster moved from the HA overlay into the base. This +# DISSOLVES the old T16(pre-overlay=12)/T17(post-overlay=13) ordering distinction: both +# read 13 now. T16 on its own asserts the real dc0 deploy input is CLEAN, every one of +# the 13 base-bundle hacluster principals carrying a VIP. (Whether dc-ha-scaleup.yaml is +# now wholly redundant is a Task-#1 post-wave-review finding -- logged, not fixed here.) +run 0 '13 hacluster principal\(s\) all carry a VIP' \ + "T16 the real dc0 deploy input is CLEAN (full HA chain in base)" "$TMP/pristine.yaml" mutate t16b.yaml 'b["applications"]["designate"]["options"].pop("vip"); b["applications"]["designate"]["options"].pop("prefer-ipv6",None)' run 1 'hacluster relation but no vip: designate' \ @@ -256,7 +274,9 @@ # REPLACED, not deleted -- the real deploy input must read 0 dual-family. The # dual-family full-count positive control moves to T21/T45 on the synthetic # dual.yaml. When D-141 rule-3 promotes v6, re-point this back to dual-family. -run 0 '13 clustered VIP\(s\).*\(0 dual-family\)' \ +# COUNT 13->12 (2026-08-05, D-020 amendment): vault is now METAL-ONLY and is counted by +# its own [ok] line, NOT the provider/admin/internal triple count -- so 13 triples -> 12. +run 0 '12 clustered VIP\(s\).*\(0 dual-family\)' \ "T19 the real dc0 deploy input is v4-only under D-141 (0 dual-family)" "$TMP/good.yaml" # T20/T21 RE-POINTED 2026-07-31 (D-101 RULING NOTE (b), operator: "Set it false on the @@ -277,7 +297,7 @@ # rejecting everything. RE-POINTED 2026-08-03 to the synthetic dual.yaml: under # D-141 the real deploy input (good.yaml) is v4-only, so the dual-family shape is # carried by the fixture promotion restores, not by good.yaml. -run 0 '13 clustered VIP\(s\).*\(13 dual-family\)' \ +run 0 '12 clustered VIP\(s\).*\(12 dual-family\)' \ "T21 v6 legs WITHOUT the option PASSES (the ruled shape)" "$TMP/dual.yaml" # T22 v6 provider leg in the NODE /64 instead of the dedicated GUA VIP /64 @@ -472,8 +492,8 @@ # T45 the RULED shape passes: the six carry full dual-family vips and no option. This is # the positive control -- without it, 9a could be satisfied by rejecting everything. # RE-POINTED 2026-08-03 to dual.yaml (good.yaml is v4-only under D-141). -run 0 '13 clustered VIP\(s\).*\(13 dual-family\)' \ - "T45 the ruled shape PASSES: six dual-family vips with no prefer-ipv6" "$TMP/dual.yaml" +run 0 '12 clustered VIP\(s\).*\(12 dual-family\)' \ + "T45 the ruled shape PASSES: dual-family vips with no prefer-ipv6 (vault metal-only excluded)" "$TMP/dual.yaml" # T46 9a AND 9b BOTH match this input, and the DIAGNOSIS must be 9a's. octavia with # prefer-ipv6 true AND a v4 triple satisfies the coupling rule's shape (prefer6 @@ -550,6 +570,32 @@ run 1 'declares no top-level default-base' \ "T53 a bundle with no default-base REFUSES (does not pass vacuously)" "$TMP/t53.yaml" +# ---- vault METAL-ONLY invariant (D-020 amendment 2026-08-05) failing-direction fixtures ---- +# The positive control is the real deploy input: T16/T18/T19 all run it and its output +# carries `vault vip is metal-only ...`. These three prove the new branch can FAIL -- a +# fresh assertion is decoration until a fixture reddens it (this repo's measured rule). +# T54 a provider-public leg on vault FAILS. vault is the PKI/CA root: it must never be +# tenant-facing. Uses a 2-leg pair so this tests the PROVIDER exclusion, not arity. +mutate t54.yaml 'b["applications"]["vault"]["options"]["vip"]="10.12.4.61 10.12.8.61"' +run 1 'vault vip carries a provider-public leg' \ + "T54 vault vip with a provider-public leg FAILS (metal-only)" "$TMP/t54.yaml" + +# T55 the OLD provider/admin/internal triple FAILS -- the pre-amendment .61 shape that +# findif-failed live is now rejected at the gate, not just in the field. +mutate t55.yaml 'b["applications"]["vault"]["options"]["vip"]="10.12.4.61 10.12.8.61 10.12.12.61"' +run 1 'vault vip must be the metal PAIR' \ + "T55 vault as a provider/admin/internal triple FAILS (must be metal pair)" "$TMP/t55.yaml" + +# T56 THE gap the amendment nearly shipped: the first draft `continue`d before the octet +# band + octet_owner registration, so vault's .61 was un-band-checked AND never +# registered -- a second app given .61 would collide with NOTHING and pass silently. +# Give glance vault's .61 and demand the collision fires. This case would PASS (rc=0, +# the gate disarmed) against that draft; it FAILS now because the metal branch +# registers octet_owner too. Proves the metal-only path still guards uniqueness. +mutate t56.yaml 'b["applications"]["glance"]["options"]["vip"]="10.12.4.61 10.12.8.61 10.12.12.61"' +run 1 'octet .61 shared by' \ + "T56 another app taking vault's .61 octet FAILS (metal branch still registers uniqueness)" "$TMP/t56.yaml" + echo echo "RESULT: PASS=$PASS FAIL=$FAIL" [[ "$FAIL" -eq 0 ]] && { echo "ALL PASS"; exit 0; } || exit 1 diff --git a/tests/render-dc-overlays/run-tests.sh b/tests/render-dc-overlays/run-tests.sh index 52a17c9..edc5114 100755 --- a/tests/render-dc-overlays/run-tests.sh +++ b/tests/render-dc-overlays/run-tests.sh @@ -144,10 +144,34 @@ else bad "T14 rendered overlay rejected by the gate: $(grep -m1 FAIL "$TMP/gate")" fi -if grep -q "13 clustered VIP(s).*13 dual-family" "$TMP/gate"; then - ok "T15 the gate sees 13 VIPs, all dual-family" +# COUNT 13->12 (2026-08-05, D-020 amendment): vault renders METAL-ONLY, so it is NOT +# in the provider/admin/internal dual-family count -- 12 clustered dual-family VIPs + +# vault's own metal-only [ok] line. The 13 VIP-carrying apps are unchanged (T12). +if grep -q "12 clustered VIP(s).*12 dual-family" "$TMP/gate" \ + && grep -q "vault vip is metal-only" "$TMP/gate"; then + ok "T15 the gate sees 12 dual-family VIPs + vault metal-only" else - bad "T15 gate did not report 13 dual-family VIPs" + bad "T15 gate did not report 12 dual-family VIPs + vault metal-only" +fi + +# T15b PROOF-OF-TEETH for the D-020 vault metal-only render exception (2026-08-05). It is +# keyed on the app NAME: rename vault in a COPY of the values and the octet-61 app renders +# the FULL dual-family sextet again -- proving the 2-leg metal pair is produced BY the +# exception, not by accident -- while the real vault stays 2 legs with NO provider (10.12.64.) leg. +python3 - "$DV" "$TMP/vault-renamed.yaml" <<'PY' +import sys, yaml +v = yaml.safe_load(open(sys.argv[1])) +for a in v["apps"]: + if a["name"] == "vault": a["name"] = "zzz-notvault" +yaml.safe_dump(v, open(sys.argv[2], "w"), sort_keys=False, width=4096) +PY +vault_legs="$(python3 "$R" render "$DV" 2>/dev/null | grep -A2 '^ vault:' | grep 'vip:' | sed 's/.*"\([^"]*\)".*/\1/' | wc -w)" +vault_prov="$(python3 "$R" render "$DV" 2>/dev/null | grep -A2 '^ vault:' | grep 'vip:' | grep -c '10.12.64.')" +renamed_legs="$(python3 "$R" render "$TMP/vault-renamed.yaml" 2>/dev/null | grep -A2 'zzz-notvault:' | grep 'vip:' | sed 's/.*"\([^"]*\)".*/\1/' | wc -w)" +if [ "$vault_legs" = "2" ] && [ "$vault_prov" = "0" ] && [ "$renamed_legs" = "6" ]; then + ok "T15b vault renders the 2-leg metal pair (no provider); renaming it restores the 6-leg sextet" +else + bad "T15b vault metal-only render (vault_legs=$vault_legs vault_prov=$vault_prov renamed_legs=$renamed_legs; want 2/0/6)" fi # ---- refusals ----------------------------------------------------------------