diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 9b7d408..61174b2 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -727,6 +727,32 @@ (section 6): ANSWERABLE from Stage 5 onward once the prefix exists, **BLOCKING at the FINAL stage close / project close**. Ruling it early would also pre-empt the UNRULED D-136 render-pipeline decision, which covers the same apex-authority ground. + **FINAL PRE-RULING MEASUREMENTS for R9/R10/R12-R15 taken 2026-07-27** (capture + `docs/audit/r9-r15-final-measurements-20260727.txt`). **TWO changed materially, both + corrections to the audit's OWN framing.** + **R9: there is only ONE failure mode, and the Stage-5 blast radius is TWO scripts, not + twenty.** `lib-net.sh:76-79` states the design -- sourcing without the selector keeps + VR0/DC0 values "completely unchanged" -- so the `unset` block fires ONLY for + selector-CALLERS. The 20 non-selector consumers therefore fail SILENTLY with dc0 literals, + while the 8 correctly-updated ones are the ones that break loudly under `set -u`. Of the + 20, only `phase-03-core-verify.sh` and `deploy-watch.sh` are invoked by Stage-5's runbooks; + the rest bite at later stages. + **R14: THE PREMISE IS WRONG and the question largely dissolves.** The three S5 power-key + asymmetries are NOT "ruled correct by SEC-016" -- SEC-016 ruled per-DC ISOLATION (dc1 gets + its own key), which is satisfied; it never blessed the filename/host/custody divergence. + That divergence is **SEC-021(b)**, an OPEN defect whose own disposition reads "needs a + naming/custody reconciliation to the dc1 shape", and whose complaint was literally "nothing + compares them". **S5 is the thing that now compares them -- the register is RIGHT and the + finding is REAL**, not noise to suppress. + **R15 refinement:** the gauntlet ALREADY has a zero-floor (`run-tests-all.sh:35` exits 2 on + `RAN -eq 0`); what is missing is a MINIMUM-count floor. `repo_lint.py:132-135` has NEITHER + -- no valid-root check and no files-scanned floor. The two gates need different fixes. + **R12 confirmed:** `dc-dc-deployment-workflow.md:206` and `dc-dc-phase4:46` both assign a + node time check to G17; `chronyc` appears ZERO times in this document. Bullet 6 IS struck + (DOCFIX-204); its REPLACEMENT is what is homeless, and the window is one-time at first boot. + **R10 confirmed:** P3 and P4-MAAS clear by running preflight on voffice1; octavia-pki and + the 7 credential findings do not. Preflight cannot usefully be RUN there yet -- that clone + is 105 commits stale. Remaining: R9-R10 blocking, R12-R15 standing; G18 deferred-and-gated. - Position inside Stage 3: deploy step A EXECUTED 2026-07-19 (6/0/6 exact; convergence zero -- diff --git a/docs/audit/r9-r15-final-measurements-20260727.txt b/docs/audit/r9-r15-final-measurements-20260727.txt new file mode 100644 index 0000000..356706c --- /dev/null +++ b/docs/audit/r9-r15-final-measurements-20260727.txt @@ -0,0 +1,162 @@ +===================================================================== +FINAL PRE-RULING MEASUREMENTS -- R9, R10, R12, R13, R14, R15 +Date: 2026-07-27 Branch: dc-dc-stage5-grounding-audit +Taken so the remaining decisions can be worked through without pausing +to gather more from the environment (operator direction). +===================================================================== + +TWO OF THESE CHANGED MATERIALLY ON MEASUREMENT. R9's blast radius is far +narrower than stated, and R14's PREMISE IS WRONG -- see below. Both are +corrections to the audit's own earlier framing. + +--------------------------------------------------------------------- +R9 -- WHERE dc1's OPENSTACK-LAYER LITERALS LIVE +--------------------------------------------------------------------- +CORRECTION 1 -- there is only ONE failure mode, not two. The audit said +consumers either "die under `set -u`" or "silently use dc0 literals". +Measured, `scripts/lib-net.sh:76-79` states the design explicitly: + "Backward compatible by construction: sourcing this file with no further + action continues to populate PLANE_CIDRS / PLANE_NAME / PLANE_GW / etc. + exactly as above (VR0/DC0's real, measured values) -- every existing + script that sources lib-net.sh keeps working completely unchanged." +The `unset` block only executes IF `lib_net_select_dc` is CALLED. So for the +20 non-selector consumers the failure mode is UNIFORM: **silent VR0/dc0 +literals.** The `set -u` death is only reachable by the 8 that DO call the +selector -- i.e. the scripts that were correctly updated are the ones that +break loudly, and the ones that were never updated fail silently. That is the +worse half, and it is the opposite of the intuitive reading. + +CORRECTION 2 -- the STAGE-5 blast radius is TWO scripts, not twenty. +Grepping the two runbooks Stage 5 actually uses +(`dc-dc-phase4-juju-bundle-per-dc.md`, `phase-01-bundle-deploy.md`) for each +non-selector consumer: + phase-03-core-verify.sh INVOKED by Stage-5 runbooks + deploy-watch.sh INVOKED by Stage-5 runbooks + phase-02-vault-preflight.sh -- not invoked at Stage 5 + phase-03-admin-openrc.sh -- not invoked at Stage 5 + phase-04-network-create.sh -- not invoked at Stage 5 + phase-04-network-verify.sh -- not invoked at Stage 5 + phase-04-internal-cert-san-verify.sh -- not invoked at Stage 5 + phase-05-octavia-verify.sh -- not invoked at Stage 5 + juju-spaces-check.sh -- not invoked at Stage 5 +The rest bite at LATER stages (their own phase runbooks), not at Stage 5. +So R9 is smaller than presented for Stage 5 and unchanged in size overall. + +Variables the dc1 arm unsets (relevant only to selector-callers): + FIP_POOL_END FIP_POOL_START KEYSTONE_VIP_DEFAULT METAL_INTERNAL_IFACE + METAL_INTERNAL_VID VIP_COUNT_EXPECT VIP_PREFIX_ADMIN VIP_PREFIX_INTERNAL + VIP_PREFIX_PROVIDER +Reminder from R11's guard: `METAL_INTERNAL_VID`/`IFACE` are CORRECTLY unset +per D-133 and must STAY unset. Only the VIP/FIP/keystone group is superseded. + +--------------------------------------------------------------------- +R10 -- STAGE-5 ENTRY BASIS AGAINST A RED PREFLIGHT +--------------------------------------------------------------------- +Client availability, measured: + vcloud (jumphost) voffice1 (D-128 Plane-2 host) + maas ABSENT PRESENT + juju ABSENT PRESENT + openstack ABSENT ABSENT +Red set disposition: + P1 lint ALREADY CLEAR (the earlier FAIL was this session's own + uncommitted captures) + P3 33 warns CLEARS on voffice1 -- caused by the missing `juju` binary + P4 MAAS CLEARS on voffice1 -- caused by the missing `maas` binary, + NOT by unreachability. The message actively misleads. + P4 octavia-pki DOES NOT clear by host. Gitignored secret; and per R7 the + generator is dc0-frozen and needs its DOCFIX first. + P5 creds 7 DOES NOT clear by host. See R13/R14. +NOT MEASURABLE TODAY: preflight cannot usefully be RUN on voffice1, because +that clone is 105 commits stale (precondition 0.1). The host-availability +matrix above is the evidence, not a preflight run there. + +--------------------------------------------------------------------- +R12 -- G17 SCOPE (the node time-source check) +--------------------------------------------------------------------- +CONFIRMED, three surfaces, two of which assign a time check that the gate row +itself omits: + `docs/dc-dc-deployment-workflow.md:206` -- "Node-side reachability **and the + node time source** are gate **G17**, split out 2026-07-27". The SAME line + confirms "`NTP from edge` was STRUCK -- superseded by D-129(iv) ... + (DOCFIX-204)". + `runbooks/dc-dc-phase4-juju-bundle-per-dc.md:46` -- "both: `chronyc sources` + shows the MAAS-served time source, NOT the DC edge (D-129(iv) removed the + edge from that path)", with `:48-50` warning that "If first boot happens + without these captures being taken, the observation window has passed". + `docs/CURRENT-STATE.md` G17 row -- `chronyc` appears ZERO times in the whole + document. The row's [V] column names only the two artifact fetches. +So bullet 6 IS struck; what is homeless is its REPLACEMENT (MAAS-hierarchy +time verification). The observation window is one-time, at first boot. + +--------------------------------------------------------------------- +R13 -- CREDENTIAL REPRODUCIBILITY +--------------------------------------------------------------------- +30 rows / 16 distinct ids carry `mint-ref=operator-terminal`. The +UNRECOVERABLE-IN-PLACE subset is SEVEN keypair ids -- loss means losing the +only access path: + dc0-edge-key, dc1-edge-key (edge SSH; SEC-007/-015 make this the ONLY + management path to the DC edges) + dc0-svc-key, dc1-svc-key (rack service access) + dc0-maas-power-key, dc1-maas-power-key (MAAS -> libvirt power) + office1-svc-key (headend access) +Plus `office1-edge-root` (password + hash, not a keypair), and the remaining +office1/netbox/vault/rgw rows. +Lens 7's proposed minimum fix needs NO new tool: record each mint as a +numbered runbook step and flip those rows' `mint-ref` from +`operator-terminal` to `runbook::` -- the existing S4 check +already resolves `runbook:` references, so the debt becomes convergent. + +--------------------------------------------------------------------- +R14 -- **THE PREMISE IS WRONG. THIS QUESTION LARGELY DISSOLVES.** +--------------------------------------------------------------------- +The audit framed R14 as: the matrix cannot express a RULED exception, so the +three S5 power-key asymmetries -- "ruled correct by SEC-016" -- report as +findings forever and a reader learns to ignore a permanent red. + +**MEASURED: they are NOT ruled correct. They are a genuine OPEN defect.** +SEC-016 ruled that dc1 gets its OWN dedicated power key rather than reusing +dc0's -- per-DC ISOLATION, which is satisfied. It did NOT bless the FILENAME, +HOST and CUSTODY divergence. That divergence is **SEC-021(b)**, verbatim from +the ledger: + "The dc0 MAAS->libvirt power key (SEC-012) is `maas-virsh_ed25519` on + **voffice1**, while dc1's equivalent (SEC-016) is + `vr1-dc1-maas-power_ed25519{,.pub}` on **vcloud** and manifest-declared: + same purpose, different filename, different host, and dc0 carries no + `.pub`. Per-DC rows that should be symmetric are not, and nothing compares + them." +and its disposition: "**(b) needs a naming/custody reconciliation to the dc1 +shape**". + +The three S5 findings say exactly that -- same logical id +(`dcN-maas-power-key`) with different filenames per site and no counterpart. +**The register is RIGHT and the finding is REAL.** It is not noise to be +suppressed; it is SEC-021(b) being correctly surfaced by the one control that +can see it. "Nothing compares them" was the complaint; S5 is the thing that +now compares them. + +WHAT REMAINS OF R14, if anything: the matrix has 12 columns +(id, cardinality, site-key, host-role, filename, access-type, principal, +custody, mint-stage, mint-ref, sec-ref, notes-ref). There is no +ruled-exception field. Whether one is ever NEEDED is now an open question +rather than a demonstrated one -- the motivating example turned out not to be +an exception at all. + +--------------------------------------------------------------------- +R15 -- GAUNTLET / REPO-LINT FLOOR +--------------------------------------------------------------------- +REFINEMENT: the gauntlet ALREADY HAS A ZERO-FLOOR. + `scripts/run-tests-all.sh:35` -- `if [ "$RAN" -eq 0 ]; then echo "GAUNTLET: + no harnesses matched '${FILTER}'"; exit 2; fi` +So "zero harnesses ran" is already caught. What is absent is a MINIMUM count +floor: 81 harnesses on disk, 81 reported, and `81` pinned nowhere executable +(`grep` of `run-tests-all.sh`: no match). A DELETED or RENAMED harness lowers +the count silently and still prints ALL GREEN -- verified in a fixture (L4-8). +`repo_lint.py:132-135` has NEITHER floor: + `argv = [a for a in argv if a not in (RECORD_FLAG, COUPLING_FLAG)]` + `R = pathlib.Path(argv[0] if argv else ".").resolve()` +Only the two KNOWN flags are stripped; any other `--flag` becomes the ROOT. +No `R.is_dir()` check and no files-scanned floor, so a one-character typo +resolves to a nonexistent directory, `rglob` yields nothing, and it reports +`PASS (0 fail, 0 warn)` over ZERO files -- reproduced twice this session. +So the two gates need DIFFERENT fixes: the gauntlet needs a minimum-count +floor; repo-lint needs a valid-root check AND a files-scanned floor.