diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 8e2ca98..dc473a0 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -477,6 +477,28 @@ on 07-11) -- that stale prose is what caused this error, and it is queued in Phase 3. Remaining: R3-R11 blocking, R12-R15 standing, in `docs/audit/queued-rulings-20260727.md`. + **UNMEASURED-GAP SWEEP 2026-07-27** (operator challenge: did the committee actually + measure live state, or take shortcuts?). Register: + `docs/audit/stage5-unmeasured-register-20260727.md`. Honest accounting -- the apex was + never polled by ANY lens (lens 2 declared it out of scope, correctly and without + overclaiming, but it is a key system it had reach to), and the synthesis then used a + two-day-old repo dump instead of polling live. **The sharpest instance of the general + problem: lens 6 declared `juju restore-backup` unverifiable because "no Juju client + exists on this host" while juju 3.6.27 was installed on voffice1 and lens 5 was + successfully running `juju help` against it in the same session.** Fourteen deferred + items were CLOSED by the sweep. Consequential results: the LIVE apex is identical to the + dump (139 prefixes / 103 IPv6, zero drift -- the conclusion was right, the method was + not); **`juju restore-backup` DOES NOT EXIST on 3.6.27**, promoting L6-14 from RISK to + CONFIRMED DEFECT and making Stage 6 Step 9's D-104 restore drill unsatisfiable as + written; dc0's compute provider MACs measured, CONFIRMING L3-8; `curl` present on both + racks so L4-13's hole is theoretical; and the pinned charm channels DO resolve + (`2024.1`, `2.4`, `squid` all present via `juju info` on voffice1), proving P3's 33 + warns are purely the missing `juju` binary on vcloud. **NEW FINDING: preflight's "MAAS + unreachable" is a MISDIAGNOSIS -- the `maas` binary is simply ABSENT on vcloud. With the + absent `juju` and absent `openstack`, THREE separate preflight/deploy failures on this + jumphost are all "the client is not installed" and each is reported as something else.** + Still-open gaps that are measurable and were NOT done are listed in the register's + section 3 rather than buried. - Position inside Stage 3: deploy step A EXECUTED 2026-07-19 (6/0/6 exact; convergence zero -- `docs/audit/outer-plan-20260719-postA-converged.txt`). **Deploy step B diff --git a/docs/audit/stage5-unmeasured-register-20260727.md b/docs/audit/stage5-unmeasured-register-20260727.md new file mode 100644 index 0000000..b51c1a4 --- /dev/null +++ b/docs/audit/stage5-unmeasured-register-20260727.md @@ -0,0 +1,123 @@ +# UNMEASURED REGISTER -- Stage-5 grounding audit (2026-07-27) + +**Why this file exists.** The operator asked whether the committee actually measured +live state or took shortcuts. The honest answer is that it did some of both, and this +register is the accounting: every UNMEASURED item the seven lenses declared, plus what +a follow-up sweep closed, plus what remains genuinely open and why. + +**The founding principle it enforces**, from the skill: *"could not look" is never +"nothing there"* -- and its corollary, which this audit had to learn the hard way: +**a declared UNMEASURED is only honest if it was genuinely unmeasurable. Declaring an +item unmeasurable that another lens measured in the same session is not scope +discipline, it is a gap.** + +--- + +## 1. What went wrong, precisely + +Two distinct failures, and they belong to different actors. + +**(a) The apex was never polled, by anyone.** Lens 2's brief was as-built-vs-live, and +NetBox is the IPAM apex per the skill's own routing table. It measured MAAS and the +nodes and declared the apex out of scope. It did so HONESTLY -- its UNMEASURED item 5 +reads: *"IPv6 state at the NetBox apex ... I measured MAAS and the nodes only. D-101's +literals may exist in NetBox and simply not be carved into MAAS -- L2-3's claim is +scoped to MAAS + node reality and does not assert the apex is empty."* That is the +contract working: no overclaim. But the apex is a key system and it had permission and +time to reach it. **Under-scoped, not dishonest.** + +**(b) The synthesis then used a two-day-old dump instead of polling live -- and that +was mine, not an agent's.** On being asked, I read +`netbox/draft/vr1-office1-current-20260725.json` and treated it as apex state. Worse, I +had already put a question to the operator (R2a, "which literals") built on D-101's +stale "Remaining open item" prose, WITHOUT acting on lens 2's explicit warning that the +literals might already exist. Trusting stale decision prose over an available +measurement is exactly the failure this audit was convened to find. + +**The sharpest single instance of the general problem:** lens 6 declared +`juju restore-backup` unverifiable because *"no Juju client exists on this host"* -- +while juju 3.6.27 was installed on voffice1 and **lens 5 ran `juju help` against it +successfully in the same session**. One lens called impossible what another was +measuring. That is not a permissions limit; that is a lens not looking. + +--- + +## 2. CLOSED by the follow-up sweep (2026-07-27, after the operator's challenge) + +| # | Item | Lens that deferred it | Measured result | +|---|---|---|---| +| U1 | **NetBox apex, LIVE** | L2 (#5) | `netbox/office1-record-dump.py` run against the live apex: **139 prefixes, 103 IPv6, zero added / zero removed** vs the 07-25 dump. The dump was accurate -- but that was luck of timing, not method. dc1's key prefixes all PRESENT: `fd50:840e:74e2:320::/64`, `:330::/64`, `2602:f3e2:f03:10::/64`, `f03:11::/64` | +| U2 | **`juju restore-backup` exists?** | L6 (#1) | **IT DOES NOT.** `juju help restore-backup` -> `ERROR juju: "restore-backup" is not a juju command. See "juju --help". Did you mean: create-backup`. **This promotes L6-14 from a flagged RISK to a CONFIRMED DEFECT** -- Stage 6 Step 9's D-104 restore drill and its DoD bullet are built on a command that does not exist. DOCFIX-204 class: unsatisfiable as written | +| U3 | `juju bind` subcommand + `=` syntax | L6 (#2) | EXISTS as documented: `Usage: juju bind [options] [] [= ...]`. **Lens 6 was right to EXCLUDE it from L6-7 rather than assert it wrong** -- a correctly-handled uncertainty | +| U4 | Is `curl` on the racks? | L4 (#1) | PRESENT on both: `/usr/bin/curl` on dc0 and dc1. **L4-13's "could not look" hole is therefore theoretical, not live** -- the artifact checkers' behavioural probe does run today | +| U5 | dc0 compute provider-public MACs | L3 (#4) | `vr1-dc0-compute-01` -> `52:54:00:1b:19:e6`, `vr1-dc0-compute-02` -> `52:54:00:18:ab:b4`. **Matches `scripts/lib-hosts.sh:121-122` exactly, and CONFIRMS L3-8**: `bundle.yaml:458-464` carries dc1's `52:54:01:d1:*` MACs in the file documented as dc0's source of truth | +| U6 | Does the `maas` CLI work from the jumphost? | L2 (#6) | **The `maas` binary is ABSENT on vcloud.** So preflight P4's `FAIL: MAAS unreachable: 'maas admin subnets read' failed` is a **MISDIAGNOSIS** -- it is not a network or region problem, it is a missing client, the same class as the absent `openstack` client (S-1) and the absent `juju` (L4-2). The message actively misleads | +| U7 | Do the pinned charm channels resolve on Charmhub? | L3 (#3) -- *"no upstream fetch performed"*, despite the session having Charmhub access granted | **They resolve.** Run via `juju info` on voffice1 (the same tool the P3 gate uses): `designate`/`designate-bind`/`cinder-backup` all list `2024.1`; `hacluster` lists `2.4`; `ceph-osd`/`ceph-mon`/`ceph-rbd-mirror`/`ceph-radosgw` all list **`squid`**. **This also proves L4-2's root cause and its fix: P3's 33 warns are purely the missing `juju` binary on vcloud. Run from the Plane-2 host, the lookups work** | +| U8 | dc1 edge egress OPEN right now? | L5 (#2) | MEASURED OPEN (earlier this session): `streams.canonical.com/juju/tools/` -> 200, `api.snapcraft.io` reachable, `archive.ubuntu.com` -> 200, `ping 1.1.1.1` 0% loss, default route via `10.12.64.1`. Probed with `--noproxy` so an apt-cacher hit could not fake it | +| U9 | Stage-5 execution host / juju location | L7 (#2) | CONFIRMED: `juju` at `/snap/bin/juju` on voffice1, ABSENT on vcloud; `~/.local/share/juju/` holds `clouds.yaml` + `credentials.yaml` | +| U10 | The three `tofu plan` captures | L2 (#2) | DONE: all three roots ZERO DIFF, with the inner pair's validity established first | +| U11 | `site-baseleg.sh check office1` | L2 (#4) | DONE: PASS | +| U12 | preflight live exit code + red set | L1 (#1) | DONE: exit 1, red set exactly the known one. L1's denial was a missing `Bash(timeout *)` rule -- a failed-to-MATCH, not a classifier override | +| U13 | `creds-matrix --tier2 --remote --privileged` | L4 (#4) | DONE: exit 1, same 7 findings, all three root-owned dirs read via `sudo -n` escalation, ZERO E3 | +| U14 | Canonical piped rack-checker invocations | L4 (#1), audit session | DONE by L2: `dc-mirror` dc0 PASS, `dc-cache-proxy` dc1 PASS, `dc-rack-net` PASS both | + +--- + +## 3. STILL OPEN -- and the honest reason for each + +**Genuinely unmeasurable today (a live artefact does not exist yet):** +- Juju's real overlay-merge behaviour for the `machines:` block, and any + `juju deploy --dry-run`. Requires a bootstrapped controller and a model; + `juju controllers` -> `ERROR No controllers registered.` `provider-bundle-check`'s + merge is a MIRROR of juju's documented behaviour, not juju itself. +- Whether Juju applies an application-level `tags=` constraint to units placed by + explicit `to:` machine ids (L5-14). No model to test against. +- `cloud-assert.sh` end-to-end. No OpenStack model exists. +- Anything node-side. All 18 nodes are powered off in `Ready` under the READY-handoff + ruling -- G17's territory by construction. +- Whether MAAS would in fact allocate out of the unreserved `.50-.99` band. Behavioural, + observable only during a real deploy. What IS measured is MAAS's own declaration that + the band is unreserved on 12/12 subnets. +- Whether `ceph-osd` accepts a directory/partition for `osd-devices`. Attempted this + sweep: `juju info ceph-osd` does not expose config keys, so it needs a charm-config + fetch. **Moot for the decision** -- R1 ruled option (a), a real second disk -- but + recorded as still-unmeasured rather than quietly dropped. + +**Measurable, NOT yet done -- these are real remaining gaps, not excuses:** +- **IPv6 state on the two DC edges.** The OPNsense API GET path is available and + permitted. Not attempted. Lower value than the planes (which ARE measured empty in + MAAS), but it is part of "full state of the configuration" and should be closed. +- **`repo-lint` and the gauntlet ON voffice1** (L5 P2). Deliberately deferred: that + clone is 105 commits behind, so running them there today measures a stale tree. Owed + immediately AFTER the clone is advanced, which is precondition 0.1. +- **Reboot-durability of voffice1's dc0/dc1 transit addressing** (L6 #4). A netplan read. + Not attempted. +- **Provenance of `RETROFIT_WAIT=30m`** and any other nested-virt-calibrated constant + (L7 #1). Not attempted. + +--- + +## 4. What this changes about the audit's conclusions + +Nothing in the readiness verdict is reversed. Two things are STRENGTHENED and one +question was withdrawn: + +- **L6-14 upgraded from RISK to CONFIRMED DEFECT** (U2). `juju restore-backup` does not + exist; Stage 6's restore drill and its DoD bullet are unsatisfiable as written. +- **L3-8 CONFIRMED by measurement** (U5): the base bundle really does carry dc1's + compute MACs while documenting itself as dc0's source of truth. +- **A new finding**: preflight's "MAAS unreachable" is a misdiagnosis (U6). Together + with the absent `juju` (L4-2) and absent `openstack` (S-1), the pattern is that + **three separate preflight/deploy failures on this jumphost are all "the client is not + installed", and each is reported as something else.** That is its own DOCFIX and its + own lesson: a tool-absence must not be reported as a target-unreachable. +- **R2a WITHDRAWN** as never open -- the literals are assigned under D-111 and the apex + carries them; the real work is propagation into `lib-net.sh` and MAAS. + +## 5. The standing lesson for the next committee + +Add to the subagent contract, alongside "report a denied call as UNMEASURED": +**before declaring an item unmeasurable, verify the constraint you are asserting.** +Lens 6 asserted "no Juju client exists on this host" without checking the host where +D-128 says the client lives. The UNMEASURED escape hatch is load-bearing for honesty, +and precisely for that reason it must not become a way to avoid looking.