| 2026-07-27 |
Second gap sweep: U15-U17 closed; DC data path has no IPv6 at ANY layer
...
Operator: "Close the remaining gaps". All three measurable items are now
measured; what remains open is blocked by a known missing credential or is a
deliberate deferral, and both are stated with cause rather than left implicit.
U15 -- voffice1 transit addressing is REBOOT-DURABLE. A positive result, which
is worth recording as clearly as a defect: live enp2s0 172.31.0.1/30 (dc0 leg)
and enp3s0 172.31.0.5/30 (dc1 leg), both netplan-persistent via
/etc/netplan/60-transit.yaml and 61-transit-dc1.yaml. The D-128 Plane-2 path
survives a headend reboot and no site-baseleg row is owed for it.
U16 -- RETROFIT_WAIT=30m has NO recorded provenance. git log -S traces it to a
single bulk commit ("New Phase Scripts") with no rationale, and a repo-wide grep
finds no constant anywhere in scripts/ documented as nested-virt or depth-4
calibrated. It is an inherited default that has never been validated against the
nested I/O it will actually run on at Stage 5 Step 9. Log-only, but it should
not be mistaken for a tuned value. Bonus finding from reading the file: that
script's own preconditions require BOTH the openstack and juju clients, and no
host currently has both -- vcloud has neither, voffice1 has juju only. It cannot
run anywhere today, which reinforces S-1.
U17 -- the DC data path carries NO IPv6 at any layer measured. On BOTH racks:
zero global v6 addresses on any plane bridge, no v6 default route, and
accept_ra=1 (so they would accept a router advertisement; none is arriving).
Combined with the existing measurements -- no v6 subnet on any of the 12 MAAS
plane fabrics, no v6 link on any of the 18 nodes -- this is a THREE-LAYER
confirmation of L2-3. It also WIDENS the R2 propagation task: it is not only
"carve into MAAS", the rack bridges have no v6 either.
STILL OPEN, with cause rather than silence: the two DC edges' own interface-level
v6 configuration. dc0 is blocked by a known missing credential, not by a failure
to look -- ~/vr1-dc0-creds/ contains no opnsense-api.txt while ~/vr1-dc1-creds/
does, which is SEC-021(a) directly visible on disk, the residual item the
2026-07-27 consolidation batch deliberately excluded because the re-mint is a
live edge mutation. dc1's API is credentialed but measured unreachable from
vcloud (timeout; the path runs from the rack, where the edge creds are correctly
not staged per SEC-015). U17 answers the substantive question from the rack side.
repo-lint/gauntlet ON voffice1 remain deliberately deferred until precondition
0.1 advances that clone -- running them against a 105-commit-stale tree would
produce a number that means nothing. That is a judgement, not an omission, and
it is recorded as such.
Revert: git revert this commit; register and CURRENT-STATE changes are additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
UNMEASURED-gap sweep: 14 deferred items closed; juju restore-backup CONFIRMED absent
...
Operator challenge: did the committee actually measure live state, or take
shortcuts? Answer: some of both. This commit is the accounting, plus the sweep
that closes what was closeable.
WHO DID WHAT, precisely, because the accounting matters:
- The apex was never polled by ANY lens. Lens 2 declared it out of scope and did
so HONESTLY -- its UNMEASURED note explicitly warned "D-101's literals may
exist in NetBox and simply not be carved into MAAS". No overclaim. But it is a
key system that lens had reach and time to measure. Under-scoped, not dishonest.
- The two-day-old dump was MINE, in the correction, not an agent's.
- Sharpest instance of the general problem: lens 6 declared juju restore-backup
unverifiable because "no Juju client exists on this host", while juju 3.6.27
was installed on voffice1 and lens 5 was running juju help against it in the
same session. One lens called impossible what another was measuring.
CLOSED BY THE SWEEP (14 items). The consequential ones:
- LIVE apex polled via netbox/office1-record-dump.py: 139 prefixes, 103 IPv6,
ZERO added, ZERO removed vs the dump. The conclusion was right; the method was
not, and that distinction is the whole point.
- juju restore-backup DOES NOT EXIST on 3.6.27 -- "not a juju command ... Did you
mean: create-backup". L6-14 goes from flagged RISK to CONFIRMED DEFECT: Stage 6
Step 9's D-104 restore drill and its DoD bullet are unsatisfiable as written.
- juju bind DOES exist as documented -- lens 6 was RIGHT to exclude it from L6-7
rather than assert it wrong. A correctly-handled uncertainty.
- dc0 compute provider MACs measured (52:54:00:1b:19:e6 / 52:54:00:18:ab:b4),
matching lib-hosts and CONFIRMING L3-8.
- curl present on both racks, so L4-13's hole is theoretical, not live.
- The pinned charm channels DO resolve (2024.1, 2.4, squid all present via juju
info on voffice1) -- proving L4-2's root cause and its fix: P3's 33 warns are
purely the missing juju binary on vcloud, not a charmhub problem.
NEW FINDING: preflight's "MAAS unreachable: 'maas admin subnets read' failed" is
a MISDIAGNOSIS -- the maas binary is simply ABSENT on vcloud. Together with the
absent juju (L4-2) and absent openstack (S-1), THREE separate preflight/deploy
failures on this jumphost are all "the client is not installed", and each is
reported as something else. A tool-absence must never be reported as a
target-unreachable. Its own DOCFIX.
STILL-OPEN gaps that are measurable and were NOT done are listed in the
register's section 3 explicitly, not buried: DC edge v6 state, repo-lint/gauntlet
on voffice1 (deferred on purpose -- that clone is 105 commits stale, so running
them today would measure a stale tree), voffice1 transit reboot-durability, and
the provenance of RETROFIT_WAIT=30m.
STANDING LESSON added for the next committee: before declaring an item
unmeasurable, VERIFY THE CONSTRAINT YOU ARE ASSERTING. The UNMEASURED escape
hatch is load-bearing for honesty and must not become a way to avoid looking.
Revert: git revert this commit; the register is new and the CURRENT-STATE
paragraph is additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
CORRECTION: the v6 literals are ASSIGNED (D-111), not pending -- R2a withdrawn
...
The operator asked whether the Office1 NetBox apex had actually been polled. It
had not. It has now been read, and the answer inverts a claim I made one commit
ago and a question I put in front of them.
WHAT IS TRUE: the v6 literals are assigned, ratified and recorded, and have been
since 2026-07-11 under D-111 (ADOPTED). Measured from
netbox/draft/vr1-office1-current-20260725.json -- 139 prefixes, 103 IPv6, every
relevant row tagged D-101/D-111: ULA fd50:840e:74e2::/48 with DC0 planes at
:220/:221/:230/:240/:250::/64 and DC1 at :320/:321/:330/:340/:350::/64; GUA
provider-public DC0 2602:f3e2:f02:10::/64 + VIP f02:11::/64, DC1
2602:f3e2:f03:10::/64 + VIP f03:11::/64.
WHAT I GOT WRONG: the R2 ruling note claimed D-101's "Remaining open item" (the
org ULA /48 and per-DC GUA carve) had become a Stage-5 precondition because
"dual-stack cannot deploy against literals that do not exist". They exist. R2a
("which literals") is WITHDRAWN as never having been open; no operator utterance
is owed on it.
THE REAL PRECONDITION IS NARROWER AND BETTER: propagation, not assignment, and it
needs no ruling. The ratified values are absent from the two places Stage 5
actually reads -- scripts/lib-net.sh carries no v6 arm at all, and MAAS carries
no v6 on any of the 12 DC plane fabrics. Both are mechanical copies from an
authoritative source, so both move from Part A (needs a decision) to the Phase-3
mechanical batch.
Note CURRENT-STATE's "DRIFT-FREE across netbox/lib-net/artifacts" is NOT
contradicted: drift-free means values appearing in more than one place agree, not
that coverage is complete. lib-net.sh simply never gained a v6 arm.
PROCESS FAILURE, OWNED AND RECORDED IN THE DECISION ITSELF: the audit's own lens 2
listed the apex as UNMEASURED and warned, in terms, that "D-101's literals may
exist in NetBox and simply not be carved into MAAS". That warning was correct and
available, and I put a question to the operator anyway on the strength of D-101's
stale prose. Trusting stale decision prose over an available measurement is
exactly what this audit was convened to catch.
NEW DOCFIX-CLASS FINDING, logged not fixed (hard rule 1): D-101's "Remaining open
item" paragraph still reads "pending NetBox assignment (gap #3)" for literals
D-111 adopted on 2026-07-11 -- a decision's own prose contradicted by a later
ruling, DOCFIX-200/204 class, and the direct cause of this error. Queued in the
Phase-3 batch.
The R2 ruling itself STANDS -- dual-stack as ruled. Only its stated effect changed.
Revert: git revert this commit to restore the prior (incorrect) framing; the R2
ruling in the preceding commit is independent and should not be reverted with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
R2 RULED: dual-stack as originally decided; the v6 literals move onto the critical path
...
GA-R5: question and exact utterance quoted in the Status block, dated, pushed
before dependent work.
Operator utterance: "Carve v6 and deploy dual-stack as ruled (Recommended)".
Recorded as a D-101 RULING NOTE -- a re-confirmation, amending nothing, in the
same shape as the 2026-07-25 note it follows.
WHY IT WAS A LIVE QUESTION AT ALL: the 07-25 ruling was measured against the
substrate for the first time by this audit and found unimplemented. Exactly ONE
IPv6 subnet exists cloud-wide and it is on the Office1 base fabric; none of the
12 DC plane fabrics carries one; zero v6 links across all 18 nodes; every node
reports default_gateways.ipv6 = NONE. D-101's matrix requires ULA on
data-tenant/storage/replication plus legs on metal-admin/metal-internal, with
nothing to bind against.
THE CONSEQUENTIAL EFFECT, and the reason this commit matters beyond recording a
choice: D-101's own "Remaining open item" -- the org ULA /48 and per-DC GUA carve
-- has been carried since authoring as "pending NetBox assignment ... not a
ratification question", i.e. non-blocking. It is now a STAGE-5 PRECONDITION.
Dual-stack cannot be deployed against literals that do not exist.
Downstream, all now inheriting dual-family: R9 (where dc1's literals live), R11
(vault/designate VIPs, which must be dual-family from the outset or their cert
SANs get re-issued on a live cloud), and the L3-9 overlay collision, which must
be reconciled BEFORE either authority location is populated. Worth restating: of
the two merge orders the audit measured, the DANGEROUS one is the one that
PASSES -- vips-overlay-last silently replaces every v6 leg and reports green.
R8 (octavia's family) is explicitly NOT resolved by this ruling.
NEW SUB-QUESTION OPENED, queued not ruled: R2a, which literals. The ruling
directs that they be assigned; it does not assign them. I have deliberately not
proposed prefixes -- that is the operator's address space, and hard rule 2
forbids inventing a literal. Only the SHAPE is presented.
Revert: git revert this commit; the ruling note and R2a are additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
R1 RULED: ceph-osd gets a real second disk on the storage nodes (D-121 amendment)
...
GA-R5: question as presented and the operator's exact utterance are both quoted
in the Status block, dated, and pushed BEFORE any dependent work.
Operator utterance: "Add an OSD volume to node-vm (Recommended)".
Recorded as a D-121 AMENDMENT rather than a new D-number. D-121 is the decision
that ratifies modules/node-vm sizing/count, and its own capacity re-validation
already records "Ceph disk re-run for 4 storage/DC = PASS 5.31 TiB" -- the second
disk was BUDGETED and never BUILT. GA-R3: architectural consequence, but it
amends a ruled node layout rather than establishing new ground.
TWO SCOPE CORRECTIONS landed with the ruling, both of which narrow it:
- It is EIGHT volumes, not eighteen. ceph-osd is placed on the four storage
nodes per DC (bundle.yaml:556-557, to: ["5","6","7","8"]). The other five
nodes per DC neither run ceph-osd nor need a second device. My earlier framing
said "all 18 nodes" -- true of the measured defect, wrong about the fix.
- Rack headroom measured before asserting feasibility: dc0 2.0T available, dc1
2.9T available under /var/lib/libvirt, thin-provisioned qcow2.
Why (a) over (b): MINIMIZE DELTA TO ROOSEVELT -- Roosevelt storage nodes have
real dedicated disks. Option (b) was also not safe to pick as presented: whether
ceph-osd at the pinned squid/stable supports a directory- or partition-backed
OSD was explicitly UNRESEARCHED, and the audit refused to assert it either way.
THE APPLY IS NOT AUTHORISED BY THIS RULING. Four preconditions recorded, the
sharpest being that re-commissioning is required for MAAS to see the new device,
and whether that preserves the D-134 statics and pinned MACs is UNVERIFIED --
the 2026-07-20 MAC-regeneration incident is the precedent for exactly that class.
modules/node-vm is shared by both DCs, so the change must be additive/opt-in or
it plans against all 18 domains.
Revert: git revert this commit; the amendment is additive and no artifact changed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
CURRENT-STATE: land the dc1 egress measurement (repairs the prior commit's L10)
...
The previous commit (c1f5dc8) touched docs/audit/ without the paired
CURRENT-STATE update that GA-R1 amendment C1 requires, so repo-lint L10 failed
on it. Owned rather than silently amended: the commit is pushed, so this is a
fix-forward.
The measurement genuinely belongs in CURRENT-STATE under amendment C2. The
Stage-5 bootstrap entry has carried the hard ordering caveat -- bootstrap fetches
the juju agent stream and juju/juju-db snaps pre-apt, outside the apt proxy, so
it works ONLY while the dc1 edge egress is open -- but it carried no measurement
of whether egress actually IS open. It now does, probed from the dc1 rack with
--noproxy so an apt-cacher hit could not fake it: streams.canonical.com/juju/tools/
returns 200, api.snapcraft.io answers, archive.ubuntu.com 200, ping 1.1.1.1 0%
loss, default route via 10.12.64.1.
Stated with its shelf life: re-probe immediately before bootstrap, because the
D-135/D-107 narrowing is the closing mutation and nothing prevents it landing
first.
Revert: git revert this commit; the edit is additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage-5 grounding audit: post-review corrections + dc1 egress MEASURED
...
Three fixes from a final review pass, one of which is a new measurement.
MEASUREMENT -- dc1 edge egress is OPEN, probed rather than assumed. Lens 5 had
correctly listed this as UNMEASURED (it rested on a record). Probed directly
from the dc1 rack with --noproxy so an apt-cacher hit could not fake it:
streams.canonical.com/juju/tools/ -> 200 (the agent stream bootstrap actually
needs), api.snapcraft.io reachable, archive.ubuntu.com -> 200, ping 1.1.1.1 0%
loss, default route via 10.12.64.1. So Phase 5.2 stays a standing caveat rather
than becoming a blocker -- but it is now measured, and the row says to re-probe
immediately before bootstrap because the measurement has a shelf life.
DELIVERABLE CORRECTIONS:
- The readiness doc said "Eleven decisions are yours", which reads as the total.
It is 11 blocking (R1-R11) plus 4 standing (R12-R15) = 15, now stated as such,
with the GA-R5 one-at-a-time rule restated at the point of use.
- Phase 3's intro still said the runbook fails "in at least five distinct ways"
from before lens 5 added ten more rows. Corrected to 21 listed corrections and
nine independent failure paths, with the rows named.
- R9 and R11 are NOT independent of R2 and were presented as if they were. Both
now carry an explicit "ANSWER R2 FIRST" dependency, matching R8. R2 decides
what the VIP literals ARE; R9 only asks where they live, and R11's VIPs are
single- or dual-family depending on it. Under dual-stack, the L3-9 overlay
collision must be solved BEFORE either authority is populated.
Also recorded for future sessions: there is no Bash(timeout *) allow rule, which
is why lens 1's `timeout 240 bash scripts/preflight.sh` was denied while the bare
command is allowed -- a failed-to-MATCH, same class as the 2026-07-26 quoted-sudo
finding, not a classifier override.
Revert: git revert this commit; all changes are to audit artifacts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage-5 grounding audit: session bookend (GA-R4) + ledger rotation
...
Close bookend for the autonomous Stage-5 grounding audit. Bounded ledger
summary (9 bullets, cap 15); full body archived in the same commit as
docs/archive/changelogs/changelog-20260727-stage5-grounding-audit.md, the ONE
changelog for this session per GA-R2/D1, carrying a per-item revert.
Ledger rotation: the bookend pushed the live ledger to 305 lines against the
300-line cap, so the oldest remaining live summary (2026-07-24, caveman
guardrails + DC1 proxy-build start) moved VERBATIM to
docs/archive/session-ledger-rotated-20260727.md as a clearly-labelled second
rotation pass. Live ledger now 289 lines. Sessions from 2026-07-25 remain live.
Reconciled against repo ground truth at close: ledger-scan reports 21 open SEC
rows and next-free D 138 / DOCFIX 205 / BUNDLEFIX 053 -- all UNCHANGED, which
is correct: this session assigned no number because it remediated nothing. The
machine-derived block needed no re-seed.
The changelog records, prominently, that a SESSION-SCOPED PERMISSION BLOCK is
live in the gitignored .claude/settings.local.json and must be removed at
close; the scope doc's section 6 is its only durable record.
Revert: git revert this commit; then restore the 2026-07-24 summary from the
rotation archive if the ledger rotation is also being undone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage-5 grounding audit: lens 5 + the three deliverables
...
VERDICT: Stage 5 would NOT run error-free today, and would fail early. The
substrate underneath is excellent -- all three OpenTofu roots ZERO DIFF, 18
nodes Ready exact to D-121 Option C, MACs and power addresses matching
lib-hosts, D-134 statics perfect, 17 fabrics, zero orphaned interfaces, both
artifact paths serving, gauntlet ALL GREEN (81). What is not ready is the
layer between the substrate and the deploy.
LENS 5 (relaunched after an API error) returned the ordered precondition list
and the largest runbook defect found: Step 4 says "follow phase-01 verbatim",
and phase-01 ACTIVELY REFUSES dc1 -- its VIP guard greps bundle.yaml for
eleven 10.12.4.x VIPs, dc1's live in an overlay on 10.12.64/68/72, so it
takes the "ABORT: VIP guard failed" branch. After the ruled VIP extraction it
aborts for dc0 too. It also carries hardcoded VR0 system_ids, a jumphost-local
libvirt loop over disks that do not exist, and a 4-machine plan gate against a
9-machine bundle.
Three Stage-5 gate commands cannot execute at all, measured against the juju
actually installed (3.6.27): juju run used for a shell command when it is the
action runner; download-backup given a backup-id when it takes a controller
path; and the geneve gate grepping ovn-central for a config key the same
runbook says does not exist. Four of the five VERIFY-LIVE gates the record
says Stage 5 owes have NO step in the runbook -- including the keystone
policyd-override check, which is RULED.
Two convergent confirmations raise confidence in the whole set: the Ceph OSD
blocker was found independently by two lenses using different methods, and the
stale-clone blocker independently by this session and lens 5.
DELIVERABLES:
- docs/audit/stage5-readiness-20260727.md -- ordered precondition checklist,
READ FIRST. 5 phases, each row with status/evidence/what-breaks.
- docs/audit/stage5-committee-raw-20260727.md -- all 7 lenses verbatim.
- docs/audit/queued-rulings-20260727.md -- 11 Stage-5-blocking + 4 standing
questions, GA-R5 shape, one exchange each, blank utterance lines. NONE
adopted; a batch answer rules NOTHING.
MECHANICAL FIX TAKEN (exactly one, deliberately): the G3 gate row read OPEN
with a standing FREEZE while its own cited evidence file records G3 CLOSED and
the freeze lifted. Left standing, that clause would have blocked the very
DOCFIX batch this audit queues. No ruling was required -- two surfaces already
declared it closed.
The 21-item DOCFIX remediation batch is LOGGED NOT EXECUTED. Nearly every
runbook fix interlocks with an unanswered ruling, so landing them now would
encode assumptions about questions the operator has not answered.
Next-free numbers unchanged (D 138 / DOCFIX 205 / BUNDLEFIX 053) -- no number
was assigned, correctly, since nothing was remediated.
Revert: git revert this commit; the deliverables are new files and the
CURRENT-STATE edits are additive plus the one G3 correction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage-5 grounding audit: lens 4 (gate integrity) + privileged creds re-run
...
LENS 4 broke seven gates with CONSTRUCTED, MEASURED fixtures. Independently
re-verified by this session before landing:
- repo-lint reports "PASS (0 fail, 0 warn)" over ZERO files when given a
one-character typo of either its documented flag or the repo path. This is
the gate whose 0-fail every GA-R6 stage-close commit cites; the verdict is
not evidence the repo was read. Reproduced here exactly.
- provider-bundle-check PASSES on decorative HA: `grep -rn cluster_count
scripts/ tests/` returns NOTHING, while the overlay carries 20 such lines.
A 3->1 rewrite of every cluster_count yields a byte-identical PASS.
- preflight's aggregator ignores every sub-gate exit code that is not 1 or 2,
so a gate that crashes (127) leaves "PREFLIGHT: PASS -- clear to deploy".
- P3 verified ZERO of 33 charm-channel pins TODAY because juju is not on the
host's PATH, and a missing BINARY is folded into the same WARN as an
unreachable charmhub. This CORRECTS this session's own earlier capture,
which waved the 33-warn line through as "expected" -- the count was
expected, the cause was not.
- G17's named check cannot fail: curl -sI exits 0 on 404/500 and the dc0 URL
is an autoindex root that answers 200 with nothing behind it.
Lens 4 also returned an 11-item COULD-NOT-BREAK list; notably the 2026-07-27
dc-mirror last-sync fix HOLDS under all five status arms, and creds-matrix's
tier-2 probe states could not be made to turn "could not look" into green.
CREDS: privileged sweep re-run (operator-authorized). Exit 1, still exactly
the same 7 findings -- no drift in a day. All three root-owned locations read
via sudo -n escalation, metadata only; ZERO E3 findings. Recorded as a
RE-CONFIRMATION of the 2026-07-26 privileged run, not a newly-closed item.
Six of seven lenses are now in docs/audit/stage5-committee-raw-20260727.md.
Lens 5 (ordered preconditions) is still running after an API-error relaunch.
Revert: git revert this commit; all files are audit artifacts plus additive
CURRENT-STATE paragraphs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage-5 grounding audit: live measurement sweep + first five lens returns
...
MEASUREMENTS (capture docs/audit/stage5-live-measurement-20260727.txt):
- ALL THREE OpenTofu roots plan ZERO DIFF -- outer on vcloud, and for the first
time both inner roots on voffice1 in the same session. Validity of the inner
pair was established FIRST by proving those roots are byte-identical to main
despite the voffice1 clone being 105 commits behind; a non-empty diff would
have made them UNMEASURED, not green.
- Gauntlet ALL GREEN (81). repo-lint 0 fail / 1 warn (the L1 legacy carve-out).
- preflight exits 1 for EXACTLY the known reasons -- nothing new has joined.
- dc0 full mirror and dc1 caching proxy both genuinely serving; dc0 sync
attested OK 2026-07-27T08:43:46Z.
CURRENT-STATE corrections under GA-R1 amendment C2 (measurement wins, corrected
in the capture's own commit -- this is also what repo-lint L10 requires):
- Juju on the headend measured 3.6.27, not the recorded 3.6.25. In-channel
patch refresh, which D-071 permits, so policy-compliant drift, not an
incident -- but it is the client that will bootstrap the controller.
- NEW pin row: the openstack client is ABSENT on BOTH hosts while ten
Stage-5/6/7 scripts invoke it. No prior surface carried this.
- Section 5 convergence history extended with the three-root re-confirmation.
OWNED MEASUREMENT ERROR, recorded rather than quietly corrected: the artifact
checkers were first run LOCALLY on vcloud and printed a comprehensive FAIL with
every probe MISS -- output indistinguishable from "never installed". Both
scripts are designed to be PIPED TO THE RACK. Corrected by direct functional
probe; lens 2 independently ran the canonical piped form and got PASS on all
three. The scripts are not at fault.
FIVE LENS RETURNS landed verbatim in docs/audit/stage5-committee-raw-20260727.md
(lenses 1, 2, 3, 6, 7). Findings are NOT yet verified -- verification is the next
step and a refuted finding stays in that file with its refutation, never deleted.
Lens 5 died on an API error and was relaunched; lens 4 still running.
Revert: git revert this commit. The two CURRENT-STATE edits are additive; the
audit files are new.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage-5 grounding audit: land the scope charter before spawning any agent
...
Operator directed a full reconciliation + grounding committee from a fresh
session before Stage 5 opens, running autonomously while they are away, with
rulings QUEUED not adopted (GA-R5).
This commit lands the charter FIRST so a session that dies mid-run still
leaves the plan behind. It fixes up front: the three deliverables (ordered
readiness checklist / committee record / GA-R5-shaped queued rulings), the
seven read-only lenses, the GA-F06 bound on the record-integrity lens
(record-churn is this project's measured failure mode -- a defect is only a
finding if it changes a Stage-5 decision or contradicts a RULED decision),
the mechanical-vs-ruling discriminator (does the fix require a choice?), and
the subagent contract (read-only; a denied call is UNMEASURED with its exact
command, never folded into a negative finding).
Section 6 records the session permission posture. The operator offered a
broad exception; a SCOPED READ-ONLY expansion was taken instead -- the task
is read-only by construction and the ask rules ARE the enforcement of hard
rule 3. Design: allow enumerates READ verbs, ask enumerates MUTATING verbs,
anything matching neither falls through and FAILS SAFE as UNMEASURED. The
settings file itself is gitignored, so section 6 is its only durable record;
the block is removed at session close.
Revert: git revert this commit (the file is new and nothing references it yet).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
close-out durability sweep: 4 transcript-only items landed; dc1-is-not-a-mirror warning
...
Operator-requested sweep for content that existed only in the session transcript, before
clearing. Precedent: docs/audit/queued-findings-20260726.txt. Capture:
docs/audit/queued-findings-20260727.txt.
THE CONSEQUENTIAL ONE. scripts/dc-mirror.sh's dc1 site row carried NO warning that dc1 is no
longer a mirror site, so `dc-mirror.sh install dc1` would have SILENTLY REBUILT everything the
same-day teardown removed -- units, nginx vhost, sync helper, and an ENABLED daily debmirror
timer that would start a fresh ~950G pull on a rack whose ruled artifact path is the proxy. That
is precisely the command someone would reach for after treating a failing `check dc1` as a
regression. The row now states dc1 is proxy-only per the D-135 amendment, that `check dc1` FAILS
BY DESIGN, and that `install dc1` is a deliberate strategy change and never a repair. The row is
RETAINED so its measured values survive a legitimate rebuild.
QUEUED, not built (hard rule 1): a RUNTIME guard in do_install for a site whose ruled path is
not the mirror. A comment is strictly weaker than a guard, and prose-only prevention has already
failed twice on this project (DoD item 8 missed dc1; D-137's wiring sat as prose and missed BOTH
DC standups). Recording that the interim is an interim.
A WRONG CLAIM CORRECTED. docs/audit/stage4-mirror-gate-20260727.txt said `systemctl reset-failed`
would re-arm the paused dc1 debmirror. It cannot: reset-failed on the SERVICE does not start an
INACTIVE TIMER. The real vector was a REBOOT -- `enabled` starts the timer at boot and
`Persistent=yes` fires it IMMEDIATELY because its window had passed. Correction APPENDED rather
than edited in place: the file is dated evidence, so a wrong analytical claim is superseded
openly, not quietly rewritten.
THE GENERALISABLE LESSON, previously unrecorded, is now platform-traps section 5:
5a "stopped is not dormant across a reboot" -- enabled + inactive + EMPTY NextElapse looks
harmless and is not; plus reset-failed does not re-arm, and Result=signal/status=15 on a
oneshot means it was SIGTERM'd and never wrote a terminal state (so a stale RUNNING is a
corpse, not progress).
5b a `oneshot` with RemainAfterExit=yes does NOT undo its work on stop, so stopping it proves
NOTHING about dependents -- which is exactly why the dc1 teardown test deleted the live
address and route instead of stopping the mirror net unit.
Three rows added to the verbatim-error index. Note dc0-mirror-sync.timer is still enabled with
Persistent=yes, so 5a describes dc0 today -- benign while its syncs succeed, same shape.
HARD RULE 2. dc-cache-proxy.sh stated that the two net units coexist safely on one host as
FACT. It is REASONED (idempotent replace verbs, identical measured values), never MEASURED --
the proxy has only run on dc1, where the mirror unit was then removed, so nobody has run both.
Marked accordingly, with an instruction to verify rather than trust the sentence.
VERIFICATION: dangling-reference sweep over docs/, runbooks/, scripts/ and the skill -- every
path this session introduced or cited RESOLVES; all pre-existing dangles are legitimate
(deleted-as-history, not-yet-built, or the deliberately-absent octavia PKI overlay preflight P4
fails on). ledger-scan reconciled against the machine-derived block. Gauntlet ALL GREEN (81),
repo-lint 0-fail, dc-mirror 24/24, dc-cache-proxy 20/20.
Revert: text + comments only; the capture correction is an append.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
session-ledger: rotate to restore the GA-R4 cap (317 -> 260); re-seed the scan header
...
CORRECTION to the previous commit's message, which said "ledger 317 lines, under the 300-line
cap". That is wrong and self-contradictory -- 317 is OVER the 300-line cap, so GA-R4 F1
rotation was OWED, not deferred. Recording the correction rather than quietly fixing it,
because mis-stating an owed obligation as satisfied is the failure mode the cap exists to
prevent (the 2026-07-25 close had to record exactly this breach as OWED).
ROTATED oldest-first per GA-R4 rule 3, matching the 2026-07-26 precedent: the three oldest live
summaries (all 2026-07-23 -- G12 dc1 edge->commission+merge, Stage 4 OPEN+carve+mirrors, queue
pass + D-068 rulings) moved VERBATIM to docs/archive/session-ledger-rotated-20260727.md. Each
still points at its own archived full body; only the summaries moved. Live ledger 317 -> 260,
under the cap. Sessions from 2026-07-24 onward remain live.
Re-seeded the machine-derived block header, which still credited a 2026-07-26 scan while
carrying 2026-07-27 values. Verified against `bash scripts/ledger-scan.sh` this session:
21 open SEC rows, D next-free 138, DOCFIX 205, BUNDLEFIX 053 -- all match.
repo-lint 0-fail. The session is now durably bookended: Stage 4 CLOSED and merged (6f5701d),
ledger under cap, scan values reconciled, every finding on a surface rather than in a commit
message. Safe to clear.
Revert: text-only; the rotation is a verbatim move.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
session bookend: correct the stale merge status + land a finding that lived only in a commit
...
Two defects in the durable record, both because the bookend was written BEFORE the merge.
Fixing them before the session is cleared, since the ledger is what the next session reads.
1. The bookend said "Merge to `main` is the ONLY remaining step and is operator-gated" -- stale
the moment the merge landed. Now records the actual end state: merge commit 6f5701d (2
parents, not squashed, 77 commits), branch retired local + remote, post-merge gauntlet ALL
GREEN (81) + repo-lint 0-fail ON main, close recorded by 1023596.
2. A finding was logged only in commit c29059b's MESSAGE, which is findable but is not a surface
any session reads at open: `tests/creds-matrix` T24's finding-class baseline covers TIER 1
ONLY, so tier-2/3 classes (E1/E3/E4/V1/V2) have no baselined red state and a future false
green there would not turn the gauntlet red. That is the same class as the two false greens
this session fixed, so burying it in a commit message was the wrong home. Now in the
bookend, together with the still-open `creds-mint.sh` advice -- worth ruling BEFORE Stage 5,
which is the largest credential-minting event in the deployment.
Bookend stays within the GA-R4 15-line cap (9 body lines); ledger 317 lines, under the 300-line
cap's rotation trigger for the next close to handle. repo-lint 0-fail.
Revert: text-only.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
CURRENT-STATE: Stage 4 CLOSED 2026-07-27 -- merged 6f5701d, branch retired
...
GA-R1 C1: the merge is a status change, so the status authority records it in the same change
set. Stage 4 / Phase 3 flips OPEN -> CLOSED with its evidence: operator-gated merge to `main`
as merge commit 6f5701d (2 parents, NOT squashed; 77 commits), branch
`dc-dc-stage4-phase3-maas-deploy` retired local + remote after confirming containment with
`git branch --merged main`. Post-merge verification ON main: gauntlet ALL GREEN (81),
repo-lint 0-fail.
Recorded as travelling forward BY DESIGN rather than silently dropped: gate G17 (node-side
artifact reachability, triggers at Stage 5 first boot), 21 open SEC rows, and 7 residual
credential-register findings including the dc0 edge-API re-mint that was deliberately excluded
from the consolidation batch.
The next stage branches off post-merge `main`.
Revert: text-only. The merge itself is 6f5701d.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage 4 close-out set: GA-R2 consolidation, skill sweep, snapshot, GA-R7 review
...
Every substantive Stage 4 item is closed or split to gate row G17. This is the GA-R6 close-out
set; only the operator-gated merge to `main` remains.
GA-R2 CONSOLIDATION: the 7 stage changelogs archived to docs/archive/changelogs/ (top-level
docs/ 25 -> 18, 113 archived total) and the stage record written at
docs/archive/stage-records/vr1-stage4-record.md. It records what the stage actually delivered
(READY handoff, not deployed nodes), the three things a future session should read first, and
the false-green pattern this stage kept surfacing. 12 stale `docs/changelog-*` paths in live
surfaces rewritten to the archive -- including several already dangling from the G12 close, so
every changelog reference in CURRENT-STATE / session-ledger / security-ledger now resolves.
SKILL SWEEP: three new INVARIANTS folded in --
- per-DC ARTIFACT DELIVERY is a per-DC STRATEGY (dc0 full mirror / dc1 caching proxy, D-135
amendments); checking dc1 as a mirror fails BY DESIGN. Corollary, learned the hard way: a
per-DC utility service OWNS ITS OWN network prerequisites -- never let one strategy's
teardown be able to break the other's.
- Stage 4 hands off READY nodes, NOT deployed ones (DOCFIX-200). Any surface saying "nodes
Deployed" as a Stage-4 exit is a defect; four were found and fixed. Consequence: nothing
node-side is probeable in Stage 4, which is why G17 exists.
- "A CHECKER THAT CANNOT FAIL IS NOT A GATE", with the working rules that came out of two
measured false greens: assert on CONTENT not existence; an unrecognised state REFUSES;
"could not look" is never "nothing there"; ENUMERATE what exists rather than only auditing
what is declared, because a register cannot report the absence of a row it does not have;
and when a test asserts a literal finding string, REPLACE the assertion, never delete it.
Plus two routing rows (per-DC artifact checks; D-137 as the credential-lifecycle authority).
SNAPSHOT REGENERATED: .claude/skills/openstack-cloud-ops-consolidated-20260727.md (1589 lines,
ASCII + LF byte-verified, all 7 sections). The superseded 20260725 snapshot is REMOVED rather
than kept alongside -- a stale snapshot being uploaded is the exact failure
docs/audit/skill-divergence-20260725.md records (a failed clone + a mis-shaped handoff), and it
is a derived artifact regenerable from any commit.
GA-R7 MEMORY REVIEW: NO new memory written -- everything durable from this stage graduated to
the skill and the repo, which is the correct GA-R7 outcome, not an omission. Both existing
entries verified against the repo (the finding-class baseline they cite does exist) and extended
with the two reasoning traps this stage produced: the register cannot see a credential it has no
ROW for, so enumerate what exists; and diagnose a denied call by checking whether an existing
rule failed to MATCH before blaming the classifier (the measured 2026-07-26 root cause). Also
marked the ask-vs-allow precedence claim UNVERIFIED so no future session relies on it.
Noted, not actioned (hard rule 1): T24's finding-class baseline covers TIER 1 only, so tier-2/3
classes have no baselined red state and a future false green there would not be caught.
Gauntlet ALL GREEN (81), repo-lint 0-fail.
Revert: git revert this commit; the changelog moves are renames and the snapshot is derived.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
set-interface-v4: reload the pf filter on --commit (RULED); appendix-A rewritten
...
RULING (GA-R5) 2026-07-27. Question as presented: (a) add `configctl filter reload` to the
script's --commit path; (b) leave the script and rely on DC standup DoD item 8; (c) put it behind
an opt-in flag. Operator answer, exact utterance: "a". OPS under GA-R3 -- a script fix completing
an already-documented operation, governed by D-113; no new D-number.
DEFECT (appendix-A, measured on the fresh dc1 edge 2026-07-23): --commit ran `configctl interface
reconfigure`, which applies the address to the kernel but does NOT regenerate the pf ruleset, so
OPNsense never creates its automatic outbound-NAT rules for the (re-)addressed LAN. Forwarded LAN
packets left the WAN un-NAT'd and replies never returned -- the edge passed NO LAN traffic while
looking perfectly configured. Self-masking: any later plugin install or config apply reloads the
filter as a side effect, so dc0's 07-20 qga work hid it there until dc1 exposed it.
Option (b) rejected on evidence, not taste: DoD item 8 already existed and did NOT fire at dc1,
and prose-only controls have a measured failure record here (D-137's own enforcement sat as prose
at phase-3:498 and missed BOTH DC standups).
PLACEMENT IS THE SUBSTANCE OF THE FIX, and my first attempt got it wrong -- caught before
shipping. Putting the reload on the next line of the `interface reconfigure` heredoc would have
been a fix that fixes nothing exactly when it matters: re-addressing the interface you arrived on
drops the session DURING the reconfigure, so a following command in that stream never executes --
and that drop case IS this incident's own scenario (dc1's LAN moved 192.168.1.1 -> 10.12.64.1
over the LAN itself). As shipped the reload runs over a FRESH connection to PROBE_HOST.
Also added a `pfctl -s nat` read-back with the `proto carp` boilerplate filtered out, since
boilerplate-only is precisely appendix-A's broken-state signature. REPORTED, not gated: an edge's
first addressing may legitimately have no gateway and so no NAT to generate, and hard-failing
would block a correct run. The hard gate stays address-on-the-kernel.
Harness 59/59 (was 53), cases 15-15f. Case 15f caught a real inconsistency -- appendix-A still
said "not yet ruled". Case 15e caught a bug in my own first draft, which scanned to EOF and so
failed the script for having the very hard gate the case exists to preserve.
appendix-A rewritten: the FIX stays for an edge already in this state, plus a PREVENTED-IN-THE-TOOL
block recording the placement detail and why prose prevention was rejected.
Gauntlet ALL GREEN (81), repo-lint 0-fail. No live edge touched -- both DC edges are already
addressed, so this affects only the NEXT edge addressing.
With this, every substantive Stage 4 in-stage item is closed or split to gate row G17; only the
GA-R6 close-out set remains.
Revert: git revert this commit's script + harness + appendix-A changes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
SEC-024 retention RULED: keep both state-surgery snapshots (GA-R5)
...
Question as presented 2026-07-27: delete both `pre-*` state-surgery snapshots (cleanest, closes
SEC-024 fully), keep both and let preflight P5 keep watching them, or keep pre-G6 and delete
pre-G16 -- noting both gates are CLOSED, both carry the Office1 MAAS API key in plaintext per
DOCFIX-175, and deletion is irreversible with no other copy of that pre-surgery state.
Operator answer, exact utterance: "Keep both".
opentofu/terraform.tfstate.pre-G6-20260719 and .pre-G16-20260721 are RETAINED at 0600, neither
git-tracked, as the only record of state prior to the two direct tfstate edits (G6 autostart
injection, G16 channels reconcile).
SEC-024 stays OPEN as a standing WATCH rather than an open remediation: the mode defect is
fixed, but the umask CAUSE was out of ruled scope so a future apply may rewrite 0664, and P5 is
the detection. Recorded in the SEC-024 disposition cell and in CURRENT-STATE. Open SEC count
unchanged at 21.
repo-lint 0-fail.
Revert: text-only; the ruling record reverts with git revert. No file was deleted (that is the
point of the ruling).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage 4 cleanup: carve residue 108 fabrics -> 17; dc1 leftover nginx purged
...
Both items were queued to the stage close at the 2026-07-23 carve.
CARVE RESIDUE. Deleted subnet id=8 192.168.1.0/24 (the superseded OPNsense FACTORY LAN -- both
edges were long since re-addressed to 10.12.4.1 / 10.12.64.1), then fabric-5, then the 90
auto-created empties (ids 6..95). 108 fabrics -> 17.
The audit caught an ORDERING DEPENDENCY the original flag did not state: 192.168.1.0/24 was the
ONLY occupant of fabric-5, so deleting the fabric first would have cascaded the subnet away
instead of removing it cleanly. Subnet first, then the fabric.
METHOD, chosen because of this repo's own history: a MAAS fabric delete cascades to its VLANs,
and the 2026-07-21 pod-delete incident cost 9 machine records precisely because the association
check ran AFTER the delete. So emptiness was PROVEN per fabric -- zero subnets, zero ipranges,
zero node interfaces across all its VLANs -- against a fresh occupancy snapshot (fabrics +
subnets + ipranges + every machine's interface_set) taken immediately before the batch, and the
cascade check was re-run afterwards too.
KEPT, all verified occupied: office1 base + GUA (fabric-0), office1 compose with its 2 LXD
guests (fabric-1), both transits (fabric-2 / fabric-141), both metal-admin/boot fabrics with 9
nodes each (fabric-4 / fabric-142), libvirt default (fabric-3), and the 10 named plane fabrics.
POST-STATE: 18 Ready + 2 Deployed office1 guests, all 18 still power_type=virsh, 7 interfaces
each (6 flat planes per D-133 + br-ex), ZERO orphaned interfaces, placement tags 9+9 intact,
both DCs' artifact paths re-verified PASS.
NGINX ON dc1. Purged nginx + nginx-common (dry-run first: nothing else depended on them, no
nginx-core was installed), plus autoremove of liblockfile-simple-perl and liblog-agent-perl --
debmirror's orphaned perl deps. /etc/nginx gone, nothing on :80, proxy still PASS on :3142. dc0
keeps its nginx and was re-verified unaffected (it serves the mirror).
Capture docs/audit/stage4-carve-residue-cleanup-20260727.txt. repo-lint 0-fail.
Revert: fabrics/subnets are MAAS records, not config -- no undo, and nothing needs them (90
were auto-created empties; 192.168.1.0/24 was a superseded factory address). nginx reverts with
apt-get install nginx, which dc1 has no use for while it is the proxy DC.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
dc1 full mirror REMOVED (D-135 amendment); dc-cache-proxy.sh now owns its net layer
...
Operator clarified intent, exact utterance: "DC0 is the test of a full mirror. DC1 is the test
of the mirror proxy. DC1 needed the partially created full mirror and associated
script/files/settings removed." Recorded as a D-135 AMENDMENT 2026-07-27 -- the per-DC split
is the DELIBERATE EXPERIMENT, not an interim state. That supersedes both the "PAUSED ... kept
dormant as the fallback" record and the "DC1's proxy is REPLACED by a full mirror afterward"
claim, which sat in THREE places (D-135, CURRENT-STATE:109, dc-cache-proxy.sh:6); all corrected.
dc0 FIRST (gated): sync re-run -> OK 2026-07-27T08:43:46Z ubuntu=0 uca=0 in 20s, confirming
the transient-500 diagnosis. The fixed check now reports PASS -- trustworthy precisely because
the same check FAILED on the same rack twenty minutes earlier. Stage 4 DoD bullet 5 is now MET
for both DCs (dc0 mirror PASS, dc1 proxy PASS), node-side half at G17.
WHY THE ORDER MATTERED. Asked to weigh rip-down-and-rebuild against surgical removal, the
rebuild path turned out to be BLOCKED, not merely slower: dc-cache-proxy.sh did not own its
utility net layer -- it REUSED dc-mirror.sh's <site>-mirror-net.service and refused to install
without it. So on dc1, whose ruled path is the PROXY, three files named dc1-mirror-* (net unit,
apply helper, resolved DNS drop-in) were load-bearing FOR THE PROXY. Consequences: the recorded
removal procedure would have silently killed the proxy, the bullet-5 PASS and the Stage-5 dc1
apt path; and the proxy could not be REBUILT afterward, since nothing but `dc-mirror.sh
install` created the net layer -- rebuilding a PROXY meant reinstalling the whole MIRROR
apparatus, enabled daily debmirror timer included. Operator direction: net layer first, then
rip it all down.
1. NET LAYER MOVED: <site>-cache-proxy-net.service + -net-apply + its own resolved drop-in,
reusing dc-mirror's proven mechanism (bridge from the libvirt net NAME at runtime, hard
rule 3). Proxy-named so a proxy-only DC has NO mirror-named dependency. Idempotent
replace verbs, so both net units coexist safely where a site runs both -- dc0 unaffected.
check asserts the ARTIFACTS, not just the live address/route: asserting only the live
effect would PASS while the layer was really the mirror's, which is the exact failure.
Harness 20/20 (was 15). T12 REPLACED not deleted -- it asserted the superseded design, so
it failed by design; now inverted, plus T12b-T12f. T12b is scoped to non-comment lines on
purpose: the header's account of the removed trap cites dc-mirror.sh install, and deleting
that history to satisfy a grep would lose why the trap existed.
2. INDEPENDENCE PROVEN BEFORE DELETING: a oneshot with RemainAfterExit=yes does not undo its
work on stop, so stopping the mirror unit would have proven nothing. Removed the live state
outright (addr count -> 0, default route count -> 0), applied the proxy-owned unit ALONE,
both restored; check dc1 PASS with dc1-mirror-net.service disabled.
3. WHOLESALE REMOVAL: all mirror units, timer, sync helper, nginx vhost, resolved drop-in, the
debmirror package, and 330G of partial mirror (33,037 files, rm -rf 47.6s; disk 339G ->
9.0G). Zero mirror-named residue, every proxy artifact present, proxy PASS throughout and
never down. Capture docs/audit/dc1-mirror-teardown-20260727.txt.
nginx left installed deliberately: a mirror prereq, but with the vhost gone it is a generic
server on its stock default vhost; removing the metapackage while nginx-core runs is a
half-measure and a full purge is scope creep on a rack host.
Gauntlet ALL GREEN (81), repo-lint 0-fail, dc-cache-proxy 20/20, dc-mirror 24/24.
Revert: `dc-mirror.sh install dc1` (needs debmirror reinstalled) then `sync dc1` to re-pull
~950G; git revert dc-cache-proxy.sh + harness to restore the mirror-owned layer. The 330G
partial is NOT recoverable -- deliberately, per the amendment.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
G17 opened by operator ruling; Stage 4 DoD bullets 5-6 repaired (DOCFIX-204)
...
RULING (GA-R5), 2026-07-27. Question as presented: "The node-side half of bullet 5. Nodes are
powered off by the READY-handoff ruling, so no node-side probe can run as things stand. Either a
gated rescue-boot check on one node per DC now (closes it inside Stage 4), or split it into its
own gate row targeted at Stage 5 first boot (GA-R6 E3 explicitly permits this; a conditional
close is not permitted)." Operator answer, exact utterance: "split it into its own gate row".
Recorded in the G17 row of docs/CURRENT-STATE.md (the authority); pushed before dependent work.
G17 "Per-DC artifact source reachable FROM A NODE", [V], OPEN. dc0 -> curl 200 from a booted
node against the D-135 item-1 mirror; dc1 -> apt fetches through the proxy at 10.12.68.4:3142,
since dc1's ruled path is the CACHING PROXY with no node-facing mirror (D-135 amendment) and
checking it as one fails by design. Trigger: Stage 5 first boot. The row states that G17 is NOT
a Stage-5 precondition -- Stage 5's bootstrap needs open edge egress for the juju agent stream
and snaps (D-135 items 2-3 unbuilt), a different path from the apt artifact source.
Stage 4 retains bullet 5's rack-side half: the source answers on its own address with an
attested-current sync. dc1's proxy PASSES; dc0 pends the sync re-run.
DOCFIX-204 -- bullet 6 was UNSATISFIABLE and its stale text had spread to four surfaces. "NTP
from the DC's own OPNsense edge working" is superseded by D-129(iv) (RULED 2026-07-21, "Keep
MAAS hierarchy", no NTP role on the edge): the DoD asked for confirmation the edge is the time
source, which the ruling had already refused, so a correctly-built DC could never meet it.
Repaired in the phase-3 DoD (bullet 6 struck, bullet 5 rewritten per-DC), the phase-3 Step 7
(REWRITTEN -- it branched on gap #5 that D-135 resolved, applied one mirror check to both DCs
which fails on dc1 by design, and asked for checks "from a deployed node" that the READY handoff
makes impossible), the workflow-doc Gate cell (which also still said "Nodes deployed",
contradicting DOCFIX-200), the buildout design, and the phase-4 prerequisites (same "nodes
Deployed" error -- exactly the expectation DOCFIX-200 removed to stop Stage 5 breaking).
phase-4 also gains a block making it the OWNER of G17's captures, warning that first boot is the
observation window and missing it forces a deliberate rescue-boot.
Text and gate-record only; no live state touched. repo-lint 0-fail. DOCFIX next-free -> 205.
Revert: git revert this commit; all changes are documentation and the G17 gate row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage 4 close-out: fix dc-mirror.sh check false-greening the mirror gate
...
Operator directed closing Stage 4 before resuming the DC1 Stage-5 chain. Verifying the DoD's
fifth bullet ("per-DC mirror reachable") found that the check the gate would close on COULD
NOT FAIL.
scripts/dc-mirror.sh:271-273 tested `[ -f last-sync.status ]` and printed the file's contents
behind an unconditional "OK" -- the status word was displayed but never asserted on, so the
check exited PASS regardless. MEASURED on both racks:
- dc0 read "FAIL 2026-07-27T00:54:54Z ubuntu=255 uca=0" -> printed OK, PASS
- dc1 read "RUNNING 2026-07-23T21:49:35Z", four days stale,
left by the debmirror the D-135 amendment killed by signal -> printed OK, PASS
GA-R6 requires a stage to close on a NAMED executable check whose captured output the closing
commit cites. A check that cannot distinguish "synced" from "failed" or "died mid-run" is not
that check.
FIXED: the status word is case-analysed. OK* passes. FAIL* misses. An ABSENT file misses (it
was a passing `note`, but nothing has attested the content). An unrecognised word REFUSES
instead of defaulting to success. RUNNING* is deliberately neither pass nor plain fail --
in-flight and died-mid-run are indistinguishable from the file alone, so it reports an explicit
UNKNOWN and cross-checks the unit, naming a RUNNING marker with no active unit as a corpse.
Harness 19 -> 24; T20-T24 lock each status word and T24 locks out the return of the exact
defective shape.
PROOF, re-run against both live racks -- the gate is now honestly NOT MET for the first time:
dc-mirror check (dc0): FAIL / dc-mirror check (dc1): FAIL. Capture
docs/audit/stage4-mirror-gate-20260727.txt.
SUBSTANCE (measured, not inferred):
- dc0's mirror CONTENT is complete: 949G ubuntu + 342M cloud-archive, all three dists plus
pool. Only the overnight INCREMENTAL failed -- "Download of dists/jammy/Release failed: 500
read timeout" aborted the ubuntu leg with 255; the UCA leg succeeded. Transient upstream
fetch failure, not mirror damage. Timer re-arms 2026-07-28 00:14:48.
- dc1's PROXY -- its RULED artifact path per the D-135 amendment -- checks PASS genuinely
(apt-cacher-ng on .4:3142 serving archive + UCA Release 200). Its dormant fallback debmirror
is dormant only ACCIDENTALLY: the timer is `enabled` with an EMPTY next-elapse because the
unit sits in failed/Result=signal, so the ruling is enforced by nothing and a reset-failed
would re-arm a 330G -> ~949G pull.
ALSO RECORDED, not yet actioned: DoD bullet 6 ("NTP from the DC's own OPNsense edge") is STALE
and unsatisfiable -- superseded by D-129(iv) 2026-07-21 ("Keep MAAS hierarchy", no NTP role on
the edge) -- and survives in four surfaces needing a DOCFIX before it can be checked at all.
The node-side half of bullet 5 needs an operator decision (gated rescue-boot check vs its own
gate row per GA-R6 E3); nodes are powered off by the READY-handoff ruling, so no node-side
probe can run as things stand. There is no conditional close.
No live state was altered: both racks were read only. repo-lint 0-fail, dc-mirror 24/24.
Revert: git revert the scripts/dc-mirror.sh + tests/dc-mirror/run-tests.sh changes; the check
returns to passing unconditionally on last-sync.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
permission allowlist: 13 read-only entries; ssh/scp/virsh and creds-matrix DELIBERATELY excluded
...
Transcript scan of the 44 most-recent session JSONLs across both project dirs, counting Bash
command+subcommand pairs and splitting each candidate into LOCAL vs nested-inside-ssh (a
command that only ever runs inside `ssh host '...'` gains nothing from its own rule -- the
outer ssh is what the permission layer matches).
ADDED to permissions.allow only, 31 -> 44 entries. ask/deny/env/hooks untouched, nothing
removed or reordered, JSON re-validated and de-duplicated:
bash scripts/creds-audit.sh * ; bash scripts/dc-dc-ceph-disk-budget.sh * ;
bash scripts/site-baseleg.sh check* ; tofu validate* (+ -chdir variant) ;
tofu state list* (+ -chdir variant) ; tofu state show* ; tofu providers schema* ;
tofu version* ; ip route* ; systemctl is-active *
The substantive half of this change is what was NOT added:
- ssh (693 calls), scp (47), virsh (234) -- the three biggest prompt sources, left alone.
They are this cloud's MUTATION TRANSPORT; CLAUDE.md hard rule 3 makes the ask rules the
enforcement mechanism for per-mutation approval and says not to work around them, and the
`Bash(ssh *)` disposition is an OPEN operator item queued at the 2026-07-23 close.
Allowlisting any ssh shape would bypass the gate and pre-empt an unresolved decision.
THIS COMMIT DOES NOT DISCHARGE THAT QUEUED PRUNE -- it is still open.
- python3 scripts/creds-matrix.py (76 local runs, the largest genuine miss) -- skipped
because `Bash(*creds-matrix.py *--privileged*)` is an ask rule added 2026-07-26 right after
the auto-mode classifier incident, and every prefix pattern wide enough to cover the
observed `... 2>&1 | tail` forms would ALSO match --privileged. Allow-vs-ask precedence was
not verified, and a rule that might silently downgrade an operator-installed gate is the
class of change hard rule 3 forbids. Needs the precedence answer, then exact forms.
- tofu plan (54) -- read-only in effect, but CURRENT-STATE section 10 says in terms "Do NOT
re-run `tofu plan` casually against live state". The prompt is deliberate friction.
- python3 -c/heredocs (324) and shells: arbitrary code execution. curl (114): mutation vector
via -X/-d. tofu fmt (26): rewrites files. tofu init (10): mutates .terraform/. nft list
(10): normally needs sudo, which correctly stays ask.
Flagged not fixed: six existing allow entries are redundant with Claude Code's built-in
auto-allow (git status/log, grep, ls, jq, cat scripts/*), and `Bash(git pull*)` sits in allow
though it is a fetch-and-merge. Removing entries was out of scope.
Folded into today's session changelog as item 6 (GA-R2/D1, ONE changelog per session) with a
per-item revert. repo-lint 0-fail; no executable surface changed, so the gauntlet result from
this session's earlier commit (ALL GREEN, 81) still stands.
Revert: drop the 13 appended lines from permissions.allow -- a contiguous deletion.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
creds consolidation batch: SEC-021(b) dc0 power key, SEC-025 NetBox GUI, V2 ruled deferral
...
Operator asked whether vcloud holds a consolidated set of login creds for every account that
exists, and what is missing / needs creating / was never moved in. Audited, then cleared the
consolidation batch on operator direction ("clear the whole consolidation batch first").
The audit's own headline: creds-audit read CLEAN on all three sites while the matrix returned
13 findings -- and the two most consequential gaps were in NEITHER surface. They were found by
enumerating the accounts that ACTUALLY EXIST. Absence of a ROW is invisible to the register,
which generalises D-137's founding argument one level up.
VERIFIED POSITIVE (both previously only asserted):
- MAAS account set COMPLETE: all 6 live accounts accounted for; admin + operator passwords on
vcloud, juju-vr1-dc0/dc1 random+unstored BY RULING with API keys present, MAAS +
maas-init-node MAAS-internal.
- tier-3 V1 now MEASURES maas-admin-password byte-identical to the headend source-of-record,
so SEC-020's stale-trap risk is clear as of this date.
DONE:
- dc0 SEC-012 power key consolidated to vcloud + .pub DERIVED (SEC-021(b) as written).
MEASURED FIRST, because the naming looked like a defect and was not: the headend
maas-virsh_ed25519 and the snap's id_ed25519 are the SAME key; it IS dedicated (distinct
from the dc0 service key, which is what SEC-012 requires); and dc0 using the snap's default
identity is SEC-016's RULED design. So no re-mint, no live power path touched.
- dc1 svc .pub backfilled to the headend store (public-key material, sha256-verified).
- NetBox web-GUI admin password consolidated -- a HUMAN login that had never left the VM it
was minted on, the third instance of the SEC-009/SEC-020 miss class. SEC-025 OPENED for the
at-rest exposure the copy CREATES, not the gap it closed. Open rows 20 -> 21.
- V2 taught the ruled-deferral state: SEC-006's standing "revoke at completion of this
deployment" ruling is now ACKNOWLEDGED (still naming the credential live and exposed)
instead of failing every run. Reissuing the token would have CONTRAVENED that ruling, so the
register was what needed to change. Match demands the cased state word AND explicit operator
ruling attribution, and cannot cross a ledger table cell -- T58/T59 lock the evasion out.
- Added --ledger (default unchanged): V2 read the register via --repo, which the harness must
point at the real repo for S4, so V2 had shipped with ZERO harness cases. T56-T59 added.
Findings 13 -> 7. Matrix 82 rows, harness 60/60 (was 56), creds-audit CLEAN x3, gauntlet ALL
GREEN (81), repo-lint 0-fail. The register is NOT green and is not expected to be.
RESIDUAL, all recorded: dc0-edge-api x2 (the opnsense-api.txt re-mint is a live edge mutation,
deliberately EXCLUDED from this batch), S5 x3 (RULED by SEC-016 -- needs a ruled-exception
mechanism, operator decision), S6 conflation x1 (the SEC-020 defect), E4 x2 (Stage-5/6 rows).
LOGGED NOT ACTIONED (hard rule 1): no registered root/console credential at EITHER DC edge
(measured absence of row/manifest/SEC row; what those passwords ARE is UNKNOWN and
deliberately unprobed per hard rule 2 -- vector is the LAN-reachable GUI and serial console,
not SSH, which is key-only and proven); plus the two structural blind spots that let it hide
-- S5 compares only cardinality=per-DC while all six per-site rows are office1-only, and
vm-secret-locations declares no rack/edge/cloud/unit/client location though the checker
accepts them.
Revert: per-item reverts in docs/changelog-20260727-creds-consolidation.md. The headend, snap
and NetBox VM source copies were never modified, so every file move is reversible by deletion.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
close-out sweep: capture what existed only in the session transcript
...
Operator-requested final sweep before clearing. Two classes of content lived
only in the transcript and are now durable in
docs/audit/queued-findings-20260726.txt.
PART A -- the secrets-storage advice. Checked FIRST what the repo already
carries, and most of it was already there: OpenBao/BUSL/the fork question and
auto-unseal are in docs/D-068-vault-1.8-vs-1.16-analysis.md, and creds-mint.sh
is specified in detail in D-137 item 2(a). Only three items were genuinely
unrecorded: Tang/Clevis as the no-HSM unseal mechanism; MAAS 3.7's Vault
integration measured `status: disabled` together with the MAAS/Vault circular
dependency that must be designed around on bare metal; and a Vault SSH CA to
retire the static keypairs. The capture is framed so a future session does not
re-propose what is already ruled -- re-proposing settled decisions is this
project's measured failure mode.
PART B -- ten committee findings acknowledged but deliberately not acted on.
Most consequential: mint-ref line pins rot SILENTLY (S4 checks existence and
EOF, never content, so every pin becomes wrong-but-passing when the runbooks
are rewritten for Roosevelt), and ruling-5 remediation is indistinguishable
from ruling-5 evasion to S6. cardinality (B6) needs an operator ruling.
PART C -- items deferred by ruling, recorded so a later reader does not mistake
them for oversights.
Also repaired a dangling sentence fragment in CURRENT-STATE left by an earlier
edit this session, and noted the repair rather than making it silently.
Gauntlet ALL GREEN (81), repo-lint 0-fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
SESSION CLOSE 2026-07-26 -- D-137 build + committee audit + remediation (GA-R4)
...
Bounded 15-line summary in docs/session-ledger.md; full body archived to
docs/archive/session-20260726-d137-build.md in this same commit (GA-R4
rule 2). Status remains CURRENT-STATE.md's alone (GA-R1).
GA-R4 F1 cap handled as designed: writing the close summary took the ledger
to 300 against the 300-line cap, so the next-oldest summary rotated out --
a long session forces older summaries out rather than being truncated
itself. Ledger 280/300, close summary 15/15.
GA-R7 memory review: the creds-folder-convention entry asserted the D-137
matrix was NOT BUILT, which this session falsified. Corrected, and the
reasoning-trap half kept (creds-audit CLEAN is still not evidence of
completeness, because absence is invisible to a declaration-based sweep).
Memory index line updated to match.
Stage 4 stays OPEN; no stage closed by this session. The live mission did
not move -- the DC1 Stage-5 chain is where it was at session open, and the
next session resumes it per CURRENT-STATE.
Gauntlet ALL GREEN (81), repo-lint 0-fail, ledger-scan exit 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
| 2026-07-26 |
D-137 tier 3 behavioural half: V2 declared-state + creds-probe.sh
...
Boundary and cadence operator-ruled: "Full -- including OpenStack service
credentials", "Stage close + DC standup close". I had recommended
passive-only and flagged the costs; the operator ruled otherwise.
V2 (passive, zero authentication) fires TODAY on a real defect: SEC-006 has
recorded the NetBox upstream token as BURNED since 2026-07-12 while the
matrix carried it as a healthy consolidated row. Nothing reconciled the two
surfaces. Keyword matching on ledger prose is fragile, so it is conservative
and reports the matched text as its basis.
creds-probe.sh is SEPARATE by design: a probe must USE a credential, and
creds-matrix.py is metadata-only by an enforced invariant (T22/T51). Folding
probes in would have relaxed that guard, and a guard relaxed once stays
relaxed.
Safety: never retry a failed auth (retry loops lock accounts; the MAAS admin
identity backs 19 call sites), never argv, never echo, stage-aware skips, and
per-tenant credentials REFUSED BY CONSTRUCTION -- a declaration cannot opt
them back in (D-069, ruling 3). Not a deploy gate.
FALSE GREEN in my own first run: ssh inside a while-read loop ate the loop's
stdin, so 1 of 13 probes ran and the run exited 0. A partial run reporting
clean is the exact failure class this tool exists to catch. Fixed with ssh -n
and locked by a total-accounting check that makes a partial run VOID.
13 probes declared: 3 active, 4 stage-pending, 6 declared-only with reasons
recorded. OpenStack credentials declared now, skipping until Stage 5.
tests/creds-probe 9/9; gauntlet 80 -> 81; phase-3 DoD wired.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
D-137 tier 2: local half blocking in P5; --pending-stage -> coupled projection
...
Two operator decisions, both taking the recommendation.
GATE SHAPE: local-only blocking, remote operator-run. preflight P5 now runs
creds-matrix.py --tier2 -- tier 1 STATIC plus tier 2 LOCAL, hermetic (no ssh,
no sudo). It already catches SEC-021's three dc0 on-disk absences without
leaving the jumphost. The remote half stays an operator-run check at
DC-standup close and stage close.
Wiring --remote --privileged into the deploy gate would let a host outage, a
NOPASSWD change, or a MAAS snap refresh turn the gate red for reasons that
are not about credentials, and a gate that fails environmentally is one
people learn to bypass. The repo already made this call once
(run-tests-all.sh:7-8, "different cadence"), and preflight already shows the
failure mode: P4 reads "MAAS unreachable" because it runs on vcloud while
MAAS lives on the headend.
STAGE INPUT: --pending-stage replaced by creds-manifests/stages-reached, a
machine-readable projection of CURRENT-STATE, coupled by new repo-lint L12
(the L10 pattern). A hand-typed gate parameter can be widened silently, and
GA-R1 puts status in CURRENT-STATE alone. A drifted projection would make the
gate defer REAL absences -- a false green.
Exhaustive both ways: an unclassified mint-stage is a hard FAIL, a
classified-but-unused one is a stale-declaration FAIL. That caught `stage4`
immediately -- Stage 4 is open and executing but mints no credential.
Self-inflicted break caught by the existing harness: I first renamed L10's
message to "L10/L12", breaking five cases that grep the established L10 id.
Reverted -- new rules get their own message id, they do not rebrand old ones.
Harnesses: creds-matrix 52 -> 56, repo-lint 45 -> 47, preflight 10.
Gauntlet ALL GREEN (80), repo-lint 0-fail, baseline unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
SEC-024 REMEDIATED (mode): chmod 600, operator-approved; retention deferred
...
Operator-approved mutation, executed and read-back verified: chmod 600 on
terraform.tfstate.backup and terraform.tfstate.pre-G16-20260721. All four
state files now 0600, matching the live state file, and the checker's
E2 WORLD-READABLE finding cleared.
Severity was corrected in the preceding commit BEFORE any remediation: the
row as opened said "group- and world-readable", but opentofu/ is 0700 and ~
is 0750, so no other account could reach it, and neither file is git-tracked.
Defence-in-depth defect, not live exposure. A ledger row that overstates an
exposure is its own defect.
Retention of the two pre-* state-surgery snapshots is operator-DEFERRED to
the stage close-out: both gates (G6, G16) are CLOSED, but deletion is
irreversible with no other copy of that state.
Deliberately NOT done: the umask cause (out of ruled scope -- a future apply
may write 0664 again and the P5 gate will catch it), and the underlying
DOCFIX-175 issue that the key reaches state files in plaintext at all, whose
rotation is owed under SEC-018/-019.
Caveat recorded in creds-matrix-notes.md: E2 reports file MODE, not
reachability. That is the right default for a gate, but whoever writes up an
E2 finding owes a reachability check before using the word "exposed" -- this
defect is exactly why.
Gauntlet ALL GREEN (80), repo-lint 0-fail, finding-class baseline unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
