| 2026-07-27 |
R2 RULED: dual-stack as originally decided; the v6 literals move onto the critical path
...
GA-R5: question and exact utterance quoted in the Status block, dated, pushed
before dependent work.
Operator utterance: "Carve v6 and deploy dual-stack as ruled (Recommended)".
Recorded as a D-101 RULING NOTE -- a re-confirmation, amending nothing, in the
same shape as the 2026-07-25 note it follows.
WHY IT WAS A LIVE QUESTION AT ALL: the 07-25 ruling was measured against the
substrate for the first time by this audit and found unimplemented. Exactly ONE
IPv6 subnet exists cloud-wide and it is on the Office1 base fabric; none of the
12 DC plane fabrics carries one; zero v6 links across all 18 nodes; every node
reports default_gateways.ipv6 = NONE. D-101's matrix requires ULA on
data-tenant/storage/replication plus legs on metal-admin/metal-internal, with
nothing to bind against.
THE CONSEQUENTIAL EFFECT, and the reason this commit matters beyond recording a
choice: D-101's own "Remaining open item" -- the org ULA /48 and per-DC GUA carve
-- has been carried since authoring as "pending NetBox assignment ... not a
ratification question", i.e. non-blocking. It is now a STAGE-5 PRECONDITION.
Dual-stack cannot be deployed against literals that do not exist.
Downstream, all now inheriting dual-family: R9 (where dc1's literals live), R11
(vault/designate VIPs, which must be dual-family from the outset or their cert
SANs get re-issued on a live cloud), and the L3-9 overlay collision, which must
be reconciled BEFORE either authority location is populated. Worth restating: of
the two merge orders the audit measured, the DANGEROUS one is the one that
PASSES -- vips-overlay-last silently replaces every v6 leg and reports green.
R8 (octavia's family) is explicitly NOT resolved by this ruling.
NEW SUB-QUESTION OPENED, queued not ruled: R2a, which literals. The ruling
directs that they be assigned; it does not assign them. I have deliberately not
proposed prefixes -- that is the operator's address space, and hard rule 2
forbids inventing a literal. Only the SHAPE is presented.
Revert: git revert this commit; the ruling note and R2a are additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
R1 RULED: ceph-osd gets a real second disk on the storage nodes (D-121 amendment)
...
GA-R5: question as presented and the operator's exact utterance are both quoted
in the Status block, dated, and pushed BEFORE any dependent work.
Operator utterance: "Add an OSD volume to node-vm (Recommended)".
Recorded as a D-121 AMENDMENT rather than a new D-number. D-121 is the decision
that ratifies modules/node-vm sizing/count, and its own capacity re-validation
already records "Ceph disk re-run for 4 storage/DC = PASS 5.31 TiB" -- the second
disk was BUDGETED and never BUILT. GA-R3: architectural consequence, but it
amends a ruled node layout rather than establishing new ground.
TWO SCOPE CORRECTIONS landed with the ruling, both of which narrow it:
- It is EIGHT volumes, not eighteen. ceph-osd is placed on the four storage
nodes per DC (bundle.yaml:556-557, to: ["5","6","7","8"]). The other five
nodes per DC neither run ceph-osd nor need a second device. My earlier framing
said "all 18 nodes" -- true of the measured defect, wrong about the fix.
- Rack headroom measured before asserting feasibility: dc0 2.0T available, dc1
2.9T available under /var/lib/libvirt, thin-provisioned qcow2.
Why (a) over (b): MINIMIZE DELTA TO ROOSEVELT -- Roosevelt storage nodes have
real dedicated disks. Option (b) was also not safe to pick as presented: whether
ceph-osd at the pinned squid/stable supports a directory- or partition-backed
OSD was explicitly UNRESEARCHED, and the audit refused to assert it either way.
THE APPLY IS NOT AUTHORISED BY THIS RULING. Four preconditions recorded, the
sharpest being that re-commissioning is required for MAAS to see the new device,
and whether that preserves the D-134 statics and pinned MACs is UNVERIFIED --
the 2026-07-20 MAC-regeneration incident is the precedent for exactly that class.
modules/node-vm is shared by both DCs, so the change must be additive/opt-in or
it plans against all 18 domains.
Revert: git revert this commit; the amendment is additive and no artifact changed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage-5 grounding audit: post-review corrections + dc1 egress MEASURED
...
Three fixes from a final review pass, one of which is a new measurement.
MEASUREMENT -- dc1 edge egress is OPEN, probed rather than assumed. Lens 5 had
correctly listed this as UNMEASURED (it rested on a record). Probed directly
from the dc1 rack with --noproxy so an apt-cacher hit could not fake it:
streams.canonical.com/juju/tools/ -> 200 (the agent stream bootstrap actually
needs), api.snapcraft.io reachable, archive.ubuntu.com -> 200, ping 1.1.1.1 0%
loss, default route via 10.12.64.1. So Phase 5.2 stays a standing caveat rather
than becoming a blocker -- but it is now measured, and the row says to re-probe
immediately before bootstrap because the measurement has a shelf life.
DELIVERABLE CORRECTIONS:
- The readiness doc said "Eleven decisions are yours", which reads as the total.
It is 11 blocking (R1-R11) plus 4 standing (R12-R15) = 15, now stated as such,
with the GA-R5 one-at-a-time rule restated at the point of use.
- Phase 3's intro still said the runbook fails "in at least five distinct ways"
from before lens 5 added ten more rows. Corrected to 21 listed corrections and
nine independent failure paths, with the rows named.
- R9 and R11 are NOT independent of R2 and were presented as if they were. Both
now carry an explicit "ANSWER R2 FIRST" dependency, matching R8. R2 decides
what the VIP literals ARE; R9 only asks where they live, and R11's VIPs are
single- or dual-family depending on it. Under dual-stack, the L3-9 overlay
collision must be solved BEFORE either authority is populated.
Also recorded for future sessions: there is no Bash(timeout *) allow rule, which
is why lens 1's `timeout 240 bash scripts/preflight.sh` was denied while the bare
command is allowed -- a failed-to-MATCH, same class as the 2026-07-26 quoted-sudo
finding, not a classifier override.
Revert: git revert this commit; all changes are to audit artifacts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage-5 grounding audit: lens 5 + the three deliverables
...
VERDICT: Stage 5 would NOT run error-free today, and would fail early. The
substrate underneath is excellent -- all three OpenTofu roots ZERO DIFF, 18
nodes Ready exact to D-121 Option C, MACs and power addresses matching
lib-hosts, D-134 statics perfect, 17 fabrics, zero orphaned interfaces, both
artifact paths serving, gauntlet ALL GREEN (81). What is not ready is the
layer between the substrate and the deploy.
LENS 5 (relaunched after an API error) returned the ordered precondition list
and the largest runbook defect found: Step 4 says "follow phase-01 verbatim",
and phase-01 ACTIVELY REFUSES dc1 -- its VIP guard greps bundle.yaml for
eleven 10.12.4.x VIPs, dc1's live in an overlay on 10.12.64/68/72, so it
takes the "ABORT: VIP guard failed" branch. After the ruled VIP extraction it
aborts for dc0 too. It also carries hardcoded VR0 system_ids, a jumphost-local
libvirt loop over disks that do not exist, and a 4-machine plan gate against a
9-machine bundle.
Three Stage-5 gate commands cannot execute at all, measured against the juju
actually installed (3.6.27): juju run used for a shell command when it is the
action runner; download-backup given a backup-id when it takes a controller
path; and the geneve gate grepping ovn-central for a config key the same
runbook says does not exist. Four of the five VERIFY-LIVE gates the record
says Stage 5 owes have NO step in the runbook -- including the keystone
policyd-override check, which is RULED.
Two convergent confirmations raise confidence in the whole set: the Ceph OSD
blocker was found independently by two lenses using different methods, and the
stale-clone blocker independently by this session and lens 5.
DELIVERABLES:
- docs/audit/stage5-readiness-20260727.md -- ordered precondition checklist,
READ FIRST. 5 phases, each row with status/evidence/what-breaks.
- docs/audit/stage5-committee-raw-20260727.md -- all 7 lenses verbatim.
- docs/audit/queued-rulings-20260727.md -- 11 Stage-5-blocking + 4 standing
questions, GA-R5 shape, one exchange each, blank utterance lines. NONE
adopted; a batch answer rules NOTHING.
MECHANICAL FIX TAKEN (exactly one, deliberately): the G3 gate row read OPEN
with a standing FREEZE while its own cited evidence file records G3 CLOSED and
the freeze lifted. Left standing, that clause would have blocked the very
DOCFIX batch this audit queues. No ruling was required -- two surfaces already
declared it closed.
The 21-item DOCFIX remediation batch is LOGGED NOT EXECUTED. Nearly every
runbook fix interlocks with an unanswered ruling, so landing them now would
encode assumptions about questions the operator has not answered.
Next-free numbers unchanged (D 138 / DOCFIX 205 / BUNDLEFIX 053) -- no number
was assigned, correctly, since nothing was remediated.
Revert: git revert this commit; the deliverables are new files and the
CURRENT-STATE edits are additive plus the one G3 correction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage-5 grounding audit: lens 4 (gate integrity) + privileged creds re-run
...
LENS 4 broke seven gates with CONSTRUCTED, MEASURED fixtures. Independently
re-verified by this session before landing:
- repo-lint reports "PASS (0 fail, 0 warn)" over ZERO files when given a
one-character typo of either its documented flag or the repo path. This is
the gate whose 0-fail every GA-R6 stage-close commit cites; the verdict is
not evidence the repo was read. Reproduced here exactly.
- provider-bundle-check PASSES on decorative HA: `grep -rn cluster_count
scripts/ tests/` returns NOTHING, while the overlay carries 20 such lines.
A 3->1 rewrite of every cluster_count yields a byte-identical PASS.
- preflight's aggregator ignores every sub-gate exit code that is not 1 or 2,
so a gate that crashes (127) leaves "PREFLIGHT: PASS -- clear to deploy".
- P3 verified ZERO of 33 charm-channel pins TODAY because juju is not on the
host's PATH, and a missing BINARY is folded into the same WARN as an
unreachable charmhub. This CORRECTS this session's own earlier capture,
which waved the 33-warn line through as "expected" -- the count was
expected, the cause was not.
- G17's named check cannot fail: curl -sI exits 0 on 404/500 and the dc0 URL
is an autoindex root that answers 200 with nothing behind it.
Lens 4 also returned an 11-item COULD-NOT-BREAK list; notably the 2026-07-27
dc-mirror last-sync fix HOLDS under all five status arms, and creds-matrix's
tier-2 probe states could not be made to turn "could not look" into green.
CREDS: privileged sweep re-run (operator-authorized). Exit 1, still exactly
the same 7 findings -- no drift in a day. All three root-owned locations read
via sudo -n escalation, metadata only; ZERO E3 findings. Recorded as a
RE-CONFIRMATION of the 2026-07-26 privileged run, not a newly-closed item.
Six of seven lenses are now in docs/audit/stage5-committee-raw-20260727.md.
Lens 5 (ordered preconditions) is still running after an API-error relaunch.
Revert: git revert this commit; all files are audit artifacts plus additive
CURRENT-STATE paragraphs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage-5 grounding audit: live measurement sweep + first five lens returns
...
MEASUREMENTS (capture docs/audit/stage5-live-measurement-20260727.txt):
- ALL THREE OpenTofu roots plan ZERO DIFF -- outer on vcloud, and for the first
time both inner roots on voffice1 in the same session. Validity of the inner
pair was established FIRST by proving those roots are byte-identical to main
despite the voffice1 clone being 105 commits behind; a non-empty diff would
have made them UNMEASURED, not green.
- Gauntlet ALL GREEN (81). repo-lint 0 fail / 1 warn (the L1 legacy carve-out).
- preflight exits 1 for EXACTLY the known reasons -- nothing new has joined.
- dc0 full mirror and dc1 caching proxy both genuinely serving; dc0 sync
attested OK 2026-07-27T08:43:46Z.
CURRENT-STATE corrections under GA-R1 amendment C2 (measurement wins, corrected
in the capture's own commit -- this is also what repo-lint L10 requires):
- Juju on the headend measured 3.6.27, not the recorded 3.6.25. In-channel
patch refresh, which D-071 permits, so policy-compliant drift, not an
incident -- but it is the client that will bootstrap the controller.
- NEW pin row: the openstack client is ABSENT on BOTH hosts while ten
Stage-5/6/7 scripts invoke it. No prior surface carried this.
- Section 5 convergence history extended with the three-root re-confirmation.
OWNED MEASUREMENT ERROR, recorded rather than quietly corrected: the artifact
checkers were first run LOCALLY on vcloud and printed a comprehensive FAIL with
every probe MISS -- output indistinguishable from "never installed". Both
scripts are designed to be PIPED TO THE RACK. Corrected by direct functional
probe; lens 2 independently ran the canonical piped form and got PASS on all
three. The scripts are not at fault.
FIVE LENS RETURNS landed verbatim in docs/audit/stage5-committee-raw-20260727.md
(lenses 1, 2, 3, 6, 7). Findings are NOT yet verified -- verification is the next
step and a refuted finding stays in that file with its refutation, never deleted.
Lens 5 died on an API error and was relaunched; lens 4 still running.
Revert: git revert this commit. The two CURRENT-STATE edits are additive; the
audit files are new.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage-5 grounding audit: land the scope charter before spawning any agent
...
Operator directed a full reconciliation + grounding committee from a fresh
session before Stage 5 opens, running autonomously while they are away, with
rulings QUEUED not adopted (GA-R5).
This commit lands the charter FIRST so a session that dies mid-run still
leaves the plan behind. It fixes up front: the three deliverables (ordered
readiness checklist / committee record / GA-R5-shaped queued rulings), the
seven read-only lenses, the GA-F06 bound on the record-integrity lens
(record-churn is this project's measured failure mode -- a defect is only a
finding if it changes a Stage-5 decision or contradicts a RULED decision),
the mechanical-vs-ruling discriminator (does the fix require a choice?), and
the subagent contract (read-only; a denied call is UNMEASURED with its exact
command, never folded into a negative finding).
Section 6 records the session permission posture. The operator offered a
broad exception; a SCOPED READ-ONLY expansion was taken instead -- the task
is read-only by construction and the ask rules ARE the enforcement of hard
rule 3. Design: allow enumerates READ verbs, ask enumerates MUTATING verbs,
anything matching neither falls through and FAILS SAFE as UNMEASURED. The
settings file itself is gitignored, so section 6 is its only durable record;
the block is removed at session close.
Revert: git revert this commit (the file is new and nothing references it yet).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
close-out durability sweep: 4 transcript-only items landed; dc1-is-not-a-mirror warning
...
Operator-requested sweep for content that existed only in the session transcript, before
clearing. Precedent: docs/audit/queued-findings-20260726.txt. Capture:
docs/audit/queued-findings-20260727.txt.
THE CONSEQUENTIAL ONE. scripts/dc-mirror.sh's dc1 site row carried NO warning that dc1 is no
longer a mirror site, so `dc-mirror.sh install dc1` would have SILENTLY REBUILT everything the
same-day teardown removed -- units, nginx vhost, sync helper, and an ENABLED daily debmirror
timer that would start a fresh ~950G pull on a rack whose ruled artifact path is the proxy. That
is precisely the command someone would reach for after treating a failing `check dc1` as a
regression. The row now states dc1 is proxy-only per the D-135 amendment, that `check dc1` FAILS
BY DESIGN, and that `install dc1` is a deliberate strategy change and never a repair. The row is
RETAINED so its measured values survive a legitimate rebuild.
QUEUED, not built (hard rule 1): a RUNTIME guard in do_install for a site whose ruled path is
not the mirror. A comment is strictly weaker than a guard, and prose-only prevention has already
failed twice on this project (DoD item 8 missed dc1; D-137's wiring sat as prose and missed BOTH
DC standups). Recording that the interim is an interim.
A WRONG CLAIM CORRECTED. docs/audit/stage4-mirror-gate-20260727.txt said `systemctl reset-failed`
would re-arm the paused dc1 debmirror. It cannot: reset-failed on the SERVICE does not start an
INACTIVE TIMER. The real vector was a REBOOT -- `enabled` starts the timer at boot and
`Persistent=yes` fires it IMMEDIATELY because its window had passed. Correction APPENDED rather
than edited in place: the file is dated evidence, so a wrong analytical claim is superseded
openly, not quietly rewritten.
THE GENERALISABLE LESSON, previously unrecorded, is now platform-traps section 5:
5a "stopped is not dormant across a reboot" -- enabled + inactive + EMPTY NextElapse looks
harmless and is not; plus reset-failed does not re-arm, and Result=signal/status=15 on a
oneshot means it was SIGTERM'd and never wrote a terminal state (so a stale RUNNING is a
corpse, not progress).
5b a `oneshot` with RemainAfterExit=yes does NOT undo its work on stop, so stopping it proves
NOTHING about dependents -- which is exactly why the dc1 teardown test deleted the live
address and route instead of stopping the mirror net unit.
Three rows added to the verbatim-error index. Note dc0-mirror-sync.timer is still enabled with
Persistent=yes, so 5a describes dc0 today -- benign while its syncs succeed, same shape.
HARD RULE 2. dc-cache-proxy.sh stated that the two net units coexist safely on one host as
FACT. It is REASONED (idempotent replace verbs, identical measured values), never MEASURED --
the proxy has only run on dc1, where the mirror unit was then removed, so nobody has run both.
Marked accordingly, with an instruction to verify rather than trust the sentence.
VERIFICATION: dangling-reference sweep over docs/, runbooks/, scripts/ and the skill -- every
path this session introduced or cited RESOLVES; all pre-existing dangles are legitimate
(deleted-as-history, not-yet-built, or the deliberately-absent octavia PKI overlay preflight P4
fails on). ledger-scan reconciled against the machine-derived block. Gauntlet ALL GREEN (81),
repo-lint 0-fail, dc-mirror 24/24, dc-cache-proxy 20/20.
Revert: text + comments only; the capture correction is an append.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage 4 cleanup: carve residue 108 fabrics -> 17; dc1 leftover nginx purged
...
Both items were queued to the stage close at the 2026-07-23 carve.
CARVE RESIDUE. Deleted subnet id=8 192.168.1.0/24 (the superseded OPNsense FACTORY LAN -- both
edges were long since re-addressed to 10.12.4.1 / 10.12.64.1), then fabric-5, then the 90
auto-created empties (ids 6..95). 108 fabrics -> 17.
The audit caught an ORDERING DEPENDENCY the original flag did not state: 192.168.1.0/24 was the
ONLY occupant of fabric-5, so deleting the fabric first would have cascaded the subnet away
instead of removing it cleanly. Subnet first, then the fabric.
METHOD, chosen because of this repo's own history: a MAAS fabric delete cascades to its VLANs,
and the 2026-07-21 pod-delete incident cost 9 machine records precisely because the association
check ran AFTER the delete. So emptiness was PROVEN per fabric -- zero subnets, zero ipranges,
zero node interfaces across all its VLANs -- against a fresh occupancy snapshot (fabrics +
subnets + ipranges + every machine's interface_set) taken immediately before the batch, and the
cascade check was re-run afterwards too.
KEPT, all verified occupied: office1 base + GUA (fabric-0), office1 compose with its 2 LXD
guests (fabric-1), both transits (fabric-2 / fabric-141), both metal-admin/boot fabrics with 9
nodes each (fabric-4 / fabric-142), libvirt default (fabric-3), and the 10 named plane fabrics.
POST-STATE: 18 Ready + 2 Deployed office1 guests, all 18 still power_type=virsh, 7 interfaces
each (6 flat planes per D-133 + br-ex), ZERO orphaned interfaces, placement tags 9+9 intact,
both DCs' artifact paths re-verified PASS.
NGINX ON dc1. Purged nginx + nginx-common (dry-run first: nothing else depended on them, no
nginx-core was installed), plus autoremove of liblockfile-simple-perl and liblog-agent-perl --
debmirror's orphaned perl deps. /etc/nginx gone, nothing on :80, proxy still PASS on :3142. dc0
keeps its nginx and was re-verified unaffected (it serves the mirror).
Capture docs/audit/stage4-carve-residue-cleanup-20260727.txt. repo-lint 0-fail.
Revert: fabrics/subnets are MAAS records, not config -- no undo, and nothing needs them (90
were auto-created empties; 192.168.1.0/24 was a superseded factory address). nginx reverts with
apt-get install nginx, which dc1 has no use for while it is the proxy DC.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
dc1 full mirror REMOVED (D-135 amendment); dc-cache-proxy.sh now owns its net layer
...
Operator clarified intent, exact utterance: "DC0 is the test of a full mirror. DC1 is the test
of the mirror proxy. DC1 needed the partially created full mirror and associated
script/files/settings removed." Recorded as a D-135 AMENDMENT 2026-07-27 -- the per-DC split
is the DELIBERATE EXPERIMENT, not an interim state. That supersedes both the "PAUSED ... kept
dormant as the fallback" record and the "DC1's proxy is REPLACED by a full mirror afterward"
claim, which sat in THREE places (D-135, CURRENT-STATE:109, dc-cache-proxy.sh:6); all corrected.
dc0 FIRST (gated): sync re-run -> OK 2026-07-27T08:43:46Z ubuntu=0 uca=0 in 20s, confirming
the transient-500 diagnosis. The fixed check now reports PASS -- trustworthy precisely because
the same check FAILED on the same rack twenty minutes earlier. Stage 4 DoD bullet 5 is now MET
for both DCs (dc0 mirror PASS, dc1 proxy PASS), node-side half at G17.
WHY THE ORDER MATTERED. Asked to weigh rip-down-and-rebuild against surgical removal, the
rebuild path turned out to be BLOCKED, not merely slower: dc-cache-proxy.sh did not own its
utility net layer -- it REUSED dc-mirror.sh's <site>-mirror-net.service and refused to install
without it. So on dc1, whose ruled path is the PROXY, three files named dc1-mirror-* (net unit,
apply helper, resolved DNS drop-in) were load-bearing FOR THE PROXY. Consequences: the recorded
removal procedure would have silently killed the proxy, the bullet-5 PASS and the Stage-5 dc1
apt path; and the proxy could not be REBUILT afterward, since nothing but `dc-mirror.sh
install` created the net layer -- rebuilding a PROXY meant reinstalling the whole MIRROR
apparatus, enabled daily debmirror timer included. Operator direction: net layer first, then
rip it all down.
1. NET LAYER MOVED: <site>-cache-proxy-net.service + -net-apply + its own resolved drop-in,
reusing dc-mirror's proven mechanism (bridge from the libvirt net NAME at runtime, hard
rule 3). Proxy-named so a proxy-only DC has NO mirror-named dependency. Idempotent
replace verbs, so both net units coexist safely where a site runs both -- dc0 unaffected.
check asserts the ARTIFACTS, not just the live address/route: asserting only the live
effect would PASS while the layer was really the mirror's, which is the exact failure.
Harness 20/20 (was 15). T12 REPLACED not deleted -- it asserted the superseded design, so
it failed by design; now inverted, plus T12b-T12f. T12b is scoped to non-comment lines on
purpose: the header's account of the removed trap cites dc-mirror.sh install, and deleting
that history to satisfy a grep would lose why the trap existed.
2. INDEPENDENCE PROVEN BEFORE DELETING: a oneshot with RemainAfterExit=yes does not undo its
work on stop, so stopping the mirror unit would have proven nothing. Removed the live state
outright (addr count -> 0, default route count -> 0), applied the proxy-owned unit ALONE,
both restored; check dc1 PASS with dc1-mirror-net.service disabled.
3. WHOLESALE REMOVAL: all mirror units, timer, sync helper, nginx vhost, resolved drop-in, the
debmirror package, and 330G of partial mirror (33,037 files, rm -rf 47.6s; disk 339G ->
9.0G). Zero mirror-named residue, every proxy artifact present, proxy PASS throughout and
never down. Capture docs/audit/dc1-mirror-teardown-20260727.txt.
nginx left installed deliberately: a mirror prereq, but with the vhost gone it is a generic
server on its stock default vhost; removing the metapackage while nginx-core runs is a
half-measure and a full purge is scope creep on a rack host.
Gauntlet ALL GREEN (81), repo-lint 0-fail, dc-cache-proxy 20/20, dc-mirror 24/24.
Revert: `dc-mirror.sh install dc1` (needs debmirror reinstalled) then `sync dc1` to re-pull
~950G; git revert dc-cache-proxy.sh + harness to restore the mirror-owned layer. The 330G
partial is NOT recoverable -- deliberately, per the amendment.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage 4 close-out: fix dc-mirror.sh check false-greening the mirror gate
...
Operator directed closing Stage 4 before resuming the DC1 Stage-5 chain. Verifying the DoD's
fifth bullet ("per-DC mirror reachable") found that the check the gate would close on COULD
NOT FAIL.
scripts/dc-mirror.sh:271-273 tested `[ -f last-sync.status ]` and printed the file's contents
behind an unconditional "OK" -- the status word was displayed but never asserted on, so the
check exited PASS regardless. MEASURED on both racks:
- dc0 read "FAIL 2026-07-27T00:54:54Z ubuntu=255 uca=0" -> printed OK, PASS
- dc1 read "RUNNING 2026-07-23T21:49:35Z", four days stale,
left by the debmirror the D-135 amendment killed by signal -> printed OK, PASS
GA-R6 requires a stage to close on a NAMED executable check whose captured output the closing
commit cites. A check that cannot distinguish "synced" from "failed" or "died mid-run" is not
that check.
FIXED: the status word is case-analysed. OK* passes. FAIL* misses. An ABSENT file misses (it
was a passing `note`, but nothing has attested the content). An unrecognised word REFUSES
instead of defaulting to success. RUNNING* is deliberately neither pass nor plain fail --
in-flight and died-mid-run are indistinguishable from the file alone, so it reports an explicit
UNKNOWN and cross-checks the unit, naming a RUNNING marker with no active unit as a corpse.
Harness 19 -> 24; T20-T24 lock each status word and T24 locks out the return of the exact
defective shape.
PROOF, re-run against both live racks -- the gate is now honestly NOT MET for the first time:
dc-mirror check (dc0): FAIL / dc-mirror check (dc1): FAIL. Capture
docs/audit/stage4-mirror-gate-20260727.txt.
SUBSTANCE (measured, not inferred):
- dc0's mirror CONTENT is complete: 949G ubuntu + 342M cloud-archive, all three dists plus
pool. Only the overnight INCREMENTAL failed -- "Download of dists/jammy/Release failed: 500
read timeout" aborted the ubuntu leg with 255; the UCA leg succeeded. Transient upstream
fetch failure, not mirror damage. Timer re-arms 2026-07-28 00:14:48.
- dc1's PROXY -- its RULED artifact path per the D-135 amendment -- checks PASS genuinely
(apt-cacher-ng on .4:3142 serving archive + UCA Release 200). Its dormant fallback debmirror
is dormant only ACCIDENTALLY: the timer is `enabled` with an EMPTY next-elapse because the
unit sits in failed/Result=signal, so the ruling is enforced by nothing and a reset-failed
would re-arm a 330G -> ~949G pull.
ALSO RECORDED, not yet actioned: DoD bullet 6 ("NTP from the DC's own OPNsense edge") is STALE
and unsatisfiable -- superseded by D-129(iv) 2026-07-21 ("Keep MAAS hierarchy", no NTP role on
the edge) -- and survives in four surfaces needing a DOCFIX before it can be checked at all.
The node-side half of bullet 5 needs an operator decision (gated rescue-boot check vs its own
gate row per GA-R6 E3); nodes are powered off by the READY-handoff ruling, so no node-side
probe can run as things stand. There is no conditional close.
No live state was altered: both racks were read only. repo-lint 0-fail, dc-mirror 24/24.
Revert: git revert the scripts/dc-mirror.sh + tests/dc-mirror/run-tests.sh changes; the check
returns to passing unconditionally on last-sync.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
creds consolidation batch: SEC-021(b) dc0 power key, SEC-025 NetBox GUI, V2 ruled deferral
...
Operator asked whether vcloud holds a consolidated set of login creds for every account that
exists, and what is missing / needs creating / was never moved in. Audited, then cleared the
consolidation batch on operator direction ("clear the whole consolidation batch first").
The audit's own headline: creds-audit read CLEAN on all three sites while the matrix returned
13 findings -- and the two most consequential gaps were in NEITHER surface. They were found by
enumerating the accounts that ACTUALLY EXIST. Absence of a ROW is invisible to the register,
which generalises D-137's founding argument one level up.
VERIFIED POSITIVE (both previously only asserted):
- MAAS account set COMPLETE: all 6 live accounts accounted for; admin + operator passwords on
vcloud, juju-vr1-dc0/dc1 random+unstored BY RULING with API keys present, MAAS +
maas-init-node MAAS-internal.
- tier-3 V1 now MEASURES maas-admin-password byte-identical to the headend source-of-record,
so SEC-020's stale-trap risk is clear as of this date.
DONE:
- dc0 SEC-012 power key consolidated to vcloud + .pub DERIVED (SEC-021(b) as written).
MEASURED FIRST, because the naming looked like a defect and was not: the headend
maas-virsh_ed25519 and the snap's id_ed25519 are the SAME key; it IS dedicated (distinct
from the dc0 service key, which is what SEC-012 requires); and dc0 using the snap's default
identity is SEC-016's RULED design. So no re-mint, no live power path touched.
- dc1 svc .pub backfilled to the headend store (public-key material, sha256-verified).
- NetBox web-GUI admin password consolidated -- a HUMAN login that had never left the VM it
was minted on, the third instance of the SEC-009/SEC-020 miss class. SEC-025 OPENED for the
at-rest exposure the copy CREATES, not the gap it closed. Open rows 20 -> 21.
- V2 taught the ruled-deferral state: SEC-006's standing "revoke at completion of this
deployment" ruling is now ACKNOWLEDGED (still naming the credential live and exposed)
instead of failing every run. Reissuing the token would have CONTRAVENED that ruling, so the
register was what needed to change. Match demands the cased state word AND explicit operator
ruling attribution, and cannot cross a ledger table cell -- T58/T59 lock the evasion out.
- Added --ledger (default unchanged): V2 read the register via --repo, which the harness must
point at the real repo for S4, so V2 had shipped with ZERO harness cases. T56-T59 added.
Findings 13 -> 7. Matrix 82 rows, harness 60/60 (was 56), creds-audit CLEAN x3, gauntlet ALL
GREEN (81), repo-lint 0-fail. The register is NOT green and is not expected to be.
RESIDUAL, all recorded: dc0-edge-api x2 (the opnsense-api.txt re-mint is a live edge mutation,
deliberately EXCLUDED from this batch), S5 x3 (RULED by SEC-016 -- needs a ruled-exception
mechanism, operator decision), S6 conflation x1 (the SEC-020 defect), E4 x2 (Stage-5/6 rows).
LOGGED NOT ACTIONED (hard rule 1): no registered root/console credential at EITHER DC edge
(measured absence of row/manifest/SEC row; what those passwords ARE is UNKNOWN and
deliberately unprobed per hard rule 2 -- vector is the LAN-reachable GUI and serial console,
not SSH, which is key-only and proven); plus the two structural blind spots that let it hide
-- S5 compares only cardinality=per-DC while all six per-site rows are office1-only, and
vm-secret-locations declares no rack/edge/cloud/unit/client location though the checker
accepts them.
Revert: per-item reverts in docs/changelog-20260727-creds-consolidation.md. The headend, snap
and NetBox VM source copies were never modified, so every file move is reversible by deletion.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
close-out sweep: capture what existed only in the session transcript
...
Operator-requested final sweep before clearing. Two classes of content lived
only in the transcript and are now durable in
docs/audit/queued-findings-20260726.txt.
PART A -- the secrets-storage advice. Checked FIRST what the repo already
carries, and most of it was already there: OpenBao/BUSL/the fork question and
auto-unseal are in docs/D-068-vault-1.8-vs-1.16-analysis.md, and creds-mint.sh
is specified in detail in D-137 item 2(a). Only three items were genuinely
unrecorded: Tang/Clevis as the no-HSM unseal mechanism; MAAS 3.7's Vault
integration measured `status: disabled` together with the MAAS/Vault circular
dependency that must be designed around on bare metal; and a Vault SSH CA to
retire the static keypairs. The capture is framed so a future session does not
re-propose what is already ruled -- re-proposing settled decisions is this
project's measured failure mode.
PART B -- ten committee findings acknowledged but deliberately not acted on.
Most consequential: mint-ref line pins rot SILENTLY (S4 checks existence and
EOF, never content, so every pin becomes wrong-but-passing when the runbooks
are rewritten for Roosevelt), and ruling-5 remediation is indistinguishable
from ruling-5 evasion to S6. cardinality (B6) needs an operator ruling.
PART C -- items deferred by ruling, recorded so a later reader does not mistake
them for oversights.
Also repaired a dangling sentence fragment in CURRENT-STATE left by an earlier
edit this session, and noted the repair rather than making it silently.
Gauntlet ALL GREEN (81), repo-lint 0-fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
| 2026-07-26 |
D-137 tier 3 behavioural half: V2 declared-state + creds-probe.sh
...
Boundary and cadence operator-ruled: "Full -- including OpenStack service
credentials", "Stage close + DC standup close". I had recommended
passive-only and flagged the costs; the operator ruled otherwise.
V2 (passive, zero authentication) fires TODAY on a real defect: SEC-006 has
recorded the NetBox upstream token as BURNED since 2026-07-12 while the
matrix carried it as a healthy consolidated row. Nothing reconciled the two
surfaces. Keyword matching on ledger prose is fragile, so it is conservative
and reports the matched text as its basis.
creds-probe.sh is SEPARATE by design: a probe must USE a credential, and
creds-matrix.py is metadata-only by an enforced invariant (T22/T51). Folding
probes in would have relaxed that guard, and a guard relaxed once stays
relaxed.
Safety: never retry a failed auth (retry loops lock accounts; the MAAS admin
identity backs 19 call sites), never argv, never echo, stage-aware skips, and
per-tenant credentials REFUSED BY CONSTRUCTION -- a declaration cannot opt
them back in (D-069, ruling 3). Not a deploy gate.
FALSE GREEN in my own first run: ssh inside a while-read loop ate the loop's
stdin, so 1 of 13 probes ran and the run exited 0. A partial run reporting
clean is the exact failure class this tool exists to catch. Fixed with ssh -n
and locked by a total-accounting check that makes a partial run VOID.
13 probes declared: 3 active, 4 stage-pending, 6 declared-only with reasons
recorded. OpenStack credentials declared now, skipping until Stage 5.
tests/creds-probe 9/9; gauntlet 80 -> 81; phase-3 DoD wired.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
D-137 tier 3 (VALIDITY) BUILT: cross-copy sha256 provenance
...
Closes the gap the plan names: creds-audit PARSES declared provenance and
never verifies it, so SEC-020's stale-trap warning has been unchecked prose
and its sha256 equality was a ONE-TIME manual act on 2026-07-25.
sha256sum runs as an EXTERNAL command on both sides so only a 64-hex digest
enters the checker -- hashing bytes in Python would meet the letter of "never
transfer content" and not the spirit.
RESULT: 4 groups compared, all byte-identical, INCLUDING the SEC-020 pair
(maas-region-admin, region source-of-record vs jumphost copy). Now
machine-verified instead of remembered. The [ok] line names the groups.
TWO FALSE POSITIVES IN MY OWN DRAFT, caught by verifying before reporting.
Keying digests by (role, site) let an identity's DISTINCT artifacts at one
scope overwrite each other, so it compared unrelated files and reported drift
on maas-region-admin and dc1-svc-key. Manual sha256sum showed both IDENTICAL.
Reporting that would have raised a false credential-rotation alarm.
Root cause was a schema gap: nothing expressed which rows are byte-copies.
custody gains `verbatim-copy` [OPS]. V1 now compares only rows sharing
(id, filename) across scopes, or a source-of-record row against declared
verbatim-copy rows -- SEC-020's exact shape. Everything else is skipped with
a reason: a pair we cannot justify comparing is not a pair we may report
drift on. verbatim-copy is manifest-bearing; it asserts byte-identity, it
does not move the file.
Harness 44 -> 52: T47/T48 regression-lock both false positives; T49 keeps
SEC-020's case from falling into T48's skip; T51 asserts the digest path uses
sha256sum and that no content-transfer verb exists anywhere in the checker.
NOT built: tier 3's behavioural probes. Liveness, expiry, revocation state
and grant/scope drift remain unverified.
Gauntlet ALL GREEN (80), repo-lint 0-fail, finding-class baseline unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
D-137 phase 3: re-measure the register; SEC-024 opened
...
~21 inferred filenames corrected against their own mint-refs. The session had
owned six; those were only the rows the first sweep physically touched.
Octavia's 8 real basenames plus its THREE SUBDIRECTORIES (the bare
~/octavia-pki/* pattern matched ZERO of the 8 artifacts, two of them CA
private keys); vault's init.txt; the tenant rows' <client>- instance prefix,
with a placeholder matcher so they can match a real basename at all;
kube-config -> config; .maas.cli -> .maascli.db (measured: ~/.maas.cli does
not exist).
sec-ref mis-attribution corrected: both juju-maas-user rows cited SEC-020;
the on-subject rows are SEC-018/-019. This is the SEC-015 class recurring --
one instance was fixed by hand earlier without checking whether it was
systemic. It was.
mint-ref corrected: admin.pass cited a CONSUMER (:452), not the mint (:450);
the RGW key cites a line that takes it as INPUT, so it is operator-terminal.
Rows added: the libvirt SSH power password (standing rotation obligation from
a 2026-06-26 plaintext exposure, in neither the matrix nor the exclusions),
vault-ca-root, and the tfstate siblings. ~/as-executed/ deliberately NOT
declared, reason recorded: it holds session logs, so declaring it made every
log a false UNDECLARED.
Four further defects in my own phase-2 changes, found by RUNNING the sweep:
shlex.quote broke ~ expansion; a non-existent parent read as unreadable
rather than absent; site-key `-` needed shared-location semantics; the
tfstate siblings needed rows.
SEC-024 OPENED -- new exposure found by the fixed checker on its first clean
run: opentofu/terraform.tfstate.backup is mode 0664 and carries the MAAS API
key in plaintext (DOCFIX-175); the live state file is 0600. Invisible to
every prior control because the world-readable check was custody-gated and
the siblings were undeclared. LOGGED, NOT ACTIONED (hard rule 1).
Findings 35 -> 13, all true. Matrix 81 rows. Harness 44/44, creds-audit
15/15, gauntlet ALL GREEN (80), repo-lint 0-fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
D-137 committee audit + phase 1: correct the record
...
Six-lens read-only review found the delivered tier-1/tier-2 build has eight
reproduced false-greens and that the records overstated what was verified.
Four lenses converged independently on tier 2 being SITE-BLIND.
Record corrections only in this commit (no code, no data):
- NEW docs/audit/d137-location-listing-20260726.txt: the actual stat listing.
CURRENT-STATE asserted remote filenames/modes citing the tier-2 capture,
which is a VERDICT file containing none of them -- a GA-R1 rule 2 defect.
- .maas.cli was NEVER MEASURED and is wrong; the MAAS snap CLI profile is at
~/snap/maas/current/.maascli.db (measured). It sat inside the fix for the
inferred-filename miss, so the corrected count is ~21, not six.
- WITHDRAWN: "ZERO undeclared files ... fully accounted for" (rested on the
site-blind check). CORRECTED: "SEC-021 on-disk half REPRODUCED" now reads
power-key artifacts only -- opnsense-api.txt's absence was masked.
- CURRENT-STATE's "virsh gap NOT fixed" corrected; it was fixed in 6d43619,
whose commit did not touch the status authority because L10's trigger set
excludes .claude/settings.json.
- Superseding figures recorded; stale "14 NOT JUDGED" -> 2; glob is *.pass.
- 12-column amendment recorded on D-137 in design-decisions.md; it previously
lived only in a changelog (non-citable per GA-R2 rule 1) and the build spec.
- Repaired citations my own edits broke (guard moved 71-73 -> 90-92), the
plan's 12-vs-27 operator-terminal count, the "two the ruling did not name"
claim (it is one), and an unsupported ruling date in the notes.
Code and data remediation follows in later commits.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
D-137 tier 2: privileged sweep completed; permission gap fixed; 6 inferred filenames corrected
...
The block was a PATTERN GAP, not a policy conflict. Bash(ssh * sudo *) was
already in the project ask list, but it needs a literal space before `sudo`
and the command was ssh <host> 'sudo ...' -- the quote meant no rule matched,
so it fell through to the classifier. Added the quoted variants plus a
targeted rule for the privileged invocation; all `ask`, never `allow`.
FLAGGED NOT FIXED (hard rule 1): Bash(ssh * virsh *) has the identical gap.
MEASURED (docs/audit/d137-tier2-privileged-20260726.txt): the region secrets
dir holds exactly admin.apikey/admin.pass/db.pass/lxd-trust.pass at 0600 --
precisely the SEC-020(i) carve-out list; netbox holds admin.pass/api.token/
secret_key at 0600; the SEC-022 shadow stores are confirmed and now audited.
ZERO undeclared files at any declared location: SEC-020 and SEC-022 are
accounted for.
OWNED: the first matrix draft carried SIX filenames I authored plausibly
instead of measuring -- a hard-rule-2 miss. The checker's undeclared-detection
caught every one on first contact with real hosts. Corrected to measured
values, real locations declared, and five colocated non-secret artifacts
declared so the check stays meaningful. Matrix 72 -> 77 rows.
9 findings remain, all true: SEC-021 (3 declaration + 2 on-disk), the ruling-5
conflation, and 3 S5 asymmetries -- dc0's snap power key is the generic
id_ed25519 while dc1's is the dedicated id_dc1_power, the SEC-012/SEC-016
per-DC-isolation divergence made visible.
Harness 35/35, gauntlet ALL GREEN (80), repo-lint 0-fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
D-137 tier 2: live read-only sweep + two probe false-greens it exposed
...
SEC-021's ON-DISK half REPRODUCED: vr1-dc0-maas-power_ed25519{,.pub} are
genuinely absent from the dc0 jumphost creds folder, not merely undeclared.
Capture: docs/audit/d137-tier2-sweep-20260726.txt (read-only, no sudo,
stat-over-ssh metadata only; jumphost + voffice1 + office1-netbox).
Running it against real hosts exposed two false-greens in the probe:
1. "absent" vs "could not look". /root/maas-secrets and /root/netbox-secrets
reported "does not exist" -- they exist; SEC-020 is precisely about
admin.pass living in the first. An unprivileged [ -d ] on a root-owned
parent is indistinguishable from a missing directory. The probe now tests
parent traversability and reports UNREADABLE as a FAIL, not a skip.
2. Role-level aggregation manufactured false absences: skipping one location
let a role's other locations cover it, so credentials in the unprobed
location reported EXPECTED-BUT-ABSENT. Absence is now asserted only over
fully-probed roles (14 rows report NOT JUDGED). Refinement: "absent" is a
conclusive observation and does not gate a role; only unreadable and
unreachable do.
Harness 33 -> 35 (T33 unreadable location, T34 absent location).
Gauntlet ALL GREEN (80), repo-lint 0-fail.
OUTSTANDING: the two root-owned dirs need a privileged read, so SEC-022's
shadow-store contents and the SEC-020 region secrets stay UNCONFIRMED. That
is a remote-sudo shape and was not run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
audit: transcribe the full 55-row credential inventory in-repo (matrix seed)
...
The capture's body previously said the per-row tables lived in the session
transcript. The operator is clearing that session to start the build, so the
matrix SEED would have been lost and would have cost another full inventory
run to re-derive.
Appendix now carries all of it: section A (25 scripted MINT rows), section B
(30 runbook/prose MINT rows), section D (charm-minted, with the keystone admin
password called out as in-scope per the materialization ruling), section E
(MATERIALIZE/UPLOAD checked and excluded, with file:line so the exclusions are
auditable), and section G (coverage boundary, incl. what was deliberately not
opened). Corrected the body pointer that referenced the transcript.
Notable rows the build will need: A6 the MAAS rack-enrollment secret (a mint
nothing else records), B5-B12 the eight Octavia PKI artifacts, B13 the overlay
that lands a CA key + plaintext passphrase inside the repo clone, B14 the
juju-<dc> superusers stored nowhere, B22 the per-DC power key whose dc0 copy is
missing (SEC-021), and A21-A25 which DUPLICATE A11-A14 -- one credential set,
two creation locations, do not double-count.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
| 2026-07-25 |
audit: credential creation-point inventory -- 12 declared secrets have NO mint command
...
Read-only research capture seeding the D-137 credential matrix. Nothing built;
the five D-137 forks remain unruled.
FINDING 1 (qualifies the premise that creation points are already known):
`grep -rnI "ssh-keygen" .` returns ZERO hits repo-wide -- no mkpasswd, pwgen,
uuidgen, random_password or tls_private_key either. Yet the manifests declare
six SSH keypairs plus the OPNsense root password/hash, whose only provenance
record is a manifest COMMENT. Twelve standing credentials therefore have no
reproducible mint. If the jumphost is rebuilt, no repo artifact says how to
recreate them -- a Roosevelt-transfer defect, not merely hygiene. Consequence
for the design: mint-ref must admit three provenance kinds (script:line,
runbook:step, operator-terminal), and the operator-terminal rows are the debt
the matrix exists to surface, not parse errors.
FINDING 2: three credential DIRECTORIES sit outside the SEC-009 per-site creds
convention and outside creds-audit entirely -- the Vault init directory (5
unseal shares + root token), the Octavia PKI directory (8 artifacts including
two CA private keys), and the per-tenant directories. All are named in
CLAUDE.md, so they are known; they are simply uncontrolled. Materially extends
SEC-023. Also flags overlays/octavia-pki.yaml, which lands a CA key blob plus a
plaintext passphrase inside the repo clone, gitignored -- with SEC-004 the repo
still PUBLIC.
FINDING 4 corroborates SEC-020 independently: phase-4 runbook:128 mints the
juju-<dc> superusers with no --password (stored nowhere), and
site-headend-install.sh:452 uses --password in argv.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
SEC-020: MAAS admin-account recovery -- consolidate `admin`, mint `operator`, creds-audit CLEAN x3
...
Operator task: recover the MAAS admin accounts and save the creds retrievably.
The task premise was FALSIFIED by measurement before any mutation: the ledger's
2026-07-25 "MAAS web-GUI login does NOT exist / never minted" was wrong. The
`admin` password was minted 2026-07-13 by site-headend-install.sh:452 and is
MEASURED working (login 204 vs 400 wrong-password control). It was merely
root-only on voffice1 and un-consolidated -- the SEC-009 miss class, not a
missing account. Scope re-ruled by the operator after that finding:
"New account + consolidate only" -- no existing password rotated.
- NEW MAAS superuser `operator` for human GUI login; password generated on
vcloud, set via stdin (never argv -- SEC-018 discipline; changepassword has
no --password and createadmin --password would expose it).
- `admin` password CONSOLIDATED byte-identical (sha256-verified) to
~/vr1-office1-creds/. NOT rotated, so the VM copy REMAINS source-of-record;
a future rotation must update both or the VM copy becomes a stale trap.
- juju-vr1-dc0/dc1 untouched BY RULING: their SEC-018/019 API keys are
load-bearing for the blocked Stage-5 bootstrap.
- Manifest backfill -> creds-audit CLEAN on all three sites (was RED: dc0 3 +
dc1 1 undeclared), closing the dc0 edge-keypair queued finding named in
vr1-dc1.manifest:18-20.
- Root cause recorded: `admin` doubles as the automation identity (19 `maas
admin` call sites in 4 scripts), so automation never needed the password and
nothing forced it into the folder. Proposable as the next-free D-number
[ARCH]; NOT assigned (GA-R3 doubt-resolves-DOWN).
- Falsified ledger claim struck+corrected; machine-derived block re-seeded
(was stale on every line: SEC 12->16, next-free D 133->137, DOCFIX 200->204).
Evidence: docs/audit/maas-admin-recovery-20260725.txt (generated, no secret
values; scanned for password bytes = 0 hits with a planted-control check).
repo-lint 0 fail; gauntlet ALL GREEN (79).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
D-136 PROPOSED (NetBox-coupled render pipeline) + Ruling 3 + DOCFIX-201
...
Register append + CURRENT-STATE same commit (GA-R1/C1); DOCFIX-201's docs/audit
edits ride the same CURRENT-STATE touch (L10).
- D-136 PROPOSED [ARCH]: NetBox-coupled per-DC render pipeline; option (C)
recommended (values-file back half now for both DCs, NetBox front half for
Roosevelt); non-gating for dc1; prerequisite = extend sandbox-fidelity-check.py.
- Ruling 3 (GA-R5) recorded INLINE in D-136 with the verbatim question + utterance
("Your question about 1 or 2, go with 1. Base bundle and everything else is
applied via overlay") -- committee BLOCKER-1 fix (durable home, not the
delete-after-use pack file).
- CURRENT-STATE Stage-4: bullet (a) vault-VIP D-020 fix + per-DC widen, bullet (c)
artifact-shape RULED, coupling bullet -> PROPOSED as D-136, FOLLOW-UP gate
blockers (pre-flight-checks CHECK 1, provider-bundle-check vault VIP, VIP_OCTET_MAX).
- DOCFIX-201: dual-VIP decision is D-020 not D-036, SEVEN sites (7th caught by the
pack-review committee; verify -> 0 VIP-context D-036 remain).
- Chat correction (2026-07-25): D-136 open Q5 rewritten -- Jenkins IS the runner
(gap void); real requirements webhooks + status-back; placement OPEN.
repo-lint 0 fail. No cloud state mutated. D-136 PROPOSED -- operator rules the
mechanism (GA-R5).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
|
|
audit: full decision reconciliation record (IP/VLAN/node/decision-status)
...
4-lens read-only recon of every ruled decision vs the authoritative surfaces
(office1-netbox apex snapshot, lib-net/lib-hosts, bundle/overlays, MAAS carve).
Headline: IP prefix-layer COMPLETE + drift-free; VLANs correctly absent
(untagged-per-fabric, D-133 -- MAAS construct, not NetBox); the GAP is sub-prefix
-- the D-134 per-DC bands + the 33 VIP addresses live only in prose/overlays, not
the apex (blocks the NetBox->deploy coupling). Node/MAC/power config drift-free;
Fork-3 role tags + 10th controller VM pending gated (deploy-blocking).
Decision-status integrity flagged STALE surfaces (fixes QUEUED, operator-deferred):
CURRENT-STATE section 4 comprehensively stale (HIGH), G14 SEC count 12->15,
phase-4 runbook designate line. Reconciliation backlog (P0-P2) + the "what else to
queue" answer in the record; extend sandbox-fidelity-check.py first (gates the
apex recons).
CURRENT-STATE updated same change set (GA-R1/C1) with a pointer + a section-4-stale
warning. No cloud state mutated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
| 2026-07-24 |
vr1-dc1 bundle render (9-node role-separated) + expansion review record
...
Render (committee Fork-2 hand-render), validated by the extended checker:
- bundle.yaml: machines "8"-"11" (tags=openstack) -> "0"-"8" role+DC tags (block
style); every `to:` re-homed (quorum trio -> control 0/1/2; nova-compute 3->2 on
compute 3/4; ceph-osd -> storage 5-8; scaled apps + utilities -> control);
ovn-chassis -> the 2 dc1 compute provider MACs (52:54:01:d1:04:02/05:02).
- overlays/vr1-dc1-machines.yaml (NEW): per-DC retag (machines-overlay merge is
VERIFY-LIVE at `juju deploy --dry-run`).
- overlays/dc-ha-scaleup.yaml: tokens rendered to logical control ids 0/1/2
(DC-neutral); header updated; stale ceph-osd-3 comment -> 4.
- provider-bundle-check.py: ovn-chassis MAC check skips the VR0 Pattern-A set on a
role-separated bundle (the validator CAUGHT a real YAML flow-comma render bug).
Validated: base + FULL dc1 deploy input (3 overlays, --dc vr1-dc1) PASS; harness
15/15; gauntlet ALL GREEN (78); repo-lint 0-fail.
Expansion review (3 read-only reviewers) recorded in
docs/audit/stage5-expansion-review-20260724.md: relations/bindings/subordinates
HOLD under role-sep (uniform 6-NIC, SEC-011); no channel/pin change; open
resolve-before-deploy items (vault VIP, v4-vs-dual-stack phasing) + deploy-gate
script gaps + VERIFY-LIVE gates. NetBox->deploy coupling routed to Chat as a new
PROPOSED D-number (Fork-2 escalation). No cloud state mutated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
Stage-5 dc1: committee review + Fork-1 ruling (D-104 amendment) + BUNDLEFIX-052
...
Track-2 bundle-render blocker, committee-adjudicated (4 read-only lenses:
Roosevelt-delta / HA-DR / operational-risk / Juju-MAAS-mechanism). Every
load-bearing claim re-verified against the repo before adoption.
- Committee record: docs/audit/committee-20260724-track2-bundle-render.md
- Fork 1 RULED (operator, GA-R5): dedicated 10th per-DC controller VM ->
D-104 amendment. Capacity gate PASS (measured): 10-node x 2-DC =
854/1024 GiB (83%, FIT, 170 GiB headroom);
docs/audit/stage5-controller-capacity-20260724.txt
- BUNDLEFIX-052: removed an orphaned `bindings:` block absorbed into
ceph-rbd-mirror (clobbered its D-108 replication bindings + injected a
phantom `dashboard` endpoint -> juju deploy reject). provider-bundle-check PASS.
- CURRENT-STATE Stage-4 bullet updated same change set (GA-R1/C1).
No cloud state mutated. Fork 2/3 = OPS committee-unanimous, executed under gating.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
| 2026-07-23 |
dc-mirror.sh shipped (D-135 item 1): per-DC apt+UCA mirror, harness 18/18, gauntlet 77 ALL GREEN
...
Site-keyed (dc0/dc1 MEASURED rows), check/install/sync; net layer = D-134
utility .4 alias + default route via the DC edge (egress path MEASURED
working; edge DNS answers). debmirror GPG-verified jammy triple + UCA
caracal, nginx on the utility address, daily timer, niced/idle-IO. Install
fails loud on missing packages with the exact apt line (net-first ordering
resolves the no-egress chicken-and-egg). Gauntlet capture + CURRENT-STATE
same-commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
Stage 4 carve EXECUTED both DCs: D-133 flat statics + D-134 bands live in MAAS
...
Logged window stage4-carve (index entry added). Sets 0-5: dynamic ranges
moved .201-.254, 10 plane fabrics, 90 NIC re-homes (pinned MACs matched),
2 provider subnets moved off fabric-5 + 8 created, 6 spaces + 12 VLANs,
18 nodes' statics + br-ex OVS (provider raw, D-100). All 18 nodes Ready.
Verification capture docs/audit/stage4-carve-verify-20260723.txt.
Changelog items 5-6 (incl. session permission-rule change + flagged
residue: 192.168.1.0/24, emptied fabric-5, empty auto-fabrics).
CURRENT-STATE same-commit (GA-R1 C1).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
Stage 4 OPEN + D-133 ADOPTED: flat per-NIC carve (VR1), hw-faithful NICs next deployment
...
Stage 4 (Phase 3, MAAS deploy per DC) opened on this branch: precondition
PASS, discovery captured read-only (docs/audit/stage4-discovery-20260723.txt),
runbook-vs-reality deltas + open decision points recorded in CURRENT-STATE
(same commit, GA-R1 C1). D-133 records the operator ruling (utterance quoted,
GA-R5): VR1 planes carve flat on dedicated NICs -- no VLAN-103 stack; next
deployment builds node VMs with target-hardware-exact NIC topology
(bonds/trunks). Affected stale surfaces queued in the entry, not edited.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
