| 2026-07-29 |
Phase-3 batch re-measured and 13 items delivered into the Stage-5 runbook
...
The 21-item Phase-3 remediation batch was written 2026-07-27 and logged-not-executed.
Re-verified against HEAD: 2 FIXED / 19 REMAIN / 0 SUPERSEDED, plus 5 NEW. Zero superseded
is load-bearing -- ruling 3 CREATED 3.3's hazard rather than retiring it, and the
no-DC-ordering ruling promoted 3.8 to live rather than closing it.
Two measurements corrected the readiness doc instead of inheriting it: 3.6's "21
non-selector consumers" is really 23 consumers / 8 selector-calling / 15 not / only 5
carrying DC-dependent values (the doc counted grep hits, including four files that never
source lib-net), and the D-133 guard is already satisfied at lib-net.sh:175.
Delivered into runbooks/dc-dc-phase4-juju-bundle-per-dc.md (434 -> 929 lines): 3.1 (the
DC1/DC2 namespace, which meant vr1-dc0 in one place and vr1-dc1 in another), 3.2, 3.4
(controller-tag constraint derived from maas-role-tags.sh, gated on exactly one matching
machine), 3.11 phase-4 side, 3.13, 3.14 (asserts the ovs-vsctl observable, refuses on
no-encap), 3.15, 3.16 (four VERIFY-LIVE gates), 3.17 (dry-run now precedes the deploy it
gates), 3.18, 3.19, 3.20, 3.21. Per-DC octavia overlay name threaded through to match F1.
Three items deliberately NOT claimed as fixed: the dc0 apt-mirror model-config key rests on
a single repo comment with no client here to verify it; the juju create-backup flag shape is
corrected only where established and marked unverified-at-authoring, with the gate moved onto
the resulting file; and whether --unit <app>/leader resolves for a subordinate could not be
established, so the ovn-chassis step probes a named unit per existing repo precedent.
NEW-6/7/8 logged not fixed. A human read of the expanded runbook is owed -- it is lint-clean
and its claims were re-grepped, but length is not correctness.
repo-lint 0 fail / 615 files, zero non-ASCII.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
F1+F4 BUILT: the Octavia PKI generator is per-DC end to end, and the gitignore glob follows it
...
F1. The D-109 amendment rules per-DC independent Octavia CAs. R7 parameterised the CA
subject and the VIP gate but left every PATH fixed, so generating a second DC's PKI
overwrote the first DC's issuing-CA key, controller-CA key and both passphrases, and left
one fixed-name overlay carrying the wrong DC's CA for any later redeploy to pick up.
Step 1.0-GEN is now per-DC end to end: DC/DC_LABEL/REPO/VIP_OVERLAY/OCTAVIA_PKI_OVERLAY
export once in a rewritten 1.0-GEN.0; WORKDIR="$HOME/octavia-pki/$DC"; every WORKDIR
re-derivation and the overlay write carry ${DC:?} so an unset DC refuses rather than
silently reusing the old shared path.
Two gates added that did not exist:
- REFUSE-IF-PRESENT: aborts if this DC already has PKI material. Regeneration invalidates
every amphora already issued against that CA, so it must be a deliberate act rather than
the default outcome of re-running a step.
- F4 gate: `git check-ignore -q "$OUT"` immediately before writing key material, asserting
the ACTUAL ignore decision for that path rather than a pattern someone has to remember.
F4. .gitignore:40 pinned the single exact path overlays/octavia-pki.yaml, so the per-DC
rename would have left a file holding CA key blobs plus a plaintext issuing-CA passphrase
UNIGNORED and committable, in a repo SEC-004 records as PUBLIC. Widened to
overlays/*octavia-pki.yaml and proven BOTH ways, not assumed: all per-DC and legacy names
read IGNORED, and a negative control (overlays/vr1-dc0-vips.yaml) reads NOT ignored, so the
glob is not swallowing tracked overlays.
Also fixed, same dc0-freeze: Step 1.3's VIP guard was a hand-rolled grep triple anchored on
the literal 10\.12\.4\. -- it hard-aborted on dc1, counted IPv4 only so R2's ruled v6 legs
were guarded by nothing (chain-audit finding 22), and its prose said 11/11/0 while its code
demanded 13. It now calls provider-bundle-check.py on the merged per-DC input, which already
encodes the bands, the count and the 2026-07-28 dual-family arity coupling.
The 2026-06-03 as-built line keeps the old command verbatim: it is history, not instruction.
repo-lint 0 fail / 615 files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Chain-audit deploy path: DC-aware preflight, dc1's real overlay set, Fork-3 role tags
...
Findings 16-20 from the D-136 chain audit, actioned.
preflight P2 is DC-AWARE and validates the deploy's ACTUAL overlay set. It
hardcoded --dc vr1-dc0 with no dc1 branch while dc-dc-phase4 invokes it BARE
as the dc1 gate and tells the operator to expect PASS -- so a dc1 operator
got a green gate that validated dc0's numbers against dc0's bands and never
parsed a dc1 artifact. It also passed one overlay where the deploy passes
two. DC=vr1-dc1 now merges vips + machines for that site.
dc1's deploy command names ALL its overlays. phase-4 Step 4 named only the
vips overlay, so a dc1 deploy run as written merged a machines block still
saying tags=openstack-vr1-dc0 -- grep for vr1-dc1-machines across runbooks
and scripts returned ZERO, i.e. that overlay was in no executable path. The
step now carries the full command, states dc-ha-scaleup is deliberately
excluded per R6, and adds the --dry-run machines-merge confirmation the
overlay's own VERIFY-LIVE note required and no step performed.
phase-6 deployed with NO VIP overlay and named a file that never existed.
Both blocks -- including the real apply against a live model -- passed
overlays/${DC}-hostnames.yaml, never authored, and no VIP overlay;
post-ruling-3 that is a merged input with zero VIPs including the designate
those commands exist to add.
scripts/maas-role-tags.sh ships Fork 3 (harness 9/9). Every machine
constrains on two tags; only the site tag is ever created, so the three role
tags exist nowhere and juju's allocation constraint matches no machine.
Role is DERIVED from two signals that must agree -- the ruled D-134 octet
band and the host's own name token -- and disagreement REFUSES rather than
picking one. Nodes match by pinned boot MAC, never hostname. Dry by default,
every write read back, absent CLI distinguished from unreachable service.
apply --commit is a live MAAS mutation and is NOT run here.
OWNED, third appearance of the IFS trap: ROLES="control compute storage"
under IFS=$'\n\t' does not word-split, so the script reported one absurd tag
named 'control compute storage'. Caught by its own harness pre-ship. Same
class as commit 1's CHECK 1 death and phase-05's aborting captures -- these
strict-bash gates should use arrays, never space-separated string lists.
Also fixed: the role-derivation refusal swallowed its own diagnostic.
Gauntlet ALL GREEN (87) on vcloud; repo-lint 0 fail / 614 files; preflight
verdict unchanged in kind.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
D-109 ruling note: the Octavia controller cert gains IPv6 IP SANs
...
Operator ruling (GA-R5), exact utterance: "Yes, add the v6 IP sans."
Recorded as a D-109 RULING NOTE (2026-07-29); OPS under GA-R3 -- a
generator change extending an already-ruled principle to a surface D-109
governs -- so no new D-number and next-free stays 138.
The generator now emits IP.1 = this DC's provider v4 leg and IP.2 = its
provider v6 leg, both derived from the same per-DC overlay under R7's $DC
selector. Verified: dc0 -> 10.12.4.57 + 2602:f3e2:f02:11::57; dc1 ->
10.12.64.57 + 2602:f3e2:f03:11::57; and a v4-only control emits NO v6 SAN,
so the generator stays correct on a tree where the dual-stack ADD has not
landed. Admin and internal legs stay EXCLUDED exactly as before --
DOCFIX-067's design has always been provider-leg-only.
Two repo guards fired and both were right: repo-lint L5 rejected a heading
that LED with the D-number (it reads as a second definition -- the third
time this trap has bitten on this branch), and L10 rejected the change for
touching a design-decisions Status line without CURRENT-STATE in the same
commit.
Observation logged, not acted on: amphorae reach the controller on o-hm0's
charm-generated fc00::/64 ULA, not on any VIP leg, so neither SAN matches
that path. Measured upstream, the controller verifies the amphora by UUID;
the reverse direction was not established from source. Settle by inspection
at the Octavia step rather than assuming.
Gauntlet ALL GREEN (85) on vcloud; repo-lint 0 fail / 611 files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Octavia: R8a BUILT (record was false), two stale surfaces, R7 generator de-frozen
...
Follows an upstream/vendor research pass answering "can Octavia run full
IPv6" -- yes; nothing inside Octavia requires IPv4. charm-octavia creates
the lb-mgmt subnet with ip_version: 6 and has no IPv4 code path at all; the
health manager is v6-capable both directions; the amphora agent binds '::'
and verifies certs by amphora UUID, not IP; and Nova metadata is not a
dependency (config_drive=True). Surviving v4 pressure is soft or external.
R8a BUILT. phase-05-octavia-verify.sh now compares o-hm0's MTU against
lb-mgmt-net's (resolved BY TAG, never by name), fails in either direction,
and REFUSES when a value cannot be read. This closes the LP#2018998
obligation, live on our exact pin after its 2025-12-31 recurrence.
THE DECISION RECORD HAD CLAIMED THIS SHIPPED. design-decisions.md carried a
present-tense "Delivery: the assertion ships in ..." written on the day of
the ruling, while the file held zero MTU references and its last commit
predated the ruling by a month. Corrected in place. Sharper than the usual
ruled-but-not-built class: the decision doc was the false witness.
TWO OWNED ERRORS:
- I concluded "no octavia harness exists" from a NAME grep. The script's
harness has always been tests/phase-05. Duplicate deleted, R8a cases
folded into the real one.
- My first MTU draft ABORTED the script: a $( ) under set -euo pipefail with
inherit_errexit exits the run rather than reaching the refusal branch. The
pre-existing harness I had just declared nonexistent caught it. Same class
as commit 1's IFS word-splitting trap; both now locked by cases.
Two live surfaces still contradicted R8 -- dc-dc-phase4 and the workflow doc
both called lb-mgmt IPv6 "a real, open risk" and recommended keeping it
v4-only, citing the two refuted bugs. A reader would have re-litigated a
closed ruling in the wrong direction. Both superseded in place.
R7 executed: phase-01 Step 1.0-GEN gains a $DC selector; the baked
/CN=VR0 DC0 CA subjects derive from it, and the VIP gate derives this DC's
provider prefix from the same overlay it read the VIP from -- R7's explicit
"read the MERGED input" caveat. dc1's 10.12.64.57 used to hard-ABORT, so no
dc1 artifact could be produced. Verified both DCs, negative control still
aborts.
Logged not built: the controller cert's CN/DNS SANs still carry dc0.vr0
(outside R7's ruled scope, inert while os-public-hostname is unset), and it
gains no IPv6 IP SAN though the VIP is dual-family -- a gap, not a break,
and adding v6 SANs needs its own ruling.
tests/phase-05 14 cases (was 9); gauntlet ALL GREEN (85) on vcloud;
repo-lint 0 fail / 611 files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Ruling 3 commit 2: dual-stack ADD -- R2 and R11 are now BUILT, L3-9 resolved
...
Both per-DC VIP overlays re-rendered dual-family: every app carries
prefer-ipv6: true and a six-address vip, and vault .61 + designate .62 are
built. Measured both DCs: 13 clustered VIPs (13 dual-family) + 12 hacluster
principals all carrying a VIP -> PASS. With dc-ha-scaleup stacked: 13
principals, PASS -- R6's ruled ordering (VIPs before the HA overlay) is now
satisfied and executable. Two ruled-but-never-built decisions became
artifacts in one pass.
L3-9 RESOLVED. dc-dc-ipv6-family-matrix.yaml is narrowed to ceph-mon only;
its ten duplicate vip + prefer-ipv6 pairs (every value an unrendered token)
are gone, removed TOGETHER -- removing the vips alone would have recreated
the defect deliberately. Both merge orders now PASS, i.e. order-independent,
which is the actual fix. ceph-mon's ULA-only networks survive because they
exist in no other file. The two Launchpad citations arguing octavia lb-mgmt
IPv6 was an open risk are removed: both were read in full and both failed,
and R8 closed that question on 2026-07-27.
Also corrected at source: CURRENT-STATE described L3-9 as "the dangerous
merge order is the one that PASSES". Measured -- that stopped being true
when invariant 9 shipped on 2026-07-28. The merge semantics it describes
remain accurate; only the green-over-loss conclusion is retired.
Gate repairs the agents caught BEFORE the edit:
- phase-01's deploy guard would have ABORTED THE DEPLOY AGAIN -- its HI regex
(5[0-9]|60) excludes .61/.62, so it would have read 13/11/0 against a
required 11/11/0. Widened, counts moved to 13, verified 13/13/0 -> DEPLOY.
Second near-miss on this one guard across two commits.
- pre-flight-checks CHECK 1 hardcoded if(n!=3) and would have failed all 13.
Now takes a triple OR a sextet and asserts a sextet's last three legs are
really v6. Proven able to fail on both new paths.
Harnesses re-pointed, never deleted. T16/T17 asserted R11's gaps were OPEN;
inverted to the surviving invariant and PAIRED with new T16b/T17b that remove
a VIP and demand the check still fires. T20/T21 had become no-ops that would
have gone GREEN testing nothing -- inverted. A v4-only twin fixture keeps the
pre-dual-stack cases testing their own invariants. The renderer harness now
proves reproduction on both shapes, including new T3b against the live
dual-family overlay.
provider-bundle-check 33/33; render-dc-overlays 18/18; render-baseline 10/10;
preflight 16/16; gauntlet ALL GREEN (85) on vcloud; repo-lint 0 fail / 611.
PREFLIGHT: BOTH STANDING REDS CLEARED, 3 fatal -> 2 fatal. The remaining two
are the deliberately-absent octavia-pki overlay and MAAS unreachable from
vcloud, which R10 measured clears on voffice1.
FLAGGED, not hidden: octavia's dual-family API VIP is an inference from R2
applied evenly, not a quoted ruling. Worth one operator confirmation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Ruling 3 commit 1: bundle.yaml is VIP-free; dc0's VIPs are now a RENDERED overlay
...
The 11 inline vip: lines are gone from bundle.yaml and live in
overlays/vr1-dc0-vips.yaml, GENERATED by scripts/render-dc-overlays.py from
render/values/vr1-dc0-vips.yaml -- the renderer's first real output.
placement's then-empty options: block was dropped rather than left parsing
as null. Ruling 3's dual-stack ADD is commit 2 and is NOT in here.
PROVEN NEUTRAL by the ruling's own named check: provider-bundle-check on the
pre-extraction bundle and on the merged pair produce line-for-line identical
output, same known designate failure included. A structural diff separately
confirmed only vip moved -- applications 56 -> 56, relations 108 -> 108,
every other key byte-identical.
The 11 inline comments were carried, not dropped: extracted verbatim by
parser rather than retyped, via new optional per-app comment support in the
renderer. An app with no comment emits none, so dc1 still reproduces
byte-for-byte at 1634 bytes. The B1/B5 tokens stay DEFINED in bundle.yaml's
header and the overlay header names that, so the references do not dangle.
The blast-radius repair is the bulk of this, and every item was found BEFORE
the edit by the read-only agent sweep:
- preflight.sh P2 validated the bare base (11 phantom failures post-move) ->
merged input.
- pre-flight-checks.sh CHECK 1 read the base as raw text and would have seen
ZERO VIPs. This is BLOCKER-1, fixed the ruled way and not by bumping the
count. Measured trap: the file sets IFS=$'\n\t', so a space-joined string
reached grep as ONE filename, grep exited 2, pipefail propagated and the
gate died silently mid-check. Fixed with a bash array.
- phase-01's RUN block gates juju deploy on 11/11/0 and would have read
0/0/0 and ABORTED THE DEPLOY. Both guards now span base + overlay, and a
second latent bug was fixed in the same edit: grep -c over two files prints
one count PER FILE, so the bare $( ) captured a two-line string and every
numeric test would have failed anyway. Now awk-summed; verified 11/11/0.
- phase-01's Octavia SAN derivation raised KeyError, swallowed by || true
into an empty VIP. Now reads the overlay, deriving 10.12.4.57.
- phase-00 teardown, dc-dc-phase4's "NO VIP edits" guidance, appendix-A L3,
and render-baseline's now-unverifiable bundle hash all repaired.
Harness 30 -> 31. The fixture base is now bundle + dc0 overlay, since that
pair is the deploy input; T16 renamed to match. NEW T31 asserts the bare base
fails for every clustered principal, so "deployed without its overlay" is a
tested state rather than an assumption.
Gauntlet ALL GREEN (85) on vcloud; repo-lint 0 fail / 610 files scanned.
Still red and expected: CHECK 1 at OK=11 want 13, and designate -- R11's
.61/.62 remain ruled-but-not-built, which is commit 2's job.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
| 2026-07-27 |
set-interface-v4: reload the pf filter on --commit (RULED); appendix-A rewritten
...
RULING (GA-R5) 2026-07-27. Question as presented: (a) add `configctl filter reload` to the
script's --commit path; (b) leave the script and rely on DC standup DoD item 8; (c) put it behind
an opt-in flag. Operator answer, exact utterance: "a". OPS under GA-R3 -- a script fix completing
an already-documented operation, governed by D-113; no new D-number.
DEFECT (appendix-A, measured on the fresh dc1 edge 2026-07-23): --commit ran `configctl interface
reconfigure`, which applies the address to the kernel but does NOT regenerate the pf ruleset, so
OPNsense never creates its automatic outbound-NAT rules for the (re-)addressed LAN. Forwarded LAN
packets left the WAN un-NAT'd and replies never returned -- the edge passed NO LAN traffic while
looking perfectly configured. Self-masking: any later plugin install or config apply reloads the
filter as a side effect, so dc0's 07-20 qga work hid it there until dc1 exposed it.
Option (b) rejected on evidence, not taste: DoD item 8 already existed and did NOT fire at dc1,
and prose-only controls have a measured failure record here (D-137's own enforcement sat as prose
at phase-3:498 and missed BOTH DC standups).
PLACEMENT IS THE SUBSTANCE OF THE FIX, and my first attempt got it wrong -- caught before
shipping. Putting the reload on the next line of the `interface reconfigure` heredoc would have
been a fix that fixes nothing exactly when it matters: re-addressing the interface you arrived on
drops the session DURING the reconfigure, so a following command in that stream never executes --
and that drop case IS this incident's own scenario (dc1's LAN moved 192.168.1.1 -> 10.12.64.1
over the LAN itself). As shipped the reload runs over a FRESH connection to PROBE_HOST.
Also added a `pfctl -s nat` read-back with the `proto carp` boilerplate filtered out, since
boilerplate-only is precisely appendix-A's broken-state signature. REPORTED, not gated: an edge's
first addressing may legitimately have no gateway and so no NAT to generate, and hard-failing
would block a correct run. The hard gate stays address-on-the-kernel.
Harness 59/59 (was 53), cases 15-15f. Case 15f caught a real inconsistency -- appendix-A still
said "not yet ruled". Case 15e caught a bug in my own first draft, which scanned to EOF and so
failed the script for having the very hard gate the case exists to preserve.
appendix-A rewritten: the FIX stays for an edge already in this state, plus a PREVENTED-IN-THE-TOOL
block recording the placement detail and why prose prevention was rejected.
Gauntlet ALL GREEN (81), repo-lint 0-fail. No live edge touched -- both DC edges are already
addressed, so this affects only the NEXT edge addressing.
With this, every substantive Stage 4 in-stage item is closed or split to gate row G17; only the
GA-R6 close-out set remains.
Revert: git revert this commit's script + harness + appendix-A changes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
G17 opened by operator ruling; Stage 4 DoD bullets 5-6 repaired (DOCFIX-204)
...
RULING (GA-R5), 2026-07-27. Question as presented: "The node-side half of bullet 5. Nodes are
powered off by the READY-handoff ruling, so no node-side probe can run as things stand. Either a
gated rescue-boot check on one node per DC now (closes it inside Stage 4), or split it into its
own gate row targeted at Stage 5 first boot (GA-R6 E3 explicitly permits this; a conditional
close is not permitted)." Operator answer, exact utterance: "split it into its own gate row".
Recorded in the G17 row of docs/CURRENT-STATE.md (the authority); pushed before dependent work.
G17 "Per-DC artifact source reachable FROM A NODE", [V], OPEN. dc0 -> curl 200 from a booted
node against the D-135 item-1 mirror; dc1 -> apt fetches through the proxy at 10.12.68.4:3142,
since dc1's ruled path is the CACHING PROXY with no node-facing mirror (D-135 amendment) and
checking it as one fails by design. Trigger: Stage 5 first boot. The row states that G17 is NOT
a Stage-5 precondition -- Stage 5's bootstrap needs open edge egress for the juju agent stream
and snaps (D-135 items 2-3 unbuilt), a different path from the apt artifact source.
Stage 4 retains bullet 5's rack-side half: the source answers on its own address with an
attested-current sync. dc1's proxy PASSES; dc0 pends the sync re-run.
DOCFIX-204 -- bullet 6 was UNSATISFIABLE and its stale text had spread to four surfaces. "NTP
from the DC's own OPNsense edge working" is superseded by D-129(iv) (RULED 2026-07-21, "Keep
MAAS hierarchy", no NTP role on the edge): the DoD asked for confirmation the edge is the time
source, which the ruling had already refused, so a correctly-built DC could never meet it.
Repaired in the phase-3 DoD (bullet 6 struck, bullet 5 rewritten per-DC), the phase-3 Step 7
(REWRITTEN -- it branched on gap #5 that D-135 resolved, applied one mirror check to both DCs
which fails on dc1 by design, and asked for checks "from a deployed node" that the READY handoff
makes impossible), the workflow-doc Gate cell (which also still said "Nodes deployed",
contradicting DOCFIX-200), the buildout design, and the phase-4 prerequisites (same "nodes
Deployed" error -- exactly the expectation DOCFIX-200 removed to stop Stage 5 breaking).
phase-4 also gains a block making it the OWNER of G17's captures, warning that first boot is the
observation window and missing it forces a deliberate rescue-boot.
Text and gate-record only; no live state touched. repo-lint 0-fail. DOCFIX next-free -> 205.
Revert: git revert this commit; all changes are documentation and the G17 gate row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
| 2026-07-26 |
D-137 tier 3 behavioural half: V2 declared-state + creds-probe.sh
...
Boundary and cadence operator-ruled: "Full -- including OpenStack service
credentials", "Stage close + DC standup close". I had recommended
passive-only and flagged the costs; the operator ruled otherwise.
V2 (passive, zero authentication) fires TODAY on a real defect: SEC-006 has
recorded the NetBox upstream token as BURNED since 2026-07-12 while the
matrix carried it as a healthy consolidated row. Nothing reconciled the two
surfaces. Keyword matching on ledger prose is fragile, so it is conservative
and reports the matched text as its basis.
creds-probe.sh is SEPARATE by design: a probe must USE a credential, and
creds-matrix.py is metadata-only by an enforced invariant (T22/T51). Folding
probes in would have relaxed that guard, and a guard relaxed once stays
relaxed.
Safety: never retry a failed auth (retry loops lock accounts; the MAAS admin
identity backs 19 call sites), never argv, never echo, stage-aware skips, and
per-tenant credentials REFUSED BY CONSTRUCTION -- a declaration cannot opt
them back in (D-069, ruling 3). Not a deploy gate.
FALSE GREEN in my own first run: ssh inside a while-read loop ate the loop's
stdin, so 1 of 13 probes ran and the run exited 0. A partial run reporting
clean is the exact failure class this tool exists to catch. Fixed with ssh -n
and locked by a total-accounting check that makes a partial run VOID.
13 probes declared: 3 active, 4 stage-pending, 6 declared-only with reasons
recorded. OpenStack credentials declared now, skipping until Stage 5.
tests/creds-probe 9/9; gauntlet 80 -> 81; phase-3 DoD wired.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
D-137 ruling 4 EXECUTED: SEC-009 demoted to a pointer, policy moves to D-137
...
Ruling 4 was already RULED and its stated trigger (tier 2 built) had passed,
so this is execution, not a decision.
The standing consolidation rule moved from docs/security-ledger.md into
D-137, restated as six numbered points and updated to D-137's own mechanisms
(matrix is the register; manifests are DERIVED; discovery bounded by
vm-secret-locations; one identity one principal; enforcement is blocking P5).
The ledger keeps a pointer and returns to being purely an exposure/rotation
register -- which is what lets a gate cite a D-number instead of a SEC row.
Kept as history: SEC-009's founding addendum (the NetBox token that lived
only on the VM and went un-consolidated until needed). That miss established
the convention, and history is what the register is for.
Carried forward WITH CORRECTIONS: the moved text's "11 entries", "7/7", and
its description of manifests as hand-declared were all stale and are not
reproduced.
Callers repointed in the same commit, because a demotion that leaves callers
citing the moved text is not done:
- dc-dc-phase3-maas-enlist-deploy.md:491 is the ONLY credential DoD in any
runbook and it told operators to hand-edit creds-manifests/$DC.manifest --
a GENERATED artifact under sub-ruling 2. The live runbook was teaching the
anti-pattern. It now directs a matrix row plus a locations row, and its
close-out requires both creds-audit CLEAN and no new creds-matrix finding
class for that DC.
- docs/vr1-office1-as-built.md restated the convention; now a pointer.
Gauntlet ALL GREEN (80), repo-lint 0-fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
| 2026-07-24 |
phase-4 runbook: per-DC MAAS service-credential step (Step 2.0)
...
Operator-directed deployment-time task: Juju's MAAS API key is created by no
prior stage. Added Step 2.0 (before bootstrap) -- dedicated per-DC MAAS admin
service user + own API key (per-DC isolation, SEC-012/-016; Roosevelt: per-DC
cloud creds), staged in the JUMPHOST per-site creds folder
(~/vr1-<dc>-creds/maas-api-key.txt on vcloud, SEC-009) NOT a new region location,
generated from the jumphost so the key never persists on the region, never
printed, consumed by path/stdin. Corrects the earlier walkthrough's location.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
| 2026-07-23 |
Delivery batch: lib-hosts VR1 arms, D-133 guards, appendix-A pf entry, DoD 8-9 (gauntlet 77 GREEN)
...
lib-hosts vr1-dc0/vr1-dc1 populated from the measured identity table
(tofu-name keys, pinned boot MACs, D-134 octets, per-DC power URIs +
tags) + host_sysid_by_bootmac (the VR1 resolver) + one-selection guard;
dc-selector rewritten, 48 PASS. reenroll-hosts + carve-host-interfaces
gain explicit vr1-* refusals (VR0 flows superseded by D-133 -- populated
arms must not enable them). lib-net vr1-dc0 VID caveat. Appendix-A:
edge pf-never-regenerated symptom (masking trap named). Phase-2 DoD:
items 8 (filter reload + LAN egress probe) + 9 (dc-mirror), item 4
range corrected to D-134 .201-.254. CURRENT-STATE same-commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
Deploy-model ruled (Ready+jammy, DOCFIX-200), per-DC placement tags applied, mirror ruled BUILD-NOW
...
Step 4 rewritten: nodes stay Ready, Juju provisions at Stage 5 (phase-01
precondition precedent); jammy boot images synced to the region; tags
openstack-vr1-dc0/-dc1 applied 9+9. Operator REJECTED the D-107 rehearsal
exception -- the per-DC mirror gets built in-stage; design decision next.
Changelog item 7 carries the three rulings verbatim + queued findings
(phase-00-maas-standup VR0-VLAN false-DRIFT; Step 7 NTP text vs D-129(iv)).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
D-068 Q3 RULED + delivered: monthly-review checks in ops-update-procedure 0c
...
Two standing read-only checks (Vault 1.8.x CVE scan, migration-signal probes);
design-time V1-V5 re-verify trigger recorded. Same-vehicle stale claims fixed
(bundle-pins-1.16 CAUTION, D-071 PROPOSED tail). D-068 item 1 remainder is now
only the Q2 path selection at Roosevelt Vault design time.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
Queue pass: post-G12 queued findings delivered (5 script/harness fixes + DOCFIX-198/-199)
...
- opnsense-prep-image.sh: survives bash -s (BASH_SOURCE unbound fix; T3b)
- site-headend-install.sh: version/site-neutral hints; nftables-sec010
idempotent reload preamble; serial-log staging dir in --host-nodes (+check)
- opnsense-bootstrap-apikey.sh: sshd readiness wait + first-scp retry +
ssh -n stdin guard + minter-beside-script usage note (T9/T10)
- DOCFIX-198: phase2 runbook Steps 9-10 SUPERSEDED banners (pod route refuted,
D-103/D-123 amendments) + DC standup definition-of-done section (lands the
staging-dir + SEC-016 snap-refresh re-assert DoD obligations)
- DOCFIX-199: D-112(c) console-bootstrap payload requirements (util.inc et al)
Gauntlet ALL GREEN (76); repo-lint 0 fail. Per-item reverts in
docs/changelog-20260723-queue-pass.md. No cloud mutation this session;
queued: gated live re-assert of the nftables-sec010 files on all three hosts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
| 2026-07-21 |
STAGE 3 CLOSE (operator-ruled): GA-R2 consolidation + skill sweep; gauntlet 76 ALL GREEN
...
- Four execution-session changelogs -> docs/archive/changelogs/;
vr1-stage3-record.md finalized (close summary + session table)
- CURRENT-STATE stage line CLOSED (vr1-dc0 scope; dc1 = HELD G12);
workflow Stage-3 State row + runbooks/README corrected
- Skill sweep: D-131 forwarder invariant + MAAS pod traps + merge
precedent folded; no stale facts found
- Close-out evidence: gauntlet 76 ALL GREEN, repo-lint 0 fail, GA-R7
memory review done (changelog items 11, 17)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XRtm5sDQgUsZyTJk3k7dz
|
Recovery COMPLETE: 9/9 Ready again (shapes exact, power=virsh); retire-fully fully executed; SEC-013 narrowed
...
- Operator-ruled recovery: PXE re-enlist (~2 min) -> maas-node-power.sh
dry+commit (9/9 verified) -> re-commission -> ALL 9 READY ~3 min
(docs/audit/incident-20260721-recovery-verify.txt; new MAAS hostnames)
- voffice1 cleanup: vr1-dc0-maas dir + tfstate + SEC-013 key file removed,
absence verified; SEC-013 row narrowed to CLI-profile-only
- CURRENT-STATE incident paragraph RESOLVED same commit (GA-R1 C1/C2)
- NEW appendix-A entry: pod delete cascades to linked machine records --
read the pod's machine list BEFORE vm-host delete; non-empty = STOP
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XRtm5sDQgUsZyTJk3k7dz
|
Stage-close batch: ledger-scan PARTIAL fix; DOCFIX-197; D-103/D-123 pod-refutation amendments; GA-R7 memory review
...
- scripts/ledger-scan.sh: PARTIALLY-RULED decisions now surface (D-131
blind spot); harness regression fixture, 46 checks ALL PASS
- DOCFIX-197: phase2 runbook Step 11 leg selection corrected to executed
reality (dc0<->dc1 leg; office1 leg carries live transit)
- D-103/D-123 amendments + modules/maas-vm-host header: 2026-07-20
measured pod refutation + per-machine-virsh ruling RECORDED (cited to
captures; no new ruling made)
- GA-R7 memory review DONE: no memory-only facts remain
- Read-only: stale MAAS pod object confirmed (id=4 vr1-dc0-inner, virsh)
for the pending retire-or-keep ruling
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XRtm5sDQgUsZyTJk3k7dz
|
Incident docs: 2 appendix-A entries, platform-traps MAC-regen corollary, LP draft
...
Closes the remaining documentation queue from the 2026-07-21
commissioning incident. Appendix-A gains the MAC-drift and
agent-resolver symptoms (verbatim, with checks + recorded fixes);
platform-traps 1e gains the unpinned-MAC regeneration corollary + index
row; LP draft ready for the operator to file (sanitization warning
included). Still queued: stale pod cleanup at stage close.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STHXiHfoxHqq8fGRVvb66G
|
| 2026-07-20 |
appendix-A: new incident class -- MAAS says dhcp_on=True but no dhcpd runs (Temporal wedged)
...
Records the 2026-07-20 incident by verbatim symptom: config vs running state
disagree, MAAS's own service_set reports healthy, and the region journal
shows Temporal 'Not enough hosts to serve the request'. Check is pgrep -c
dhcpd on EVERY controller; fix is snap restart maas on the region, verified
behaviorally. Includes the scope trap that made it invisible for days: an
already-built site keeps working because deployed VMs hold their addresses,
so it only surfaces when new nodes try to PXE.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KiUu1oqt76tWvV4vEC3NAr
|
| 2026-07-19 |
Batch 4 C6: docs disposition -- 24 history docs archived, 16 retained, refs rewritten
...
DISPOSITION MANIFEST (archive, one-line justifications): clientdocs-
workflow-review (v1 tenant-docs era) / D-057-DECIDED-append +
D-057-REVIEW-ITEMS + README-D057-PACK + D-058-renumber (executed v1
decision packs) / D-068-openbao-assessment-DRAFT (supplementary; the
cited analysis retained) / DOCFIX-064-phase08-changelist + docfix-draft-
20260702 (executed/draft) / handoff-20260703 + handoff-20260705
(superseded handoffs) / incident-20260712-triplefault (trap routed to
platform-traps 1b/1c long ago) / netbox-vip-queue + phase-00-maas-
standup-notes + script-quality-findings + session-findings-2026-07-02 +
tenant-cidr-overlap-PLAN + v1-pre-deploy-fixes (v1-era executed/dated) /
repo-lint-nextfree-bug-FINDING (absorbed DOCFIX-105/107; repo_lint.py
pointer updated) / upstream-bug-draft-dashboard-tls (draft, dated) /
model-a-fallback-plan (D-123 Model B adopted; fallback preserved in
archive) / stage3-adversarial-review-20260716 (R3-F register,
dispositioned in 2.10) / dc-dc-replication-DR-seed (absorbed per
workflow) / dc-dc-ipv6-charm-research + dc-dc-netem-and-ula-gua-proposal
(stage 4+ research, refs updated). RETAINED 16 top-level, justified in
the session changelog. Live-surface refs to all moved files rewritten to
archive paths (18 files). G3 cell + ledger + changelog updated same
commit (the L10 fire on a Status-line path rewrite is satisfied by the
CURRENT-STATE touch -- the check worked as designed).
Revert: git revert this commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZbUeRE8weySLSGY1odrCi
|
Batch 4 C1: GA-R4 rule-6 ledger rotation -- 1187 -> 131 lines (GA-F13)
...
Bodies rotated VERBATIM to docs/archive/session-ledger-rotated-20260719.md
(Live state 2026-07-13, OPEN WORK VR1+VR0, NetBox write-path bugs, PINNED
rulings, standing lessons, state facts, project-completion, NetBox import
narrative). Routed-before-rotating (durable homes): juju format=line 3.6
trap -> appendix-A; guards-are-load-bearing -> operating-discipline; VR0
operational facts + admin/controller -> maas-as-built-reference.md
(lessons 1-6 verified already covered by platform-traps/L8). Live ledger
keeps header + machine-derived fence + two compact still-live blocks
(NetBox write-path bugs pointer, project-completion) + the bounded session
summaries. 131 < 300: the F1 cap is now ENFORCEABLE. Fences OK; ledger-
scan counters unchanged; repo-lint 0 fail.
Revert: git revert this commit (restores the 1187-line ledger).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZbUeRE8weySLSGY1odrCi
|
GA-F10/H1 + GA-R1/R6: token migration, State-line demotions, E1 type tags (2.5+2.6)
...
Mapping record: docs/audit/h1-token-map-20260719.md -- all distinct
tokens dispositioned, none unmapped. Live-surface migration: workflow
doc (26 flags -> 0; header drops the tracker model, 8 State lines demote
to identity + CURRENT-STATE pointers), runbooks (18 -> 0; close-out
instructions RETARGETED to record status in CURRENT-STATE, not the
workflow tracker), readiness doc (5 -> 0), CURRENT-STATE itself (6 -> 0:
G9 STOPPED -> BLOCKED with named blocker; stale "GA-R1 NOT yet ratified"
header + pre-D-130 divergence rows corrected per C2). Gate rows gain
GA-R6/E1 type tags ([V]/[R]) with legend. CLAUDE.md expectation line
demoted per S5b. History classes (ledger, changelogs, design-decisions)
deliberately untouched -- their disposition is Batch 4/GA-R3 structural.
Extractor re-run: zero flagged occurrences on live non-history surfaces.
Batch 2 items 2.5 + 2.6.
Revert: git revert this commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZbUeRE8weySLSGY1odrCi
|
| 2026-07-18 |
D-129: build the OPNsense edge plugin profile-installer + sharpen pins
...
Autonomous processing of D-129 items that need no operator keyboard. Repo-only;
NOTHING applied to the cloud (live plugin install stays classifier-gated).
- NEW scripts/opnsense-plugins.sh + tests/opnsense-plugins/ (17/17): named-profile
installer over the D-113 firmware API. Installs ONE plugin at a time + polls
upgradestatus (the firmware-lock lesson), delegates auth to opnsense-api.sh,
warns os-qemu-guest-agent != a working agent, offline --dry-run. Closes the
"build the reusable installer" pin.
- D-129 + dc-dc-phase2 pin: installer recorded as BUILT (use it, not build it);
qga channel logged as an OPT-IN-var design (default OFF -> no office1/DC0-plan
side-effect; dmacvicar 0.9.8 mechanism-to-verify); os-git-backup operator ask.
- Gauntlet ALL GREEN (68), repo-lint 0-fail.
Revert: git rm the script+harness; revert the doc/runbook edits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JUBUyinU4sBAT4HV1b9vSz
|
D-129 verify: qga installed-but-non-functional; iperf lost to firmware lock
...
Reconcile records to the measured 2026-07-18 install verification (operator ran
the firmware POSTs via `!`; agent verified read-only). No cloud mutation here.
- os-qemu-guest-agent: package installed, NON-FUNCTIONAL -- daemon off + the
opnsense-edge module defines no org.qemu.guest_agent.0 libvirt channel
(guest-ping fails). Functional enablement = channel-in-module + service-enable
+ edge restart -> DEFERRED; new DC edges born with the channel.
- os-iperf: did NOT install -- back-to-back firmware POSTs collided on the
OPNsense firmware lock. Re-run pending; profile-installer must be 1-at-a-time
+ poll upgradestatus.
- D-129 Verification note + phase2 pin + as-built row + ledger reconciled.
Revert: git revert; nothing installed/applied by this commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JUBUyinU4sBAT4HV1b9vSz
|
D-129 partial-rule: adopt VR1 edge plugin profile + pin the rest at deploy steps
...
Operator directive 2026-07-18 ("install the recommendations + pin the rest").
Doc/runbook-only; the LIVE install is operator-gated (auto-mode classifier
blocked the agent POST) and pending verification -- no cloud mutation here.
- D-129 PARTIALLY RULED (status line kept OPEN so ledger-scan still tracks the
4 sub-decisions: COS-edge scope, os-frr, os-tailscale, edge-NTP). VR1 profile
{os-qemu-guest-agent, os-iperf} adopted; pinned-at-steps mapping added.
- dc-dc-phase2 (DC-edge section): pin the VR1 profile at edge bring-up + build
the reusable opnsense-api profile-installer there.
- dc-dc-phase6 Step 10: pin os-node_exporter gated on the COS-edge scope call.
- vr1-office1-as-built: FreeBSD base 14.3->15.1 recorded; directed plugins noted.
- session-ledger + machine block synced.
Revert: git revert this commit; nothing was installed/applied on the cloud.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JUBUyinU4sBAT4HV1b9vSz
|
| 2026-07-17 |
D-127 IaC autostart matrix baked into deploy tooling + Office1 guest updates
...
Operator directive: bake the autoboot-vs-manual matrix into the IaC so DC0/DC1 get
correct autostart at creation, not by hand. Added an `autostart` bool var to modules
cloudinit-vm/opnsense-edge/node-vm (wired to libvirt_domain.autostart); set explicitly
per instance (shared modules => IaC is source of truth): voffice1=true,
office1-opnsense=true, vvr1-dc0=FALSE, vr1-dc0-opnsense=true, node-vm=FALSE.
HARD GATE verified: the two existing Office1 domains (voffice1, office1-opnsense) plan
as autostart false->true UPDATE-IN-PLACE, NOT replacement (replacement would destroy the
live host/edge). vvr1-dc0 is a new create with autostart=false baked in. Outer plan now
5 add / 2 in-place / 6 destroy; the 2 collapse to 0 once the bookend sets those two
autostart live pre-apply. tests/opentofu-validate T11-T15 pin the matrix against drift.
Standup runbook records the composed service VMs' LXD boot.autostart step. Confirmed
DC0/DC1 create NO LXD instances (inner root = node-vm + opnsense-edge + planes/pool/
wan-bridge) -- DC side is purely this VM matrix.
Office1 guest updates (agent sudo inside the guests): office1-netbox apt upgrade (no
reboot, netbox healthy); office1-tailscale full-upgrade -> kernel 6.8.0-136 + reboot
(back, re-advertising 10.10.0.0/22); voffice1 0-upgradable; LXD boot.autostart=true set
on both. vcloud + office1-opnsense stay in the operator's bookend.
Gauntlet ALL GREEN (66); opentofu-validate clean; repo-lint 0-fail.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ck6xh3jWQi5b3Su8Dx1LEH
|
| 2026-07-16 |
D-127: VR1 host-VM autostart policy (foundational autostart, DC containment manual)
...
Recorded before a vcloud OS-update/reboot window (open now: only Office1 VMs live,
pre-DC0). voffice1 autostarts; office1-opnsense autostarts AFTER one verified post-fix
cold boot (DOCFIX-188 mem-unit fix -- the triple-fault was a fixed config bug, not live
fragility); vvr1-dc0/dc1 containment stay MANUAL (gated depth-4/site-down bring-up).
Live via `virsh autostart`; IaC layer (module autostart arg) queued. ops-restart-
procedure.md pointer added. Ledger records the post-reboot TODO (verify clean startup,
THEN set autostart) since a host reboot ends the running agent session.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ck6xh3jWQi5b3Su8Dx1LEH
|
D-125 correction: uplink /24 is the ruled 172.30.2.0/24, not a HELD tfvar (no new NetBox object)
...
The initial D-125 records wrongly modeled the vcloud ISP /24 as a NEW,
NetBox-assigned HELD value (`var.vr1_dc0_uplink_cidr`) with an OPNsense WAN
re-address. That is factually wrong: bridge-in is single-NAT, so there is
exactly ONE simulated-ISP /24 for vr1-dc0, and D-115 already ruled it
(172.30.2.0/24, role edge, site vr1-dc0). NetBox models the prefix by role+site
and does not care which libvirt host runs the NAT, so the /24's IPAM identity is
UNCHANGED from Model A; only the libvirt realization moved (a vcloud NAT bridged
through vvr1-dc0). OPNsense WAN stays .2. (Caught reconciling the "create netbox
objects" request against dc-edge-wan-import.py, which already registers this /24.)
- opentofu/main.tf: module vr1_dc0_uplink cidr = "172.30.2.0/24" inlined (ruled
literal, matching the inner root's site-wan pattern) instead of the var.
- opentofu/variables.tf: REMOVED variable vr1_dc0_uplink_cidr (replaced with a
note explaining why it is a ruled literal, not a tfvar).
- Records corrected everywhere the false claim propagated: D-125 body + D-122
amendment, changelog-phaseC2-D125, model-a-fallback (B->A delta item 8 +
revert), session-ledger, dc-dc-deployment-workflow, phase-2 runbook.
Net effect: D-125 adds NO new NetBox object and NO new HELD gate. The only
NetBox work left for the DC substrate is D-124's transit /30 + rack IP.
Both roots validate; gauntlet ALL GREEN (63); repo-lint 0 fail.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ck6xh3jWQi5b3Su8Dx1LEH
|
