| 2026-07-30 |

dc-node-v6-carve.py: honour MAAS_PROFILE; refuse instead of tracebacking
...
The script was env-blind -- default 'admin', --profile only, no environment
read anywhere. Every bash tool here honours MAAS_PROFILE, so
`export MAAS_PROFILE=vr1-dc0-region` left THIS tool silently targeting the
Office1 region. During the per-DC region migration that is the
unrecoverable direction: against the wrong region a carve is an idempotent
no-op that prints PASS while the region you meant stays untouched.
It now reads MAAS_PROFILE, keeps --profile as an override, and prints which
region it targets and where that came from -- a capture that does not name
the profile cannot be audited for wrong-region work.
An absent maas CLI raised FileNotFoundError and exited 1 with a traceback,
indistinguishable from a real failure and contrary to this script's own
REFUSE contract. It now refuses with exit 2.
Harness 14/14 (was 9); both fixes proven able to fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

dc-region-topology: RENAME the auto fabric instead of moving the subnet
...
Moving a subnet between fabrics does NOT bring interface links along. The
first apply created vr1-dc0-provider-public and moved 10.12.4.0/22 onto it,
leaving the region VM's enp2s0 holding a static 10.12.4.6 on a VLAN whose
subnet had left -- the region's provider-public leg, which it needs for
image sync, and the same under-carve class that cost three bootstrap
attempts on the Juju controller. Live connectivity was unaffected because
the OS netplan is independent of MAAS's model, so every other assertion
passed 39/39 with the model inconsistent.
Renaming the auto fabric that already carries the subnet moves nothing. A
MOVE is kept only for the case where the fabric carries other planes, and
now prints a warning that links do not follow.
Adds the assertion that would have caught it: every interface link must be
on its subnet's VLAN. Harness 47/47; the rename path and the stranded-link
check each proven able to fail.
Live remediation already applied to the dc0 region: subnet returned to
vlan 5002, the wrongly-created fabric deleted, fabric-1 renamed, space
re-bound. enp2s0 now reads fabric=vr1-dc0-provider-public with its static
intact.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

dc0 region plane topology BUILT and verified (39/0); DHCP cutover blocked
...
dc-region-topology.sh applied to the new dc0 region: 5 named plane fabrics,
6 spaces, 4 plane subnets, 10.12.4.0/22 MOVED off the auto fabric-1 onto
vr1-dc0-provider-public, 6 VLAN->space bindings, openstack-vr1-dc0 tag.
Named check now reads 39 passed / 0 failed, exit 0.
metal-admin service config set: dns_servers=10.12.8.6, allow_dns=false,
D-134 dynamic range .201-.254, subnet resolved by CIDR.
The DNS target was measured rather than carried over, and my first reading
was wrong: a +time=3 +tries=1 probe suggested the new region's BIND could
not resolve externally. That was a cold-cache timeout. At +time=5 +tries=3
it answers archive.ubuntu.com, streams.canonical.com and api.snapcraft.io
with flags qr rd ra. Pointing node DNS at the DC-local region is correct --
it is authoritative for the zone the migrated nodes will live in and drops
the cross-fiber dependency D-132 q1 exists to remove.
Measured in passing, and worth keeping: VLAN row id 5005 is dc0's
metal-admin VLAN in Office1 and vr1-dc0-data-tenant in the new region.
Same integer, different DC plane.
The DHCP cutover was REFUSED by the harness classifier and was not retried
in an altered shape. Verified read-only afterwards that nothing is
half-applied: Office1 still reads dhcp_on=True primary_rack=7chphy and the
rack still runs exactly one dhcpd on virbr2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
dc-region-topology: explicit dry-run placeholders instead of empty ids
...
An empty `vlan=` in a dry run reads like a defect and would hide a real
one. On a from-scratch build the target fabric does not exist yet when the
MOVE and space-bind lines are planned, so their ids cannot resolve; they
now print <vlan-of-NAME> / <fabric-for-PLANE>. Four assertions added.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

dc-region-topology.sh: the plane-topology builder the repo never had
...
A read-only survey found THREE dc0 carve components with no repo tool at
all -- the named plane fabrics, the six v4 plane subnets, and the site
placement tag were created ad-hoc in the Stage-4 window, with the mutations
landing only in ~/as-executed/2026-07-23-stage4-carve.log, which is not in
the repo. When D-132 q1 required rebuilding that carve on a new region
there was nothing to re-run. This closes the gap so dc1 and every Roosevelt
DC standup rebuild from a tool rather than a transcript.
check/apply over: five named plane fabrics, six spaces, six v4 plane
subnets on the right VLANs with the ruled gateways, VLAN->space bindings,
and the openstack-<site> tag. Expected values derive from lib-net.sh;
fabric names derive from the site token. Numeric MAAS ids are resolved
per-run by name or CIDR and never persisted -- the pre-migration capture's
id= values are database row ids, not 802.1Q tags, and nothing reads them.
metal-admin deliberately keeps MAAS's own auto-created fabric: it is a
discovery artifact created when the rack registers, Juju binds spaces
rather than fabrics, and renaming it would fight MAAS for ownership of its
own object for no functional gain. The check asserts only that it does not
share a fabric with a named plane.
Every mutating path is gated by maas-profile-assert.sh first -- against
the wrong profile this apply is an idempotent no-op that prints PASS.
--profile has NO default, because the repo-wide default 'admin' resolves
to the Office1 region.
Harness 35/35; 4 mutations each killed tests (region gate removed, profile
defaulted to admin, space-binding assertion, fabric-placement assertion).
Fixtures corrected against a live read: MAAS reports an unbound VLAN as
the string "undefined", not null, so the first fixtures graded a payload
MAAS never emits.
Live read-only check against the new dc0 region: 5 passed, 22 failed,
exit 1 -- the expected shape of a region that has only auto-discovered its
own two NICs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Per-region power address, region power-key tool, and two red gates fixed
...
lib-hosts.sh: MAAS power ops originate from the REGION, so the reachable
rack address depends on which region dials. Measured from vr1-dc0-maas-01:
the rack's transit leg 172.31.0.2:22 is CLOSED, its metal-admin leg
10.12.8.2:22 is OPEN. Adds VIRSH_POWER_ADDRESS_FROM_{OFFICE1,DCREGION} for
both DCs (dc1's 10.12.68.2 measured on the dc1 rack, not inferred from
dc0's .2). VIRSH_POWER_ADDRESS still aliases the Office1 form so existing
callers are byte-for-byte unchanged. dc-selector 81 checks; 4 mutations
each killed tests.
New scripts/maas-region-power-key.sh (harness 44/44) installs and verifies
the per-DC MAAS->libvirt key inside a region's snap, ending on a real virsh
call rather than on config. Runbook prerequisite 5 carried this as prose,
and prose is not a gate -- it must also be re-asserted after every snap
refresh. LIVE at dc0: 9/9 exit 0, virsh enumerates 12 domains. The key is
the SEC-012 dc0 key already authorized on that rack, so per-DC isolation
holds.
Three defects its own testing found: ssh-keygen -lf reads an adjacent .pub
and reports ITS fingerprint; a snap's data root (/var/snap) and exec root
(/snap) are not interchangeable; the snap's virsh must run inside the snap
runtime or it fails on missing libraries.
tests/node-vm T8/T9 were RED at HEAD before any of this session's work
(proven by stashing and re-running): commits 086c827 + 447315f added the
region VMs, moving the dc0 root to 11 macs lists / 66 MAC literals against
assertions reading 10 / 60, and the gauntlet was not re-run. Re-pointed to
the new invariant, reason recorded in-file, re-proven able to fail.
HARNESS-MANIFEST re-recorded (91) for the two new harnesses.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Region-resolution gate: maas-profile-assert.sh + named dc0 region profile
...
The wrong-REGION hazard is the migration's unrecoverable failure mode: 13
repo scripts default to MAAS_PROFILE=admin, which on voffice1 resolved to
the OFFICE1 region. Against the wrong profile a carve recreate is a silent
no-op that prints PASS, and machine delete destroys the real record.
New scripts/maas-profile-assert.sh asserts which region a profile resolves
to by rack-controller IDENTITY (a machine count is not proof -- two regions
can hold the same number). Harness 20/20; mutation pass found one of its own
assertions could not fail (nameless rack entries were caught by a different
branch) and it was made branch-specific.
Live both directions: admin -> 23 machines / office1 trio (exit 0);
vr1-dc0-region -> 0 machines / hot-kid (exit 0); admin asserted as hot-kid
(exit 1). Capture docs/audit/dc0-region-profile-assert-20260730.txt.
Tunnel voffice1 -> rack -> 10.12.8.6:5240 because no single host has repo +
CLI + new-region reach + NetBox apex reach. Application-layer and
rack-originated, so SEC-010 is not punctured; nothing was rerouted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

GA-R4 session close: Stage 5 opened, D-138 + D-132 ruled, dc0 region live
...
Sweep first: every in-session finding grepped against CURRENT-STATE and the
changelog. THREE lived only in the transcript and would have been lost -- the
absent openstack CLI on the DC client host (blocks Step 7+), the metal-admin
v6 rack-leg gap, and the plane-fabric mtu=1500 vs libvirt 9000 mismatch. All
now in docs/audit/queued-findings-20260730-stage5.txt (F1-F12), with six tool
traps collected as platform-traps candidates.
F6 recorded honestly: the as-executed log for this window is PARTIAL. The
classifier refused several `script -aqe ... -c` wrapped forms while the plain
ssh form passed, so those calls ran unwrapped. The index row says so -- a log
that silently looks complete is worse than one declaring its gap.
Bookend appended (14 lines, under the GA-R4 cap; ledger 250/300 so no
rotation owed). Counters: 3 decisions, SEC 23, D 139 / DOCFIX 206 /
BUNDLEFIX 053.
GA-R7 memory review: one entry added (as-executed vs classifier), indexed.
Nothing in memory claims operator policy.
Stage 5 remains OPEN. Nothing is half-applied -- Office1 still owns all 9 dc0
nodes, correctly carved and Ready.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Region images synced (jammy+noble); pre-migration carve captured
...
Both releases now Synced on the dc0 region, 6 rows each, matching Office1.
jammy is NOT seeded by `maas init` (noble only) and the OpenStack nodes
require it. The first import raced the selection's creation and returned
noble-only while reporting is-importing=false -- assert on Synced ROWS, never
on the import call returning.
Measured and recorded so the two are not conflated: MAAS boot images do NOT
come from the DC mirror. The region pulls images.maas.io; the mirror 404s on
every simplestreams index path because dc-mirror.sh mirrors debmirror APT
packages only. Three artifact classes, one local. Consequence: the
D-135/D-107 egress narrowing would BREAK image sync and juju bootstrap, not
just slow them, and nothing sequences it after them.
Pre-migration carve captured to docs/audit (179 lines): fabrics, VLANs,
spaces, subnets, D-134 bands, tags, per-node identity/power and the full
per-interface carve. MAAS cannot move machines between regions, so the
migration must recreate all of it.
The destructive half has NOT started -- the Office1 region still owns all 9
dc0 nodes, correctly carved and Ready. Nothing is half-applied.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

dc0 MAAS region LIVE; three creds minted, consolidated and REGISTERED (SEC-027)
...
Region: noble + MAAS 3.7.2 + PostgreSQL 16.14 snaps, byte-identical versions
to the Office1 region (checked, not assumed). regiond/rackd/apiserver ACTIVE,
dhcpd INACTIVE so there is no DHCP conflict with the Office1 rack still
serving that VLAN. API answers 200 on 10.12.8.6:5240 -- the address jujud
could not reach at Office1, which is the whole point of D-132's amendment.
Carve verified BOTH legs (the lesson of today): metal-admin 10.12.8.6 + v6
::6, provider-public 10.12.4.6 with gw 10.12.4.1.
Credentials, per operator direction: DB password, region admin password and
admin API key minted ON the VM, never printed, consolidated to
~/vr1-dc0-creds/ at 0600 with sha256 digests compared to prove the copies are
identical, API key format-verified parts=3. SEC-027 opened (open SEC 22->23),
recording the residual argv exposure honestly -- `maas init` has no stdin or
file option for --database-uri.
The creds MATRIX now expects them at BOTH DCs: 5 rows per DC, a new `region`
host-role token in the enum, 4 vm-secret-locations rows (definition-of-done
for a new mint site), dc0's manifest declares its three. dc1's three now FAIL
S2 BY DESIGN -- the forward register making dc1's absence detectable, which is
what D-137 exists for. Harness 65/65 PASS, repo-lint 0 fail.
Also: durable ssh aliases for DC nodes. Deployed nodes carry the
MAAS-registered vr1-office1-svc key, not the rack service key, so each hop
needs its own identity -- a bare `ssh -i <key> -J voffice1,<rack>` fails
because -i forces one key on every hop. No credential was copied anywhere.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

dc0 MAAS region VM APPLIED, MACs pinned, converged; commissioning trap recorded
...
Capacity gate re-measured PASS (RAM 870/1024 = 85%, FIT, 154 GiB headroom).
Plan asserted on content -- exactly two resources, both the new VM. Applied
2 added / 0 changed / 0 destroyed; MACs measured and pinned; converged ZERO
DIFF with live virsh domiflist byte-identical to the pin; virsh power set.
Enlisted as hot-kid / tw7ptw.
Corrected against measurement: the "1 add" figure in both new comments and in
D-104's amendment prose is wrong at resource level -- node-vm creates a
domain AND a volume, so it is 2 to add. The half that mattered, 0 change /
0 destroy, held.
Ordering trap measured and recorded as a standing rule: the first apply sets
running=true, so the VM PXE-enlists and auto-commissions within ~90s, and
pinning MACs is an in-place libvirt_domain update that BOUNCES the guest --
stalling that commission at "Loading ephemeral". After pinning MACs on a
freshly-applied VM, assume commissioning was interrupted and re-commission
deliberately. Recovery is abort -> New -> commission.
dc1's region VM is authored but NOT applied.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Pin vr1-dc0-maas-01 MACs from measurement, pre-enlistment
...
Measured with virsh domiflist on vvr1-dc0 in the same session as the first
apply, before the VM enlists -- the boundary modules/node-vm warns about and
the one the 2026-07-20 MAC regeneration made expensive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Author the per-DC MAAS region VM into both substrate roots (D-132 addendum)
...
vr1-dc0-maas-01 and vr1-dc1-maas-01: 4 vCPU / 8 GiB / 150 GiB, macs = []
for first apply. Additive on a for_each-keyed caller, so each plans
1 add / 0 change / 0 destroy.
Sizing is the shape the capacity gate actually modelled
(dc-dc-whole-host-budget.py --containment-overhead-mem-gib 32
--containment-overhead-vcpu 12 -> RAM 870/1024 = 85 pct, FIT, 154 GiB
headroom). Disk is 150 not 100 because this VM stores the region's
PostgreSQL AND its boot-image set.
Both entries carry the carve requirement in the comment: metal-admin AND
provider-public at enlistment. Carving metal-admin alone is exactly the
under-carve that cost three bootstrap attempts on the juju controller VM
today -- no default route, no image sync.
MACs deliberately empty, to be pinned from measurement immediately after
first apply, per the same rule that the 2026-07-20 MAC regeneration
established.
opentofu-validate: root + 12 modules + both extra roots all PASS.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
D-132 placement RULED: per-DC MAAS region VM at utility .6; D-134 map extended
...
GA-R5. Operator utterance: "Dedicated region VM per DC at utility .6
(Recommended)".
The region DB must not share fate with the hypervisor running every node it
manages -- also a precondition for D-132 q3 cross-site backup custody, which
stays pinned. The D-134 standing octet map now reads .4 artifact service /
.5 juju controller / .6 MAAS region, binding at every future DC standup.
Build notes carried, not ruled: author BOTH metal-admin and provider-public
legs at creation (the identical under-carve on the juju controller VM cost
three bootstrap attempts today); re-measure capacity before applying.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

D-132 q1 RULED for VR1: a MAAS region controller per DC; Stage 5 blocked
...
GA-R5, before dependent work. Operator utterance: "Put a MAAS region
controller in each DC".
Third bootstrap got PAST the agent fetch (downloaded attempt 1 -- the
controller carve closed that failure class) and failed at the next layer:
jujud, running ON the node, cannot reach the MAAS region API at
10.10.0.20:5240. Measured: rack reaches it (200, rack-originated, permitted
by SEC-010); the node cannot (forwarded, blocked); and the rack does NOT
proxy it -- nginx listens on 5248 only. All six required flows enumerated;
exactly one crosses the boundary.
The finding underneath: SEC-010 as written is incompatible with the
deployment's own control-plane topology (D-104 controller in-DC + MAAS
region at Office1). D-138 was necessary but NOT sufficient -- it moved the
cloud-facing client into the DC, and jujud is itself a provider client whose
MAAS dependency I did not enumerate when scoping it.
Per-DC regions remove the requirement rather than excepting it, so SEC-010
is PRESERVED UNAMENDED. Single, not HA (utterance is singular); D-132 q2/q3
stay pinned to the next deployment.
Consequence stated plainly: this reopens the MAAS layer of Stages 3 and 4,
both closed and merged. A region owns its own DB; nothing migrates between
regions. Open sub-question blocking the build: where each DC's region lives.
Also landed: both controllers carved symmetric with gw-bearing
provider-public legs. Trap recorded -- MAAS `interface update vlan=`
returned a full JSON object while changing nothing on a Deployed machine.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

G17 dc0 CAPTURED (both assertions PASS); bootstrap root cause = under-carved controller
...
G17's one-shot first-boot window opened during the second bootstrap attempt and
was taken rather than lost. Capture docs/audit/g17-dc0-firstboot-20260730.txt,
exit 0: mirror asserted on FOUR body fields of a real package path (not the
autoindex root), and the node time source is the MAAS rack 10.12.8.2 chaining
to the region, not the DC edge. Scope stated: the node is the dc0 CONTROLLER
VM, not a role node; G17's dc1 half stays OPEN.
Two defects in G17's own text, both measured: it names `chronyc sources` and
chrony is NOT installed on the MAAS jammy image (node runs systemd-timesyncd),
so the gate as written can only REFUSE; and a first pass false-FAILED on
`printf | grep -q` under pipefail -- the same trap recorded earlier the same
day. Capture uses here-strings.
Root cause of both bootstrap failures is measured and is NOT architectural:
the controller VMs are UNDER-CARVED. D-138's path works -- the second attempt
reached "Connected to 10.12.8.5" -- then failed fetching the juju agent
because the node has NO DEFAULT ROUTE. Role nodes carry provider-public with
gw 10.12.4.1; the controllers carry metal-admin only, because they were added
after both carves and Stage 4's 90 NIC re-homes covered the 18 role nodes.
Owned: this session's egress re-probe was run FROM THE RACK and reported the
window open. The rack holds a default route nodes lack, so it did not
discharge the caveat for a node-side fetch. Same wrong-host class as the
dc-mirror instrument note, made after writing that note.
Blocked on a harness permission wall for four MAAS interface calls, not on a
decision. Not retried in altered shapes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

D-138 ADOPTED: the cloud-facing client lives IN the DC; SEC-026 opened
...
GA-R5, before dependent work. Operator utterance: "Move the cloud-facing
client into the DC (Recommended)".
Resolves the contradiction the Stage-5 bootstrap failure exposed: SEC-010 +
D-052 make metal-admin DC-local and forbid the route juju needs, while D-100
names Juju as fiber traffic and D-128 puts the client on voffice1. SEC-010's
"pinning is free" was priced against tools that proxy at the app layer; juju
dials the machine at L3 and was not in scope.
The client moves; the boundary does not. SEC-010, D-052 and D-125 are
UNCHANGED -- nothing punctured, no plane opened. D-128 is amended to exclude
cloud-facing tools. Scope was set by enumeration first: keystone's VIP is on
provider-public, so routing would have opened two planes per DC across a
dozen ports.
SEC-026 opened for the consequence: a MAAS admin-scoped key becomes resident
on a DC-local host, over a region shared by both DCs. Isolation is the
control -- each DC's client host gets ONLY its own credential.
Counters: 22 open SEC, next-free D 139.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Stage 5 BLOCKED: no juju-client->node-plane path; D-134 .5 now BUILT both DCs
...
Bootstrap attempted and failed. Machine selection was correct (7n87bt, the
tagged VM) and MAAS deployed jammy end to end -- confirming the
--bootstrap-base ubuntu@22.04 pin against a live deploy. juju then could not
SSH the machine and released it.
Root cause: voffice1 has NO route to any DC node plane and two deliberate
controls forbid one -- SEC-010's transit FORWARD-drop on the rack, and
libvirt's blanket reject into the isolated plane bridges. The DC edge has no
metal-admin leg. Nothing regressed; this path never existed.
This is a contradiction between RULED surfaces: SEC-010/D-052 make
metal-admin DC-local and forbid the region routing to 10.12.8.0/22, while
D-100 says the fiber carries Juju traffic and D-128 puts the juju client on
voffice1. SEC-010's "pinning is free" justification was priced against MAAS,
which proxies at the app layer; juju dials the machine at L3 and was not in
scope. Needs a ruling, not a firewall edit. No reachability change made.
BUILT, both DCs (operator: "Fix now: static .5 + v6, re-bootstrap"): the
controller VMs held AUTO v4-only addresses; now static 10.12.8.5 +
fd50:840e:74e2:220::5 and 10.12.68.5 + fd50:840e:74e2:320::5. v6 prefixes
confirmed by VLAN pairing, not inferred. D-134's amendment was
ruled-but-not-built until now.
Capture: docs/audit/stage5-bootstrap-reachability-20260730.txt
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
Stage 5: Step 1 + 2.0 gates passed; bootstrap constraint-flag ruled
...
GA-R5, before dependent work. Operator utterance: "Use both flags".
juju 3.6.27's own help assigns machine-targeting to --bootstrap-constraints
and model-defaulting to --constraints; the runbook and D-104's mechanism
sentence both name only --constraints. D-104 is NOT amended -- only the flag
implementing it is clarified. DOCFIX owed, recorded in the changelog.
Read-only gates, all on voffice1: selectors exit 0 (6 planes, 10 hosts);
credential gate outcome 1 of 3 so the mint is skipped (SEC-018 sprawl);
exactly one machine carries juju-controller-vr1-dc0, Ready.
Artifact sources PASS both DCs; egress re-probed from both racks with
--noproxy '*' -- bootstrap window OPEN. Instrument note recorded: the DC
checkers run ON the rack host and read all-MISS when run from voffice1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

STAGE 5 OPEN: P5 accepted by ruling, entry gate captured on voffice1
...
Opens Stage 5 / Phase 4 (Juju controller + OpenStack bundle, per DC) on the
standing operator directive. GA-R1/C1: the status change and CURRENT-STATE
land in one commit. GA-R5: the P5 ruling is committed and pushed BEFORE any
dependent work.
Entry gate, measured ON voffice1 (the only host whose reading counts):
P1/P2/P3/P4 PASS, P7 PASS 37/0 with the literal zone line, P5 FAIL on 6
pre-existing credential-register findings. Capture
docs/audit/stage5-preflight-dc0-20260730.txt.
Operator ruling, exact utterance: "Accept and proceed to deploy (Recommended)".
The acceptance covers those six findings, enumerated, and nothing else.
Logged not executed (hard rule 1): bundle.yaml:592 gives ceph-osd the stale
VR0 constraint tags=openstack, a tag measured absent from the VR1 region.
Decided at Step 4.2's dry-run, where its impact is first observable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

GA-R4 session close: F9 closed live, the reissue tool, P7, lib-identity
...
BOOKEND (bounded, 15 lines) appended to docs/session-ledger.md. Machine-derived
block RE-SEEDED from ledger-scan (DOCFIX next-free 205 -> 206). Ledger stood at
exactly 300 lines after the append, so the two oldest live summaries (the
2026-07-27 grounding-audit close and its 14-rulings addendum) rotated VERBATIM to
docs/archive/session-ledger-rotated-20260730.md: 300 -> 235. Rotated HERE rather
than left owed, because the next session is directed straight at the deploy.
STANDING OPERATOR DIRECTIVE recorded at the TOP of CURRENT-STATE section 1, where
the next session's bookend hits it first: 'we have to continue to juju deployment
next session no matter what'. Precondition work is DONE and further hardening is
OUT OF SCOPE unless it blocks the deploy. This is the GA-F06 circuit-breaker made
explicit -- the directive removes the judgement call.
Deliberately NOT put in auto-memory: GA-R7 rule 1 forbids operator policy,
priorities and postures there. It belongs on a repo surface, so it is on the
status authority.
Entry facts recorded so the next session does not re-derive them: run FROM
voffice1; preflight FAILs on P5 ONLY (pre-existing credential register -- decide
accept-or-remediate AT the gate, do not re-audit); every other gate green and P7
[ok] both DCs; the deploy input is bundle + that DC's three overlays; G17 arms at
first boot and is a Stage-5 artefact, not a precondition.
GA-R7 memory review done: two memories, both observations/conventions, both still
accurate, neither claiming operator intent. No changes needed.
Session totals: 19 commits. F9 CLOSED live on both DCs; octavia-pki 21/21 -> 51/51;
preflight 26/26 -> 33/33; gauntlet ALL GREEN (89) both hosts; repo-lint 0 fail;
scan unchanged at 3 decisions / SEC 21.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

lib-identity.sh: the estate's name and DNS base in ONE place
...
Operator: 'The specific server and region names do not matter as they change
during every deployment and hardening of the deployment workflow.' Then, after
scoping the two options: 'Yes, take option 2.'
WITHDREW MY OWN FIRST RECOMMENDATION. I proposed retiring the vr0-dc0 selector
as removing 'the ambiguity class outright'. Measured, wrong on both halves:
D-119 already closed it by region-qualifying (the hazard was the BARE dc0, which
is rejected loudly today); lib-net.sh:110 documents that arm as a NO-OP over the
file's flat defaults; and the footprint is 43 files, mostly the separate VR0
phase-NN track the VR1 runbooks cite as precedent. Medium cost, near-zero
benefit. Logged, not executed.
NEW scripts/lib-identity.sh -- CLOUD_NAME + CLOUD_DOMAIN, no side effects,
env-overridable. With <dc> and <region> already derived from the site token,
these were the LAST typed identity in the shell surface. A rebuild that renames
the estate now edits one file and the certs, zones, SANs and P7 all follow.
DELIBERATELY NOT lib-net.sh: sourcing that bare populates a full flat plane/VIP
namespace (its own header says so). A certificate checker needs two strings, not
a network namespace, and octavia-pki.sh sources NOTHING today -- pulling in
VR0-shaped defaults it never asked for is the R9 hazard in miniature. Sourced
from SCRIPT_DIR (sibling, survives the harness REPO override) and FAILS CLOSED
with REFUSE 3 if absent: a cert gate that invents the estate's identity is worse
than none.
Consumers: octavia-pki.sh derive_zone(), phase-01 1.0-GEN.c, dc-dc-phase6 Step 0.
The tofu side was ALREADY parameterised (opentofu/variables.tf domain_suffix
passed explicitly into modules/dc-planes), so nothing there changed.
HARNESS 48/48 -> 51/51:
- T45 the centralisation must be REAL, not decorative -- overrides both values
and requires the zone to follow BOTH (acme.dc0.vr1.example.test). A constant
nothing can vary is indistinguishable from a literal.
- T46 a missing lib-identity.sh REFUSES rather than guessing.
- T47 SHELL vs OPENTOFU drift. HCL cannot source a shell file, so two copies of
one fact exist; if they part, the libvirt plane domains and the cert zones
describe different estates and nothing else would notice. PROVEN able to fail
by injecting a drifted value, then restored.
Also: repo-lint L6 flagged '. $REPO/scripts/...' as a bare invocation. Its own
docstring blesses a bash|source|python3 prefix and sourcing needs no exec bit,
so the RUNBOOK moved to the documented 'source' form -- the gate was not changed
for style. And ${REPO:?} is now guarded at its FIRST use in GEN.c, not only at
:538 below it.
STILL typed, deliberately: the site allowlists. One that accepts anything is how
a typo'd DC name becomes a wrong-target write.
gauntlet ALL GREEN (89); repo-lint 0 fail; GEN.c bash -n clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

P7: the Octavia amphora PKI becomes an actual pre-deploy gate
...
Operator: 'Yes, I accept the recommendation. Convert to a gate.'
THE GAP: octavia-pki.sh was invoked by NOTHING executable. P4's CHECK 0 asserts
only that the overlay FILE EXISTS, and the harness runs against throwaway
fixtures -- so 'gauntlet ALL GREEN' said nothing about the live PKI and the only
real check ran when a human remembered to type it. That is how F9 survived.
Three problems solved, each from a measured precedent:
1. rc 3 must not reach note()'s *) arm. octavia-pki.sh uses exit 3 for REFUSE,
but note() calls any rc>=3 'UNEXPECTED exit code'. P7 maps rc explicitly, so
a REFUSE on the headend FAILS and is NAMED as an unevaluated trust domain.
2. HOST AUTHORITY. The PKI lives on the headend (D-109 (b)); elsewhere P7 says
NOT EVALUATED (WARN) -- never a silent pass, never a hard FAIL for being on
the jumphost. The binding is read from the SAME host-identity file
octavia-pki.sh reads, so they cannot drift. P5's F6 defect designed out.
3. A PASS VERDICT IS NOT SUFFICIENT. While os-public-hostname is unset, A12/A13
RECORD a wrong-zone cert rather than failing it, so verify exits 0 on exactly
the defect this gate exists to catch. P7 also requires the literal line
'DNS SANs are all in this DC.s expected zone'. Satisfiable by construction
now that GEN.c derives the zone and both DCs were reissued.
Also skips cleanly for vr0-dc0 (a gate that fires on a target it does not apply
to is how gates get bypassed) and FAILS CLOSED if the checker is absent.
HARNESS 26/26 -> 33/33. Ten existing cases went RED when P7 landed -- correctly,
the fixtures had no octavia-pki.sh. Fixed by making mkfix model a healthy headend,
NOT by patching cases one by one: that is the moment a gate gets demoted to a
warning to go green. T27 is the case that matters -- a wrong-zone cert on which
verify exits 0 must still FAIL P7.
Two harness bugs of my own, caught by running not reading: a stub emitted DC.s
where the real line has an apostrophe, then \x27\x27\x27 produced three of them
because \x27 in a printf FORMAT already yields one. Fixed the stub, not the gate.
Preflight's overall verdict is still FAIL -- that is P5 working as designed on
the pre-existing red register. P7 itself passes on the headend.
gauntlet ALL GREEN (89); repo-lint 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

F8 + F9 fixed AT SOURCE in 1.0-GEN.c -- the next DC standup will not recreate them
...
Everything before this repaired the two EXISTING DCs. The GENERATION path still
carried both defects, so the next DC standup would have minted them again.
F9 at source: GEN.c baked omega.dc0.vr0 as a literal in THREE places (CN + both
DNS SANs). It now DERIVES DC_ZONE from $DC with the same two expansions the tool
uses (${DC%%-*} / ${DC#*-}), so runbook and gate cannot disagree, and echoes the
zone for confirmation before the sign.
F8 at source: the cat > controller.cnf heredoc is replaced by one printf per
line, values passed as %s ARGUMENTS. The 2026-07-29 note deferred this as 'an
untested rewrite ... reasonable when someone can run a real generation'. That
condition is now MET: the identical shape was exercised end to end by two real
mints (both DCs) plus 48 harness cases.
Plus structural assertions BEFORE signing -- four sections present, subjectAltName
wired, CN equals the derived zone, exactly 2 DNS entries -- because printf removes
the paste hazard but not the failure CLASS: a typo'd section name still yields a
SAN-less cert behind a wall of OK output.
The stale 'NOT changed -- outside R7's ruled scope' note is superseded in place,
keeping the reasoning worth carrying: R7 recorded this cert's SAN as 'already
DERIVED per-DC by design', true of the IP SAN and NOT the DNS names -- a claim
accurate about one half of a field, read as covering both. That is how F9 survived.
F10 HANDLED DELIBERATELY. Editing GEN.c shifts every mint-ref anchored below it.
All 13 octavia anchors re-resolved BY MARKER in a SINGLE PASS keyed by row id --
never sequential seds, because a line number can be simultaneously an old value
for one row and a new value for another -- then each verified to point at its
correct command. 12 rows rewritten; creds-matrix S4 CLEAN; block bash -n checked.
octavia-pki 48/48; creds-matrix 65/65; creds-matrix 101 rows / 5 findings, all
pre-existing; gauntlet ALL GREEN (89); repo-lint 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Q2 withdrawn: D-137 fork 1 was ruled 2026-07-25, not open
...
Third stale-premise item in queued-findings-20260730.txt (after Q1). Q2 opens
'D-137 OPEN FORK 1 (enforcement strength) -- still unruled'. Sub-ruling 1 ruled
it 2026-07-25 at option (b), utterance 'Blocking in preflight', and its
CONSEQUENCE block explicitly declines option (c): 'NOT adopted: the PreToolUse
guard (so an agent is not constrained at write time, only at the deploy gate)'.
So guard-destructive.py is NOT D-137's enforcement point -- it is the older
CLAUDE.md/DOCFIX-006 mechanism, and hardening it needs no ruling to proceed.
This was verified BEFORE the reissue minter was built, which is why 'Full script
minter now' was consistent with the ruling rather than a side-effect decision.
The misfire evidence stays valuable and the tally is now SEVEN. The two newest
were collected today: the guard blocked a command whose only purpose was to TEST
it (it read nothing), and then blocked the heredoc writing this very correction
into the audit file -- documentation prose, no file read. Both are the same class
as the earlier five: the matcher cannot distinguish reading secret material from
merely naming it.
Correction APPENDED, not rewritten (the stage4-mirror-gate precedent).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Reissue backup custody: transferred to vcloud, and REGISTERED
...
Operator: 'Update the matrix as needed. Transfer the artifacts to the vcloud.
keep the pin that during the secrets workflow planning that the creds and certs
from this step need to be included.'
CUSTODY (Step 1.0-REISSUE.4, executed): both pre-reissue archives pulled from the
headend to the jumphost SEC-009 creds folders, sha256 compared and IDENTICAL at
both ends (e30cab7f...3487 dc0, 17c831da...6fe3 dc1), 0600, each listing 15
entries including the deploy overlay and the CA serial. Headend staging copies
removed; ~/octavia-pki/backups/ gone; no .reissue-* residue. The sha256 compare
happens BEFORE the delete because a truncated scp would otherwise leave a
verified-looking backup of nothing.
REGISTER: two octavia-reissue-backup rows (per-DC, jumphost, consolidated,
templated filename <site>-octavia-pki-<stamp>.tar.gz), the DERIVED manifests
updated to match (D-137 ruling 2 -- manifests are generated, not hand-authored),
and a new n-reissue-backup note. It is deliberately a SEPARATE id from
n-pki-backup: that one is the generation-time workspace archive, this one is
per-ROTATION and additionally carries the deploy overlay, which the generation
archive does not -- restoring only the workspace would leave the half that
actually reaches the charm unrecoverable.
Measured after: tier1 101 rows / SAME 5 pre-existing findings; tier2 the same 7,
with NO reissue-attributable finding -- and tier2 FOUND both archives at their
declared location, which is the point of registering them. Harness 65/65.
HAZARD RECORDED: these archives contain controller-ca.cert.srl, the CA's issuance
state. Restoring one over a workspace that has issued since rolls the serial
BACKWARDS and the next mint reuses a serial the estate already holds. Serials
burned 2026-07-30: ...250E (dc0), ...674DE (dc1).
PINNED: the note carries the operator's standing requirement that the creds and
certs from this step be included in the secrets-storage workflow planning, with
the full scope enumerated. The jumphost creds folder is the INTERIM home only.
gauntlet ALL GREEN (89); repo-lint 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

F9 CLOSED: controller certs reissued into their own zones, both DCs
...
Executed on voffice1 via Step 1.0-REISSUE at HEAD 7b6a2e4. Operator-gated PER DC
('Go ahead' each), never batched; dc0 verified clean before dc1 was touched.
Capture: docs/audit/octavia-reissue-executed-20260730.txt
vr1-dc0 -> omega.dc0.vr1.cloud.neumatrix.local, IP 10.12.4.57 + f02:11::57,
serial ...250D -> ...250E
vr1-dc1 -> omega.dc1.vr1.cloud.neumatrix.local, IP 10.12.64.57 + f03:11::57,
serial ...674DD -> ...674DE
dc1's OLD cert carried dc0's LABEL as well as the wrong region -- the cross-DC
case A11 exists for. A11 confirms per-DC independence on both DCs after the mint.
Both: fresh EC P-256 key with the rotation PROVEN (public key differs from the
outgoing cert's), neither CA regenerated so the amphora trust domain is intact,
overlay surgery measured other_values_identical=4 changed_lines=1 line_count=8.
ACCEPTANCE was the literal positive line, not the verdict: while os-public-hostname
is unset, A12/A13 RECORD rather than fail, so PASS alone is compatible with a wrong
certificate. Both DCs now carry 'DNS SANs are all in this DC's expected zone'.
verify PASS 37/0 both DCs (was 29/0; +8 from the new assertions). creds-matrix:
same 5 pre-existing findings, no new finding, no S4 drift. No staging residue,
neither run reached exit 5.
The hardened re-run guard proved itself live: re-running dc0 immediately after
returned exit 4 'already correct ... and verify returns clean' -- the fix for the
defect that had it refusing to repair certs verify was failing.
STILL OWED, operator-gated: backup custody. Two archives on the headend hold BOTH
CA private keys, BOTH passphrases and the overlay; 1.0-REISSUE.4 pulls them to the
jumphost creds folder and removes the headend copy. They are also an unregistered
credential class (no creds-matrix row, no retention rule). Logged, not actioned.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

octavia-pki: a hardened, tool-driven controller-cert REISSUE path (F9's remedy)
...
Operator: 'Let's reissue now. Engineer the artifact, variable, based workflow to
reduce the chance of errors.' Five GA-R5 rulings, one exchange each, quoted in
the changelog; ruling 5 was 'Full script minter now'.
THE TOOL IS BUILT; THE MINT IS NOT YET RUN (gated operator action, 1.0-REISSUE).
scripts/octavia-pki.sh gains 'reissue <site> [--force] [--dry-run]': controller
LEAF only, signed by the EXISTING controller CA (neither CA regenerated, amphora
trust domain untouched), fresh P-256 key, zone DERIVED from the site token and
never typed, printf-built config (the F8 payoff), stage-assert-promote, and a
single-value overlay surgery that never puts the bundle in argv.
SIX GATE DEFECTS FOUND BY VALIDATION AGENTS, ALL CLOSED. Mutation testing is
what earned this: deleting the three assertions I had just written left the
harness fully GREEN every time, so they were decoration until each got a
failing-direction fixture.
- A15 keyUsage/EKU on the ISSUED cert. A cert with NEITHER minted, promoted and
passed everything. keyUsage appeared once in the file: the printf writing it.
- A16 validity. openssl x509 -req defaults to -days 30, and checkend/notAfter/
enddate appeared ZERO times in verify -- a 30-day cert passed forever.
- A17 the overlay must decode to what the workspace holds. A10 graded shape,
A1-A16 graded the workspace, nothing joined them; a desynced overlay read
PASS 0-failed and the overlay is what reaches the charm.
- The re-run guard REFUSED to fix broken certs: names-correct + stale IP SAN
gave verify FAIL and reissue exit 4 'nothing to fix'. Now gated on verify.
- A8's negative proves its instrument loads first (openssl verify exits non-zero
for 'could not load the CA' too, so a corrupt CA passed it vacuously).
- Two pre-existing overlay defects moved pre-mint, so they stop producing a
false exit-5 with the workspace already promoted.
Harness 23/23 -> 48/48. Runbook Step 1.0-REISSUE appended AFTER GEN.e
deliberately (F10 line-anchored mint-refs; creds-matrix reports no S4 findings
after the append). Skill gains two standing invariants for future DC standups:
every per-DC secret needs a ROTATION tool not just a generation recipe -- a
REFUSE-IF-PRESENT generation gate makes the dangerous path the only path -- and
a hand-pasted openssl chain is a defect class, with F8 and F9 as the receipts.
Also corrected: D-137 fork 1 is NOT open (sub-ruling 1, 2026-07-25, ruled
'Blocking in preflight' and explicitly declined the PreToolUse guard), so the
queued Q2 premise is stale in the same way Q1 was.
gauntlet ALL GREEN (89); repo-lint 0 fail; creds-matrix 5 findings, all
pre-existing and named; ledger-scan unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

DOCFIX-205 sweep: correct the artifact that filed Q1, and one stale residue
...
Advisor review caught that correcting CURRENT-STATE and the ledger was not
enough: docs/audit/queued-findings-20260730.txt -- the sweep capture the 07-30
bookend points at -- still presented Q1 as an open ruling that 'blocks any FQDN
cert', enforced by an A12 refusal that no longer exists. A next session reading
it would re-derive Q1 exactly as this one did, reopening the loop just closed.
- queued-findings-20260730.txt: APPENDED correction (not a rewrite; the
stage4-mirror-gate-20260727.txt precedent), naming the two claims that are now
false and pointing at the evidence. Q2 explicitly untouched.
- dc-dc-deployment-workflow.md:257: described phase-6 as proposing the retired
dc1/dc2-hostnames.yaml spelling. MEASURED: the runbook itself was already
corrected to the region-qualified form by an earlier session -- only the
workflow doc's description of it lagged. Under D-119 IS vr1-dc0, so the
-hostnames.yaml interpolation used elsewhere is correct as written.
- The DOCFIX-205 capture's section 1 quoted a 3-line span starting mid-sentence
and left a clause orphaned. Repaired to two verbatim quotes, both re-verified
present in design-decisions.md. CURRENT-STATE cites this file as evidence and
this repo's discipline is exact captures.
octavia-pki 23/23; repo-lint 0 fail (L10 gated this commit correctly -- the
docs/audit/ change set required CURRENT-STATE to move with it).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
L10 repair: cite the DOCFIX-205 capture in CURRENT-STATE
...
The preceding commit added docs/audit/docfix205-d117-annotation-20260730.txt
without touching CURRENT-STATE.md and FAILED repo-lint L10 (GA-R1 rule 8 / C1).
It was pushed anyway because the lint was chained with ';' rather than '&&', so
a red gate did not stop the commit -- the same silenced-error class logged in
the 2026-07-29 close. Recorded here rather than force-pushed over.
NOTE for the next session: repo-lint exits 2 on WARN, so a bare '&&' chain also
fails on the EXPECTED legacy L1 warn. Gate on '(0 fail,' in the output, not on
the exit code -- that is why ';' was reached for in the first place.
CURRENT-STATE's superseding block now cites the capture; tree is lint-clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|