| 2026-07-30 |

lib-hosts: the node carve is ROLE-DEPENDENT, not uniform
...
Measured from the pre-migration capture. Role nodes carve all six planes
plus an OVS br-ex parented on enp2s0 carrying the provider-public static,
with enp2s0 itself holding no L3 link. The juju controller carves only TWO
planes -- enp1s0 metal-admin and enp2s0 provider-public as a raw NIC WITH
the gateway -- has no br-ex, and leaves enp3s0..enp6s0 on their
auto-created VLANs.
Evidence: 7n87bt (controller) reads enp3-6 on vlans 5200-5203, which are
auto fabrics; 8mtpxq (storage-02) reads enp3-6 on 5190-5193, the named
planes, plus br-ex.
This matters because assuming uniformity fails in both directions: it would
either strand the controller without a default route -- the under-carve that
cost three bootstrap attempts on 2026-07-30 -- or give it four legs it must
not have. The node carve has no repo tool yet, so this was about to be
re-derived by hand.
dc-selector 95 checks; the depths-must-differ guard proven able to fail, and
the controller-presence assertion checked inside a per-DC subshell because
the main shell still holds the flat VR0 fleet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

lib-hosts: record the MEASURED VR1 node NIC -> plane order
...
The substrate roots pin six MACs per node and the module attaches them in
list order, so macs[i] is enp<i+1>s0. Verified end to end on
vr1-dc0-control-01 against the live carve: enp1s0 metal-admin (PXE),
enp2s0 provider-public (br-ex parent), enp3s0 metal-internal, enp4s0
data-tenant, enp5s0 storage, enp6s0 replication.
This is NOT lib-net.sh's PLANE_CIDRS order, which starts with
provider-public. Anything walking PLANE_CIDRS positionally to place NICs
swaps enp1s0 and enp2s0 and puts the PXE leg on provider-public, stranding
commissioning on a plane with no DHCP. The node carve has no repo tool yet,
so this ordering was about to be re-derived by hand; recording it measured,
with a guard that the two orders differ, is the cheap half of that tool.
dc-selector 88 checks; the trap guard proven able to fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Per-region power address, region power-key tool, and two red gates fixed
...
lib-hosts.sh: MAAS power ops originate from the REGION, so the reachable
rack address depends on which region dials. Measured from vr1-dc0-maas-01:
the rack's transit leg 172.31.0.2:22 is CLOSED, its metal-admin leg
10.12.8.2:22 is OPEN. Adds VIRSH_POWER_ADDRESS_FROM_{OFFICE1,DCREGION} for
both DCs (dc1's 10.12.68.2 measured on the dc1 rack, not inferred from
dc0's .2). VIRSH_POWER_ADDRESS still aliases the Office1 form so existing
callers are byte-for-byte unchanged. dc-selector 81 checks; 4 mutations
each killed tests.
New scripts/maas-region-power-key.sh (harness 44/44) installs and verifies
the per-DC MAAS->libvirt key inside a region's snap, ending on a real virsh
call rather than on config. Runbook prerequisite 5 carried this as prose,
and prose is not a gate -- it must also be re-asserted after every snap
refresh. LIVE at dc0: 9/9 exit 0, virsh enumerates 12 domains. The key is
the SEC-012 dc0 key already authorized on that rack, so per-DC isolation
holds.
Three defects its own testing found: ssh-keygen -lf reads an adjacent .pub
and reports ITS fingerprint; a snap's data root (/var/snap) and exec root
(/snap) are not interchangeable; the snap's virsh must run inside the snap
runtime or it fails on missing libraries.
tests/node-vm T8/T9 were RED at HEAD before any of this session's work
(proven by stashing and re-running): commits 086c827 + 447315f added the
region VMs, moving the dc0 root to 11 macs lists / 66 MAC literals against
assertions reading 10 / 60, and the gauntlet was not re-run. Re-pointed to
the new invariant, reason recorded in-file, re-proven able to fail.
HARNESS-MANIFEST re-recorded (91) for the two new harnesses.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
| 2026-07-29 |

P4 is DC-aware, 3.7 closed, and lib-net's dc1 arm populated
...
DC references in pre-flight-checks.sh went from 1 (a comment) to 45. CHECK 2/4 now iterate the
selected DC's own HOSTS and resolve system_ids by pinned boot MAC on VR1 -- the cause of 8 of
P4's 11 fatals, which were the gate probing VR0's openstack0..3. Role-node scoping reuses
maas-role-tags.sh's existing D-134 utility-band rule rather than a second copy; the D-104
controller at .5 is named and excluded with its reason, its MAAS record still asserted, and
deliberately not asserted Ready since it is legitimately Deployed after bootstrap.
The two VR0-inherited expectations were RETIRED, NOT DELETED: VID 103 (D-133) and the
metal-admin gateway pin (D-134 -- measured, 10.12.8.1/10.12.68.1 are held by nothing, so the
pin was the defect). A spurious gateway still warns and a missing provider-public gateway still
fails, so both checks kept their teeth.
Item 3.7's guards are not skips: each guarded branch asserts D-133's own invariant instead --
VID must be 0 on VR1, and the metal-internal link must be MAAS type == physical, which asserts
the flat carve and is falsifiable. An UNREADABLE VLAN now REFUSES rather than satisfying the
untagged branch: `.vlan.vid // 0` had rendered "missing" and "untagged" identically.
F5: DC is validated against the D-119 token set and refuses before any gate runs, is exported,
and appears in the header and verdict -- keeping the PREFLIGHT: <verdict> token contiguous,
which ten existing cases grep.
R9 residue: dc1 arm populated (KEYSTONE_VIP_DEFAULT=10.12.64.50). The D-133 guard HELD --
METAL_INTERNAL_VID/IFACE stay unset on both VR1 arms, verified independently here, not from
reported output. Two dc-selector assertions were replaced with the new invariant and explained
in-file, never deleted to go green. The arm is hand-authored with a drift check; R9's generator
half is still owed and is recorded as such.
Corrected from prediction to measurement, and it changes the remaining work: all four other
DC-dependent consumers carry set -u, so adding the selector ALONE to
phase-04-network-{create,verify}.sh makes them ABORT with PLANE_GW[...]: unbound variable on
dc1 (reproduced). The hardcoded PROVIDER_CIDR must be derived in the same edit.
Harnesses: pre-flight-checks 31/31 (new), preflight 26/26 (was 16), dc-selector 69 (was 45).
repo-lint 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Pin juju-01 MACs, add both controllers to lib-hosts at the ruled .5
...
Twelve measured MACs (six per DC, virsh domiflist) pinned into both
substrate roots; <dc>-juju-01 added to lib-hosts.sh with HOST_OCTET=5 per
the D-134 amendment and its measured boot MAC. Done now because both VMs are
ENLISTED -- the boundary modules/node-vm warns about, past which an in-place
apply can regenerate unpinned MACs and strand the node, as happened to the
nine role nodes on 2026-07-21.
No race with the running agents: they operate on the voffice1 clone; this
edit is on vcloud's, and nothing reaches their tree until someone pulls
there.
Recorded rather than smoothed over: dc1's juju-01 MACs are libvirt-generated
52:54:00: values, not dc1's schematic 52:54:01:d1: scheme -- a direct
consequence of the deliberate macs = [] at first apply.
Three exact-count assertions re-pointed to the new invariant and kept EXACT
rather than relaxed: dc-selector fleet 9 -> 10 (label corrected, since Option
C is a 9-node ruling and the 10th is D-104's), node-vm T8 9 -> 10 and T9
54 -> 60.
A real harness defect surfaced doing it: T8's grep -c 'macs = [' counts
COMMENTS, and my own comment explaining the deliberate macs = [] inflated the
count to 11 -- which reads exactly like a spurious extra node. Fixed to strip
comment lines. Same class as the ledger-scan next-free defect where a doc
merely quoting a token inflated the counter.
maas-role-tags.sh taught the utility band: adding juju-01 to HOSTS made it
refuse, because .5 is in no ROLE band -- correct behaviour meeting a new
fact. Per the amendment a utility-band host with no role token is now SKIPPED
and REPORTED, while the refusal keeps its teeth for genuine disagreement.
New cases T10 and T11; harness 9 -> 11.
Gauntlet ALL GREEN (87) on vcloud; opentofu-validate PASS; repo-lint 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
| 2026-07-28 |
dc1 FIP pool RULED 10.12.65.0-10.12.67.254 (GA-R5); lib-net dc1 arm + assertions re-pointed
...
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|
| 2026-07-23 |
Delivery batch: lib-hosts VR1 arms, D-133 guards, appendix-A pf entry, DoD 8-9 (gauntlet 77 GREEN)
...
lib-hosts vr1-dc0/vr1-dc1 populated from the measured identity table
(tofu-name keys, pinned boot MACs, D-134 octets, per-DC power URIs +
tags) + host_sysid_by_bootmac (the VR1 resolver) + one-selection guard;
dc-selector rewritten, 48 PASS. reenroll-hosts + carve-host-interfaces
gain explicit vr1-* refusals (VR0 flows superseded by D-133 -- populated
arms must not enable them). lib-net vr1-dc0 VID caveat. Appendix-A:
edge pf-never-regenerated symptom (masking trap named). Phase-2 DoD:
items 8 (filter reload + LAN egress probe) + 9 (dc-mirror), item 4
range corrected to D-134 .201-.254. CURRENT-STATE same-commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
| 2026-07-22 |
G12 dc1: item-3 substrate landed + harness reconcile (gauntlet 76 ALL GREEN)
...
Lands the disconnected session's authored tree (vr1_dc1 vars, vvr1-dc1 +
uplink modules, inner root vr1-dc1-substrate/ with 54 pre-pinned MACs,
lib-net dc1 arm + one-selection guard) and delivers changelog item 4:
six harnesses reconciled to the ratified dc1 selector arm, phase-00
PLANES/lib-net parity guard (exit 2, refuses cross-DC planning),
rbd-mirror + radosgw parent-shell selection (dc1 reminder now shows
10.12.84.0/22, not dc0's). Gauntlet ALL GREEN 76
(docs/audit/gauntlet-20260722-g12-reconcile.txt); repo-lint 0-fail.
G12 next gated step unchanged: operator-gated apex --commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUjEb7onHpdqFHUnio6iVw
|
| 2026-07-14 |

D-119: region-qualify the VR1 DC namespace -- C3 / gap #19 CLOSED (Stage 3 unblocked)
...
THE BUG: the token `dc0` meant TWO DIFFERENT CLOUDS depending on the file --
VR0's LIVE testcloud in scripts/lib-net.sh, VR1's FIRST DC in the NetBox
importer. One string, two clouds, one of them in production.
THE RULING (operator): the repo adopts the apex's names verbatim.
vr0-dc0 VR0's DC0 the LIVE testcloud (a DIFFERENT region)
vr1-dc0 VR1's FIRST DC GUA 2602:f3e2:f02::/48
vr1-dc1 VR1's SECOND DC GUA 2602:f3e2:f03::/48
Bare dc0/dc1/dc2 are now REJECTED LOUDLY everywhere.
D-119 COMPLETES D-117, it does not reverse it: it executes D-117's own amendment
across the three surfaces D-117 never touched. ZERO NetBox writes -- the apex was
already correct and self-consistent (MEASURED: f02::/48 -> vr1-dc0, f03::/48 ->
vr1-dc1); the REPO was the only surface out of step. Renaming the apex to match
the repo was considered and REJECTED (production IPAM write; VR1 would become the
only 1-indexed region; and vr1-dc1 would mean VR1's FIRST DC while vr0-dc1 means
VR0's SECOND).
THE PRIZE: the importer's DC->site map is now an IDENTITY -- there is no offset
table left to get wrong, and an assert enforces it. The original defect was
precisely a WRONG LOOKUP TABLE. The bug class is deleted, not defended.
TWO REAL BUGS the naming fix ALONE would NOT have closed (found by review sweep):
- BREAK-1: descriptions were built with f"VR1 {dc.upper()} ..." -- under D-119 that
renders "VR1 VR1-DC0 provider-public" on all 36 prefixes. Same class as the
original bug (deriving a label by munging a token), hidden in the description
field where slug-focused review missed it. Now looked up from SITES[name].
- BREAK-2 (the important one): DC_GUA_PREFIX was NEVER cross-checked against --dc.
`--dc vr1-dc0 DC_GUA_PREFIX=<f03>` was ACCEPTED: it writes the SECOND DC's GUA,
carved with the FIRST DC's ULA nibble, scoped to the FIRST DC's site -- a
silently mis-bound datacenter assembled from two disagreeing sources. Identity-
mapping the slug leaves the ADDRESSING free-floating. Now guarded by EXPECTED_GUA.
ALSO: vdc1/vdc2 -> vvr1-dc0/vvr1-dc1 (D-114 amendment). D-114's own DR primitive is
`virsh destroy vdc1`, which read as "destroy VR1 DC1" while MEANING "destroy VR1
DC0". A mislabelled destroy command in a DR drill is not cosmetic. Neither VM is
built, so it is free. voffice1/Office1 KEEP their number (operator ruling).
Env var: DC1_/DC2_V4_SUPERNET -> VR1_DC1_V4_SUPERNET (both old names rejected).
rbd-mirror's --site-name aligned: `--dc vr1-dc0 --site-name dc1` was a re-created
two-namespace collision on one command line.
TESTS: dc-selector 21->30, prefixes-import 40->82 (identity invariant; EXPECTED_GUA
in BOTH mismatch directions; mismatched run writes NOTHING; matching pair still
succeeds; no munged description). GAUNTLET ALL GREEN (57). repo-lint 0 fail.
The old harnesses PINNED THE WRONG MAPPING -- they would have gone green while
enforcing the bug.
THE tofu apply IS NOT DONE -- IT IS GATED. Plan: 11 to add, 0 to change, 11 to
destroy (libvirt object names are ForceNew; moved{} makes the plan reviewable but
cannot suppress a replace). MEASURED SAFE: all 11 are EMPTY -- no guests, no
volumes, Stage 3 hasn't run. Office1's live objects are not in the plan.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
| 2026-07-10 |
DOCFIX-151: $DC selector convention for lib-net.sh/lib-hosts.sh (gap #1 closed)
...
Adds lib_net_select_dc() (dc0/dc1 no-op per D-101 inherited-layout ruling,
dc2 fails loud pending NetBox) and lib_hosts_select_dc() (dc0 no-op, dc1+dc2
both fail loud -- no per-DC host enrollment exists yet). Backward compatible;
21/21 new harness tests green; full gauntlet unaffected (A/B via git stash).
Unblocks the Stage 5 runbook's "high reuse" claim in the DC-DC deployment
workflow.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QrcJx8TUar7pYAvpGJw57A
|