RULED 2026-07-31 (GA-R5): sync jammy-backports into the dc0 mirror (D-135 amendment) ...

Operator utterance, verbatim: "Sync the backports into the mirror".

Question as presented: 22 of 33 units failed their install hook on one measured
cause -- the mirror serves jammy/jammy-security/jammy-updates 200 and
jammy-backports 404, exactly dc-mirror.sh:179's "jammy triple" scope, while
every node's sources.list carries jammy-backports because the Step-3.5
apt-mirror model-config rewrites EVERY suite to the DC mirror. Options: (a) add
the suite and re-sync, (b) drop backports from the nodes' sources.

Recorded as a D-135 AMENDMENT. OPS under GA-R3, no new D-number; D-135's per-DC
strategy split (dc0 full mirror / dc1 caching proxy) is unchanged.

Cost measured BEFORE the ruling was put: ~1.00 GiB / 461 packages against a
951 GB mirror with 1.8 T free -- ~0.1%.

Recorded as owed, not built: dc-mirror.sh check verifies sync STATUS, not SUITE
COVERAGE against what the deployed image's sources.list requests, which is why
it read PASS throughout. Same class the mirror gate was already fixed for once.

Committed and pushed BEFORE the dependent work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Deploy blocker: mirror lacks jammy-backports. Queue per-DC Tailscale (item 21) ...

BLOCKER, logged not fixed (D-135 scope is a ruled surface). 22 of 33 units in
error, all hook failed: "install", all one cause:
  E: The repository 'http://10.12.8.4/ubuntu jammy-backports Release'
     does not have a Release file.
Measured: the mirror serves jammy/jammy-security/jammy-updates 200 and
jammy-backports 404 -- exactly D-135's "jammy triple" scope -- while the node's
sources.list carries jammy-backports, because the Step-3.5 apt-mirror
model-config rewrites EVERY suite to the DC mirror. Machines are all
started/running; this is purely the artifact layer and units retry.
dc-mirror.sh check dc0 PASSES while this is true -- it verifies sync STATUS,
not suite COVERAGE against what the deployed image asks for.

QUEUED BY OPERATOR DIRECTION, not built: a per-DC Tailscale subnet router is
now a standing DC-standup requirement, dc0 first. Home of record is the tooling
gap register item 21 (new), with a per-DC definition-of-done, the
vendor-documented build constraints, and four sub-decisions needing GA-R5
rulings first: the utility-band OCTET (D-134's map is a standing cross-DC
standard), the admin-reachability model (star vs mesh -- the real architecture
question at region-region scale), HA count, SNAT on/off. This is EXECUTION of
the already-ruled D-129(iii) shape, not a new decision.

Measured while scoping: SEC-010 does NOT need relaxing (the forwarding to
permit is on the router VM, not the transit leg); voffice1 holds no route to
10.12.x at all, so the tailnet stops at Office1 by construction; the existing
Office1 node is UNTAGGED and carries a 180-day key-expiry clock on the
operator's only tailnet path; and D-129(iii) cites D-107 as governing Office1
while D-107 rules nothing about Tailscale -- the second instance of the F7
miscitation class, this time inside a ruling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

>>> BUNDLE DEPLOY SUCCEEDED: "Deploy of bundle completed." exit 0 <<< ...

Option D half 2. The 23 application definitions were removed INDIVIDUALLY,
never batched (hard rule 3), each read back, with the precondition re-verified
immediately before the first removal and the model read back empty afterwards.
Step-3.5 state survived the clear -- apt-mirror still set, all six spaces still
bound with both address families, checked not assumed.

MEASURED IMMEDIATELY AFTER: juju 56 applications / 9 machines / 33 units,
machines pending+allocating; MAAS 10 machines -- 9 Deploying, 1 Deployed (the
controller).

Three defects closed in the order they were hit: attempt 1's unknown option
prefer-ipv6 (D-101 ruling note); attempt 2a's missing policies/overrides.zip on
the client host; attempt 2b's barbican 265-vs-261 downgrade refusal (explicit
base on all 56 AND a clean model).

Operational note, because the first removal read as a failure: juju
remove-application PROMPTS by default and aborts on non-interactive stdin.
--no-prompt is required.

WHAT IS NOT CLAIMED: this means juju ACCEPTED and QUEUED the bundle. It does
NOT mean the cloud is up. 9 machines are allocating, 33 units pending; the
settle takes hours and nothing here asserts unit health or any service verdict.

G17's dc0 half is now genuinely armable -- the existing capture was taken on
the CONTROLLER VM and says so. The nine role nodes are booting for the first
time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Option D half 1: explicit base on all 56 apps, gated by new invariant 12 ...

PROVEN AGAINST THE REAL MODEL, NOT INFERRED. The earlier one-app test only
showed the error MOVING to the next app. The fully-based bundle was staged as a
throwaway on the rack and --dry-run against the LIVE 23-application model --
the same command that errored on barbican 265-vs-261 -- exits 0.

Subordinates are in scope, measured: mysql-router is 1154 at 22.04 and 1178 at
24.04. hacluster sharing rev 166 across both bases is not a property to depend
on.

TWO SILENT UNDER-MATCHES, both caught by cross-checking the edit against the
parsed file rather than by review:
 (1) a first pass keyed on the block form inserted 44 of 56 -- the twelve
     -hacluster apps are single-line FLOW MAPPINGS. The inserter asserts its
     app list equals what yaml.safe_load sees, and refuses otherwise.
 (2) overlays/dc-ha-scaleup.yaml DEFINES vault-hacluster, an application that
     exists nowhere in bundle.yaml, and it had no base. That overlay is a LATER
     deploy step, so this would have re-opened the trap AFTER the deploy
     succeeded. Found by the new gate on its FIRST run, in a file the fix was
     not looking at -- the same later-step-overlay class as the ceph-mon miss
     earlier today.

INVARIANT 12 (NEW): every application carries an explicit base equal to the
bundle's OWN default-base -- keyed on that value, not a literal. A bundle with
no default-base REFUSES rather than passing vacuously. Without this, deleting
the 56 lines is a byte-identical PASS and the trap returns at the next partial
failure, at any DC.

Five cases, each mutation-proven individually (harness 50 -> 55/55): neutering
nobase kills T49/T50/T52, wrongbase kills T51, the refuse branch kills T53.

Gauntlet ALL GREEN (93); repo-lint 0 fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

RULED 2026-07-31 (GA-R5): option D -- fix the bundle AND clear the model ...

Operator utterance, verbatim: "I want to do option D: Fix the bundle by
defining the base 22.04 on each app and then clearing the modeling and
deploying from a clear model. It is very cheap to clear and redeploy."

I offered three options; the operator composed a fourth that is strictly
stronger than any of them. The durable fix lands (explicit per-app base, so
the bundle survives a partial failure), AND the deploy runs the path that is
MEASURED to work -- a clean deploy onto an empty model, exactly what run 2a
did before it hit the missing resource file -- rather than juju's incremental
re-run path, which is the one carrying the defect.

OPS under GA-R3, no D-number: an explicit base per application states what
bundle.yaml:85's default-base already means. No ruled surface changes.

The invariant it establishes is skill material at stage close: a bundle whose
applications rely solely on default-base is deployable exactly ONCE, and a
production bundle that cannot survive a partial failure is a trap.

Committed and pushed BEFORE the dependent work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Deploy attempt 2: prefer-ipv6 fixed, two new defects, model partially populated ...

The prefer-ipv6 failure did NOT recur. Two new ones did, and for the first time
the model is not empty: MEASURED 23 applications, 0 machines, 0 units.
Attempt 1 aborted during VALIDATION; this one reached EXECUTION, which is why
it left residue. NOTHING IS PROVISIONED -- no MAAS machine left Ready, no disk
written.

DEFECT 1: the rack's ~/repo-stage is a PARTIAL COPY of the repo with no
policies/, and bundle.yaml:214 is the only local-file reference in the deploy
input. juju resolves it relative to the bundle, so it must exist ON THE HOST
THAT DEPLOYS. Nothing caught it: --dry-run does not upload resources (the
SECOND --dry-run blind spot in two days), attempt 1 aborted before resource
upload, and preflight runs on voffice1 where policies/ DOES exist -- the same
wrong-host instrument class already recorded twice, and the second instance
this session. Fixed forward, both files sha256-verified.

DEFECT 2: a bundle relying on default-base is deployable exactly ONCE. The
re-run failed on barbican 265-vs-261. Measured: 265 is the ubuntu@22.04
revision, 261 the ubuntu@24.04 one. The 23 existing apps are all 22.04, so run
2a honoured bundle.yaml:85 -- but on a re-run, for an app that ALREADY EXISTS,
juju resolves without that default. A model-level default-base has NO EFFECT
(tested, then RESET). An explicit per-application base WORKS (tested read-only
on a throwaway; the error moved to the next app).

OWNED: two of those read-only tests were staged under /tmp and failed with
"no charm was found" -- the juju snap's PRIVATE /tmp, a trap this repo has
already recorded in CURRENT-STATE and the phase-4 runbook.

The rollback decision tree is written for tofu apply and defaults to
fix-forward, which is measured not to work here. A partial juju deploy is a
failure mode the repo does not cover. Recovery is an operator decision;
options are enumerated in the capture and the model residue is untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

RULED 2026-07-31 (GA-R5): the five new P5 findings accepted; dc0 deploy is clear ...

Operator utterance, verbatim: "Accept the five and proceed to the dc0 deploy".

The 2026-07-30 acceptance covered SIX findings and says it covers "these six,
enumerated, and nothing else". This preflight run reports eleven: all six still
present, plus FIVE new -- maas-region-db-password, maas-region-admin-password,
maas-region-api-key.txt (SEC-027) and maas-juju-api-key.txt,
maas-juju-user-password (SEC-028), all vr1-dc1. Every one is the D-137 forward
register correctly reporting that dc1's MAAS region is authored but not applied.

Carried as known, by-design absences on their existing SEC rows. preflight keeps
exiting FAIL on P5 for the rest of this stage; that RED is ruled-accepted and
must NOT be made green by deleting or weakening a matrix row. Like its
predecessor this covers these FIVE, enumerated, and nothing else -- total
accepted at P5 is now ELEVEN across the two rulings.

Also corrects this changelog's header, which claimed the session was read-only
throughout. True for items 1-5; after the ruling landed it edited the renderer,
the gate, both overlays and both values files. No cloud state changed at any
point.

Committed and pushed BEFORE the deploy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Pre-deploy: rack input refreshed + hash-verified, preflight captured, dry-run green ...

(a) The dc0 rack's ~/repo-stage is a COPY, not a clone -- nothing updates it, and
it still held the pre-ruling vips overlay. Exactly one of four files differed;
copied, then ALL FOUR hash-compared against repo HEAD. The gitignored
octavia-pki overlay was deliberately not touched (SEC-029) and verifies
byte-identical at 5fc117f1, still 0600. A sweep is evidence about the deploy
only if the swept bytes ARE the deployed bytes.

(b) Preflight captured (242 lines, exit 1), run with
MAAS_PROFILE=vr1-dc0-region -- without it preflight is REGION-BLIND and emits 19
false negatives. Instrument proven current first: maas-profile-assert.sh
vr1-dc0-region hot-kid exits 0. P1-P4 and P7 PASS, incl. aligned VIPs 13 OK / 0
bad against the RE-RENDERED overlay. P5 FAIL, 11 findings.

(c) The P5 delta is ENUMERATED. The 2026-07-30 acceptance covered six findings
and says it covers "these six, enumerated, and nothing else". All six are still
present; FIVE NEW are all vr1-dc1 S2 rows (SEC-027/-028) -- the D-137 forward
register correctly reporting that dc1's region is not built. Diff appended to
the capture. Needs its own GA-R5 exchange before the deploy.

(d) Step 4.2 --dry-run against the fixed input: exit 0, 56 apps / 108 relations
/ 33 unit placements. The ceph-osd tags=openstack exposure is now MEASURED, not
reasoned: the plan places ceph-osd/0..3 on machines 5-8 by explicit id, so the
absent tag never has to match. The residual (a later UNPLACED add-unit) stands.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Fix a defect in my own build: invariant 9a was VIP-scoped and the charm list was short ...

Found by a repo-wide grep, not by any gate. Two faults, one cause -- I
measured and gated the population the QUESTION named (the 13 VIP charms)
rather than the population the INVARIANT covers (every application juju
validates).

(1) PREFER_IPV6_CHARMS was WRONG. Re-measured across ALL 33 charms in
bundle.yaml: TWELVE declare the option, not seven. The five missing are
ceph-mon 491, ceph-osd 953 (squid/stable), nova-compute 894, hacluster 166
(2.4/stable) and mysql-innodb-cluster 164 (8.0/stable).

(2) 9a was written INSIDE the VIP loop, so an application with no vip never
reached it -- and the repo ALREADY has that case:
overlays/dc-dc-ipv6-family-matrix.yaml sets prefer-ipv6 on ceph-mon, which
carries no VIP. That overlay is a LATER deploy step, so the miss would have
passed attempt 2 cleanly and surfaced at the step after, looking like a new
fault.

9a now runs over EVERY application, before the VIP loop; the VIP loop skips an
app 9a already flagged so the diagnosis is not doubled. ceph-mon's own in-file
claim of the option is now INDEPENDENTLY VERIFIED from the artifact rather
than taken from the comment.

Harness 48 -> 50/50. T47 (non-VIP app whose charm lacks the option FAILS) and
T48 (ceph-mon PASSES) both mutation-proven: re-scoping 9a back to the VIP loop
kills T47; dropping ceph-mon from the list kills T48.

Gauntlet ALL GREEN (93); repo-lint 0 fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Sweep both deploy inputs for option NAMES and VALUE TYPES: clean, and proven able to fail ...

The two extensions owed after the ruling, now run:
  dc0  75 assignments / 20 schemas / 0 unknown name / 0 type mismatch / 0 note
  dc1  75 assignments / 20 schemas / 0 unknown name / 0 type mismatch / 0 note
Both EXIT 0. dc1 introduces no new charm@channel pair -- checked, not assumed.

PROOF OF TEETH, because a clean sweep across two inputs is exactly the
implausibly-uniform result this repo has been burned by. Three defects planted
in a THROWAWAY copy, all three caught, EXIT 1:
  [FAIL name] barbican  does not declare option 'prefer-ipv6'   (attempt-1 defect)
  [FAIL type] keystone  'prefer-ipv6': declared boolean, got str ('true')
  [note]      glance    'vip': declared string, value is unquoted float

Type judgement is deliberately conservative: a mismatch is reported only where
juju's own coercion cannot save it. A scalar where a STRING is declared is a
NOTE, not a failure -- but it is still surfaced, because an unquoted value is
how a version string silently becomes a float. An UNRECOGNISED declared type
REFUSES rather than passing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Build the 2026-07-31 ruling: renderer + invariant 9 replaced, overlays re-rendered ...

Three parts, all mutation-proven.

(i) provider-bundle-check.py gains PREFER_IPV6_CHARMS -- the measured authority,
keyed by CHARM name, carrying the revisions and a re-measure warning -- and
invariant 9 is REPLACED, never deleted:
  9a  prefer-ipv6 on a charm that does NOT declare it FAILS. Asserted on
      PRESENCE, not truthiness: 'prefer-ipv6: false' is the same fatal
      'unknown option' to juju as true.
  9b  on a DECLARING charm it still travels with the v6 legs -- the L3-9
      protection is retained unchanged for those seven.
  9c  arity.
The old text's mechanism claim ("prefer-ipv6 makes HAProxy bind :::port") is
refuted by this session's measurement and is recorded as such in-file.

(ii) render-dc-overlays.py emits the option only for a declaring charm, READING
the set from provider-bundle-check with ast rather than restating it (the same
rule APP_OCTET already uses). render() keeps its purity property: the set is a
PARAMETER and the default REFUSES rather than defaulting to empty, because an
empty set renders a plausible artifact with the option nowhere. Both overlays
re-rendered: exactly SIX lines removed per DC and nothing else -- every vip
string byte-identical, so all v6 legs are retained.

(iii) tests/provider-bundle-check 44 -> 48/48. T43 reproduces the attempt-1
defect; T44 proves presence-not-truthiness; T45 is the positive control; T46
asserts the DIAGNOSIS.

MUTATION PASS -- six mutations, every one killed tests. And one NEW assertion
was found to be DECORATION and was REPLACED rather than kept: T46's first form
could not fail under any mutation, because a non-declaring charm with no option
reaches neither branch. Re-written to catch the fold-9a-into-9b simplification
and re-proven.

Gauntlet ALL GREEN (93); repo-lint 0 fail. The fixed dc0 deploy input now
sweeps CLEAN on option names: 75 assignments, 20 schemas, 0 unknown.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

RULED 2026-07-31 (GA-R5): the six drop prefer-ipv6, the v6 VIP legs STAY ...

Operator utterance, verbatim: "I want it to use IPv6, if there is spam
mechanisms being applied then that is bad. Even if it doesn't cause an issue
now, it might in the future. A clean IPv6 network is better than one with
unusable and possibly future breaking configurations."

That stated a PRINCIPLE rather than selecting an option, so a confirming
exchange was taken rather than adopting an inferred ruling (GA-R5 forbids
adopting from an inferred or reconstructed ruling). Confirming utterance,
verbatim: "Yes -- keep the v6 legs, remove only the option".

Recorded as a D-101 RULING NOTE dated 2026-07-31. OPS under GA-R3 -- a
renderer + gate change, doubt resolves DOWN, no D-number assigned. D-101's
family matrix is UNAMENDED: the dual-stack posture does not change and every
v6 VIP leg is retained.

Effect: barbican, designate, magnum, octavia, placement and vault keep all
three v6 VIP legs each; prefer-ipv6 stops being emitted for them; both DCs,
symmetric.

The SEVEN charms that DO declare the option are NOT covered by this ruling
and were deliberately not bundled into it -- one decision per exchange.

Committed and pushed BEFORE any dependent work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

prefer-ipv6 research: size option (b) and scope the sweep claim ...

Two corrections to my own framing, both measured:

(1) Option (b) is NOT a values-file edit. 'prefer-ipv6: true' is INJECTED
UNCONDITIONALLY by scripts/render-dc-overlays.py:243-246 whenever fam ==
"dual", and 'family: dual' is a single TOP-LEVEL key in
render/values/vr1-dc{0,1}-vips.yaml:3 -- not per-app data. Implementing (b)
means teaching the renderer which charms declare the option, re-pointing
tests/render-drift, and replacing invariant 9: a code change with a harness.
Recorded before the fork is put to the operator so the ruling is taken on the
real cost.

(2) The option-name sweep's claim is scoped inline. It checked NAMES, dc0
only. Option VALUE TYPES (the 'type:' field in every schema already
downloaded) and the dc1 input are both owed before the redeploy, and each
could cost an attempt the same way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

prefer-ipv6 research DONE: the option never drove the v6 HAProxy bind ...

Answers the 2026-07-31 ruled question ("Research what those 6 charms do with
v6 VIPs first"). Measured by downloading each charm at its pinned channel for
amd64/ubuntu@22.04 and reading its templates and code -- not Charmhub's
config-yaml, which answers only whether the knob is declared.

FINDING: in BOTH template families the ':::port' frontend bind is gated on
ipv6_enabled = not is_ipv6_disabled(), a read of the kernel
net.ipv6.conf.all.disable_ipv6 sysctl -- MEASURED 0 on a live MAAS-deployed
jammy node in this DC. The v6 VIP is assigned by pacemaker on per-address
family detection. vault ships no haproxy template at all; its listener is the
hardcoded literal [::]:8200. prefer-ipv6 is a UNIT ADDRESS-FAMILY switch, and
the one value it sets that sounds like a bind address (haproxy_host) is
consumed by no template in any charm downloaded.

So option (b) -- drop the option from the six, keep their v6 VIP legs -- is
what the measurement supports. NOT TAKEN: the ruling is the operator's, the
overlay is untouched and R2 is unamended.

Also recorded: provider-bundle-check invariant 9's mechanism claim is wrong
and must be REPLACED (never deleted) when a ruling lands; a new and separate
question about the SEVEN charms that do declare the option; a full
option-name sweep of the dc0 deploy input (56 apps / 81 assignments / 20
schemas / 0 unresolvable) showing prefer-ipv6 on the six is the ONLY unknown
option, so nothing else is waiting for attempt 2; and a runbook defect --
the dc0 deploy block omits vr1-dc0-machines.yaml, which is NOT a no-op (it
carries ovn-chassis bridge-interface-mappings).

Read-only session: no cloud mutation, no overlay edit, no ruling adopted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Session close: GA-R4 bookend, memory review, gauntlet 93 green ...

dc0 node carve complete (v4 134/0, v6 54/54, diff vs pre-migration EMPTY);
juju controller vr1-dc0-controller LIVE on the fourth attempt; model, spaces
gate and apt-mirror done. Bundle deploy attempt 1 failed on prefer-ipv6 with
nothing half-applied; operator ruled research-first, so R2 is NOT amended and
Stage 5 is blocked on that question.

SEC-028 + SEC-029 opened and registered before use. DOCFIX-206 fixed a
credential gate that was not region-scoped. preflight region-blindness logged.

GA-R7 memory review: instrument-currency memory updated with three new
instances and a new DETECTOR -- an implausibly uniform result is itself the
tell. Other three memories reviewed, accurate, none claim operator intent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Bundle deploy attempt 1 FAILED: prefer-ipv6 unsupported on 6 of 13 VIP charms ...

ERROR cannot deploy application 'barbican': unknown option 'prefer-ipv6'.
Model measured EMPTY afterwards -- juju validated and aborted atomically, so
nothing is half-applied and Steps 1-3.5 remain good.

The dry-run passed and the deploy did not: --dry-run resolves charms and plans
relations but does NOT validate config option names. A green dry-run is not a
gate on option validity.

Measured against charmhub's own config-yaml per pinned channel: 7 of 13 VIP
charms declare prefer-ipv6, 6 do not. barbican never had it at any channel, so
it was not an obsoleted option -- R2's uniform application was never valid.

Not fixed: invariant 9 couples prefer-ipv6 to the v6 VIP legs, so this needs a
GA-R5 ruling, not a mid-deploy edit.

Owned: my first charmhub query used a wrong field and returned a uniform
'no config' across 13 charms -- implausible, which is what caught it.

Capture: docs/audit/stage5-dc0-deploy-attempt1-20260731.txt

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

JUJU CONTROLLER BOOTSTRAPPED: vr1-dc0-controller live, active/idle ...

Fourth attempt, first success. Each earlier failure was a real defect one layer
deeper and all three are now closed: D-138 (client into the DC), the controller
under-carve fix (default route), and D-132 q1 (per-DC MAAS region).

controller/0 active/idle on juju-controller rev 311; machine 0 started
ubuntu@22.04 as subtle-grouse; MAAS reads arfr7p Deployed ubuntu/jammy.

Transcript evidence: agent binaries resolved on attempt 1 (339-retry class stays
closed); targeting took the tagged VM not a role node; juju offered the v6 leg
restored earlier this session before connecting on metal-admin.

Capture: docs/audit/stage5-bootstrap-dc0-20260730.txt

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Juju re-pointed at the dc0 region: credential minted, proven, registered (SEC-028) ...

Operator-ruled: 'Mint juju-vr1-dc0 on the new region (Recommended)' and
'Add it to allow'. juju-vr1-dc0 minted in the dc0 region as a superuser,
mirroring the measured Office1 shape. Never printed; moved host-to-host with
all three sha256 digests compared; staging shredded.

Proven to AUTHENTICATE before bootstrap -- login + users/machines/rack reads
confirming superuser scope and DC-local region identity. Cloud re-pointed to
10.12.8.6:5240 and read back; the stale Office1-keyed credential removed first.

DOCFIX-206: Step 2.0's credential gate was not region-scoped, and I hit it --
all three checks passed while the user lived in Office1, so the SKIP branch
would have led to an auth failure that reads as a network fault.

Two snap-confinement traps recorded: private /tmp, and the home interface
refusing dot-directories. Neither error names confinement.

Registered BEFORE bootstrap: SEC-028, 8 matrix rows, dc0 manifest, and the
first 'rack' rows in vm-secret-locations. creds-matrix 65/65; repo-lint 0 fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

v6 carve complete 54/54; controller v6 restored; Juju re-point blocked on a credential mint ...

v6: applied=54 errors=0, read back 54/54. Ran after the v4 carve because the
provider v6 static lands on br-ex. The tool walks only the nine tagged role
nodes, so the controller's metal-admin v6 was applied separately -- restoring
ruled state the re-enrolment dropped. Its provider GUA stays deliberately absent.

Juju survey on the rack: cloud still points at the OFFICE1 region, which is what
killed bootstrap 3. juju unregister is NOT needed -- no controllers registered.

Blocker: the new region has no juju-vr1-dc0 MAAS user. Re-establishing it mints
a new API key, so it is operator-gated. Nothing was minted; the refusal was a
permission rule failing to match, not a classifier fault.

Gauntlet ALL GREEN (93); repo-lint 0 fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

dc0 v4 node carve COMPLETE: 10/10 machines, gate 134/0, diff vs pre-migration EMPTY ...

Canary-first then node by node, never batched. 11 + 13x8 + 4 mutations, every one
read back. Nine role nodes on six planes with OVS br-ex; the juju controller on
two planes raw with no br-ex and enp3-6 left on auto VLANs.

Verified by diffing against the Office1 pre-migration capture: every node's six
v4 legs match address for address. Compared by LEG SET, since system_id and
hostname are both re-minted at re-enlistment.

Instrument note: the first extraction read the wrong capture field and returned
'-' for every old address, which would have read as 'the old region had no
statics'. Caught by the implausible shape, not by the parse.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

dc-node-carve.sh: the v4 node interface carve, mutation-proven 40/40 ...

The last no-tool gap between the dc0 region migration and the Juju deploy:
60 NIC re-homes, 9 OVS br-ex bridges, 54 statics. The Stage-4 equivalent was
done ad-hoc and logged outside the repo, so there was nothing to re-run when
D-132 q1 forced a re-enrolment.

Proved the four-call MAAS cycle on ONE live interface (canary 6q4syf enp3s0)
before writing the tool -- nothing had exercised `interface update vlan=` on a
link_up-only interface on an auto fabric, which is the state all 60 NICs are in.
Measured in passing: link-subnet REPLACES a link_up link, so only the br-ex
member needs an explicit unlink.

No default profile; maas-profile-assert.sh before any read or write; every id
resolved live by stable key (boot MAC / CIDR / interface name); every mutation
read back and compared. Role asymmetry derived from the host token, and the
controller's deliberately-uncarved NICs are asserted ABSENT.

Mutation pass found the br-ex static compare could not fail -- Pattern B was
being caught by the type/parent assertions instead. Added the brexip fixture
and re-proved the kill. Also found that neutering an assertion's MESSAGE only
proves it exists, not that its predicate works.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Close sweep: queued findings F1-F10, memory review, skill invariants ...

Sweep capture docs/audit/queued-findings-20260730-dc0-region-migration.txt.
Three items lived ONLY in the transcript and would have been lost on a
clear:

F1 -- this session's permission rules exist only in the GITIGNORED
.claude/settings.local.json, so their verbatim text is now recorded,
including a broad Bash(ssh vr1-dc0-rack *) that an interactive approval
auto-added and which deserves review.

F2 -- the Office1 region STILL registers a rack controller on the dc0 rack
(vvr1-dc0/7chphy) and still holds the dc0 region VM's machine record. Not
harmful (Office1's DHCP for that VLAN is off, verified by process) but
dual-registration the operator should rule on.

F10 -- the five instrument errors of this session as one pattern, with the
rule that catches them.

F9 carries the next session's first commands and every measured input the
carve tool needs.

GA-R7 memory review: sharpened the classifier memory with the profile-token
lesson (a rule pinning 'maas admin' would have matched step 1 of the DHCP
cutover and silently missed step 3), and added an instrument-currency
memory. Both graduate to repo surfaces; neither is the only copy.

Skill swept: the per-DC region migration invariants folded in -- region
identity by rack, ids do not cross regions, the per-node ordering, batches
of three, zero SSH keys, the role-dependent carve, the role-tags gate that
passes on an incomplete tag set, and the two scripts that must not be
pointed at a new region.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Plane IPAM complete on the new region: 24/0 PASS, existing tooling only ...

dc-plane-ipam.sh check vr1-dc0 goes from pass 7 / fail 17 to pass 24 /
fail 0, exit 0. Two idempotent applies, dry-run first, every write read
back: 13 D-134 reserved bands plus the FIP pool, then the 5 missing v6
plane subnets each created on the same MAAS vlan as its v4 twin.

No new code was needed. The tool's forced sequencing also proved itself --
the reserve run skipped every v6 band with 'not in MAAS yet, run carve-v6
first' rather than failing or silently no-opping, so running the two out of
order is safe.

This is the step that justifies the tunnel architecture: carve-v6 derives
its prefixes from the NetBox apex, reachable only from voffice1, while the
target region is reachable only via the rack. Neither host alone could have
run it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

All ten dc0 machines Ready in the per-DC region; task 6 complete ...

Enlisted as canary + three batches of three. Shapes match D-121 Option C
exactly (3x16/64, 2x12/48, 4x8/24, 1x4/8), all with 6 interfaces, every one
matched to its libvirt domain by pinned boot MAC -- hostname and system_id
were both re-minted.

maas-role-tags.sh check: 0 tags missing, 0 nodes needing a tag, 0 not in
MAAS, exit 0.

But it reported that clean PASS while the machines were still missing the
openstack-vr1-dc0 placement tag, which is not in its ROLES set and has no
creator anywhere in the repo. bundle.yaml places on that tag and
dc-node-v6-carve.py dies without it, so both the deploy and the v6 carve
would have failed. Caught only by diffing against the pre-migration
snapshot. Applied to the nine role nodes and deliberately not to the
controller -- reproducing the Office1 shape row-for-row.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Correct my own wording: every machine passes through New ...

I wrote that batch 1 went 'straight to Ready with no New detour'. Wrong --
I aborted them to New myself to clear the enlistment-time wedge. Every
machine passes through New, either by self-settling there (batch 2 did,
before I could abort) or by an explicit abort.

The real distinction is whether the commission ISSUED FROM New succeeds on
the first try, and that is what power_type decides. The conclusion stands;
the phrasing overstated it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Batch 1 all Ready; the New detour was ORDERING, not chance ...

Batch of three enlisted in 2.3 min against a 15-minute zero-record stall for
nine, and all three went straight to Ready with no New detour -- because
power was set BEFORE commission was issued, where the canary was
commissioned first and powered afterwards.

That supersedes the earlier 'one-off' reading with a stronger and more
useful conclusion: the detour is caused by ordering. Revised standing
procedure is delete -> virsh start -> self-enlist -> SET POWER -> abort ->
commission, and never commission a machine whose power_type is unset.

Region now holds 4 machines all Ready, each matched to its libvirt domain by
pinned MAC.

Role tags need no new tool: maas-role-tags.sh creates the four missing ones
and refuses while any pinned MAC lacks a record, so it enforces the ordering
itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Nine records deleted; found a fresh region has ZERO SSH keys ...

All nine remaining Office1 dc0 records deleted individually. A scripted loop
over them was correctly refused by the guard -- hard rule 3 says destructive
steps are never batched, and I had written exactly that. Office1 is now at
13 machines with zero dc0-tagged.

Found by accident while probing stalled nodes: the new region has no SSH
keys at all. MAAS injects registered keys into every machine it deploys and
juju bootstrap needs client->machine SSH, so this is the same signature that
consumed three bootstrap attempts today and would have recurred at task 8
looking like a network fault. Imported the same vr1-office1-svc key Office1
carries, deliberately rather than minting a per-DC one, because the node ssh
aliases built this session depend on that identity.

Enlistment does not scale to nine concurrent nodes: all alive and leasing,
cloud-init silent for 15 minutes, zero records created. Egress, MAAS proxy,
region load and node liveness all ruled out by measurement. Restarted in
batches of three.

I misread the stall twice in five minutes -- first off a bad elapsed-time
estimate, then by counting 104 enlist lines that were DHCP renewals rather
than cloud-init progress. Counting lines from a broad grep is not measuring
the thing you care about.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Canary reaches Ready; two theories refuted by measurement ...

6q4syf/civil-bug Ready at 22:05:33 -- cpu=8 mem=24576 (exact D-121 Option C
storage shape), power=virsh, all six pinned MACs. The re-enrolment ordering
is now a working sequence.

Root cause of the earlier New: a commission issued immediately after an
abort can land in New once; re-issuing from a settled New reached Ready in
~3.5 min, matching the dc1 timing. One-off, not systematic. The repeat was
justified as the experiment that distinguished those two cases -- not as a
blind retry.

Refuted by measurement, recorded so neither is re-proposed: smartctl on
virtio disks (it was Skipped and its result set Passed, so
testing_scripts=none would have taught nothing), and a config delta between
the independently-configured regions (all 13 commissioning-relevant
settings identical).

Standing consequence for the remaining nine: a single New after a passing
commission is not a fault -- re-issue once; only a second New is evidence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Correct an over-claim: the re-enrolment ordering has NOT reached Ready ...

I wrote the five-step re-enrolment sequence into CURRENT-STATE and the
changelog as a standing invariant. Each step is individually measured, but
the sequence has not yet produced a Ready machine, so calling it a working
invariant was ahead of the evidence. Reworded to separate what is proven
(delete leaves the domain; manual power-on is required; self-enlistment
carries the pinned MAC; power config is a prerequisite, and without it the
machine wedges in Commissioning with zero events; abort+commission has MAAS
power it on) from what is not (that it reaches Ready).

Opens item 18 with the real diagnostic, which I had skipped:
node-script-results returned unparseable on first read and I moved on --
"could not look" treated as "nothing there", against my own rule. Read
properly it is unambiguous: commissioning result set Passed with all 13
scripts exit 0, testing set Passed with smartctl-validate Skipped on both
virtio disks, hardware discovery correct at 8 cpu / 24576 MiB. Neither
phase failed; the machine went to New anyway.

This REFUTES the smartctl-on-virtio theory. Re-running with
testing_scripts=none would have fixed nothing and taught nothing.

The other nine machines stay untouched until the canary is Ready, and
dc-node-carve.sh stays unwritten because a New machine's interfaces are
rediscovered on the next commission.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Canary re-enrolment: the ordering a fresh region actually needs ...

Canary xqqwdq/moral-salmon = vr1-dc0-storage-04 (a storage node, least
central of the ten). Office1 23 -> 22; it re-enlisted into the new region
carrying boot MAC 52:54:00:2b:ed:ab -- the machine that arrived is the
machine that left. system_id AND hostname are both re-minted
(xqqwdq -> 6q4syf, moral-salmon -> civil-bug), so the boot MAC is the only
stable key; the pre-delete snapshot records it as the identity.

MEASURED ORDERING, not the obvious one: a freshly-enlisted machine has no
power config, so MAAS cannot power-cycle it and cannot drive commissioning.
delete -> power on BY HAND via virsh -> self-enlist -> SET POWER -> abort ->
commission. Skipping the power step wedges the machine in Commissioning
producing zero events, because the enlistment ephemeral powers the node off
and nothing can bring it back.

maas-node-power.sh already supports the split the migration needs: VIRSH_URI
enumerates from voffice1 over the transit /30 while the positional address
stores what the REGION dials over metal-admin. Result was a real
query-power-state, proving the region drives the rack's libvirt with the
SEC-012 key in its snap.

Found: the per-node serial log stopped capturing on 2026-07-21 while the
domain XML still configures append-mode logging. It contains a complete,
plausible boot that I read as current and reported as fact before checking
the mtime. An append log that silently stopped appending returns confident,
well-formed, wrong evidence. Logged, not actioned.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf