| 2026-07-27 |

R9 RULED: derive lib-net's dc1 arm from the overlay with a drift check (D-119 amendment)
...
GA-R5: question and exact utterance quoted, dated, pushed before dependent work.
Operator utterance: "Derive lib-net's dc1 arm from the overlay, with a drift check
(Recommended)".
Recorded as a D-119 AMENDMENT -- D-119 owns the region-qualified per-DC arms in
lib-net.sh, so the question of how the dc1 arm is populated belongs there.
RULED SHAPE: lib-net.sh's vr1-dc1 arm is GENERATED from overlays/vr1-dc1-vips.yaml
with a render-drift check that fails the gauntlet on divergence. Consumers keep
sourcing lib-net unchanged -- no script learns to parse YAML -- and exactly one
authored copy of the values exists.
WHY THIS IS NOT NEW MACHINERY: D-137 sub-ruling 2 already ruled and built the same
shape for credentials, with creds-manifests DERIVED from creds-matrix.tsv and the
gauntlet failing on rendered-vs-checked-in drift. This applies a proven in-repo
pattern to network literals rather than inventing one.
IT DOES NOT PRE-EMPT D-136, and that was the deciding property. D-136 is PROPOSED
and unruled. This is the smaller compatible step: the overlay becomes the single
authored source for script-facing values. If D-136 is later adopted the apex
becomes the source and BOTH the overlay and this derived arm become generated --
a sub-case, not a competitor.
MEASURED BASIS, which inverts the intuitive reading: lib-net.sh:76-79 states the
backward-compatibility design in terms -- sourcing without the selector keeps
VR0/DC0 values "completely unchanged". So the unset block fires ONLY for
selector-callers. The 8 correctly-updated scripts break LOUDLY under set -u; the
20 never updated fail SILENTLY on dc0's literals. The silent half is the
dangerous half.
GUARD CARRIED FROM R11, and it matters for the generator's implementation: the
dc1 arm unsets NINE variables for TWO different reasons. METAL_INTERNAL_VID and
METAL_INTERNAL_IFACE are CORRECTLY unset because D-133 abolished the VLAN-103 /
br-internal stack for VR1 -- those facts genuinely do not exist. Only the
VIP/FIP/keystone group is superseded and derivable. A generator that populated
all nine would silently reintroduce a stack D-133 retired.
COUPLED: R2 makes the derivation dual-family, not v4-only. R11 moves
VIP_COUNT_EXPECT 11 -> 13 and widens the octet band to .99, and the two band
constants (OCTET_LO/HI in provider-bundle-check.py, VIP_OCTET_MAX in lib-net.sh)
are separately named in two files.
STILL OWED AND EXPLICITLY NOT COVERED: the consumer sweep. This ruling decides
where values live; it does not make the 20 non-selector scripts call the
selector. Stage-5 exposure is two of them (phase-03-core-verify.sh,
deploy-watch.sh); carve-host-interfaces.sh:48-67 (DOCFIX-166) is the in-repo
precedent for wiring one correctly.
Execution is a separate gated step under standard delivery discipline.
Revert: git revert this commit; the amendment is additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Final pre-ruling measurements R9/R10/R12-R15: two premises corrected
...
Operator direction: gather everything remaining so the last decisions can be
worked through without stopping to measure. Capture:
docs/audit/r9-r15-final-measurements-20260727.txt.
TWO OF THESE CHANGED MATERIALLY, and both are corrections to the audit's OWN
earlier framing rather than new findings.
R9 -- there is only ONE failure mode, not two, and the Stage-5 blast radius is
TWO scripts rather than twenty. lib-net.sh:76-79 states the design explicitly:
sourcing without the selector "continues to populate PLANE_CIDRS ... exactly as
above (VR0/DC0's real, measured values) -- every existing script that sources
lib-net.sh keeps working completely unchanged". So the unset block fires ONLY
for selector-CALLERS. The 20 non-selector consumers fail SILENTLY with dc0
literals; the 8 correctly-updated ones are the ones that break loudly under
set -u. That inverts the intuitive reading -- the scripts nobody fixed are the
dangerous ones. And grepping the two runbooks Stage 5 actually uses, only
phase-03-core-verify.sh and deploy-watch.sh are invoked there; the rest bite at
later stages.
R14 -- THE PREMISE IS WRONG and the question largely dissolves. I framed it as
"the matrix cannot express a RULED exception, so three S5 asymmetries ruled
correct by SEC-016 report as a permanent red". Measured: they are NOT ruled
correct. SEC-016 ruled per-DC ISOLATION -- dc1 gets its own dedicated power key
rather than reusing dc0's -- which is satisfied. It never blessed the filename,
host and custody divergence. THAT is SEC-021(b), an OPEN defect whose own
recorded disposition reads "needs a naming/custody reconciliation to the dc1
shape", and whose stated complaint was "Per-DC rows that should be symmetric are
not, and nothing compares them". S5 is the thing that now compares them. The
register is RIGHT and the finding is REAL. Suppressing it would have hidden an
open security-ledger item.
R15 refinement: the gauntlet ALREADY has a zero-floor -- run-tests-all.sh:35
exits 2 when RAN is 0. What is missing is a MINIMUM-count floor, since 81 is
pinned nowhere executable. repo_lint.py:132-135 has NEITHER a valid-root check
nor a files-scanned floor. The two gates need different fixes, which the earlier
framing conflated.
R12 and R10 confirmed as previously stated, with exact line citations.
Revert: git revert this commit; the capture is new and CURRENT-STATE additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

G18 OPENED: IPAM apex completeness for the Octavia lb-mgmt plane (blocking, deferred)
...
Operator direction, verbatim: "leave this as an open decision that will need a
ruling once we have the cloud live and we have a better read on the network and
how everything is functioning with the addition of the new IPv6 configurations.
Make this a gated decision so we cannot close the project (or whatever phase you
think it best ruled in) without a ruling on this item."
WHAT IT IS. R8 ruled that Octavia creates and owns its own IPv6 lb-mgmt network.
The measured consequence is that the octavia charm exposes NO CIDR, address-family
or router configuration option -- create-mgmt-network (default True) is the only
related option -- so the prefix is CHARM-GENERATED and cannot come from the D-111
carve. NetBox is therefore knowingly incomplete for exactly one plane. That is the
authority-inversion concern the audit's lens 7 raised: the apex being back-filled
to match a deploy rather than driving it.
WHY DEFERRED RATHER THAN RULED NOW. It is not answerable from artifacts -- it
needs the cloud live and an observed read on IPv6 behaviour. It is also adjacent
to the UNRULED D-136 NetBox-coupled render pipeline, which covers the same
apex-authority ground, so ruling it early would pre-empt that decision.
PLACEMENT. Recorded as gate G18 in CURRENT-STATE section 6, the gate authority.
[R] ruling-type: closes ONLY on a GA-R5 recorded ruling with the exact utterance,
dated, committed and pushed. ANSWERABLE from Stage 5 onward, since the prefix
exists once Octavia deploys. BLOCKING at the FINAL stage close / project close --
the deployment may not be declared complete while it is open. That placement gives
the widest window in which live evidence can accumulate while still guaranteeing
the question cannot be lost.
Options are pre-recorded in the gate row so a future session does not have to
re-derive them: (a) back-fill the charm-created prefix into NetBox post-deploy as
a documented record; (b) record the plane as charm-owned and explicitly out of
apex scope; (c) fold it into D-136's render-pipeline ruling if that is taken first.
ALSO CROSS-RECORDED, because it is exactly the kind of thing that gets "fixed" by
mistake: the absence of an lb-mgmt :x80 prefix in the VR1 ULA carve is CORRECT
under R8, not a gap. Noted in the D-101 R8 ruling note and in the G18 row.
Mirrored into docs/audit/queued-rulings-20260727.md as R16 (gated, deferred) with
CURRENT-STATE named as the authority, so the queue and the gate table agree.
Revert: git revert this commit; the gate row and cross-references are additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

R8a RULED: extend the o-hm0 verifier to compare MTUs (OPS, no D-number)
...
GA-R5: question and exact utterance quoted, dated, pushed before dependent work.
Operator utterance: "Extend the existing o-hm0 verifier to compare MTUs
(Recommended)".
Recorded as a SUB-RULING on the D-101 MTU ruling note rather than a new decision.
OPS under GA-R3 -- it is a script change, doubt resolves DOWN, so no D-number is
assigned. It discharges the LP #2018998 obligation that R8 carried forward.
MEASURED BEFORE PRESENTING, and it changed the option set. "Pre-emptive
configuration" was not actually available: the octavia charm exposes NO MTU
option, so nothing can be set in the bundle. The real choice was only how the
mismatch gets DISCOVERED.
The measurement also found a better option than either I had sketched:
scripts/phase-05-octavia-verify.sh:104 ALREADY inspects o-hm0, asserting
state != DOWN and the presence of an fc00::/ ULA. So it holds the handle already
and an MTU comparison is a natural extension rather than new machinery. Against
that, grep -i mtu across cloud-assert.sh, phase-05-octavia-verify.sh and
phase-04-network-verify.sh returns NOTHING -- MTU is asserted nowhere in any
post-deploy verifier, which is why this class would have surfaced as instability
rather than as a failed gate.
Rely-on-the-charm was refused: the 2025-12-31 recurrence against octavia 14.0.0 /
2024.1 stable is evidence the update-status self-heal is not reliable, and the
symptom -- spurious failovers under load -- reads as a Ceph, network or amphora
fault long before anyone suspects MTU. The unconditional ovs-vsctl mtu_request
workaround was refused: it fights the charm's own fix where that fix is working,
and would MASK a future regression instead of surfacing it. It stays available
as the remedy IF the check fires.
INCIDENTAL CORROBORATION OF R8, from this repo's own tooling: that o-hm0 verifier
is VR0-era and expects an fc00::/ ULA, which independently confirms the octavia
charm's default lb-mgmt-subnet is an IPv6 ULA. The as-built already assumed what
R8 ruled -- a second, internal line of evidence for a ruling whose external
evidence had to be rebuilt from scratch.
Delivery when executed: the assertion ships with its harness green, gauntlet ALL
GREEN, repo-lint 0-fail and a changelog entry carrying a revert.
Revert: git revert this commit; the sub-ruling is additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

R8 RULED: Octavia creates and owns its own IPv6 network (D-101 note; sub-ruling closed)
...
GA-R5: question and exact utterance quoted, dated, pushed before dependent work.
Operator utterance: "I want to take a Octavia creates and owns its own IPv6
network." Supporting reasoning recorded because it is load-bearing: "We have to
research to troubleshoot if we run into MTU bug issues down the road. We have
already deployed using this topology in the v1 DC test deployments that got us to
this point."
This CLOSES the octavia-family sub-ruling that the 2026-07-25 D-101 note left
explicitly open.
THE RULING REQUIRES NO ARTIFACT CHANGE, and the operator's reasoning checks out on
the artifacts: create-mgmt-network is set NOWHERE in bundle.yaml or any overlay, so
the charm default True has always applied, and Octavia's whole options block is
debug, openstack-origin, amp-image-tag, vip. VR0 deployed Octavia on precisely this
shape. This ratifies the existing topology rather than changing it.
FAMILY AGREES THREE WAYS. The charm's default lb-mgmt-subnet is IPv6 (LP #1897418,
verbatim: "By default, Octavia charm uses ipv6 for its lb-mgmt-subnet") and it is a
ULA -- the amphora address quoted in LP #1911788 is fc00:fa21:3d5c:9cfd:..., i.e.
fc00::/7. That is exactly D-101's "IPv6-only ULA ... Octavia lb-mgmt ... Internal,
no external clients".
THE APPARENT VR0 GUA CONTRADICTION WAS A PAPER ALLOCATION, checked on operator
instruction: lib-net.sh gives VR0 six IPv4 planes and no lbaas plane; lbaas is
listed in STALE_SPACES; maas-as-built-reference records that NIC as "idle
(undefined; ex-lbaas), raw NIC, no link"; and D-101's own context says VR0 is
IPv4-only. The whole VR0 v6 tree in the apex is a design record.
CONSEQUENCE FOR D-111, worth recording so it is not later "fixed": the absence of
an lb-mgmt :x80 prefix in the VR1 ULA carve is CORRECT, not a gap. The charm
exposes no CIDR option, so the prefix is charm-generated and cannot come from the
apex.
STANDING OBLIGATION carried by the operator's own reasoning: LP #2018998 (o-hm0 vs
lb-mgmt-net MTU, charm-octavia, High) is Fix Released across our lineage but
recurred 2025-12-31 against octavia 14.0.0 / 2024.1 stable -- our exact pin. A
jumbo lb-mgmt-net beside a 1500 o-hm0 silently drops health messages over 1500
bytes and causes spurious failovers, which is the direct interaction with R3's
jumbo-underlay ruling. Owed at the Octavia step: verify o-hm0's MTU MATCHES
lb-mgmt-net's by measurement, not by trusting the charm.
NOT ASSERTED: whether the charm attaches an external gateway to the Neutron router
it creates. No config surface tells it to and ULA is not globally routable, but
isolation was not proven from the documentation. Settle by inspecting the router at
deploy time; queued as an observation, not a blocker.
Revert: git revert this commit; the ruling note and research sections are additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

R8 research: both in-repo citations fail; charm default is IPv6; new MTU risk found
...
Operator declined to rule on the overlay's own citations -- "I would like
documentation and vendor specific information rather than conjecture. Research is
cheap, guesswork is expensive" -- then directed a full read of two bugs weighed
against current versions. That was the right call and it inverted the answer.
BOTH CITATIONS IN overlays/dc-dc-ipv6-family-matrix.yaml FAIL ON INSPECTION:
- LP #1913409 is Fix Released (2021) against kolla-ansible, a DIFFERENT
INSTALLER. No bearing on a Juju/charm deployment.
- LP #1911788 is Incomplete and a DUPLICATE of LP #1896630, and is NOT an IPv6
defect. Its diagnosed cause is an OVN port-binding hostname mismatch --
binding_host_id carrying the shortname instead of the FQDN for LXD containers
on MAAS, giving binding_failed instead of ovs. The IPv6 address in the timeout
was incidental; the port never bound, so nothing would have worked over any
family.
VERSION WEIGHING, verified rather than asserted. #1896630's primary fix is
ovs-record-hostname.service, which the bug places in OVS 2.15 (Focal shipped
2.13). Rather than assume, I queried THIS DEPLOYMENT'S OWN dc0 mirror for what a
jammy node would actually install: openvswitch-switch 2.17.0-0ubuntu1 (jammy) and
2.17.9-0ubuntu0.22.04.2 (jammy-updates). 2.17.9 >> 2.15, so the fix is present on
every node this deployment will build. The bug's era was Juju 2.7.8, charms 20.08
Ussuri, Focal/OVS 2.13; we are Juju 3.6.27, charms 2024.1, jammy/OVS 2.17.9. The
charm-layer-ovn task -- the one closest to this repo's OVN usage -- is closed
Invalid.
AND THE DEFAULT RUNS THE OTHER WAY: LP #1897418 records, verbatim, "By default,
Octavia charm uses ipv6 for its lb-mgmt-subnet", with a reporter noting success
"using the charm default (ipv6) network but using ipv4 everywhere else". Upstream
Octavia documents that "IPv6 subnets can be used for the LB Network". So D-101's
IPv6-only placement of lb-mgmt is ALIGNED with the charm default, and a v4-only
lb-mgmt-net would be the DEPARTURE requiring justification -- the opposite of how
R8 was first framed.
NEW LIVE RISK FOUND, COUPLED TO THE R3 RULING: LP #2018998 "MTU mismatch between
o-hm0 and lb-mgmt-net" (charm-octavia, High, Edward Hope-Morley). A jumbo
lb-mgmt-net beside a 1500 o-hm0 silently drops health messages over 1500 bytes
and triggers SPURIOUS load-balancer failovers -- evidenced in the bug as "UDP,
length 1534 > 1500 and o-hm0 never receives them" and "o-hm0: dropped over-mtu
packet: 1744 > 1500". Fix Released across our lineage, BUT a recurrence was
reported 2025-12-31 against octavia 14.0.0 / 2024.1 stable, the exact channel
bundle.yaml pins. R3 ruled the underlay be finished to jumbo, which is precisely
this bug's precondition. OWED at the Octavia step of Stage 5: verify o-hm0's MTU
MATCHES lb-mgmt-net's after deploy, not merely that the charm claims to set it.
GENERALISABLE LESSON, recorded in the capture: a bug NUMBER in a comment is an
existence claim; what the bug SAYS is the content, and only content is evidence.
Same class as the two ruled-but-never-built decisions this audit already found --
a plausible in-repo statement that nothing verifies.
R8 is NOT ruled by this commit. It will be re-presented on corrected evidence.
Revert: git revert this commit; the capture is new and CURRENT-STATE additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

R7 RULED: per-DC independent Octavia PKI (D-109 amendment)
...
GA-R5: question and exact utterance quoted, dated, pushed before dependent work.
Operator utterance: "Per-DC independent Octavia PKI; fix the generator first
(Recommended)".
Recorded as a D-109 AMENDMENT. D-109 established per-DC INDEPENDENT Vault roots
with a DR-honest rationale but never mentioned the Octavia amphora
control-plane PKI, which is a SEPARATE trust domain with its own CA generated
outside Vault by phase-01 step 1.0-GEN. This extends the same posture to it.
MEASURED REFINEMENT THAT NARROWS THE WORK. The generator is dc0-frozen in two
ways of DIFFERENT severity, and only one is a design problem:
- The CA SUBJECT is a baked literal, "/CN=VR0 DC0 Omega Cloud Octavia Controller
CA/O=Neumatrix". Reusing it roots dc1's amphora chain in a VR0-DC0-named CA.
- The controller cert's SAN is ALREADY DERIVED per-DC by design -- the step
states it carries the controller FQDN, octavia API FQDN and the Octavia API VIP
"derived from the bundle at generation time -- DOCFIX-067; never a baked
literal". That half needs no change at all.
- The actual blocker is the VIP gate, grep -qE '^10\.12\.4\.', which accepts only
dc0's band while dc1's octavia VIP is 10.12.64.57 and lives in an overlay.
A DOCFIX was owed regardless of the ruling: the generator cannot produce a dc1
artifact as written, and phase-01:144-145 hard-ABORTS the deploy when
overlays/octavia-pki.yaml is absent, which it is. Option (c) would therefore have
deferred only the part D-109 already answers by precedent while leaving the
required work untouched.
REUSE REFUSED ON POSTURE. The overlay carries CA private keys plus a plaintext
issuing-CA passphrase inside the repo clone, and SEC-004 records the repo as
still PUBLIC. Sharing one amphora control-plane CA private key across two clouds
D-100 defines as independent would widen an existing exposure rather than
contain it, in a commercial multi-tenant cloud with hard tenant isolation.
Independent per-DC CAs keep that blast radius to one DC.
CAVEAT CARRIED FORWARD, because it will bite otherwise: the generator's VIP gate
must be re-pointed at the MERGED deploy input rather than bundle.yaml. Left
reading the base bundle it breaks again the moment ruling-3's VIP extraction and
R11's new .61/.62 allocations land.
Roosevelt analog: per-DC amphora CAs match per-DC IPMI/BMC credentials and the
per-DC MAAS power keys of SEC-012/-016 -- the same no-cross-DC-shared-secret
principle already ruled twice in this deployment.
Revert: git revert this commit; the amendment is additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

R11 RULED: vault .61 and designate .62, dual-family triples (D-020 amendment)
...
GA-R5: question and exact utterance quoted, dated, pushed before dependent work.
Operator utterance: "Both full triples (.61 vault, .62 designate), dual-family,
and fix the gate (Recommended)".
THE FINDING THAT MEASURING FIRST PRODUCED: vault was ALREADY RULED and never
built. D-020's decision text enumerates vault BY NAME among the clustered
applications carrying both a provider and a metal VIP; measured, base vault is
num_units:1 with an EMPTY options block. This is a conformance repair of a
2026-era decision, not a new choice -- and it is the SECOND ruled decision this
audit has found unimplemented, after D-134's address bands (R4). Worth stating
plainly: the pattern is ruled-but-never-built, and nothing in the repo could
detect either case.
designate is genuinely new -- absent from D-020's enumeration -- so it needed a
ruling rather than a repair, and this amendment adds it. Its dnsaas endpoint is
already bound provider-public, so a provider leg is coherent for it.
Shape is the ESTABLISHED triple, not a new form. The measured octet map is
consecutive: keystone .50 through ceph-radosgw .60, all provider/admin/internal
triples. vault takes .61, designate .62, both dual-family per R2 -- adding
v4-only now and re-doing them later would mean re-issuing certificate SANs on a
live cloud, and for vault that is 23 certificate relations.
Option (b), vault metal-only per the 2026-07-25 expansion review, was refused and
the conflict is RECORDED so that proposal is not later mistaken for the ruled
position: it has a real technical argument (all 23 vault consumers are internal)
but it contradicts D-020's own enumeration and would leave vault the single
non-triple in the bundle, a permanent special case for the checker.
Mechanical consequences, no choice in them: .61/.62 are legal under D-134's
amended .50-.99 band but REJECTED by the gate today -- provider-bundle-check.py
holds OCTET_LO/HI = 50,60 and lib-net.sh holds VIP_OCTET_MAX=60. These are
SEPARATELY NAMED constants in two files, so widening the band is a two-file
change (L3-7). VIP_COUNT_EXPECT moves 11 -> 13.
Gate hardening ruled IN SCOPE rather than deferred: the checker learns to FAIL on
an application with an hacluster relation and no vip. Justification is measured --
grep -rn cluster_count scripts/ tests/ returns NOTHING, and a constructed overlay
rewriting all 20 cluster_count values 3 -> 1 yields a byte-identical PASS (L4-3).
That is exactly why decorative HA was found by an audit instead of by a gate.
Per the R6 ruling, these VIPs land BEFORE dc-ha-scaleup.yaml is applied.
Execution is a separate gated step.
Revert: git revert this commit; the amendment is additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

R6 RULED: close the two VIP gaps, then apply the HA overlay whole (D-121 note)
...
GA-R5: question and exact utterance quoted, dated, pushed before dependent work.
Operator utterance: "Close the two VIP gaps first, then apply the overlay whole
(Recommended)".
THE TENSION IT RESOLVES. D-121 is titled "VR1 makes HA real -- scale the
decorative single-unit control plane to 3". Deferring the overlay would have
deployed VR1 in exactly the shape D-121 was written to retire. Applying it as-is
would have shipped, for vault, precisely the defect D-121 exists to remove -- a
pacemaker cluster with nothing to manage. Only the ruled sequence satisfies the
decision rather than half of it.
MEASURED BEFORE PRESENTING, and it reframed the question. The overlay scales 14
applications and moves all 12 base hacluster subordinates to cluster_count 3, but
ELEVEN OF TWELVE principals already carry proper VIP triples. The gap is exactly
two applications, and they are not the same shape:
- designate is in the base bundle WITH an hacluster and an ha binding, no vip.
- vault has NO hacluster in base at all and an EMPTY options block. The overlay
INTRODUCES the vault-hacluster application, its cluster_count 3, and the
vault:ha relation -- and still no vip.
Vault is the deciding case because of blast radius: 23 relations consume
vault:certificates, plus barbican-vault:secrets-storage. With no VIP every
consumer binds a unit address and there is nothing for a failover to move.
Remediating that on a live 3-unit vault means re-pointing 23 certificate
relations and re-issuing SANs on a running cloud, which is why the faster option
was refused.
SEQUENCING CONSEQUENCE: R11 becomes a HARD Stage-5 precondition ordered BEFORE
the overlay, not a parallel item.
Scope is tractable and the pattern is already in-repo: octavia carries a correct
triple (10.12.4.57 10.12.8.57 10.12.12.57), so R11 has a shape to copy rather
than design.
NOT resolved here, tracked separately: the Stage-6 radosgw multisite path is
single-unit-shaped while this scales ceph-radosgw to 3; and
provider-bundle-check.py checks cluster_count NOWHERE, which is precisely why
decorative HA was found by an audit instead of by a gate.
Address demand is not an argument against this -- R4's band ruling already covers
the overlay's growth from 27 to ~55 LXD units.
Execution is a separate gated step, not authorised by this ruling.
Revert: git revert this commit; the ruling note is additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Pre-ruling measurements for R6-R15: measure before asking, all ten
...
Operator: "Measure all the rest then we can work through them with relevant
data at hand." Taken directly by the session rather than delegated, after the
UNMEASURED-gap sweep showed lens findings need measuring before they become
questions. Capture: docs/audit/r6-r15-measurements-20260727.txt.
THE TWO RESULTS THAT CHANGE A QUESTION'S SHAPE:
R6/R11 -- vault's ENTIRE HA apparatus exists only in dc-ha-scaleup.yaml. Base
vault is num_units:1 with an EMPTY options block: no vip, no hacluster, no
relation. The overlay INTRODUCES the vault-hacluster application (not present
in base at all), sets cluster_count:3, adds the vault:ha relation -- and still
no vip. Applying the overlay therefore creates a 3-node pacemaker cluster with
nothing to manage. The blast radius is what makes it sharp: 23 relations
consume vault:certificates, plus barbican-vault:secrets-storage. Vault is the
CA for the whole cloud and every consumer binds a unit address. Of the 12 base
hacluster subordinates exactly ONE principal lacks a vip (designate), and
octavia carries a proper triple -- so this is a 2-app gap, not a pattern.
R10 -- the question largely DISSOLVES. Preflight has been run on the wrong
host. Measured: vcloud has neither maas nor juju nor openstack; voffice1 has
maas AND juju. So P3's 33 warns and P4's "MAAS unreachable" both clear simply
by running preflight on the D-128 Plane-2 host where it belongs. Only the
octavia-pki absence and the 7 credential findings are host-independent. Same
"tool absence reported as something else" class as U6, now three instances.
OTHERS: R7 the octavia CA subject is a baked VR0-DC0 literal while the SAN is
already derived by design (DOCFIX-067) -- the two halves differ in severity.
R8 the v4-only lb-mgmt shape is already pre-analysed in-repo with LP #1911788
and #1913409 cited and a drafted block ready (NOT verified upstream by me).
R9 only 8 of 28 lib-net consumers call the DC selector; the 20 that do not
include the entire phase-02..phase-06 family. R12 chronyc appears ZERO times
in CURRENT-STATE, confirming G17 omits the time check. R13 30 rows / 16 ids
are operator-terminal, of which SEVEN are keypairs whose loss is
unrecoverable-in-place. R14 the matrix has no exception field at all. R15 81
harnesses on disk and 81 reported, with 81 pinned nowhere executable.
METHOD NOTE, recorded because it nearly published a wrong figure: parsing
creds-matrix.tsv with awk -F'\t' returned ZERO operator-terminal rows,
contradicting lens 7's 30. The file is SPACE-ALIGNED, not tab-separated.
Re-measured correctly it is 30/16 and lens 7 was right. A disagreement with a
prior finding was treated as a reason to re-check my instrument rather than to
publish the new number.
Revert: git revert this commit; the capture is new and CURRENT-STATE additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

R5 RULED: designate deploys at Stage 5, is configured at Stage 7 (D-106 note)
...
GA-R5: question and exact utterance quoted, dated, pushed before dependent work.
Operator utterance: "Accept at Stage 5; rewrite Stage 7 Step 5 to
configure-not-deploy (Recommended)".
CORRECTION RECORDED IN THE DECISION ITSELF, not just this message. When R5 was
first put to the operator, the audit claimed this option "inverts D-106's
bootstrap order, which puts os-public-hostname + FQDN-SAN certs BEFORE
Designate". That was WRONG. D-106's order is a CONFIGURATION sequence -- static
hosts, then os-public-hostname, then Vault FQDN-SAN certs, then zones and A/AAAA,
then neutron, then tenant subnets. It governs when the DNS wiring happens, not
when the charm is installed. Deploying the app earlier does not invert it: the
zones still follow the certs. Conflating "install the charm" with "do D-106's
work" nearly cost a decision made on a false constraint, so the correction is
recorded in D-106 where a future session will meet it.
MEASURED BEFORE PRESENTING. All four designate applications and all EIGHT
relations are deploy-ready at Stage 5 -- every relation peer (mysql-innodb-cluster,
keystone, rabbitmq-server, vault, memcached, designate-bind) is created by Stage
5. But they will be FUNCTIONALLY INERT: os-public-hostname is set in NO deploy
artifact, and bundle.yaml:11 records the current posture as IP-ONLY with the dual
VIPs as the catalog endpoint. The charm runs; the feature does not. That is
exactly the state Stage 7 exists to resolve.
NEW SURFACE DEFECT FOUND BY THE MEASUREMENT: the phase-6 runbook contradicts
ITSELF. Line 172 states "There is no designate: or designate-bind: application
block anywhere in ..."; line 178 states "Designate is deployed in-bundle in each
DC". Both cannot be true. The bundle settles it -- DOCFIX-167 put designate there
on 2026-07-10 -- and the runbook needs rewriting regardless of this ruling.
Option (c), setting os-public-hostname at Stage 5, was refused and the reason is
worth keeping: it is the ONE branch that genuinely collides with D-106. Publishing
a public FQDN endpoint before Vault has issued FQDN-SAN certs recreates the exact
D-019 root cause D-106 was written to remove -- metal-only charms pulling a public
FQDN endpoint they cannot resolve, which is also the D-021 amphora constraint.
Option (b), suppressing designate at Stage 5, was refused because it needs
machinery that does not exist and partially undoes DOCFIX-167 -- building a
mechanism to satisfy a stale gate rather than fixing the gate.
Execution is a DOCFIX against the phase-6 runbook (Step 1 inventory prose, Step 5
verb and gate), logged to the Phase-3 batch, not executed under this ruling.
Revert: git revert this commit; the ruling note is additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

R4 RULED: reserved bands become enforced in MAAS via a DC-aware tool (D-134 amendment)
...
GA-R5: question and exact utterance quoted, dated, pushed before dependent work.
Operator utterance: "Build a DC-aware tool; full v4 scheme + FIP now, v6 bands
after the carve (Recommended)".
MEASURED BEFORE ASKING, which sharpened the question considerably.
The defect is worse than "bands not enforced": D-134's band table has never
existed anywhere but prose. maas admin ipranges read returns THREE ranges
cloud-wide, ALL type=dynamic, ZERO reserved.
The collision is now QUANTIFIED rather than hypothetical. MAAS's own
subnet unreserved-ip-ranges for dc1 metal-admin reports its lowest free span as
10.12.68.5-.99 (95 addrs) -- precisely the .4-.49 utility band and the whole
.50-.99 VIP band. Against that, the dc1 bundle places 27 LXD units in base form
and ~55 once dc-ha-scaleup scales 14 applications, each needing a MAAS-allocated
address, and the VIP band is where the 33 per-DC VIPs live. Zero 10.12.*
addresses are allocated today, so this is a PRE-EMPTION and not an incident.
MAAS's exact allocation ORDER is deliberately NOT asserted -- lens 2 was right to
refuse that claim and I have not added it.
Confirmed there is genuinely NO VR1 path: only site-headend-install.sh
(office1-only) and phase-00-maas-standup.sh can create an iprange, and the latter
REFUSES for any non-VR0 DC. That refusal is CORRECT -- it is the cross-DC mixing
guard lib-net's selector exists to enforce. The gap is that nothing replaced it.
NEW ARCHITECTURAL CONTENT, and it exists because of R2: D-134's bands are v4-only.
Ruling dual-stack made the 33 VIPs dual-family, which left the v6 planes with no
band discipline at all and the same collision waiting in the other family. The
amendment establishes that the v6 planes inherit an equivalent scheme. The exact
v6 octet mapping is NOT ruled -- mirroring the v4 host-part layout is the obvious
mechanical default, not a fresh design question.
The v6 pass following the carve is FORCED SEQUENCING, not a deferral: a reserved
range cannot be created on a subnet that does not yet exist, and the v6 plane
subnets are not in MAAS.
Worth noting what the tool buys beyond the reservation itself: a `check` arm gives
D-134 an EXECUTABLE gate. Today nothing in the repo can detect that the bands are
unenforced, which is why this survived from 2026-07-23 to now.
Execution is a separate gated step. Standard delivery discipline applies to the
tool (harness green, gauntlet, repo-lint, changelog with revert); the 12-subnet
reservation pass is an operator-gated live MAAS mutation and is not batched with it.
Revert: git revert this commit; the amendment is additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

R3 RULED: finish the jumbo underlay; first recorded MTU budget verdict
...
GA-R5: question and exact utterance quoted, dated, pushed before dependent work.
Operator utterance: "Raise the two lagging segments to 9000 (Recommended)".
Recorded as a D-101 RULING NOTE. D-102 is the original MTU sub-policy but is
MERGED INTO D-101 and its body directs amendments there.
MEASURED BEFORE ASKING, which materially re-framed the question. D-101 requires
"the measured underlay MTU is a Phase-0 gate -- do not assume jumbo", and
scripts/dc-dc-mtu-geneve-budget.sh had existed without ever being run to a
recorded verdict. Running it both ways, plus measuring the host bridges, showed
the jumbo branch is nearly complete already rather than a large project:
- Every vcloud MESH leg is ALREADY 9000, including mesh-vr1-dc0-vr1-dc1
(virbr5), the inter-DC path itself. All six plane bridges on both racks: 9000.
- The four 1500 legs are the D-125 SIMULATED-ISP uplinks. They model the
internet, D-125's egress gate is defined against them, and they must STAY
1500. Not a defect.
- Exactly TWO segments lag: enp1s0 inside both containment VMs, and all 17 MAAS
VLAN records.
The MAAS record is the one that bites silently: MAAS renders VLAN MTU into node
netplan, so a jumbo bridge beneath a 1500 record still yields 1500 node
interfaces. Jumbo bridges alone do not deliver a jumbo underlay.
Why (a): tenant MTU stays 1500, so no per-charm MTU coordination is needed at
all. Option (b) required ovn geneve + tenant-network MTU + amphora to agree
exactly and permanently across both DCs, and NOTHING in this repo checks MTU --
neither cloud-assert.sh nor provider-bundle-check.py has any MTU assertion -- so
drift would be silent, which is precisely what D-101 calls "the classic
nested-OpenStack failure mode". Option (c) was the worst: jumbo at the plane
layer with a still-capped transit throttles D-108 rbd-mirror and radosgw
multisite behind a chokepoint invisible where an operator would look.
COUPLING TO R2, recorded because it changes the arithmetic: the 56-byte overhead
is the IPv6 figure and applies BECAUSE dual-stack was ruled. Under v4-only it
would have been 42, and the 1500-underlay tenant MTU 1458. R2 and R3 are not
independent.
Execution is a SEPARATE gated step and is NOT authorised by this ruling. The
verification owed is BEHAVIOURAL: an end-to-end large-frame test with DF set
across the inter-DC path. An interface claiming 9000 is not proof a 9000-byte
frame survives the path -- the assert-on-content rule applied to MTU.
Also worth recording: invoked bare, the budget script correctly REFUSES ("FAIL:
--underlay-mtu is REQUIRED -- no default, measure it this session"). It does not
guess. That discipline is why this verdict is trustworthy.
Revert: git revert this commit; the ruling note and capture are additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Second gap sweep: U15-U17 closed; DC data path has no IPv6 at ANY layer
...
Operator: "Close the remaining gaps". All three measurable items are now
measured; what remains open is blocked by a known missing credential or is a
deliberate deferral, and both are stated with cause rather than left implicit.
U15 -- voffice1 transit addressing is REBOOT-DURABLE. A positive result, which
is worth recording as clearly as a defect: live enp2s0 172.31.0.1/30 (dc0 leg)
and enp3s0 172.31.0.5/30 (dc1 leg), both netplan-persistent via
/etc/netplan/60-transit.yaml and 61-transit-dc1.yaml. The D-128 Plane-2 path
survives a headend reboot and no site-baseleg row is owed for it.
U16 -- RETROFIT_WAIT=30m has NO recorded provenance. git log -S traces it to a
single bulk commit ("New Phase Scripts") with no rationale, and a repo-wide grep
finds no constant anywhere in scripts/ documented as nested-virt or depth-4
calibrated. It is an inherited default that has never been validated against the
nested I/O it will actually run on at Stage 5 Step 9. Log-only, but it should
not be mistaken for a tuned value. Bonus finding from reading the file: that
script's own preconditions require BOTH the openstack and juju clients, and no
host currently has both -- vcloud has neither, voffice1 has juju only. It cannot
run anywhere today, which reinforces S-1.
U17 -- the DC data path carries NO IPv6 at any layer measured. On BOTH racks:
zero global v6 addresses on any plane bridge, no v6 default route, and
accept_ra=1 (so they would accept a router advertisement; none is arriving).
Combined with the existing measurements -- no v6 subnet on any of the 12 MAAS
plane fabrics, no v6 link on any of the 18 nodes -- this is a THREE-LAYER
confirmation of L2-3. It also WIDENS the R2 propagation task: it is not only
"carve into MAAS", the rack bridges have no v6 either.
STILL OPEN, with cause rather than silence: the two DC edges' own interface-level
v6 configuration. dc0 is blocked by a known missing credential, not by a failure
to look -- ~/vr1-dc0-creds/ contains no opnsense-api.txt while ~/vr1-dc1-creds/
does, which is SEC-021(a) directly visible on disk, the residual item the
2026-07-27 consolidation batch deliberately excluded because the re-mint is a
live edge mutation. dc1's API is credentialed but measured unreachable from
vcloud (timeout; the path runs from the rack, where the edge creds are correctly
not staged per SEC-015). U17 answers the substantive question from the rack side.
repo-lint/gauntlet ON voffice1 remain deliberately deferred until precondition
0.1 advances that clone -- running them against a 105-commit-stale tree would
produce a number that means nothing. That is a judgement, not an omission, and
it is recorded as such.
Revert: git revert this commit; register and CURRENT-STATE changes are additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

UNMEASURED-gap sweep: 14 deferred items closed; juju restore-backup CONFIRMED absent
...
Operator challenge: did the committee actually measure live state, or take
shortcuts? Answer: some of both. This commit is the accounting, plus the sweep
that closes what was closeable.
WHO DID WHAT, precisely, because the accounting matters:
- The apex was never polled by ANY lens. Lens 2 declared it out of scope and did
so HONESTLY -- its UNMEASURED note explicitly warned "D-101's literals may
exist in NetBox and simply not be carved into MAAS". No overclaim. But it is a
key system that lens had reach and time to measure. Under-scoped, not dishonest.
- The two-day-old dump was MINE, in the correction, not an agent's.
- Sharpest instance of the general problem: lens 6 declared juju restore-backup
unverifiable because "no Juju client exists on this host", while juju 3.6.27
was installed on voffice1 and lens 5 was running juju help against it in the
same session. One lens called impossible what another was measuring.
CLOSED BY THE SWEEP (14 items). The consequential ones:
- LIVE apex polled via netbox/office1-record-dump.py: 139 prefixes, 103 IPv6,
ZERO added, ZERO removed vs the dump. The conclusion was right; the method was
not, and that distinction is the whole point.
- juju restore-backup DOES NOT EXIST on 3.6.27 -- "not a juju command ... Did you
mean: create-backup". L6-14 goes from flagged RISK to CONFIRMED DEFECT: Stage 6
Step 9's D-104 restore drill and its DoD bullet are unsatisfiable as written.
- juju bind DOES exist as documented -- lens 6 was RIGHT to exclude it from L6-7
rather than assert it wrong. A correctly-handled uncertainty.
- dc0 compute provider MACs measured (52:54:00:1b:19:e6 / 52:54:00:18:ab:b4),
matching lib-hosts and CONFIRMING L3-8.
- curl present on both racks, so L4-13's hole is theoretical, not live.
- The pinned charm channels DO resolve (2024.1, 2.4, squid all present via juju
info on voffice1) -- proving L4-2's root cause and its fix: P3's 33 warns are
purely the missing juju binary on vcloud, not a charmhub problem.
NEW FINDING: preflight's "MAAS unreachable: 'maas admin subnets read' failed" is
a MISDIAGNOSIS -- the maas binary is simply ABSENT on vcloud. Together with the
absent juju (L4-2) and absent openstack (S-1), THREE separate preflight/deploy
failures on this jumphost are all "the client is not installed", and each is
reported as something else. A tool-absence must never be reported as a
target-unreachable. Its own DOCFIX.
STILL-OPEN gaps that are measurable and were NOT done are listed in the
register's section 3 explicitly, not buried: DC edge v6 state, repo-lint/gauntlet
on voffice1 (deferred on purpose -- that clone is 105 commits stale, so running
them today would measure a stale tree), voffice1 transit reboot-durability, and
the provenance of RETROFIT_WAIT=30m.
STANDING LESSON added for the next committee: before declaring an item
unmeasurable, VERIFY THE CONSTRAINT YOU ARE ASSERTING. The UNMEASURED escape
hatch is load-bearing for honesty and must not become a way to avoid looking.
Revert: git revert this commit; the register is new and the CURRENT-STATE
paragraph is additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

CORRECTION: the v6 literals are ASSIGNED (D-111), not pending -- R2a withdrawn
...
The operator asked whether the Office1 NetBox apex had actually been polled. It
had not. It has now been read, and the answer inverts a claim I made one commit
ago and a question I put in front of them.
WHAT IS TRUE: the v6 literals are assigned, ratified and recorded, and have been
since 2026-07-11 under D-111 (ADOPTED). Measured from
netbox/draft/vr1-office1-current-20260725.json -- 139 prefixes, 103 IPv6, every
relevant row tagged D-101/D-111: ULA fd50:840e:74e2::/48 with DC0 planes at
:220/:221/:230/:240/:250::/64 and DC1 at :320/:321/:330/:340/:350::/64; GUA
provider-public DC0 2602:f3e2:f02:10::/64 + VIP f02:11::/64, DC1
2602:f3e2:f03:10::/64 + VIP f03:11::/64.
WHAT I GOT WRONG: the R2 ruling note claimed D-101's "Remaining open item" (the
org ULA /48 and per-DC GUA carve) had become a Stage-5 precondition because
"dual-stack cannot deploy against literals that do not exist". They exist. R2a
("which literals") is WITHDRAWN as never having been open; no operator utterance
is owed on it.
THE REAL PRECONDITION IS NARROWER AND BETTER: propagation, not assignment, and it
needs no ruling. The ratified values are absent from the two places Stage 5
actually reads -- scripts/lib-net.sh carries no v6 arm at all, and MAAS carries
no v6 on any of the 12 DC plane fabrics. Both are mechanical copies from an
authoritative source, so both move from Part A (needs a decision) to the Phase-3
mechanical batch.
Note CURRENT-STATE's "DRIFT-FREE across netbox/lib-net/artifacts" is NOT
contradicted: drift-free means values appearing in more than one place agree, not
that coverage is complete. lib-net.sh simply never gained a v6 arm.
PROCESS FAILURE, OWNED AND RECORDED IN THE DECISION ITSELF: the audit's own lens 2
listed the apex as UNMEASURED and warned, in terms, that "D-101's literals may
exist in NetBox and simply not be carved into MAAS". That warning was correct and
available, and I put a question to the operator anyway on the strength of D-101's
stale prose. Trusting stale decision prose over an available measurement is
exactly what this audit was convened to catch.
NEW DOCFIX-CLASS FINDING, logged not fixed (hard rule 1): D-101's "Remaining open
item" paragraph still reads "pending NetBox assignment (gap #3)" for literals
D-111 adopted on 2026-07-11 -- a decision's own prose contradicted by a later
ruling, DOCFIX-200/204 class, and the direct cause of this error. Queued in the
Phase-3 batch.
The R2 ruling itself STANDS -- dual-stack as ruled. Only its stated effect changed.
Revert: git revert this commit to restore the prior (incorrect) framing; the R2
ruling in the preceding commit is independent and should not be reverted with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

R2 RULED: dual-stack as originally decided; the v6 literals move onto the critical path
...
GA-R5: question and exact utterance quoted in the Status block, dated, pushed
before dependent work.
Operator utterance: "Carve v6 and deploy dual-stack as ruled (Recommended)".
Recorded as a D-101 RULING NOTE -- a re-confirmation, amending nothing, in the
same shape as the 2026-07-25 note it follows.
WHY IT WAS A LIVE QUESTION AT ALL: the 07-25 ruling was measured against the
substrate for the first time by this audit and found unimplemented. Exactly ONE
IPv6 subnet exists cloud-wide and it is on the Office1 base fabric; none of the
12 DC plane fabrics carries one; zero v6 links across all 18 nodes; every node
reports default_gateways.ipv6 = NONE. D-101's matrix requires ULA on
data-tenant/storage/replication plus legs on metal-admin/metal-internal, with
nothing to bind against.
THE CONSEQUENTIAL EFFECT, and the reason this commit matters beyond recording a
choice: D-101's own "Remaining open item" -- the org ULA /48 and per-DC GUA carve
-- has been carried since authoring as "pending NetBox assignment ... not a
ratification question", i.e. non-blocking. It is now a STAGE-5 PRECONDITION.
Dual-stack cannot be deployed against literals that do not exist.
Downstream, all now inheriting dual-family: R9 (where dc1's literals live), R11
(vault/designate VIPs, which must be dual-family from the outset or their cert
SANs get re-issued on a live cloud), and the L3-9 overlay collision, which must
be reconciled BEFORE either authority location is populated. Worth restating: of
the two merge orders the audit measured, the DANGEROUS one is the one that
PASSES -- vips-overlay-last silently replaces every v6 leg and reports green.
R8 (octavia's family) is explicitly NOT resolved by this ruling.
NEW SUB-QUESTION OPENED, queued not ruled: R2a, which literals. The ruling
directs that they be assigned; it does not assign them. I have deliberately not
proposed prefixes -- that is the operator's address space, and hard rule 2
forbids inventing a literal. Only the SHAPE is presented.
Revert: git revert this commit; the ruling note and R2a are additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

R1 RULED: ceph-osd gets a real second disk on the storage nodes (D-121 amendment)
...
GA-R5: question as presented and the operator's exact utterance are both quoted
in the Status block, dated, and pushed BEFORE any dependent work.
Operator utterance: "Add an OSD volume to node-vm (Recommended)".
Recorded as a D-121 AMENDMENT rather than a new D-number. D-121 is the decision
that ratifies modules/node-vm sizing/count, and its own capacity re-validation
already records "Ceph disk re-run for 4 storage/DC = PASS 5.31 TiB" -- the second
disk was BUDGETED and never BUILT. GA-R3: architectural consequence, but it
amends a ruled node layout rather than establishing new ground.
TWO SCOPE CORRECTIONS landed with the ruling, both of which narrow it:
- It is EIGHT volumes, not eighteen. ceph-osd is placed on the four storage
nodes per DC (bundle.yaml:556-557, to: ["5","6","7","8"]). The other five
nodes per DC neither run ceph-osd nor need a second device. My earlier framing
said "all 18 nodes" -- true of the measured defect, wrong about the fix.
- Rack headroom measured before asserting feasibility: dc0 2.0T available, dc1
2.9T available under /var/lib/libvirt, thin-provisioned qcow2.
Why (a) over (b): MINIMIZE DELTA TO ROOSEVELT -- Roosevelt storage nodes have
real dedicated disks. Option (b) was also not safe to pick as presented: whether
ceph-osd at the pinned squid/stable supports a directory- or partition-backed
OSD was explicitly UNRESEARCHED, and the audit refused to assert it either way.
THE APPLY IS NOT AUTHORISED BY THIS RULING. Four preconditions recorded, the
sharpest being that re-commissioning is required for MAAS to see the new device,
and whether that preserves the D-134 statics and pinned MACs is UNVERIFIED --
the 2026-07-20 MAC-regeneration incident is the precedent for exactly that class.
modules/node-vm is shared by both DCs, so the change must be additive/opt-in or
it plans against all 18 domains.
Revert: git revert this commit; the amendment is additive and no artifact changed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

CURRENT-STATE: land the dc1 egress measurement (repairs the prior commit's L10)
...
The previous commit (c1f5dc8) touched docs/audit/ without the paired
CURRENT-STATE update that GA-R1 amendment C1 requires, so repo-lint L10 failed
on it. Owned rather than silently amended: the commit is pushed, so this is a
fix-forward.
The measurement genuinely belongs in CURRENT-STATE under amendment C2. The
Stage-5 bootstrap entry has carried the hard ordering caveat -- bootstrap fetches
the juju agent stream and juju/juju-db snaps pre-apt, outside the apt proxy, so
it works ONLY while the dc1 edge egress is open -- but it carried no measurement
of whether egress actually IS open. It now does, probed from the dc1 rack with
--noproxy so an apt-cacher hit could not fake it: streams.canonical.com/juju/tools/
returns 200, api.snapcraft.io answers, archive.ubuntu.com 200, ping 1.1.1.1 0%
loss, default route via 10.12.64.1.
Stated with its shelf life: re-probe immediately before bootstrap, because the
D-135/D-107 narrowing is the closing mutation and nothing prevents it landing
first.
Revert: git revert this commit; the edit is additive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Stage-5 grounding audit: lens 5 + the three deliverables
...
VERDICT: Stage 5 would NOT run error-free today, and would fail early. The
substrate underneath is excellent -- all three OpenTofu roots ZERO DIFF, 18
nodes Ready exact to D-121 Option C, MACs and power addresses matching
lib-hosts, D-134 statics perfect, 17 fabrics, zero orphaned interfaces, both
artifact paths serving, gauntlet ALL GREEN (81). What is not ready is the
layer between the substrate and the deploy.
LENS 5 (relaunched after an API error) returned the ordered precondition list
and the largest runbook defect found: Step 4 says "follow phase-01 verbatim",
and phase-01 ACTIVELY REFUSES dc1 -- its VIP guard greps bundle.yaml for
eleven 10.12.4.x VIPs, dc1's live in an overlay on 10.12.64/68/72, so it
takes the "ABORT: VIP guard failed" branch. After the ruled VIP extraction it
aborts for dc0 too. It also carries hardcoded VR0 system_ids, a jumphost-local
libvirt loop over disks that do not exist, and a 4-machine plan gate against a
9-machine bundle.
Three Stage-5 gate commands cannot execute at all, measured against the juju
actually installed (3.6.27): juju run used for a shell command when it is the
action runner; download-backup given a backup-id when it takes a controller
path; and the geneve gate grepping ovn-central for a config key the same
runbook says does not exist. Four of the five VERIFY-LIVE gates the record
says Stage 5 owes have NO step in the runbook -- including the keystone
policyd-override check, which is RULED.
Two convergent confirmations raise confidence in the whole set: the Ceph OSD
blocker was found independently by two lenses using different methods, and the
stale-clone blocker independently by this session and lens 5.
DELIVERABLES:
- docs/audit/stage5-readiness-20260727.md -- ordered precondition checklist,
READ FIRST. 5 phases, each row with status/evidence/what-breaks.
- docs/audit/stage5-committee-raw-20260727.md -- all 7 lenses verbatim.
- docs/audit/queued-rulings-20260727.md -- 11 Stage-5-blocking + 4 standing
questions, GA-R5 shape, one exchange each, blank utterance lines. NONE
adopted; a batch answer rules NOTHING.
MECHANICAL FIX TAKEN (exactly one, deliberately): the G3 gate row read OPEN
with a standing FREEZE while its own cited evidence file records G3 CLOSED and
the freeze lifted. Left standing, that clause would have blocked the very
DOCFIX batch this audit queues. No ruling was required -- two surfaces already
declared it closed.
The 21-item DOCFIX remediation batch is LOGGED NOT EXECUTED. Nearly every
runbook fix interlocks with an unanswered ruling, so landing them now would
encode assumptions about questions the operator has not answered.
Next-free numbers unchanged (D 138 / DOCFIX 205 / BUNDLEFIX 053) -- no number
was assigned, correctly, since nothing was remediated.
Revert: git revert this commit; the deliverables are new files and the
CURRENT-STATE edits are additive plus the one G3 correction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Stage-5 grounding audit: lens 4 (gate integrity) + privileged creds re-run
...
LENS 4 broke seven gates with CONSTRUCTED, MEASURED fixtures. Independently
re-verified by this session before landing:
- repo-lint reports "PASS (0 fail, 0 warn)" over ZERO files when given a
one-character typo of either its documented flag or the repo path. This is
the gate whose 0-fail every GA-R6 stage-close commit cites; the verdict is
not evidence the repo was read. Reproduced here exactly.
- provider-bundle-check PASSES on decorative HA: `grep -rn cluster_count
scripts/ tests/` returns NOTHING, while the overlay carries 20 such lines.
A 3->1 rewrite of every cluster_count yields a byte-identical PASS.
- preflight's aggregator ignores every sub-gate exit code that is not 1 or 2,
so a gate that crashes (127) leaves "PREFLIGHT: PASS -- clear to deploy".
- P3 verified ZERO of 33 charm-channel pins TODAY because juju is not on the
host's PATH, and a missing BINARY is folded into the same WARN as an
unreachable charmhub. This CORRECTS this session's own earlier capture,
which waved the 33-warn line through as "expected" -- the count was
expected, the cause was not.
- G17's named check cannot fail: curl -sI exits 0 on 404/500 and the dc0 URL
is an autoindex root that answers 200 with nothing behind it.
Lens 4 also returned an 11-item COULD-NOT-BREAK list; notably the 2026-07-27
dc-mirror last-sync fix HOLDS under all five status arms, and creds-matrix's
tier-2 probe states could not be made to turn "could not look" into green.
CREDS: privileged sweep re-run (operator-authorized). Exit 1, still exactly
the same 7 findings -- no drift in a day. All three root-owned locations read
via sudo -n escalation, metadata only; ZERO E3 findings. Recorded as a
RE-CONFIRMATION of the 2026-07-26 privileged run, not a newly-closed item.
Six of seven lenses are now in docs/audit/stage5-committee-raw-20260727.md.
Lens 5 (ordered preconditions) is still running after an API-error relaunch.
Revert: git revert this commit; all files are audit artifacts plus additive
CURRENT-STATE paragraphs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Stage-5 grounding audit: live measurement sweep + first five lens returns
...
MEASUREMENTS (capture docs/audit/stage5-live-measurement-20260727.txt):
- ALL THREE OpenTofu roots plan ZERO DIFF -- outer on vcloud, and for the first
time both inner roots on voffice1 in the same session. Validity of the inner
pair was established FIRST by proving those roots are byte-identical to main
despite the voffice1 clone being 105 commits behind; a non-empty diff would
have made them UNMEASURED, not green.
- Gauntlet ALL GREEN (81). repo-lint 0 fail / 1 warn (the L1 legacy carve-out).
- preflight exits 1 for EXACTLY the known reasons -- nothing new has joined.
- dc0 full mirror and dc1 caching proxy both genuinely serving; dc0 sync
attested OK 2026-07-27T08:43:46Z.
CURRENT-STATE corrections under GA-R1 amendment C2 (measurement wins, corrected
in the capture's own commit -- this is also what repo-lint L10 requires):
- Juju on the headend measured 3.6.27, not the recorded 3.6.25. In-channel
patch refresh, which D-071 permits, so policy-compliant drift, not an
incident -- but it is the client that will bootstrap the controller.
- NEW pin row: the openstack client is ABSENT on BOTH hosts while ten
Stage-5/6/7 scripts invoke it. No prior surface carried this.
- Section 5 convergence history extended with the three-root re-confirmation.
OWNED MEASUREMENT ERROR, recorded rather than quietly corrected: the artifact
checkers were first run LOCALLY on vcloud and printed a comprehensive FAIL with
every probe MISS -- output indistinguishable from "never installed". Both
scripts are designed to be PIPED TO THE RACK. Corrected by direct functional
probe; lens 2 independently ran the canonical piped form and got PASS on all
three. The scripts are not at fault.
FIVE LENS RETURNS landed verbatim in docs/audit/stage5-committee-raw-20260727.md
(lenses 1, 2, 3, 6, 7). Findings are NOT yet verified -- verification is the next
step and a refuted finding stays in that file with its refutation, never deleted.
Lens 5 died on an API error and was relaunched; lens 4 still running.
Revert: git revert this commit. The two CURRENT-STATE edits are additive; the
audit files are new.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

close-out durability sweep: 4 transcript-only items landed; dc1-is-not-a-mirror warning
...
Operator-requested sweep for content that existed only in the session transcript, before
clearing. Precedent: docs/audit/queued-findings-20260726.txt. Capture:
docs/audit/queued-findings-20260727.txt.
THE CONSEQUENTIAL ONE. scripts/dc-mirror.sh's dc1 site row carried NO warning that dc1 is no
longer a mirror site, so `dc-mirror.sh install dc1` would have SILENTLY REBUILT everything the
same-day teardown removed -- units, nginx vhost, sync helper, and an ENABLED daily debmirror
timer that would start a fresh ~950G pull on a rack whose ruled artifact path is the proxy. That
is precisely the command someone would reach for after treating a failing `check dc1` as a
regression. The row now states dc1 is proxy-only per the D-135 amendment, that `check dc1` FAILS
BY DESIGN, and that `install dc1` is a deliberate strategy change and never a repair. The row is
RETAINED so its measured values survive a legitimate rebuild.
QUEUED, not built (hard rule 1): a RUNTIME guard in do_install for a site whose ruled path is
not the mirror. A comment is strictly weaker than a guard, and prose-only prevention has already
failed twice on this project (DoD item 8 missed dc1; D-137's wiring sat as prose and missed BOTH
DC standups). Recording that the interim is an interim.
A WRONG CLAIM CORRECTED. docs/audit/stage4-mirror-gate-20260727.txt said `systemctl reset-failed`
would re-arm the paused dc1 debmirror. It cannot: reset-failed on the SERVICE does not start an
INACTIVE TIMER. The real vector was a REBOOT -- `enabled` starts the timer at boot and
`Persistent=yes` fires it IMMEDIATELY because its window had passed. Correction APPENDED rather
than edited in place: the file is dated evidence, so a wrong analytical claim is superseded
openly, not quietly rewritten.
THE GENERALISABLE LESSON, previously unrecorded, is now platform-traps section 5:
5a "stopped is not dormant across a reboot" -- enabled + inactive + EMPTY NextElapse looks
harmless and is not; plus reset-failed does not re-arm, and Result=signal/status=15 on a
oneshot means it was SIGTERM'd and never wrote a terminal state (so a stale RUNNING is a
corpse, not progress).
5b a `oneshot` with RemainAfterExit=yes does NOT undo its work on stop, so stopping it proves
NOTHING about dependents -- which is exactly why the dc1 teardown test deleted the live
address and route instead of stopping the mirror net unit.
Three rows added to the verbatim-error index. Note dc0-mirror-sync.timer is still enabled with
Persistent=yes, so 5a describes dc0 today -- benign while its syncs succeed, same shape.
HARD RULE 2. dc-cache-proxy.sh stated that the two net units coexist safely on one host as
FACT. It is REASONED (idempotent replace verbs, identical measured values), never MEASURED --
the proxy has only run on dc1, where the mirror unit was then removed, so nobody has run both.
Marked accordingly, with an instruction to verify rather than trust the sentence.
VERIFICATION: dangling-reference sweep over docs/, runbooks/, scripts/ and the skill -- every
path this session introduced or cited RESOLVES; all pre-existing dangles are legitimate
(deleted-as-history, not-yet-built, or the deliberately-absent octavia PKI overlay preflight P4
fails on). ledger-scan reconciled against the machine-derived block. Gauntlet ALL GREEN (81),
repo-lint 0-fail, dc-mirror 24/24, dc-cache-proxy 20/20.
Revert: text + comments only; the capture correction is an append.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

CURRENT-STATE: Stage 4 CLOSED 2026-07-27 -- merged 6f5701d, branch retired
...
GA-R1 C1: the merge is a status change, so the status authority records it in the same change
set. Stage 4 / Phase 3 flips OPEN -> CLOSED with its evidence: operator-gated merge to `main`
as merge commit 6f5701d (2 parents, NOT squashed; 77 commits), branch
`dc-dc-stage4-phase3-maas-deploy` retired local + remote after confirming containment with
`git branch --merged main`. Post-merge verification ON main: gauntlet ALL GREEN (81),
repo-lint 0-fail.
Recorded as travelling forward BY DESIGN rather than silently dropped: gate G17 (node-side
artifact reachability, triggers at Stage 5 first boot), 21 open SEC rows, and 7 residual
credential-register findings including the dc0 edge-API re-mint that was deliberately excluded
from the consolidation batch.
The next stage branches off post-merge `main`.
Revert: text-only. The merge itself is 6f5701d.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Stage 4 close-out set: GA-R2 consolidation, skill sweep, snapshot, GA-R7 review
...
Every substantive Stage 4 item is closed or split to gate row G17. This is the GA-R6 close-out
set; only the operator-gated merge to `main` remains.
GA-R2 CONSOLIDATION: the 7 stage changelogs archived to docs/archive/changelogs/ (top-level
docs/ 25 -> 18, 113 archived total) and the stage record written at
docs/archive/stage-records/vr1-stage4-record.md. It records what the stage actually delivered
(READY handoff, not deployed nodes), the three things a future session should read first, and
the false-green pattern this stage kept surfacing. 12 stale `docs/changelog-*` paths in live
surfaces rewritten to the archive -- including several already dangling from the G12 close, so
every changelog reference in CURRENT-STATE / session-ledger / security-ledger now resolves.
SKILL SWEEP: three new INVARIANTS folded in --
- per-DC ARTIFACT DELIVERY is a per-DC STRATEGY (dc0 full mirror / dc1 caching proxy, D-135
amendments); checking dc1 as a mirror fails BY DESIGN. Corollary, learned the hard way: a
per-DC utility service OWNS ITS OWN network prerequisites -- never let one strategy's
teardown be able to break the other's.
- Stage 4 hands off READY nodes, NOT deployed ones (DOCFIX-200). Any surface saying "nodes
Deployed" as a Stage-4 exit is a defect; four were found and fixed. Consequence: nothing
node-side is probeable in Stage 4, which is why G17 exists.
- "A CHECKER THAT CANNOT FAIL IS NOT A GATE", with the working rules that came out of two
measured false greens: assert on CONTENT not existence; an unrecognised state REFUSES;
"could not look" is never "nothing there"; ENUMERATE what exists rather than only auditing
what is declared, because a register cannot report the absence of a row it does not have;
and when a test asserts a literal finding string, REPLACE the assertion, never delete it.
Plus two routing rows (per-DC artifact checks; D-137 as the credential-lifecycle authority).
SNAPSHOT REGENERATED: .claude/skills/openstack-cloud-ops-consolidated-20260727.md (1589 lines,
ASCII + LF byte-verified, all 7 sections). The superseded 20260725 snapshot is REMOVED rather
than kept alongside -- a stale snapshot being uploaded is the exact failure
docs/audit/skill-divergence-20260725.md records (a failed clone + a mis-shaped handoff), and it
is a derived artifact regenerable from any commit.
GA-R7 MEMORY REVIEW: NO new memory written -- everything durable from this stage graduated to
the skill and the repo, which is the correct GA-R7 outcome, not an omission. Both existing
entries verified against the repo (the finding-class baseline they cite does exist) and extended
with the two reasoning traps this stage produced: the register cannot see a credential it has no
ROW for, so enumerate what exists; and diagnose a denied call by checking whether an existing
rule failed to MATCH before blaming the classifier (the measured 2026-07-26 root cause). Also
marked the ask-vs-allow precedence claim UNVERIFIED so no future session relies on it.
Noted, not actioned (hard rule 1): T24's finding-class baseline covers TIER 1 only, so tier-2/3
classes have no baselined red state and a future false green there would not be caught.
Gauntlet ALL GREEN (81), repo-lint 0-fail.
Revert: git revert this commit; the changelog moves are renames and the snapshot is derived.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

set-interface-v4: reload the pf filter on --commit (RULED); appendix-A rewritten
...
RULING (GA-R5) 2026-07-27. Question as presented: (a) add `configctl filter reload` to the
script's --commit path; (b) leave the script and rely on DC standup DoD item 8; (c) put it behind
an opt-in flag. Operator answer, exact utterance: "a". OPS under GA-R3 -- a script fix completing
an already-documented operation, governed by D-113; no new D-number.
DEFECT (appendix-A, measured on the fresh dc1 edge 2026-07-23): --commit ran `configctl interface
reconfigure`, which applies the address to the kernel but does NOT regenerate the pf ruleset, so
OPNsense never creates its automatic outbound-NAT rules for the (re-)addressed LAN. Forwarded LAN
packets left the WAN un-NAT'd and replies never returned -- the edge passed NO LAN traffic while
looking perfectly configured. Self-masking: any later plugin install or config apply reloads the
filter as a side effect, so dc0's 07-20 qga work hid it there until dc1 exposed it.
Option (b) rejected on evidence, not taste: DoD item 8 already existed and did NOT fire at dc1,
and prose-only controls have a measured failure record here (D-137's own enforcement sat as prose
at phase-3:498 and missed BOTH DC standups).
PLACEMENT IS THE SUBSTANCE OF THE FIX, and my first attempt got it wrong -- caught before
shipping. Putting the reload on the next line of the `interface reconfigure` heredoc would have
been a fix that fixes nothing exactly when it matters: re-addressing the interface you arrived on
drops the session DURING the reconfigure, so a following command in that stream never executes --
and that drop case IS this incident's own scenario (dc1's LAN moved 192.168.1.1 -> 10.12.64.1
over the LAN itself). As shipped the reload runs over a FRESH connection to PROBE_HOST.
Also added a `pfctl -s nat` read-back with the `proto carp` boilerplate filtered out, since
boilerplate-only is precisely appendix-A's broken-state signature. REPORTED, not gated: an edge's
first addressing may legitimately have no gateway and so no NAT to generate, and hard-failing
would block a correct run. The hard gate stays address-on-the-kernel.
Harness 59/59 (was 53), cases 15-15f. Case 15f caught a real inconsistency -- appendix-A still
said "not yet ruled". Case 15e caught a bug in my own first draft, which scanned to EOF and so
failed the script for having the very hard gate the case exists to preserve.
appendix-A rewritten: the FIX stays for an edge already in this state, plus a PREVENTED-IN-THE-TOOL
block recording the placement detail and why prose prevention was rejected.
Gauntlet ALL GREEN (81), repo-lint 0-fail. No live edge touched -- both DC edges are already
addressed, so this affects only the NEXT edge addressing.
With this, every substantive Stage 4 in-stage item is closed or split to gate row G17; only the
GA-R6 close-out set remains.
Revert: git revert this commit's script + harness + appendix-A changes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

SEC-024 retention RULED: keep both state-surgery snapshots (GA-R5)
...
Question as presented 2026-07-27: delete both `pre-*` state-surgery snapshots (cleanest, closes
SEC-024 fully), keep both and let preflight P5 keep watching them, or keep pre-G6 and delete
pre-G16 -- noting both gates are CLOSED, both carry the Office1 MAAS API key in plaintext per
DOCFIX-175, and deletion is irreversible with no other copy of that pre-surgery state.
Operator answer, exact utterance: "Keep both".
opentofu/terraform.tfstate.pre-G6-20260719 and .pre-G16-20260721 are RETAINED at 0600, neither
git-tracked, as the only record of state prior to the two direct tfstate edits (G6 autostart
injection, G16 channels reconcile).
SEC-024 stays OPEN as a standing WATCH rather than an open remediation: the mode defect is
fixed, but the umask CAUSE was out of ruled scope so a future apply may rewrite 0664, and P5 is
the detection. Recorded in the SEC-024 disposition cell and in CURRENT-STATE. Open SEC count
unchanged at 21.
repo-lint 0-fail.
Revert: text-only; the ruling record reverts with git revert. No file was deleted (that is the
point of the ruling).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

Stage 4 cleanup: carve residue 108 fabrics -> 17; dc1 leftover nginx purged
...
Both items were queued to the stage close at the 2026-07-23 carve.
CARVE RESIDUE. Deleted subnet id=8 192.168.1.0/24 (the superseded OPNsense FACTORY LAN -- both
edges were long since re-addressed to 10.12.4.1 / 10.12.64.1), then fabric-5, then the 90
auto-created empties (ids 6..95). 108 fabrics -> 17.
The audit caught an ORDERING DEPENDENCY the original flag did not state: 192.168.1.0/24 was the
ONLY occupant of fabric-5, so deleting the fabric first would have cascaded the subnet away
instead of removing it cleanly. Subnet first, then the fabric.
METHOD, chosen because of this repo's own history: a MAAS fabric delete cascades to its VLANs,
and the 2026-07-21 pod-delete incident cost 9 machine records precisely because the association
check ran AFTER the delete. So emptiness was PROVEN per fabric -- zero subnets, zero ipranges,
zero node interfaces across all its VLANs -- against a fresh occupancy snapshot (fabrics +
subnets + ipranges + every machine's interface_set) taken immediately before the batch, and the
cascade check was re-run afterwards too.
KEPT, all verified occupied: office1 base + GUA (fabric-0), office1 compose with its 2 LXD
guests (fabric-1), both transits (fabric-2 / fabric-141), both metal-admin/boot fabrics with 9
nodes each (fabric-4 / fabric-142), libvirt default (fabric-3), and the 10 named plane fabrics.
POST-STATE: 18 Ready + 2 Deployed office1 guests, all 18 still power_type=virsh, 7 interfaces
each (6 flat planes per D-133 + br-ex), ZERO orphaned interfaces, placement tags 9+9 intact,
both DCs' artifact paths re-verified PASS.
NGINX ON dc1. Purged nginx + nginx-common (dry-run first: nothing else depended on them, no
nginx-core was installed), plus autoremove of liblockfile-simple-perl and liblog-agent-perl --
debmirror's orphaned perl deps. /etc/nginx gone, nothing on :80, proxy still PASS on :3142. dc0
keeps its nginx and was re-verified unaffected (it serves the mirror).
Capture docs/audit/stage4-carve-residue-cleanup-20260727.txt. repo-lint 0-fail.
Revert: fabrics/subnets are MAAS records, not config -- no undo, and nothing needs them (90
were auto-created empties; 192.168.1.0/24 was a superseded factory address). nginx reverts with
apt-get install nginx, which dc1 has no use for while it is the proxy DC.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

dc1 full mirror REMOVED (D-135 amendment); dc-cache-proxy.sh now owns its net layer
...
Operator clarified intent, exact utterance: "DC0 is the test of a full mirror. DC1 is the test
of the mirror proxy. DC1 needed the partially created full mirror and associated
script/files/settings removed." Recorded as a D-135 AMENDMENT 2026-07-27 -- the per-DC split
is the DELIBERATE EXPERIMENT, not an interim state. That supersedes both the "PAUSED ... kept
dormant as the fallback" record and the "DC1's proxy is REPLACED by a full mirror afterward"
claim, which sat in THREE places (D-135, CURRENT-STATE:109, dc-cache-proxy.sh:6); all corrected.
dc0 FIRST (gated): sync re-run -> OK 2026-07-27T08:43:46Z ubuntu=0 uca=0 in 20s, confirming
the transient-500 diagnosis. The fixed check now reports PASS -- trustworthy precisely because
the same check FAILED on the same rack twenty minutes earlier. Stage 4 DoD bullet 5 is now MET
for both DCs (dc0 mirror PASS, dc1 proxy PASS), node-side half at G17.
WHY THE ORDER MATTERED. Asked to weigh rip-down-and-rebuild against surgical removal, the
rebuild path turned out to be BLOCKED, not merely slower: dc-cache-proxy.sh did not own its
utility net layer -- it REUSED dc-mirror.sh's <site>-mirror-net.service and refused to install
without it. So on dc1, whose ruled path is the PROXY, three files named dc1-mirror-* (net unit,
apply helper, resolved DNS drop-in) were load-bearing FOR THE PROXY. Consequences: the recorded
removal procedure would have silently killed the proxy, the bullet-5 PASS and the Stage-5 dc1
apt path; and the proxy could not be REBUILT afterward, since nothing but `dc-mirror.sh
install` created the net layer -- rebuilding a PROXY meant reinstalling the whole MIRROR
apparatus, enabled daily debmirror timer included. Operator direction: net layer first, then
rip it all down.
1. NET LAYER MOVED: <site>-cache-proxy-net.service + -net-apply + its own resolved drop-in,
reusing dc-mirror's proven mechanism (bridge from the libvirt net NAME at runtime, hard
rule 3). Proxy-named so a proxy-only DC has NO mirror-named dependency. Idempotent
replace verbs, so both net units coexist safely where a site runs both -- dc0 unaffected.
check asserts the ARTIFACTS, not just the live address/route: asserting only the live
effect would PASS while the layer was really the mirror's, which is the exact failure.
Harness 20/20 (was 15). T12 REPLACED not deleted -- it asserted the superseded design, so
it failed by design; now inverted, plus T12b-T12f. T12b is scoped to non-comment lines on
purpose: the header's account of the removed trap cites dc-mirror.sh install, and deleting
that history to satisfy a grep would lose why the trap existed.
2. INDEPENDENCE PROVEN BEFORE DELETING: a oneshot with RemainAfterExit=yes does not undo its
work on stop, so stopping the mirror unit would have proven nothing. Removed the live state
outright (addr count -> 0, default route count -> 0), applied the proxy-owned unit ALONE,
both restored; check dc1 PASS with dc1-mirror-net.service disabled.
3. WHOLESALE REMOVAL: all mirror units, timer, sync helper, nginx vhost, resolved drop-in, the
debmirror package, and 330G of partial mirror (33,037 files, rm -rf 47.6s; disk 339G ->
9.0G). Zero mirror-named residue, every proxy artifact present, proxy PASS throughout and
never down. Capture docs/audit/dc1-mirror-teardown-20260727.txt.
nginx left installed deliberately: a mirror prereq, but with the vhost gone it is a generic
server on its stock default vhost; removing the metapackage while nginx-core runs is a
half-measure and a full purge is scope creep on a rack host.
Gauntlet ALL GREEN (81), repo-lint 0-fail, dc-cache-proxy 20/20, dc-mirror 24/24.
Revert: `dc-mirror.sh install dc1` (needs debmirror reinstalled) then `sync dc1` to re-pull
~950G; git revert dc-cache-proxy.sh + harness to restore the mirror-owned layer. The 330G
partial is NOT recoverable -- deliberately, per the amendment.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|

G17 opened by operator ruling; Stage 4 DoD bullets 5-6 repaired (DOCFIX-204)
...
RULING (GA-R5), 2026-07-27. Question as presented: "The node-side half of bullet 5. Nodes are
powered off by the READY-handoff ruling, so no node-side probe can run as things stand. Either a
gated rescue-boot check on one node per DC now (closes it inside Stage 4), or split it into its
own gate row targeted at Stage 5 first boot (GA-R6 E3 explicitly permits this; a conditional
close is not permitted)." Operator answer, exact utterance: "split it into its own gate row".
Recorded in the G17 row of docs/CURRENT-STATE.md (the authority); pushed before dependent work.
G17 "Per-DC artifact source reachable FROM A NODE", [V], OPEN. dc0 -> curl 200 from a booted
node against the D-135 item-1 mirror; dc1 -> apt fetches through the proxy at 10.12.68.4:3142,
since dc1's ruled path is the CACHING PROXY with no node-facing mirror (D-135 amendment) and
checking it as one fails by design. Trigger: Stage 5 first boot. The row states that G17 is NOT
a Stage-5 precondition -- Stage 5's bootstrap needs open edge egress for the juju agent stream
and snaps (D-135 items 2-3 unbuilt), a different path from the apt artifact source.
Stage 4 retains bullet 5's rack-side half: the source answers on its own address with an
attested-current sync. dc1's proxy PASSES; dc0 pends the sync re-run.
DOCFIX-204 -- bullet 6 was UNSATISFIABLE and its stale text had spread to four surfaces. "NTP
from the DC's own OPNsense edge working" is superseded by D-129(iv) (RULED 2026-07-21, "Keep
MAAS hierarchy", no NTP role on the edge): the DoD asked for confirmation the edge is the time
source, which the ruling had already refused, so a correctly-built DC could never meet it.
Repaired in the phase-3 DoD (bullet 6 struck, bullet 5 rewritten per-DC), the phase-3 Step 7
(REWRITTEN -- it branched on gap #5 that D-135 resolved, applied one mirror check to both DCs
which fails on dc1 by design, and asked for checks "from a deployed node" that the READY handoff
makes impossible), the workflow-doc Gate cell (which also still said "Nodes deployed",
contradicting DOCFIX-200), the buildout design, and the phase-4 prerequisites (same "nodes
Deployed" error -- exactly the expectation DOCFIX-200 removed to stop Stage 5 breaking).
phase-4 also gains a block making it the OWNER of G17's captures, warning that first boot is the
observation window and missing it forces a deliberate rescue-boot.
Text and gate-record only; no live state touched. repo-lint 0-fail. DOCFIX next-free -> 205.
Revert: git revert this commit; all changes are documentation and the G17 gate row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf
|