Author: the Phase-0 administrator (multi-agent pass, SCOPE-AND-EXECUTION-PLAN.md Section 4). Date: 2026-08-09. Inputs: pass0-w1-substrate-map.md, pass0-w2-network-map.md, pass0-w3-consumers.md, pass0-w4-targets.md -- read in full, adversarially cross-checked against repo ground truth (checks logged in Section 3). READ-ONLY synthesis; no mutation.
This report ends in the Phase-0 OPERATOR GATE (Section 7). Do not start Phase 1 before the operator rules on it.
vcloud (outer libvirt, qemu:///system) <- OUTER ROOT: opentofu/main.tf
|-- Office1 arm: office1_network / office1-wan / office1_opnsense / voffice1 (NOT container-layer)
|-- ubuntu_noble_base (shared base image) (persists)
|-- D-100 mesh TRIANGLE (mesh-link x3, MTU 9000) + netem on dc0<->dc1 (persists)
| office1<->dc0 leg = D-124 transit: voffice1 NIC2 172.31.0.1/30 <-> vvr1-dc0 NIC1 172.31.0.2/30
| office1<->dc1 leg = transit: voffice1 NIC3 172.31.0.5/30 <-> vvr1-dc1 NIC1 172.31.0.6/30
|-- vr1_dc0_uplink / vr1_dc1_uplink (site-wan NAT 172.30.2.0/24 / 172.30.3.0/24, MTU 1500, D-125)
|-- vr1_dc0_storage / vr1_dc1_storage (outer pool -- holds ONLY the containment VM's own disk)
|
`-- vvr1-dc0 / vvr1-dc1 ==== CONTAINMENT VM (D-123 Model B) ====
108 vCPU / 480 GiB / ~3000 GiB (opentofu/variables.tf:137-156; RAM raised 416->480 2026-08-01)
NIC1 = transit (SEC-010 FORWARD-drop keys here); NIC2 = IP-less, enslaved into the
br-vr1-dcN-wan netplan bridge (D-125 bridge-in); expose_nested_virt = true
|
|== BOOTSTRAP GATE (scripts/site-headend-install.sh --role rack --host-nodes):
| nested libvirtd + inner pool dir + kvm nested=1 + OPNsense image staging +
| SEC-010 nftables writer + D-125 bridge verify. Bash, NOT tofu -- sits BETWEEN the roots.
|
`== INNER ROOT: opentofu/vr1-dcN-substrate/ (qemu+ssh into the containment VM, run
from voffice1 per D-128; D-126 per-env key auth)
|-- inner_storage pool (inside the containment VM's own disk image)
|-- the SIX planes (dc-planes, isolated L2, MTU 9000) -- RELOCATED here by D-123
|-- vr1_dcN_wan (wan-bridge onto br-vr1-dcN-wan) + vr1_dcN_opnsense (DC edge)
`-- vr1_dcN_node x 12: 9 D-121 role nodes (3 control / 2 compute / 4 storage)
+ vr1-dcN-juju-01 (.5, D-104) + vr1-dcN-maas-01 (.6, D-132 addendum)
+ vr1-dcN-tailscale-01 (.7, D-129(iii)); all MAC-pinned
OUTER apply (vcloud, Plane 1) -> BOOTSTRAP GATE (bash, over SSH) -> INNER apply (voffice1, Plane 2, qemu+ssh). Forced because "a libvirt provider cannot be configured from a resource created in the same apply" (opentofu/main.tf:28-29). Eliminating the containment VM removes the forcing function entirely: one root, one state axis, no gate script, no cross-host provider dial. This is the single largest structural simplification the elimination buys (W0.1 Sec 3).
The vvr1-dcN VMs + their inner libvirtd + the inner roots + the bootstrap gate's node-host mode + the D-125 bridge-in plumbing (wan-bridge module, IP-less uplink NIC, br-vr1-dcN-wan)
Census baseline: 424 vvr1-dc occurrences across 78 canonical files; ~26 in scripts, 17 tofu files; the bulk is decision-record prose (append-only -- superseding notes only, never edits).
| # | Consumer (file:line) | Keys to the containment layer via | On flattening | Severity |
|---|---|---|---|---|
| 1 | opentofu/main.tf:410-519,537-623 (module vvr1_dc0/_dc1) + sizing vars variables.tf:137-156,175-194 + D-124 rack-addressing vars :196-244 + D-126 pubkey vars |
IS the containment VM + its sizing/addressing/keys | DELETED. Sizing/rack-addressing/pubkey vars go with it; dc-dc-whole-host-budget.py containment-overhead flags re-derived or dropped |
HIGH |
| 2 | opentofu/vr1-dc0-substrate/, vr1-dc1-substrate/ (whole inner roots + states) |
qemu+ssh provider into vvr1-dcN | RETIRED as roots; their module CALLS (planes, edge, 12 node VMs, storage) re-home into the outer/flat root -- bodies unchanged, provider changes | HIGH |
| 3 | modules/wan-bridge + vr1_dcN_wan calls + the IP-less uplink NIC + br-vr1-dcN-wan netplan |
D-125 bridge-in (exists only to fix OBS-3 nesting egress) | DELETED; DC edge WAN attaches directly to the outer vr1_dcN_uplink NAT (Model A item 8 -- same /24, no re-address) |
MEDIUM |
| 4 | scripts/lib-hosts.sh:52,157-168,212-214,246-251 (VIRSH_POWER_ADDRESS*) |
qemu+ssh power URIs dial the containment VM's libvirtd | Re-derived to vcloud's own virsh; the FROM_OFFICE1/FROM_DCREGION split may collapse. Wrong values = "MAAS unreachable" masquerading as a network fault | HIGH |
| 5 | scripts/maas-node-power.sh |
power address as ARGUMENT (topology-agnostic) | No code change; every invocation site/runbook example needs the new address | MEDIUM |
| 6 | scripts/site-headend-install.sh --host-nodes (~140 lines) + SEC-010 writer |
THE containment-VM bootstrap + SEC-010 enforcement point | node-host mode = dead code for VR1; SEC-010's implementation must be REBUILT for the new boundary (Section 5), not moved. Rack-role remainder needs a re-homing decision (Section 6) | HIGH |
| 7 | scripts/dc-rack-net.sh (rack bridge-leg IPs .2/.3 + D-131 node DNS forwarder) |
runs ON vvr1-dcN; addresses are the containment VM's identity on its own inner bridges | Needs a NEW HOME (the Option-1 client VM is the natural candidate) or a ruled retirement -- D-131 is a DNS-availability control; silently losing it reintroduces the SERVFAIL bug | HIGH |
| 8 | SEC-028 juju service key + SEC-029 Octavia PKI overlay, resident on "the rack"; vm-secret-locations rack rows; SEC-026 isolation control |
vvr1-dcN is a credential-bearing host (first rack rows in the register) |
Residencies MIGRATE to the replacement client host; matrix/register rows re-point; SEC-026 "only that DC's credential" must be re-asserted; rotation triggers ("if the rack is rebuilt") FIRE on this change | HIGH |
| 9 | runbooks/dc-dc-teardown-rollback.md (Paths A/B/C/M) |
vvr1-dcN as the teardown unit; D-122's "site-down = one virsh destroy" |
Re-authored around a scripted group-destroy of the vr1-dcN-* domain set. The one-command site-down is a REAL D-123 regression to carry honestly; module design should re-earn it (tofu-module-scoped destroy) |
MEDIUM-HIGH |
| 10 | D-128/D-138 execution-host surface (runbooks/dc-dc-phase3:424,430, phase4:165-167; ssh -J voffice1 to the rack transit IP) |
vvr1-dcN is the concrete D-138 client host | THE load-bearing decision -- answered by proposal in Section 4 (Option 1's client VM), ruled at the gate | CRITICAL (gate item) |
| 11 | netbox/dc-rack-mgmt-import.py + its harness |
imports vvr1-dcN as a NetBox DCIM device | NetBox-side decommission/repoint (system-of-record data migration, not just code) | MEDIUM |
| 12 | scripts/site-baseleg.sh:40-48 DC rows |
deferred BECAUSE DCs nest behind qemu+ssh (deferral answered by D-138) | premise disappears; new leg only if the target needs one -- currently a no-op | LOW |
| 13 | scripts/dc-mirror.sh:6, maas-profile-assert.sh:39, maas-role-tags.sh:37, dc-egress-check.sh:63,72 |
doc-comments / usage examples naming vvr1-dcN or "the rack host" | doc-currency only; logic is host-agnostic | LOW |
| 14 | lib-hosts.sh:69-72 NIC_PLANE_ORDER/BREX_PARENT_NIC; :29-34 CARVE_AUX_HOSTS |
conventions realized by the inner root's MAC pinning | Conventions carry forward; the FILE that encodes the pinning changes (row 2). MAC re-capture owed post-rebuild (moot-ish: 10.13 rebuilds the fleet anyway) | LOW-MEDIUM |
| 15 | Non-consumers, named to prevent double-counting: scripts/lib-net.sh (0 hits -- IPAM facts change under D-143 only, a separate ruled axis); plane CIDRs/families (D-139); the 66+17 design-decision/CURRENT-STATE hits (append-only record) |
-- | -- | NONE |
All checks run against repo ground truth this session; a plausible finding was not accepted as a verified one.
docs/design-decisions.md:7216-7243, RULED 2026-07-30, exact utterance "Dedicated region VM per DC at utility .6 (Recommended)") puts each DC's MAAS REGION (regiond + PostgreSQL) in its OWN VM -- vr1-dcN-maas-01, already one of the 12 inner-root node VMs. The region survives flattening as a flat sibling with no redesign. What vvr1-dcN actually carries: the MAAS rack controller (--role rack), the D-131 node-DNS forwarder, and the D-138 client + SEC-028/SEC-029 credential residencies. This RE-GRADES W0.3's CRITICAL: the execution-host question is real but narrower than "three roles orphaned," and it is now ANSWERED-BY-PROPOSAL (Option 1's client VM), pending the gate.docs/archive/model-a-fallback-plan.md exists; tag model-a-fallback resolves to 114d392, dated 2026-07-16. That PREDATES the D-132 amendment + addendum and D-138 (both 2026-07-30), D-139's IPv6 family rulings (07-31), D-143 (10.13), the three utility node VMs, and the lb-mgmt plane. Its own MAAS line -- "region on Office1 + rack" -- is contradicted by D-132 as now ruled. Its Section-4 revert procedure (git checkout model-a-fallback -- opentofu/) must NOT be run: it would resurrect a 10.12-era, 9-node, Office1-region substrate. The real build path is re-homing the CURRENT inner-root module calls (W0.1's map) into a flat root, with Model A as shape precedent -- "Model A PLUS the rulings Model A never had," freshly validated. One thing the archive DOES verify in Option 1's favor: Model A itself kept a small non-hypervisor vvr1-dc0 headend (4/8192/80, expose_nested_virt=false, metal-admin + transit legs) -- Option 1's client VM is exactly that shape, repurposed.docs/design-decisions.md:7118-7124: "the per-DC management bastion holds only its own DC's cloud credential" / "Each DC has a management entry point inside it; the NOC reaches that entry point." The RULING is "the cloud-facing client lives IN the DC" -- a principle, with vvr1-dcN only as the then-concrete host. Option 1's client VM (inside the DC's planes, holding one DC's credential) is consistent with D-138's principle; the concrete-host change still needs recording (amendment vs. new D -- Phase 4 frames it, Section 6).pass0-w4-targets.md is the credential row of the tradeoff table. Both options put both DCs' plane bridges and node VMs on vcloud's single libvirtd. See Section 5.variables.tf:143-150 shows 416->480 GiB raised 2026-08-01 (W0.1/W0.4 are correct). (b) MTU: W0.2 rules the budget analytically unaffected; W0.4 lists it OWED -- reconciled as: analytically unchanged (same underlay_mtu=9000 var; vcloud-level mesh/office1 networks already run 9000), with a live jumbo/geneve assert still owed post-build (Section 8). (c) W0.3 row 4's "MAAS region is SHARED" echo (from SEC-026's pre-amendment text) is superseded by D-132's per-DC regions -- SEC-026's regional-blast -radius note is historical context, not current topology.runbooks/dc-dc-phase4-juju-bundle-per-dc.md:167 -- tool-placement table row: "anything at all | never the vcloud jumphost ... It picks the transport only" (measured 2026-07-29: no juju/maas binaries on vcloud). Runbook doctrine, not a D-ruling -- but Option 2 would reverse it, on top of inverting SEC-026's isolation control.Both options share the flat core: 12 node VMs + DC edge as vcloud-libvirt siblings; the six planes re-homed to vcloud level (same CIDRs/families/MTU -- IPAM identity untouched); D-125 bridge-in deleted (edge WAN -> direct NAT); mesh triangle unchanged; nesting depth 4 -> 2 (VR0-proven); one tofu root/state axis, no bootstrap gate, no qemu+ssh dial.
A dedicated ~4/8192/80 non-hypervisor VM per DC (vr1-dcN-client), legs = metal-admin + transit, carrying: the D-138 client role, the SEC-028/SEC-029 credential residencies, and (pending Section 6) the D-131 forwarder / rack-controller remainder. It is a flat utility sibling -- no nested libvirt, no expose_nested_virt -- so the containment PATTERN is gone even though a small VM remains.
Same flat core; no client VM; juju/openstack + both DCs' MAAS-admin-scoped keys land on the shared jumphost via a new host-side leg into metal-admin.
Recommendation: Option 1. Every verified axis -- risk delta, credential isolation, Roosevelt fidelity, runbook doctrine -- favors it; Option 2's only advantage is a marginal ~8 GiB/DC. The cross-DC gap (Section 5) applies EQUALLY to both options, so it is a gate condition on the flattening itself, not a tiebreaker. Framing per W0.4: "eliminate the container layer" is fully satisfied by Option 1 -- no VM is a hypervisor for another VM; the retained client VM is the same class as juju-01/maas-01. If the operator's intent is literally zero additional VMs, that is Option 2 with the above accepted knowingly.
W0.2's finding (Sec 4, "the single largest wiring risk"): today dc0's and dc1's six plane bridges live on SEPARATE kernels (each containment VM's own libvirtd). Flattening puts BOTH DCs' plane bridges + node VMs co-resident on vcloud's ONE libvirtd/kernel for the first time in any deployed shape. SEC-010 -- interface-scoped to transit legs that no longer carry this function -- does NOT cover host-level cross-DC leakage (an accidental host address on a plane bridge, a forward rule, br_netfilter interactions). Neither W0.4 option addresses this; verified absent from the doc. Nuances carried honestly: the adjacency existed in Model A's committed design, but Model A was never deployed and SEC-010 was priced AFTER it against Model-B's separate-kernel shape -- so no ruled control covers the flattened adjacency. This is DISTINCT from rebuilding SEC-010's transit drop on the client VM (row 6) -- two controls.
Handling options for the gate (presented, not picked -- GA-R5):
--check gate and its own SEC-NNN row (SEC-010's proven pattern, one layer up). Designed in Phase 1/2.vr1-dcN-maas-01 (region VM), onto the Option-1 client VM, or elsewhere? Ditto the D-131 node-DNS forwarder and the .4 artifact-service placement (dc-mirror.sh says "runs on the rack host" today). W0.3 flagged; no worker resolved it..2 metal-admin identity or take a new utility-band octet (D-134 standing map is a cross-DC STANDARD -- needs a ruled octet); name should NOT be vvr1-dcN (avoid conflation with the eliminated containment class).ssh -J access and any Office1-originated flows remain -- which ends get the re-authored FORWARD-drop (client VM + voffice1?), and does the mesh triangle survive unchanged (W0.4 implicitly kept it; W0.2 flagged it as a decision)?vm-secret-locations rack rows + rotation obligations that FIRE on rack rebuild) -- Phase 1/2 work, listed so it is not lost.Decision put to the operator:
vr1-dcN-maas-01 (per D-132 addendum -- no change needed), and rule where the rack-controller remainder + D-131 forwarder land (client VM / maas-01 / retire-with-evidence).Items 4-9 of Section 6 are flagged as downstream (Phase 1-4) work, not gate blockers.
Recorded as a DIRECTIONAL PLANNING CONFIRMATION at the Phase-0 gate -- NOT a GA-R5 [ARCH] ruling and NOT a D-number (the formal container-elim ruling, D-123 amendment vs. new D, is framed in Phase 4 and ruled then, per SCOPE Section 7). The operator was shown the layer+separation diagram (artifact, Option-1 target) before confirming.
--check gate and its own SEC-NNN row. This is now a Phase-1 DESIGN ITEM, not an open gate fork.vr1-dcN-maas-01 (.6, D-132 addendum -- no change). The rack-controller remainder + D-131 forwarder + artifact-service placement (client VM vs. maas-01 vs. retire-with-evidence) is carried into Phase 1 as an open placement decision (Section 6 item 3), not resolved at this gate.Feeds forward to Phase 1: plan against Option 1; treat (a) as a required design deliverable; keep the container-elim change-set DISTINGUISHABLE from D-143's 10.12->10.13 address change (two rulable axes riding one redeploy).
dc-dc-whole-host-budget.py:66-67's committed "MEASURED host budget" (256 vCPU / 1024 GiB / 10240 GiB) needs a fresh read-only measurement before any FIT verdict.geneve-encap-assert.sh + a jumbo-path check) is owed once the planes exist at vcloud level.