Pass 0 / W0.4 -- Target-topology options (container-layer elimination)

Worker: W0.4 (target-topology options). Feeds: the Phase-0 OPERATOR GATE (SCOPE-AND-EXECUTION-PLAN.md Section 4). Scope discipline: READ-ONLY; every claim below cites path:line or a named durable record; anything not resolvable from the repo is marked OWED rather than invented (hard rule 2).

0. Framing recap (verified against the repo, not assumed)

• Current nesting (Model B, D-123): vcloud (outer libvirt, qemu:///system) -> vvr1-dcN (containment VM, inner libvirtd) -> node VMs. The outer root creates vvr1-dc0/vvr1-dc1 (opentofu/main.tf:410-519,537-623); the INNER root (opentofu/vr1-dc0-substrate/main.tf:1-266) creates the 6 planes + WAN bridge + edge + 12 node VMs (9 D-121 role nodes + the D-104 juju-controller vr1-dc0-juju-01 + the D-132 MAAS-region vr1-dc0-maas-01 + the D-129(iii) Tailscale router vr1-dc0-tailscale-01, opentofu/vr1-dc0-substrate/main.tf:96-247) via a qemu+ssh provider dialled FROM Office1 (opentofu/vr1-dc0-substrate/main.tf:12-22).
• The eliminate-and-pull-up-one target, stated in the operator's own words (quoted in SCOPE-AND-EXECUTION-PLAN.md:34-36): collapse to vcloud (libvirt) -> node VMs directly.
• This is not a green-field question. Before Model B was ruled, the exact flat shape existed, was committed, and validated: docs/archive/model-a-fallback-plan.md ("Model A", git tag model-a-fallback at 114d392, R-3-compliant, tofu validate 11/11 modules). Both options below are read against that precedent rather than invented from scratch.
• Two rulings POST-DATE Model A and change what a flat topology must additionally provide -- Model A's own spec did not have to solve them:
  • D-138 (docs/design-decisions.md:7064-7126, RULED 2026-07-30): the cloud-facing Juju client + openstack CLI (phase-03..phase-06) MUST run from a host with L3 reach to the DC's node planes (voffice1 cannot reach them; SEC-010/D-052/D-125 forbid routing there). CURRENT STATE: that host is vvr1-dc0 itself -- "the dc0 rack" (docs/CURRENT-STATE.md:1046,3014-3016; lib-hosts.sh:212-213 shows vvr1-dc0 holds a live metal-admin address, qemu+ssh://jessea123@10.12.8.2/system, the D-134 rack-utility .2 -- distinct from .5 juju-01, .6 maas-01, .7 tailscale-01).
  • D-132 amendment (docs/design-decisions.md:7135-7204, RULED 2026-07-30): each DC gets its OWN MAAS region (now a node VM, vr1-dc0-maas-01), not a migrated/shared Office1 region.
  • SEC-026/SEC-028 (docs/security-ledger.md:79,81): "the rack" is now a credential-bearing host -- it holds the DC-scoped MAAS admin key (SEC-018) and the per-DC Juju service key (SEC-028), OPEN rotation obligations, and an explicit per-DC isolation requirement ("each DC's client host receives ONLY THAT DC's credential").
  • Consequence for this pass: eliminating the container layer must still answer "where does the D-138 client + its SEC-026/SEC-028 credential live", because that requirement did not exist when Model A was archived. This is the crux item (d) below.

1. OPTION 1 (RECOMMENDED) -- "Model A, D-132/D-138-updated": flat nodes + a small per-DC client VM

vcloud (host, L0, qemu:///system)
 |-- vr1-dc0-client        small VM (4 vCPU / 8 GiB / 80 GiB, D-138 role only -- see (d))
 |                          legs: metal-admin + office1<->dc0 transit; NO nested libvirt,
 |                          NOT a hypervisor for anything -- a flat sibling VM like any other
 |-- vr1-dc0-control-01..03    node VMs (16/65536/150)   \
 |-- vr1-dc0-compute-01..02    node VMs (12/49152/100)    \  vcloud-level libvirt siblings,
 |-- vr1-dc0-storage-01..04    node VMs (8/24576/550)      >  attached DIRECTLY to the 6
 |-- vr1-dc0-juju-01           node VM  (4/8192/100)       /  vcloud-level plane networks
 |-- vr1-dc0-maas-01           node VM  (4/8192/150)      /
 |-- vr1-dc0-tailscale-01      node VM  (2/2048/25)      /
 |-- vr1-dc0-opnsense          edge (2/2048, 2-NIC: provider-public LAN + vr1-dc0-wan WAN)
 |-- vr1-dc0-{provider-public,metal-admin,metal-internal,data-tenant,storage,replication}
 |                          6 isolated-L2 libvirt networks (dc-planes), AT VCLOUD LEVEL
 |-- vr1-dc0-wan            NAT /24 simulated ISP uplink, AT VCLOUD LEVEL (site-wan, direct)
 `-- mesh-vr1-dc0-office1   transit leg (office1 <-> dc0), unchanged

nesting depth = 2  (vcloud -> node VM -> nova KVM guest)   <- VR0-PROVEN, same as Model A
site-down     = scripted group-destroy of the vr1-dc0-* domain set (Model A's mechanism)

(a) Where node VMs live

Directly on vcloud's outer libvirt provider (qemu:///system), as siblings -- identical to docs/archive/model-a-fallback-plan.md:24-39's Model A shape, but with the 3 utility node classes (juju-controller, MAAS region, Tailscale router) added as additional flat siblings using the SAME modules/node-vm bodies already authored in opentofu/vr1-dc0-substrate/main.tf:250-265 (only the provider target changes, per that file's own header note at lines 1-10: "All module bodies are UNCHANGED from the outer root -- only the provider they run against moved").

(b) Where the six planes land

Back at vcloud level as modules/dc-planes outputs, reversing the D-123 move documented at opentofu/main.tf:22-33 ("the 6 vr1-dc0 planes MOVED to the INNER root ... under Model B"). lib-net.sh's PLANE_CIDRS/PLANE_NAME (unchanged, scripts/lib-net.sh:21-30) and the D-101/ D-134 CIDR values stay IDENTICAL -- only the libvirt network objects' host moves, not their IPAM identity. metal-admin remains the PXE/boot plane, NIC_PLANE_ORDER unchanged (lib-hosts.sh:69).

(c) Where transit/uplink/mesh/br-ex land

• Mesh legs (office1<->dc0, office1<->dc1, dc0<->dc1): UNCHANGED, already vcloud-level (opentofu/main.tf:125-141).
• Uplink/WAN: the D-125 bridge-in plumbing (module "vr1_dc0_uplink" NAT + wan-bridge module + the IP-less 2nd NIC + br-vr1-dc0-wan netplan bridge, opentofu/main.tf:360-397,505-518) is REMOVED. The DC edge's WAN attaches to a vcloud-level NAT directly, exactly like Office1's edge does today (opentofu/main.tf:99-117) -- this is Model A's item 8 (docs/archive/model-a-fallback-plan.md:67-75): "same 172.30.2.0/24 ... no uplink NIC, no bridge, no wan-bridge module ... NO re-address needed".
• br-ex: unchanged in kind -- still an OVS bridge parented on each ROLE node's provider-public NIC (lib-hosts.sh:70-72), a per-node fact independent of where the node's hypervisor sits.

(d) Where the juju/openstack execution host goes (THE CRUX)

A dedicated, small, per-DC client VM on vcloud libvirt, occupying the ROLE Model A's original rack headend already had by construction: two legs (metal-admin + office1 transit, docs/archive/model-a-fallback-plan.md:25-26), 4 vCPU / 8 GiB / 80 GiB. Model A's own headend was authored BEFORE D-138/D-132 existed and was scoped as "MAAS rack headend" only -- this option REPURPOSES that same artifact shape to also satisfy D-138 (cloud-facing client execution) and SEC-026/SEC-028 (the DC-scoped credential residency), because it already has the one property those rulings require: an L3 leg on metal-admin. It is explicitly not a hypervisor for anything -- no expose_nested_virt, no inner libvirtd, no nodes attached to it -- so it does not reintroduce a container layer; it is a flat utility sibling, same class as vr1-dc0-juju-01. Per-DC credential isolation (SEC-026 "(1) ISOLATION IS THE LOAD-BEARING CONTROL") is preserved exactly as it is today, since this VM plays the identical role the current rack plays. maas-vm-host registration (Step 9, deferred DOCFIX-179) targets vcloud's own virsh, not this VM's (Model A table row 4, docs/archive/model-a-fallback-plan.md:48) -- MAAS discovers the vcloud-level node domains directly.

(e) Capacity/FIT impact on vcloud

Directionally FREES host RAM, does not need it. Today's containment VM is sized to hold BOTH the node fleet AND its own containment overhead (opentofu/variables.tf:143-150: RAM raised 416->480 GiB per DC, "384 GiB node fleet + 96 GiB overhead"; the 96 GiB is inner-libvirtd/OS/page- cache overhead that exists ONLY because of the nesting). Flat placement removes that per-DC 96 GiB overhead layer and replaces it with the small client VM's 8 GiB -- a swing of roughly (96-8) x 2 DCs ~ 176 GiB of host RAM, all else equal. vCPU impact is smaller: Model B's overhead is 4 vCPU/DC (opentofu/variables.tf:137-141, unchanged since the RAM-only raise), and Model A's headend is also 4 vCPU, so vCPU is roughly a wash. This is a qualitative, repo-grounded direction, not a verified number -- see Section 4 OWED items: the sizing calculator (scripts/dc-dc-whole-host-budget.py) does not yet have flags for the 3 utility node classes (juju/maas/tailscale) added after its authoring, so an exact FIT verdict needs either extending it or hand-totaling before the operator relies on a number.

(f) What breaks

• The inner OpenTofu root (opentofu/vr1-dc0-substrate/, its own state file, its qemu+ssh provider dialled from Office1) is retired; its module bodies fold back into the outer root.
• The "OUTER -> BOOTSTRAP GATE -> INNER" apply-ordering contract (opentofu/main.tf:301-331) goes away -- no bootstrap gate, no cross-root apply sequencing.
• site-headend-install.sh's "node-host mode" (nested libvirtd, kvm nested=1, inner pool, the SEC-010 nftables writer scoped to the inner host, D-125 --uplink-if/--wan-bridge verification) is retired; only whatever the small client VM still needs (metal-admin address assignment, the D-138 client role) remains, in a much smaller mode.
• SEC-010's CLOSED implementation must be REBUILT, not moved. Today's mechanism (docs/security-ledger.md:21) is nftables-sec010.nft + sec010-fw.service written onto vvr1-dc0's MEASURED interface names (enp1s0 transit / br-vr1-dc0-wan/enp2s0 uplink, captured live 2026-07-20). The new client VM is a DIFFERENT artifact with its own NIC-naming trap (opentofu/main.tf:473-481 documents this exact trap for vvr1-dc0's q35 shape) -- its transit-scoped FORWARD-drop has to be re-authored and re-measured, not copy-pasted.
• VIRSH_POWER_ADDRESS_FROM_OFFICE1/_FROM_DCREGION (lib-hosts.sh:212-213,246-250) currently point at the rack's transit/metal-admin address for qemu+ssh NODE POWER CONTROL through the inner libvirt. With nodes flat on vcloud, power control becomes vcloud's own local virsh (qemu:///system or a local-equivalent address) -- these constants and every consumer that reads them (maas-node-power.sh, the teardown runbook's virsh probes) need retargeting.
• The D-123 Model B ruling's headline benefit is lost: "site-down = a single virsh destroy vvr1-dc0" (opentofu/main.tf:405) reverts to Model A's scripted group-destroy of the vr1-dc0-* domain set (docs/archive/model-a-fallback-plan.md:36,64). This is a REAL regression against what D-123 was ruled for, not a free simplification -- flagged here so Phase 4's [ARCH] decision framing carries it honestly, per SCOPE-AND-EXECUTION-PLAN.md:197-200.
• MAC-pin discipline for the 12 node VMs (opentofu/vr1-dc0-substrate/main.tf:87-245) needs a fresh capture pass once nodes are re-created flat -- moot for the 10.13 redeploy itself since the whole fleet is being torn down and rebuilt regardless.

(g) What simplifies

• Nesting depth 4 -> 2 (VR0-proven), eliminating the entire "unproven depth-4 nested-virt" risk class that Model A's own fallback-trigger list names (docs/archive/model-a-fallback-plan.md:97, 100: "nova-compute guests fail to boot or are unusably slow at 3x-nested KVM"; "exposenested virt=true ... destabilises the headend").
• No inner-root/outer-root apply-ordering gate, no separate inner state file, no cross-host qemu+ssh-from-Office1 provider dial for the substrate apply.
• No D-125 bridge-in plumbing at all (wan-bridge module, the IP-less uplink NIC, the br-vr1-dc0-wan bridge, the "unprovable pre-apply" deploy-time gate at opentofu/main.tf:375-378) -- the DC edge's WAN becomes an ordinary direct NAT, one fewer bespoke module and one fewer "unprovable until you try it" gate.
• Fewer NIC-naming traps to carry (today TWO artifacts -- vvr1-dc0 and voffice1 -- each independently measured enp1s0/enp2s0 on first boot; flat placement needs this measured once, for the single small client VM, not for a containment VM AND its inner substrate).
• lib-net.sh plane CIDRs/names are untouched (still 6 planes/DC, same D-101/D-134 values) -- only the low-level libvirt/carve/power scripts (lib-hosts.sh's power-address constants, maas-node-power.sh, dc-node-carve) need retargeting, not the IPAM layer.

(h) Roosevelt-delta

Strongly POSITIVE. Bare metal has no hypervisor containment layer at all -- Roosevelt's node "VMs" are literal servers on the DC's physical planes, so flat placement is the closer bare-metal analog and nesting was always a simulation-only artifact (Model A's fallback doc never claimed Model B carried Roosevelt fidelity; its one stated advantage was the single-object site-down convenience, which is a simulation-only DR primitive, not a transferable property). The retained small client VM in (d) DOES transfer: D-138 names its own Roosevelt analog explicitly -- "each DC has a management entry point inside it; the NOC reaches that entry point ... the per-DC management bastion holds only its own DC's cloud credential" (docs/design-decisions.md:7121- 7124). So this option's client VM is not throwaway scaffolding; it is that bastion, rehearsed early, minimizing delta-to-Roosevelt exactly as SCOPE-AND-EXECUTION-PLAN.md:203-204 asks.

2. OPTION 2 (NOT RECOMMENDED, given for contrast) -- fully flat: client runs on vcloud itself

vcloud (host, L0)  <-- juju/openstack CLI + the D-138/SEC-026/SEC-028 credential live HERE,
 |                      on the jumphost OS itself, via a new host-side NIC/veth onto the
 |                      (now vcloud-level) metal-admin plane bridge
 |-- vr1-dc0-control-01..03 / compute-01..02 / storage-01..04 / juju-01 / maas-01 / tailscale-01
 |       (same flat node-VM siblings as Option 1 -- (a)/(b)/(c) IDENTICAL to Option 1)
 `-- ... (planes/uplink/mesh identical to Option 1; NO small client VM)

Everything in (a)/(b)/(c)/(f)/(g) is IDENTICAL to Option 1 -- the only difference is (d): no per-DC client VM at all; the D-138 execution host collapses one layer further, onto vcloud's own OS.

(d) Execution host -- the crux, and why this is worse

The juju/openstack CLI, and the SEC-026/SEC-028 per-DC credentials, would live directly on the shared jumphost that ALSO runs Plane-1 substrate tooling for BOTH DCs and Office1 (docs/design-decisions.md:5354-5358, D-128 Plane 1). This inverts SEC-026's own stated control: "(1) ISOLATION IS THE LOAD-BEARING CONTROL: each DC's client host receives ONLY THAT DC's credential" (docs/security-ledger.md:79) -- on vcloud, BOTH DCs' credentials would sit on the one host that already has the widest blast radius in the whole project (it is the CLAUDE.md "live operations clone", governed by the PreToolUse guard and every permission ask rule). A credential-scoping boundary that today survives a "rebuild the rack VM" event would no longer exist to survive anything short of rebuilding vcloud itself.

(e) Capacity/FIT

Same node-fleet math as Option 1 (nodes are identical either way); this option additionally saves the ~8 GiB the small client VM would have cost. Negligible vs. Option 1's ~176 GiB swing.

(h) Roosevelt-delta -- negative

No bare-metal analog exists for "run the DC client from the shared jumphost" -- Roosevelt's own management model is per-DC bastions (D-138's own analog, quoted above), which is what Option 1's small client VM already rehearses. Option 2 would need to be UNDONE and re-built as something like Option 1's shape at the pre-Roosevelt bare-metal test, making it throwaway work rather than transferable.

3. Tradeoff table

4. Recommendation

Option 1 (flat nodes + a small, single-purpose, per-DC client VM). Rationale, in order of weight:

1. It is the lower-risk delta -- Model A is a previously-validated, git-tagged artifact, not a new design; only the D-138/D-132 layer on top of it is new work, and that work is small (repurpose an existing headend shape, not invent a host class).
2. It preserves the credential-isolation property SEC-026/SEC-028 were opened to protect, where Option 2 actively erodes it.
3. It has a NAMED Roosevelt analog in D-138's own text, so it is not scaffolding to be re-done at the bare-metal test -- it satisfies SCOPE-AND-EXECUTION-PLAN.md's minimize-delta-to- Roosevelt framing directly.
4. The RAM difference between the two options (~176 vs ~184 GiB freed) is marginal next to the isolation and Roosevelt-fidelity costs of Option 2.

Framing note for the operator gate: "eliminate the container layer" is fully satisfied by Option 1 under the reading that matters -- no VM in this topology is a HYPERVISOR for another VM (no nested libvirt, no expose_nested_virt, no inner OpenTofu root). The retained small VM is a flat utility sibling, the same class of thing as the Juju-controller or MAAS-region node VMs already in the fleet, not a resurrection of the containment pattern. If the operator's intent is literally zero additional VMs of any kind, that is Option 2, with the tradeoffs above accepted knowingly.

5. OWED live measurements (not resolvable from the repo; do not infer)

1. Current vcloud host capacity. scripts/dc-dc-whole-host-budget.py:66-67 carries a committed default it labels "MEASURED host budget" (256 vCPU / 1024 GiB / 10240 GiB), but its CURRENCY for the 10.13 redeploy (same hardware? any change since it was set?) needs a fresh read-only measurement on vcloud before any FIT verdict is finalized.
2. An exact FIT number for Option 1 vs. today's Model B sizing. The calculator's --control/ --compute/--storage flags do not yet cover the 3 utility node classes added after it was authored (vr1-dc0-juju-01, vr1-dc0-maas-01, vr1-dc0-tailscale-01 -- opentofu/vr1-dc0-substrate/main.tf:133-245). Extending the script (or hand-totaling) and re-running --model A vs. --model B with the CURRENT 12-VM/DC roster is owed before citing a precise freed-capacity number in the Phase-4 change-set.
3. MTU/jumbo budget for vcloud-level plane bridges once the 6 planes move up a layer -- this is W0.2's (network/wiring map) dimension, not measured here; flag as a dependency for the Phase-0 administrator synthesis.