Fork: 0
OpenStack / openstack-caracal-dc-dc
Transfer to URL with SHA Find file
Newer
Older
openstack-caracal-dc-dc / docs / audit / container-elim-pass / pass0-w3-consumers.md
@JANeumatrix JANeumatrix 7 hours ago 23 KB container-elim planning pass RAN (phases 0-4 + advisor): FINAL-PLAN + operator-report + D-144 framing (READ-ONLY)
Raw Blame History

Pass 0 / W0.3 -- Consumer/dependency inventory for the container-layer-elimination pass

READ-ONLY planning finding. Repo HEAD at time of read: branch dc-dc-stage5-preconditions, 2026-08-09. This is worker W0.3 of Phase 0 (docs/audit/container-elim-pass/SCOPE-AND- EXECUTION-PLAN.md, phase-prompts.md). No command was executed; every row cites path:line.

0. Framing recap (do not re-derive)

Model B (D-122/D-123): vcloud (outer libvirt) -> vvr1-dcN (containment VM = inner libvirt, ALSO the DC rack: MAAS rack/region + Juju/openstack client host) -> node VMs. Eliminating the containment VM collapses this to vcloud (libvirt) -> node VMs directly. vvr1-dcN (the containment VM) is NOT the LXD API-charm containers that run on deployed OpenStack nodes -- those are untouched by this pass.

The load-bearing fact this dimension surfaces: vvr1-dcN is not only a nesting boundary, it is also the NAMED EXECUTION HOST for three separate ruled roles: (1) the D-123 nested-libvirt node host, (2) the D-132-amendment per-DC MAAS region+rack, and (3) the D-138 cloud-facing client host (juju/openstack run FROM vvr1-dcN, reached ssh -J voffice1 jessea123@<dc- rack-transit-ip> -- runbooks/dc-dc-phase3-maas-enlist-deploy.md:424,430, runbooks/dc-dc-phase4-juju-bundle-per-dc.md:165,357,1108). Flattening removes the host that carries all three roles simultaneously -- each must be re-homed, and they need not all land on the same place.

1. Repo-wide grep census -- vvr1-dc

Canonical tree (excludes the stale .claude/worktrees/skill-repackage-20260727/ git-worktree snapshot, which mirrors ~384 additional hits from an unrelated 2026-07-27 skill-repackage task and is not part of the live repo surface):

grep -rn "vvr1-dc" --include="*.sh" --include="*.py" --include="*.md" --include="*.yaml" \
     --include="*.tf" .   (from repo root, worktree copy excluded)

424 occurrences across 78 files.

Heaviest files (occurrence count): | File | Count | Nature | |---|---|---| | docs/design-decisions.md | 66 | D-122/123/124/125/126/128/138 rulings, amendments | | opentofu/main.tf | 34 | outer root: creates the vvr1_dc0/vvr1_dc1 containment VM modules | | scripts/site-headend-install.sh | 18 | --role rack = installs on vvr1-dcN | | docs/CURRENT-STATE.md | 17 | living status references | | opentofu/variables.tf | 16 | vvr1_dc0/vvr1_dc1 module var blocks ("Model B: holds the inner libvirt pool") | | opentofu/vr1-dc0-substrate/main.tf | 13 | inner root: qemu+ssh provider targets vvr1-dc0 | | runbooks/dc-dc-phase2-tofu-dc-substrate.md | 13 | outer/inner apply runbook | | docs/dc0-deploy-readiness.md | 12 | readiness record | | opentofu/vr1-dc1-substrate/main.tf | 9 | inner root, dc1 | | docs/dc-dc-deployment-workflow.md | 9 | workflow doc, plane-split doctrine | | runbooks/dc-dc-teardown-rollback.md | 8 | Paths A/B/C/M name vvr1-dcN as the teardown target | | opentofu/vr1-dc0-substrate/variables.tf | 6 | | | docs/audit/record-inventory.md | 6 | | | netbox/dc-rack-mgmt-import.py | 5 | imports vvr1-dcN as a NetBox rack-mgmt device | | tests/dc-rack-mgmt-import/test_logic.py | 5 | harness for the above | | ... | | remaining 64 files carry 1-4 hits each (changelogs, security-ledger, archive) |

scripts/*.sh + scripts/*.py combined: 26 occurrences across site-headend-install.sh, dc-rack-net.sh, site-baseleg.sh, dc-mirror.sh, dc-egress-check.sh, maas-profile-assert.sh, dc-dc-whole-host-budget.py. opentofu/: 17 files. Most of the remainder is docs/ (design-decisions, CURRENT-STATE, changelogs, security-ledger, archive) and runbooks/ -- i.e. DECISION RECORD, not live tooling; those do not need code changes on flattening, only a superseding note (GA-R1/R2 append-only discipline -- do not edit history).

2. Consumer table

Consumer (file:line) What it keys to the containment layer What breaks / changes on flattening Severity
scripts/lib-hosts.sh:52,157-168,212-214,246-251 VIRSH_POWER_ADDRESS, VIRSH_POWER_ADDRESS_FROM_OFFICE1, VIRSH_POWER_ADDRESS_FROM_DCREGION Both forms are qemu+ssh:// URIs that dial the containment VM's libvirtd (172.31.0.2/10.12.8.2 for dc0, 172.31.0.6/10.12.68.2 for dc1) -- this is how MAAS powers on/off node VMs, because the node VMs live INSIDE vvr1-dcN's libvirt, not vcloud's. If node VMs move to vcloud's own libvirt, the power address becomes vcloud's own libvirt (qemu:///system locally, or a qemu+ssh back to vcloud from wherever MAAS's region runs). The FROM_OFFICE1/FROM_DCREGION split (which exists because "the reachable rack address differs by region", lib-hosts.sh:159-167) may collapse entirely if there is no separate inner libvirt host to be reachable FROM. Every value in both DC arms (vr1-dc0/vr1-dc1 cases, lines 212-214, 246-251) needs re-derivation, not a rename. HIGH -- this is the mechanism MAAS uses to power every node; wrong here = "MAAS unreachable" masquerading as a network fault (a failure mode this repo has already hit once, maas-role-tags.sh:48-51).
scripts/lib-hosts.sh:69-72 NIC_PLANE_ORDER, BREX_PARENT_NIC Describes the NIC-to-plane mapping the INNER tofu root pins per node (main.tf macs[0..5] order) -- a property of how opentofu/vr1-dcN-substrate/main.tf (the inner root, applied via qemu+ssh to vvr1-dcN) attaches MACs to node VM NICs. If nodes are created directly by the (now single) tofu root against vcloud libvirt, the NIC/MAC-to-plane pinning logic must be re-authored in that root, but the ORDER CONVENTION itself (metal-admin first, not PLANE_CIDRS order) is a fact about how MAAS/libvirt XML enumerates interfaces, independent of nesting -- it likely CARRIES FORWARD unchanged. Confirm at Phase 2 (tooling), not a Phase-0 blocker. LOW-MEDIUM -- convention likely survives; the FILE that encodes it (inner substrate root) is what changes.
scripts/lib-hosts.sh:29-34 CARVE_AUX_HOSTS Declares the aux (non-Juju) -tailscale-01/-maas-01 VMs as --host-only carve targets; these ALSO live inside vvr1-dcN's inner libvirt today. No conceptual change on flattening -- these become ordinary flat-libvirt VMs alongside the 9 role nodes + juju controller. Their power address inherits the same VIRSH_POWER_ADDRESS fix above. LOW -- mechanical, rides the same fix as the row above.
scripts/lib-net.sh (whole file) No vvr1-dc string appears here (verified: grep vvr1-dc scripts/lib-net.sh = 0 hits). PLANE_CIDRS/PLANE_GW/VIP prefixes are IP-plan facts, not containment-topology facts. UNCHANGED by container elimination as such. It WILL change under the concurrent D-143 re-IP (10.12->10.13), but that is a SEPARATE, already-ruled axis this pass must keep distinguishable (SCOPE-AND-EXECUTION-PLAN.md Section 7 says exactly this). NONE for this dimension -- flag as a non-consumer so Phase 4's change-set does not double-count it.
scripts/maas-node-power.sh:2-46 (usage, POWER_ADDRESS arg, VIRSH_URI) Takes a qemu+ssh:// power address as an argument (not hardcoded) and virsh-lists domains AT THAT URI to MAC-match them to MAAS machines. Every worked example in the header (qemu+ssh://jessea123@172.31.0.2/system vr1-dc0) targets the containment VM. The script itself is topology-agnostic (it takes the URI as input) -- it needs NO code change. But every CALLER that supplies 172.31.0.2/10.12.8.2-class addresses must supply the new (flat) address instead, which is exactly the lib-hosts.sh fix above. Also note: the header's D-103 rationale for per-machine (not pod) power was ALREADY about avoiding MAAS-virsh-pod storage incompatibilities (domblkinfo on volume-ref disks) -- that reasoning is unrelated to nesting and stays valid post-flatten. MEDIUM -- no code change, but every invocation site needs the new address; a stale hardcoded address in a runbook/changelog would silently power the WRONG (or a nonexistent) host.
scripts/site-headend-install.sh:6-20,82-129,204-343,371-409 --role rack, --host-nodes, node_host_check(), node_host_setup() This is THE script that turns vvr1-dcN into a nested-libvirt node host: installs qemu-kvm/libvirtd, turns on nested KVM, creates the inner pool dir, the SEC-010 FORWARD-drop keyed on $TRANSIT_IF ("mgmt"), and verifies the D-125 WAN bridge/uplink. --role rack (no --host-nodes) is the narrower "just a MAAS rack enrolling to Office1's region" case, ALSO run on vvr1-dcN today. If nodes run directly on vcloud, --host-nodes (node_host_setup/node_host_check, ~140 lines) becomes DEAD CODE for VR1 -- there is no separate "rack that also hosts nested KVM" anymore; vcloud itself already IS the outer libvirt host and does not need this bootstrap. The rack role itself (MAAS enrollment) may still be needed if D-132's "MAAS region controller per DC" is KEPT as a normal flat VM (see D-132 row below) -- but it would no longer carry --host-nodes. The SEC-010 FORWARD-drop (TRANSIT_IF="mgmt") is keyed to a transit LEG that exists BECAUSE of the containment VM's two-leg (transit + inner-plane) design; a flat topology may not have a "transit" interface to drop-FORWARD on at all, or the DC-LOCAL boundary the drop enforces may need to move to a different device (a vcloud-host-level nftables rule, or a per-VM firewall) -- this is a design question for Phase 0's target-topology worker (W0.4), flagged here as a dependency. HIGH -- largest single file by consumer surface (18 vvr1-dc hits, ~140 lines of --host-nodes logic); also the SEC-010 enforcement point (a security control, not just wiring) lives here.
scripts/dc-rack-net.sh:2-14,55-85 (whole script; site table for dc0/dc1) RUNS ON THE DC RACK HOST (i.e. vvr1-dcN) per its own header; persists rack bridge-leg IPs (vr1-dc0-metal-admin=10.12.8.2/22 etc.) and the D-131 node-facing DNS forwarder, both keyed to addresses the CONTAINMENT VM itself holds on its own inner bridges. If there is no containment VM, "the rack's own inner bridges" do not exist in the same shape -- these addresses (.2, .3 on metal-admin) were the containment VM's identity on its OWN nested networks. Under a flat topology this functionality (rack-leg persistence + node DNS forwarder) needs a new HOME: either the per-DC MAAS region VM (if D-132's region-per-DC survives as a flat VM) or is retired if the forwarder's reason (rack-only resolver SERVFAILs, dc-rack-net.sh:27-30) no longer applies once "the rack" is gone. Cannot be resolved without W0.4's target topology. HIGH -- a security/DNS-availability control (D-131), not cosmetic; silently losing it would reintroduce the SERVFAIL bug this script was built to fix.
scripts/site-baseleg.sh:40-48 LEGS["office1"], the commented-out [vr1-dc0] row The office1 row is UNRELATED to the containment VM (it's the vcloud<->voffice1 base leg). The DC rows are explicitly DEFERRED with the comment "the DCs nest inside vvr1-dc0 and are reached by qemu+ssh ... add a row ONLY if/when a DC needs one" -- and D-138 (design-decisions.md:7126-7131) already closed this deferral: "the DC rows stay DEFERRED and their MEASURE-first note is now ANSWERED: ... no host-side leg is wanted." If nodes are flat on vcloud, the "DC rows deferred because reached by qemu+ssh through vvr1-dc0" premise disappears -- vcloud already has an L3 view of its own libvirt guests without an SSH hop. Whether a NEW base-leg row is needed depends entirely on whether the flat node VMs get bridged IPs vcloud itself must route to, which again is W0.4's call. LOW -- currently an explicit no-op; only becomes live work if the target topology needs a new leg.
scripts/maas-role-tags.sh:37,48-51 The comment "Run this where maas lives (the D-128 Plane-2 headend)" and the REFUSE message both assume the operator KNOWS which host that is -- today, resolvable to voffice1 (region) OR vvr1-dcN (rack), a distinction this pass changes. No vvr1-dc LITERAL string appears in this script's logic (it is genuinely site/host-agnostic, driven by lib-hosts.sh/MAAS_PROFILE). No code change required. Purely a DOC/comment currency issue once the execution-host question (Section 3 below) is answered -- the "headend" language should be corrected to match whatever host actually runs maas post-flatten. LOW -- comment-only.
opentofu/main.tf:25-33,187,234,309-422 (outer root, module "vvr1_dc0"/"vvr1_dc1") This IS the resource that creates the containment VM (vm_name = "vvr1-dc0", line 412) -- the single biggest and most literal consumer. Line 405: "Site-down = a single virsh destroy vvr1-dc0 (the D-122 intent)" -- i.e. the outer root's entire per-DC unit of work today IS this one VM. The ENTIRE outer-root module for a DC either (a) is deleted and replaced by direct node-VM resources targeting vcloud libvirt (the flattened design), or (b) is repurposed to create something smaller (e.g. just a MAAS-region VM, if that role is kept). This is squarely Phase-0-W0.1's (substrate/tofu map) territory, cross-referenced here because it is also the top literal-reference file in the census. HIGH (cross-referenced to W0.1, not re-owned here).
opentofu/vr1-dc0-substrate/main.tf, vr1-dc1-substrate/main.tf (inner roots) Provider block targets qemu+ssh INTO vvr1-dc0/vvr1-dc1 (per main.tf:25-33 comment on the outer root, and the inner root's own versions.tf/provider config) -- this is THE apply that creates the 9+1(+2 aux) node VMs, currently nested. If flattened, there is no longer an "inner root run via qemu+ssh into the containment VM" -- either this root's provider target becomes qemu:///system on vcloud directly (merging outer+inner into one tofu apply), or it becomes qemu+ssh FROM wherever the new execution host is, straight to vcloud. Either way this is a provider-config + apply-ordering change, not a pure rename. Cross-referenced to W0.1. HIGH (cross-referenced to W0.1).
netbox/dc-rack-mgmt-import.py, tests/dc-rack-mgmt-import/* Imports vvr1-dcN into NetBox as a rack-management device record (a physical/virtual asset in the DCIM model representing the containment VM as "the rack"). If there is no containment VM, this NetBox device either needs to be retired (with a NetBox-side decommission, not silent deletion -- NetBox is the IPAM/DCIM system of record) or repointed to represent something else (e.g., vcloud itself, or nothing, if "the rack" concept is retired outright). This is a DATA-migration consumer, distinct from a code consumer -- flag for Phase 2/W2.4 (which tools become which modules) since it touches an external system of record, not just repo files. MEDIUM -- external-system consumer, easy to miss because it is not a runbook or script that gets "read" in the normal grep sweep for tooling.
scripts/dc-egress-check.sh:63,72 Comments citing MEASURED ip route output taken directly ON vvr1-dc0/vvr1-dc1 (the default-route-via-provider-public fact). Historical/measurement citations, not live logic keyed to the containment layer. No functional change -- the DEFAULT ROUTE FACT itself (default via .4.1/.64.1) is a plane-gateway fact (lib-net.sh PLANE_GW), not a containment fact, and would still hold once nodes are flat as long as the same plane/gateway design is kept. Re-verify empirically post-flatten rather than assume; the citation just needs a currency note. LOW.
scripts/dc-mirror.sh:6 Doc-comment: "RUNS ON THE DC RACK HOST (e.g. vvr1-dc0), not on vcloud/voffice1." Same class as dc-rack-net.sh above but for the apt mirror service -- if "the rack host" no longer exists as a distinct entity, this script's stated execution host must be re-pointed to wherever the mirror service actually runs post-flatten (likely still a dedicated per-DC utility VM at the .4 octet, per lib-hosts.sh:96-100 REGION_HOST_SUFFIX/utility-band convention -- that utility VM is unaffected by containment removal, it just stops being INSIDE a nested host). LOW-MEDIUM -- doc-currency + confirm the utility VM's own libvirt parent changes from "inner" to "vcloud direct", nothing else.
scripts/maas-profile-assert.sh:39 Usage example lists voffice1,vvr1-dc0,vvr1-dc1 as the three MAAS-profile-bearing hosts whose region identity this script proves ("a machine count is NOT proof"). If the per-DC MAAS region (D-132 amendment) no longer lives ON the containment VM (because there is no containment VM) but instead on a flat per-DC utility VM, the usage example needs updating but the SCRIPT'S JOB (prove which region a profile resolves to, by rack identity) is unchanged and arguably MORE necessary post-flatten (more hosts look alike without the containment boundary as a visual/structural cue). LOW -- doc-currency; the script's function is robust to the topology change.
D-128/D-138 execution-host assumption (see Section 3, its own severity call-out below) vvr1-dcN is the D-138-RULED host for the Juju client + openstack CLI, and (per D-132 amendment, design-decisions.md:7161-7168) the per-DC MAAS region controller. Both rulings say "in the DC" / "on the rack" and CITE vvr1-dc0 transit-IP SSH as the concrete mechanism (runbooks/dc-dc-phase3-maas-enlist-deploy.md:424,430; phase4-juju-bundle-per-dc.md:165). THE OWED QUESTION THIS PASS MUST ANSWER, NOT ASSUME: with no containment VM, where do juju/openstack/the MAAS region run from? See Section 3. CRITICAL / BLOCKING -- this is not a mechanical rename; it is an unresolved architecture question that D-128/D-132/D-138 do not answer on their own once their named host disappears.
runbooks/dc-dc-teardown-rollback.md (8 hits; Paths A/B/C/M) Teardown paths name vvr1-dcN as the target of tofu destroy / virsh destroy / juju-controller-down recovery. Once the containment VM does not exist, these paths must be re-authored around whatever the new substrate shape is (destroying flat node VMs individually or via a module, not "destroy one containment VM = destroy the site"). The D-122 intent quoted in opentofu/main.tf:405 ("site-down = a single virsh destroy vvr1-dc0") is a SIMPLICITY PROPERTY that flattening explicitly TRADES AWAY -- teardown of N+1 individual VMs is inherently more steps than one. Flag as a design tradeoff for W0.4/W4.2 (module design should re-earn this simplicity via a tofu-module-scoped destroy, not lose it). MEDIUM-HIGH -- teardown is a frequently-exercised, high-consequence path (see D-061 register / docs/tool-index.md's own origin story) and losing "one command tears down a DC" without a replacement is a real regression.

3. The D-128/D-138 execution-host consequence -- flagged explicitly, per instructions

Today: vvr1-dcN simultaneously serves three ruled roles:

  1. D-123 Model B node host -- nested libvirt for the 9+1(+2) node VMs.
  2. D-132-amendment MAAS region+rack for its own DC -- installed "in the DC" so jujud's continuous MAAS-API dependency never crosses the Office1 fiber (SEC-010 preserved unamended by removing the cross-fiber requirement rather than exempting it).
  3. D-138 cloud-facing CLIENT host -- juju/openstack run FROM vvr1-dcN, reached via ssh -J voffice1 jessea123@<dc-rack-transit-ip>, because SEC-010 + D-052 make the node planes unreachable at L3 from voffice1, and D-138's own reasoning is explicit: "a MAAS rack proxies at the application layer and needs no kernel forwarding... that is TRUE of every Plane-2 tool then in use and FALSE of Juju, which dials the MACHINE at L3."

If vvr1-dcN is eliminated, all three roles lose their named host simultaneously, and they do NOT have to be re-homed together -- each has a different constraint:

  • Role 1 (node host) simply DISAPPEARS as a distinct thing -- vcloud's own libvirt takes over, no replacement host is needed.
  • Role 2 (MAAS region) needs SOME host inside "the DC's" network segment (wherever that segment's boundary now is) so jujud's L3 MAAS dependency stays local -- this could be a flat per-DC utility VM (the existing .6/REGION_HOST_SUFFIX convention already assumes a VM distinct from the role nodes, so it may not even need a new design, just a new libvirt parent).
  • Role 3 (Juju/openstack CLIENT) needs a host that can reach the flat node VMs at L3 for the full port set D-138 enumerated (22, 17070, 5000, 9292, 8774, 9696, 8776, 8778, 9876, 9311, 9511, 443+ -- design-decisions.md:7107). If node VMs are flat on vcloud and reachable from vcloud's own network without SEC-010's transit-drop boundary in the way, the client could plausibly run FROM VCLOUD ITSELF (collapsing D-128's Plane-1/Plane-2 split for this specific tool-class) -- but this is only true if the flattened topology does NOT recreate an equivalent transit/forwarding boundary between vcloud and the flat node VMs. If it DOES (e.g., to keep simulating the Roosevelt fiber-boundary realism the whole VR1 exercise exists to rehearse), then a replacement "DC-local client host" is still needed, and D-138's reasoning (why the client cannot live at the far side of that boundary) still applies verbatim to whatever plays the transit-boundary role post-flatten.

This is the single largest open question this dimension surfaces, and W0.3 cannot close it alone -- it is jointly a W0.2 (network/wiring: does a transit/SEC-010-equivalent boundary still exist post-flatten?) and W0.4 (target-topology: what replaces vvr1-dcN as a named host) question. Recorded here as the explicit hand-off the SCOPE doc's Section 4 instructs: "if there is no containment VM, where do juju/openstack run from?" has NO answer in the current repo -- D-128/D-132/D-138 all name vvr1-dcN as the concrete host, and none of them anticipates its removal. Whatever Phase 0 / W0.4 proposes as the target topology MUST include an explicit answer to this, stated as plainly as D-138's own ruling was stated, because it carries the same SEC-010/D-052 security-boundary weight D-138 was ruled to resolve -- getting it wrong reopens exactly the "narrow SEC-010 exception vs. re-plan the control-plane topology" choice D-132's amendment closed.

4. Non-consumers worth naming (to prevent Phase 4 double-counting)

  • scripts/lib-net.sh -- no containment-layer coupling; only D-143 (re-IP) touches it. Keep the two axes distinguishable per SCOPE-AND-EXECUTION-PLAN.md Section 7.
  • scripts/maas-role-tags.sh, scripts/dc-egress-check.sh, scripts/maas-profile-assert.sh logic (not comments) -- host-agnostic by construction (driven by lib-hosts.sh/env), so they need NO code change, only doc-comment currency once Section 3's question is answered.
  • The 66 design-decisions.md hits and ~17 CURRENT-STATE.md hits are the DECISION RECORD -- per GA-R1/append-only discipline these are not edited; the container-elim [ARCH] decision (owed, per SCOPE doc Section 7) will AMEND or SUPERSEDE the relevant entries (D-122/123 most directly), not rewrite them in place. Not a "consumer" in the code sense; flagged so Phase 4 does not propose editing history.

5. Summary counts for the bounded return

  • Distinct functional consumers identified (code/config, excluding decision-record docs and the stale worktree copy): 15 (table rows in Section 2, excluding the two doc-only rows for teardown-runbook and netbox-import which are procedural/data consumers, counted separately).
  • Highest severity: CRITICAL/BLOCKING -- the D-128/D-138 execution-host question (Section 3): no repo surface currently answers "where do juju/openstack run from with no containment VM", and D-132/D-138's own security reasoning (SEC-010, the app-layer-proxy vs. L3-dial distinction) applies to whatever replaces vvr1-dcN in that role.
  • HIGH severity (concrete, non-blocking but large): scripts/lib-hosts.sh power-address maps (every DC arm), scripts/site-headend-install.sh --host-nodes machinery (~140 lines becomes dead code for VR1), scripts/dc-rack-net.sh (rack-leg + D-131 DNS forwarder needs a new home), the outer/inner OpenTofu roots (cross-referenced to W0.1), and runbooks/dc-dc-teardown-rollback.md (loses the "one virsh destroy" simplicity D-122 intended, needs a re-earned equivalent).