Fork: 0
OpenStack / openstack-caracal-dc-dc
Transfer to URL with SHA Find file
Newer
Older
openstack-caracal-dc-dc / docs / audit / container-elim-pass / pass2-w2-lib-hosts-net.md
@JANeumatrix JANeumatrix 8 hours ago 17 KB container-elim planning pass RAN (phases 0-4 + advisor): FINAL-PLAN + operator-report + D-144 framing (READ-ONLY)
Raw Blame History

Pass 2 -- W2.2: lib-hosts.sh / lib-net.sh containment-keyed values (flat topology + 10.13)

Author: W2.2 (Phase 2 -- Tools review), container-layer-elimination pass. READ-ONLY. No mutation, no live commands run. Repo: /home/jessea123/openstack-caracal-dc-dc. Baseline consumed: SCOPE-AND-EXECUTION-PLAN.md; pass0-admin-report.md (Option 1 CONFIRMED: flat node VMs on vcloud libvirt + one small non-hypervisor per-DC vr1-dcN-client VM carrying the D-138 client role + that DC's credential residencies; cross-DC handling (a) CONFIRMED; MAAS region stays on vr1-dcN-maas-01); pass1-admin-report.md (client VM = L1 cloudinit-vm module type, same class as voffice1/DC edges -- NOT an L2 MAAS-managed node; rack-controller-remainder + D-131 forwarder placement OPEN, carried to Phase 2 as item #2; client-VM octet+name carried to Phase 2 as item #5, owned here). Governing rulings read in full: D-134 (docs/design-decisions.md:5870-6007, the standing octet map + its three amendments), D-143 (docs/design-decisions.md:8083-8182, the 10.12->10.13 re-IP, C.3 lib-net.sh shape).

Sources read in full: scripts/lib-hosts.sh (269 lines), scripts/lib-net.sh (264 lines), scripts/maas-node-power.sh (power-address call sites), scripts/dc-rack-net.sh (rack-leg / D-131 forwarder addresses), docs/tool-index.md (operation lookups per Hard Rule 4).

1. scripts/lib-hosts.sh -- per-value table

Value Current containment binding Option-1 flat value/shape path:line
VIRSH_POWER_ADDRESS (flat/VR0 default) NOT containment-keyed. qemu+ssh://logxen@10.12.64.1/system dials a real VR0 KVM host directly -- VR0 was never nested. This is the existing flat-topology PRECEDENT the VR1 arms are converging toward. UNCHANGED (out of scope; VR0 is a separate live cloud, D-143 does not touch it, container-elim does not touch it) lib-hosts.sh:52
VIRSH_POWER_ADDRESS_FROM_OFFICE1 (per-DC) CONTAINMENT-KEYED. qemu+ssh://jessea123@172.31.0.2/system (dc0) / ...172.31.0.6/system (dc1) dials vvr1-dcN's own libvirtd over its D-124 transit-leg NIC1 address, from voffice1 (Office1 MAAS region, historically the sole power-dial origin -- comment: "What the Office1 region has always used"). UNDETERMINED from this file alone -- flagged, not inferred. The containment VM this address dials CEASES TO EXIST under Option 1; there is no libvirtd left at a 172.31.0.x transit address to reach. Post-flatten, MAAS's power target must become vcloud's own libvirtd (whichever URI/host that resolves to for qemu+ssh from a MAAS region). Whether reaching it FROM Office1 still rides the 172.31.0.0/30 transit legs (now terminating at the vr1-dcN-client VM's transit NIC per pass1's "D-124 survives only for the client VM's transit leg") -- a non-hypervisor client VM has no libvirtd for the power dial to land on -- or whether the FROM_OFFICE1 form loses its meaning entirely, is a Phase-2 W2.1(tofu)/W2.3(scripts) DESIGN QUESTION, not resolvable from lib-hosts.sh's data alone. See Section 4, biggest open item. lib-hosts.sh:212,246; called out by lib-hosts.sh:162-172 comment block
VIRSH_POWER_ADDRESS_FROM_DCREGION (per-DC) CONTAINMENT-KEYED. qemu+ssh://jessea123@10.12.8.2/system (dc0) / ...10.12.68.2/system (dc1) dials vvr1-dcN's libvirtd over its METAL-ADMIN bridge-leg address, from the DC-local MAAS region (vr1-dcN-maas-01, .6). Currently unused by default (comment: "Flipping the default ... is OWED once BOTH DCs have migrated"). Same undetermined status as FROM_OFFICE1, with an added structural wrinkle: under Option 1, BOTH DCs' node VMs are co-resident on ONE vcloud libvirtd (pass0 Section 5, the cross-DC adjacency gap). If the region VMs dial vcloud directly, vr1-dc0-maas-01 and vr1-dc1-maas-01 may resolve to the SAME target address (one libvirtd for both DCs) rather than two distinct per-DC addresses as today -- itself a manifestation of the co-residency the cross-DC control (Section 3, pass1 admin report) is designed around, not a lib-hosts.sh-local fix. Flagging, not asserting a value. lib-hosts.sh:213,250; comment :159-172
CARVE_AUX_HOSTS (per-DC populated: vr1-dcN-tailscale-01, vr1-dcN-maas-01) Names inner-root-provisioned utility VMs, carved via --host (excluded from HOSTS so role-node consumers don't miscount them). Not itself a containment ADDRESS, but its members were built by the inner (containment-nested) root. UNCHANGED SHAPE. vr1-dcN-tailscale-01 and vr1-dcN-maas-01 persist as flat vcloud-libvirt sibling VMs (pass0: the MAAS region "survives flattening as a flat sibling with no redesign"); they stay MAAS-carved and MAC-pinned, so they stay in CARVE_AUX_HOSTS, not HOSTS. vr1-dcN-client does NOT join this array -- per pass1's layer model it is an L1 cloudinit-vm (same module class as voffice1/DC edges), not an L3 MAAS-enrolled node, so it has no MAAS carve/power identity at all and is out of lib-hosts.sh's namespace entirely (Section 2). lib-hosts.sh:34,219,257
NIC_PLANE_ORDER Realized by the INNER root's six pinned MACs per node (main.tf macs[0..5]), but the ordering CONVENTION (metal-admin first, matching PXE/boot requirements) is topology-agnostic. UNCHANGED. pass0 row 14: "Conventions carry forward; the FILE that encodes the pinning changes" (the tofu module, not this constant). The re-homed flat module calls keep the same MAC-order-to-NIC-index contract. lib-hosts.sh:53-69
BREX_PARENT_NIC="enp2s0" Role-node carve convention (OVS br-ex parented on the provider-public NIC), independent of containment. UNCHANGED. Same reasoning as NIC_PLANE_ORDER. lib-hosts.sh:72
HOST_OCTET maps (.100-.200 node bands, .5/.6/.7 utility octets) NOT containment-keyed -- these are D-121/D-134 node-identity addressing, orthogonal to WHERE the libvirtd lives. A non-consumer of the container-elim axis, same class as lib-net.sh's plane CIDRs (Section 3). UNCHANGED under container-elim; changes ONLY under D-143 (second/third octet 12->13; last octet -- the D-134 band -- is offset-relative to the plane /22 and is untouched per D-143's own reconciliation, design-decisions.md:5942). This is where the client-VM octet question lives (Section 2). lib-hosts.sh:195-199,229-233
REGION_HOST_SUFFIX="maas-01" The constant itself is a naming suffix, containment-agnostic. Its COMMENT is containment-era-specific: "bootstrapped BY Office1 (deployed there, then MAAS installed on it) ... its carve runs against --profile admin with all Office1 racks in --expect-rack" -- describes a build procedure that assumed the Office1-reachable containment topology. Constant UNCHANGED. The comment's PROCEDURE (build-via-Office1-profile) needs re-verification once the region VM is a flat vcloud sibling with no containment hop to traverse -- this is site-headend-install.sh / MAAS-profile-script territory (W2.1/W2.3), not a lib-hosts.sh value change. Flagged as a comment-currency item, not asserted as broken. lib-hosts.sh:95-100
JUJU_HOST_SUFFIX="juju-01", TAILSCALE_HOST_SUFFIX="tailscale-01" Naming constants only. UNCHANGED -- non-consumers. lib-hosts.sh:89,94
HOST_TAG / per-DC HOST_TAG="openstack-vr1-dcN" MAAS placement tag for bundle binding -- orthogonal to containment. UNCHANGED. lib-hosts.sh:112,215,252
host_sysid() / host_sysid_by_bootmac() Resolution functions against the live MAAS API -- topology-agnostic (they resolve by hostname/MAC, never by containment address). UNCHANGED. lib-hosts.sh:124-138
lib_hosts_select_dc() case-arm structure (vr1-dc0/vr1-dc1) Houses all of the above per-DC. Structure UNCHANGED; the two power-address lines inside each arm are the load-bearing edits (rows 2-3 above). lib-hosts.sh:177-269

Transit IPs, precisely: the only "transit IP" values IN lib-hosts.sh are the VIRSH_POWER_ADDRESS_FROM_OFFICE1 targets (172.31.0.2, 172.31.0.6 -- the D-124 transit /30 legs into vvr1-dcN NIC1). No other transit literal exists in this file. Rack-leg .2/.3 addresses (metal-admin MAAS/DHCP leg, D-131 DNS forwarder) live in scripts/dc-rack-net.sh:59-82, NOT lib-hosts.sh -- confirmed by direct read; that script carries its own re-homing question (pass0 row 7, still OPEN per pass1 item #2) and is W2.3's dimension, not this one.

maas-node-power.sh (row 5, pass0) -- CONFIRMED topology-agnostic, verified this session. It takes the power address as a positional ARGUMENT (scripts/maas-node-power.sh qemu+ssh://jessea123@172.31.0.2/system vr1-dc0, :4-5,46) and writes it via power_parameters_power_address (:109). No code change is needed in this script under Option 1 -- only every call-site/runbook example carrying the old containment address needs updating once the new target is decided (Section 4's open item).

2. Client-VM octet + name -- recommendation (the decision this worker owns)

Recommendation: octet .8, name vr1-dcN-client.

Octet rationale, grounded in D-134 (docs/design-decisions.md:5996-6007, the most recent amendment). The STANDING cross-DC utility-band octet map, as ruled, is:

.1  gateway (routed planes)
.2  rack (MAAS rack-controller leg)
.3  node-DNS forwarder (D-131)
.4  artifact service (mirror/proxy)
.5  Juju controller (D-134 amendment 2026-07-29)
.6  MAAS region (D-134 amendment 2026-07-29 / D-132 addendum)
.7  Tailscale subnet router (D-134 amendment 2026-08-07)
.8  <next free utility slot>

.8 is the next free slot in the .4-.49 utility band (design-decisions.md:5929, "RESERVED: future per-DC utility/infra nodes (46 slots)") -- no measurement is needed to establish freeness; the ruled table enumerates every occupied utility octet through .7 and stops there. D-134's 2026-07-29 amendment is explicit that this is not a per-DC choice: "the octet map is a STANDING CROSS-DC STANDARD... Assigning it in one DC assigns it in all of them... Divergence between DCs at the same octet is a DEFECT" (:5977-5987). So .8 must be reserved for vr1-dcN-client at every DC, not decided per-standup -- consistent with the task framing. Under D-143's octet-preserving shift the last-octet value is untouched (D-134 endorses the 1:1 shift precisely because it is "offset-relative to the plane /22, not tied to the second octet," design-decisions.md:5142-5146 reconciliation), so .8 reads as 10.12.8.8/10.12.68.8 today and 10.13.8.8/10.13.68.8 post-re-IP, on whichever plane(s) the client VM's legs attach to (metal-admin per the D-138/D-124 transit shape; confirm exact plane(s) at build time -- not inferred here).

Naming rationale. vr1-dcN-client (already the name used consistently across SCOPE-AND-EXECUTION-PLAN.md, pass0-admin-report.md, pass1-admin-report.md) satisfies the task's hard constraint -- it does NOT read as vvr1-dcN (no risk of conflation with the eliminated containment class) and it follows the existing <dc>-<role>-NN family the utility octets already use (-juju-01, -maas-01, -tailscale-01). It is also NOT MAAS-carved (Section 1's CARVE_AUX_HOSTS finding: L1 cloudinit-vm, no boot MAC / power-type dance), so the -NN suffix convention is cosmetic consistency, not a functional requirement the way it is for MAAS-enrolled siblings.

One open placement question this recommendation does NOT resolve (correctly deferred to pass1's open item #2, not re-decided here): whether the rack-controller remainder + D-131 forwarder + .4 artifact service co-locate onto .8's client VM or onto .6's region VM. That is a role-placement decision, orthogonal to the octet-map slot assignment above -- .8 is reserved for the client VM's OWN identity regardless of which additional duties later land on it.

Register/register-adjacent note (not this worker's artifact to build, flagged for Phase 2/4): once ruled, .8 needs a HOST_OCTET entry (or equivalent) added to both vr1-dc0/vr1-dc1 arms if the client VM is to be resolved by the same lib-hosts.sh machinery as the utility nodes -- but per Section 1's CARVE_AUX_HOSTS finding, it is not a MAAS/virsh-power object, so whether it belongs in lib-hosts.sh at all (vs. purely in the tofu cloudinit-vm definition + IPAM record) is itself a small open design choice for W2.1.

3. scripts/lib-net.sh -- axis-separation statement (verified, not assumed)

Confirmed by direct full read this session: lib-net.sh contains ZERO vvr1-dc / containment references. Grep-confirmed (grep -n "vvr1" scripts/lib-net.sh -> no hits; matches Phase-0's finding, pass0-admin-report.md row 15). Every value in the file is IPAM literal: PLANE_CIDRS, PLANE_NAME, PLANE_GW, DATA_PLANE_CIDRS, METAL_INTERNAL_*, VIP_PREFIX_*, VIP_OCTET_MIN/MAX, VIP_COUNT_EXPECT, FIP_POOL_START/END, KEYSTONE_VIP_DEFAULT, and the lib_net_select_dc() per-arm overrides -- none of these encode a containment VM, a qemu+ssh dial, or a nested-libvirt fact. They are entirely D-052 / D-119 / D-133 / D-134 / R9 / R11 address-plane facts.

D-143 is EXPLICIT and load-bearing here, not inferred. The ruling itself specifies the exact shape lib-net.sh takes (design-decisions.md:8116-8123, Exchange 2 C.3, operator exact utterance "(i) Keep flat defaults at 10.12; VR1 arms get full 10.13 blocks"): the flat (vr0-dc0, no-op) defaults stay at 10.12 (the live cloud's real values); the vr1-dc0 and vr1-dc1 case arms each gain a complete explicit 10.13 literal block. Owed execution item 2 of the ruling (:8167-8168) names this file directly: "scripts/lib-net.sh: keep flat defaults at 10.12; give vr1-dc0/vr1-dc1 full 10.13 literal blocks; UPDATE the now-false :124-134 'inherits VR0 unchanged' comment (F13)." Every edit this ruling requires is a value substitution (10.12.x.y -> 10.13.x.y across PLANE_CIDRS, PLANE_GW, VIP_PREFIX_*, FIP_POOL_*, KEYSTONE_VIP_DEFAULT) plus one comment-currency fix (F13) -- no shape change, no new containment-dependent value, nothing container-elim touches.

Verdict: lib-net.sh's changes are D-143 ADDRESS-AXIS ONLY. The container-elim change-set should carry ZERO line-item edits to lib-net.sh. This keeps the two axes cleanly separable for this file specifically (unlike the four [both]-tagged items pass1 found elsewhere -- G17, B.1.4, R7, B.7 -- none of which are lib-net.sh edits).

Plane-CIDR / MTU values the flat topology touches -- checked, none found IN this file. lib-net.sh carries no MTU constant at all (grep-confirmed: no MTU / mtu token in the file). The plane CIDR VALUES (PLANE_CIDRS) are untouched by flattening -- pass0/pass1 are explicit that the six planes re-home to vcloud-level bridges with "SAME CIDRs/families/MTU -- IPAM identity untouched" (pass0 Section 4) and that the containment hop was "a same-MTU bridge with no extra encapsulation... removing it changes no byte budget" (pass0 Section 1.4). Where those CIDRs are REALIZED (which host's bridges carry them) is a tofu/substrate fact (opentofu/vr1-dcN-substrate -> the flat root, W2.1's dimension), not a lib-net.sh fact -- this file only ever held the address-plane VALUES, never the hosting topology.

4. Top risks / open items (this dimension)

  1. BIGGEST FINDING: VIRSH_POWER_ADDRESS_FROM_OFFICE1/_FROM_DCREGION's target is UNDETERMINED, not just "re-derived." Both today dial vvr1-dcN's libvirtd -- an object Option 1 deletes outright. There is no drop-in replacement host with a libvirtd at a client-VM-shaped address (the client VM is explicitly non-hypervisor). The power dial must land on vcloud's own libvirtd; whether the FROM_OFFICE1/FROM_DCREGION SPLIT still means anything once both DCs' targets may collapse toward the same vcloud host is a design question for W2.1 (tofu placement) and W2.3 (script/runbook), not something resolvable by editing this file's literals alone. Getting this wrong reproduces exactly the CLAUDE.md-cited incident class (wrong power address masquerading as a network fault, pass0 row 4).
  2. Cross-DC co-residency implication surfaces here too, not just in the network-wiring dimension. If both DCs' power dials converge on one vcloud libvirtd target, that is a second, host-identity-shaped face of pass0 Section 5's cross-DC adjacency gap -- worth flagging to whichever Phase-2 worker owns the (a) isolation-control design so it accounts for the operational surface, not only the data-plane one.
  3. REGION_HOST_SUFFIX's comment procedure (build-via-Office1---profile admin) is containment-era language that needs re-verification against the flat build path (W2.1/W2.3), even though the constant itself does not change.
  4. The client-VM octet's placement in lib-hosts.sh's own data structures is undecided (Section 2, register note) -- contingent on W2.1's tofu-module classification of the client VM (whether it gets any lib-hosts.sh-visible identity at all, given it is not MAAS/virsh-power-managed).
  5. Every VIRSH_POWER_ADDRESS call-site and runbook example (row 5's finding, echoed from pass0) needs a literal update once item 1 above is resolved -- listed here as a delivery dependency, not re-enumerated (pass0 row 5 already owns the site inventory).

5. Method note

No live commands were run. All findings above are grep/Read-verified against repo HEAD this session (scripts/lib-hosts.sh, scripts/lib-net.sh, scripts/maas-node-power.sh, scripts/dc-rack-net.sh, docs/design-decisions.md D-134 section incl. all three amendments, D-143 in full, docs/tool-index.md). Where a value could not be determined from the repo as it stands (Section 4 item 1), it is stated as UNKNOWN / owed, per the pass's no-inferred-value rule, rather than asserted.