diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index e578927..f8d1323 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -90,12 +90,16 @@ traffic -- pf ruleset never regenerated after v4 addressing (the set-interface script reloads the interface, not the filter; dc0 was masked by its 07-20 plugin work). Fix: `configctl filter reload`; - dc1 rack egress now 0% loss / HTTP 301. Appendix-A entry + standup - DoD addition + set-interface amendment QUEUED to stage close. NEXT: - syncs complete (last-sync.status OK both) -> mirror node-side - reachability gate -> Step 6/7/8 verify closure -> lib-hosts.sh - per-DC population + lib-net VID comment (delivery items) -> stage - close-out set. + dc1 rack egress now 0% loss / HTTP 301. **Delivery batch LANDED + 2026-07-23 (changelog item 11):** lib-hosts VR1 arms populated + (tofu-name keys, pinned MACs, D-134 octets, per-DC power/tags, new + host_sysid_by_bootmac resolver; dc-selector 48 PASS), D-133 guards + in reenroll-hosts + carve-host-interfaces (VR0 flows refuse vr1-*), + lib-net VID caveat, appendix-A edge-pf entry, DoD items 8-9 + item-4 + range correction; gauntlet ALL GREEN (77). Still QUEUED to stage + close: set-interface-v4 reload amendment (option, unruled); + permission-rules prune. NEXT: syncs complete (last-sync.status OK + both) -> mirror node-side reachability gate -> stage close-out set. - Position inside Stage 3: deploy step A EXECUTED 2026-07-19 (6/0/6 exact; convergence zero -- `docs/audit/outer-plan-20260719-postA-converged.txt`). **Deploy step B diff --git a/docs/changelog-20260723-stage4-open.md b/docs/changelog-20260723-stage4-open.md index 8639a39..5901d3e 100644 --- a/docs/changelog-20260723-stage4-open.md +++ b/docs/changelog-20260723-stage4-open.md @@ -214,6 +214,28 @@ rm the generated files, `rm -rf /var/lib/dc-mirror`, apt remove the three packages; the edge filter reload stands (item 9). +## Item 11 -- Delivery batch: lib-hosts VR1 arms, D-133 guards, appendix-A, DoD + +- **lib-hosts.sh VR1 arms POPULATED** (the runbook's standing delivery item): + vr1-dc0/vr1-dc1 blocks with tofu-name-keyed HOSTS, D-134-amended octets, + config-pinned boot MACs, per-DC power URIs (ruled step-D shape), per-DC + tags; NEW `host_sysid_by_bootmac()` (the VR1 resolver -- MAAS hostnames are + random and re-minted; MACs survive re-enlistment); one-selection-per-shell + guard (lib-net precedent). dc-selector harness rewritten for the populated + arms + identity/guard cases: 48 checks PASS. +- **D-133 guards:** `reenroll-hosts.sh` + `carve-host-interfaces.sh` now + REFUSE vr1-* explicitly (VR0 flows: hostname-keyed re-enroll / Pattern-A + VLAN-103 carve, both superseded for VR1) -- without the guard, populated + VR1 arms would have let VR0 mechanics run against VR1 fleets. Harnesses + updated to the refusal messages. Gauntlet **ALL GREEN (77)** after. +- **lib-net vr1-dc0 arm:** D-133 caveat comment -- inherited + METAL_INTERNAL_VID/IFACE are VR0 carve artifacts; never consume for VR1. +- **appendix-A:** new entry for the edge-pf symptom (LAN 100% loss, gateway + fine, `pfctl -s nat` empty; masking trap named). **DoD:** items 8 (filter + reload + LAN egress probe) and 9 (dc-mirror install/sync) added; item 4's + dynamic-range position corrected to the D-134-amended .201-.254. +- **Revert:** `git revert ` (repo-only; no live state). + ## Item 8 -- OWNED: pipe-masked L10 lint FAIL on the D-135 commit (abd6b92) - The D-135 commit changed a design-decisions Status line WITHOUT diff --git a/runbooks/appendix-A-troubleshooting.md b/runbooks/appendix-A-troubleshooting.md index bb5f6b9..ee3e02f 100644 --- a/runbooks/appendix-A-troubleshooting.md +++ b/runbooks/appendix-A-troubleshooting.md @@ -807,3 +807,29 @@ docs/audit/incident-20260721-recovery-verify.txt). NOTE: MAAS assigns NEW random hostnames at re-enlistment -- update anything quoting the old ones. + +## Symptom: LAN clients of a freshly-addressed DC edge get 100% loss to the internet; the edge gateway pings fine; the edge ITSELF egresses fine [2026-07-23] + +CAUSE: The edge's pf ruleset was never REGENERATED after v4 addressing. + `opnsense-set-interface-v4 --commit` runs `configctl interface + reconfigure` -- which applies the address to the kernel -- but does + NOT reload the pf filter, so OPNsense's automatic outbound NAT + rules for the (re-)addressed LAN are never generated. Forwarded + LAN packets leave the WAN un-NAT'd; the upstream libvirt NAT only + masquerades sources in ITS OWN subnet, so replies never come back. + MASKING TRAP: any later plugin install / config apply reloads the + filter as a side effect -- dc0's 2026-07-20 qga/plugin work did + exactly that, which made dc0 "work" and hid the defect until dc1 + (which had no such later work) exposed it at the mirror build. + +CHECK: On the edge: `pfctl -s nat` -- a healthy edge shows automatic + `nat on inet from (:network) ...` rules; the broken + state shows ONLY the `no nat/rdr proto carp` boilerplate. Both + edges' config.xml can be IDENTICAL (same WAN_GW gateway_item) -- + this is a runtime-ruleset gap, not a config gap. + +FIX: On the edge: `configctl filter reload`, then re-test from a LAN + client (rack): ping internet IP 0% loss, `curl -sI http://1.1.1.1/` + -> HTTP 301. Standing prevention: DC standup DoD item 8 + (dc-dc-phase2 runbook); a set-interface-v4 amendment (reload in + the script itself) is a QUEUED stage-close option, not yet ruled. diff --git a/runbooks/dc-dc-phase2-tofu-dc-substrate.md b/runbooks/dc-dc-phase2-tofu-dc-substrate.md index 4013fc3..3bb5108 100644 --- a/runbooks/dc-dc-phase2-tofu-dc-substrate.md +++ b/runbooks/dc-dc-phase2-tofu-dc-substrate.md @@ -760,11 +760,13 @@ 3. **Serial-log staging dir** `/var/lib/libvirt/vr1/staging/` on the rack -- created by `site-headend-install.sh --host-nodes` (since 2026-07-23; its absence failed the dc1 inner apply at 18/28). `--check` verifies it. -4. **Region-side MAAS subnet config on the DC's metal-admin subnet:** D-120 - dynamic range (.100-.200), D-131 `dns_servers=` + - `allow_dns=false`, then DHCP `dhcp_on=true primary_rack=` - -- verify dhcpd RUNNING by process on the rack, not by MAAS self-report - (the 2026-07-20 Temporal incident class). +4. **Region-side MAAS subnet config on the DC's metal-admin subnet:** dynamic + range at **.201-.254** (D-134 AMENDMENT 2026-07-23 supersedes the original + D-120-inherited .100-.200 position -- .100-.200 is now the contiguous NODE + band), D-131 `dns_servers=` + `allow_dns=false`, then + DHCP `dhcp_on=true primary_rack=` -- verify dhcpd RUNNING + by process on the rack, not by MAAS self-report (the 2026-07-20 Temporal + incident class). 5. **Per-DC MAAS->libvirt power key (SEC-012 dc0 / SEC-016 dc1 -- standing invariant, RULED).** Each DC gets its OWN dedicated power keypair, never cross-DC reuse: private half in the region MAAS snap @@ -784,6 +786,20 @@ audit file (dc0: `commissioning-diag-20260721`; dc1: `dc1-commissioning-verify-20260723` -- READY in ~3.5 min when 1-6 are in place). +8. **Edge pf ruleset regenerated + LAN-client egress proven (2026-07-23 + incident).** After the edge's v4 addressing, run `configctl filter + reload` on the edge, then from the RACK prove LAN-side egress: ping an + internet IP 0% loss + `curl -sI http://1.1.1.1/` returning 301 (the + D-125 signature). WITHOUT the reload a fresh edge passes NO LAN traffic + (automatic outbound NAT rules are never generated) with a perfectly + reachable gateway -- appendix-A carries the symptom. Do NOT rely on + plugin installs to reload the filter as a side effect (that masking is + exactly how dc0 hid this from dc1). +9. **Per-DC artifact mirror (D-135 item 1).** `scripts/dc-mirror.sh install + ` on the rack (net layer -> printed apt line -> re-run install), + then `sync `; verify `check ` PASS and, post-sync, + `last-sync.status` reads OK. The mirror serves the D-134 utility .4 on + metal-admin; Stage 5 wires it via `juju model-config apt-mirror=`. --- diff --git a/scripts/carve-host-interfaces.sh b/scripts/carve-host-interfaces.sh index 576141f..a3bbc38 100644 --- a/scripts/carve-host-interfaces.sh +++ b/scripts/carve-host-interfaces.sh @@ -52,9 +52,16 @@ # always has, implicitly against DC0/VR0's real plane scheme and enrolled # hosts (backward compatible by construction). Set DC=vr0-dc0|vr1-dc0|vr1-dc1 to select (D-119) # explicitly; the selectors' own fail-loud behavior becomes this script's own -# exit code via set -e. No second validation layer is added. +# exit code via set -e. VR1 DCs are REFUSED even though lib-hosts' VR1 arms +# are populated (2026-07-23): this script builds the VR0 Pattern-A tree +# (br-metal + VLAN-103 br-internal stack), which D-133 SUPERSEDES for VR1 -- +# VR1 DC nodes carve every plane FLAT on its dedicated NIC (executed via the +# MAAS API at Stage 4; capture docs/audit/stage4-carve-verify-20260723.txt). DC="${DC:-}" if [ -n "$DC" ]; then + case "$DC" in vr1-*) + echo "FAIL: $DC refused -- this script carves the VR0 Pattern-A VLAN-103 stack, superseded for VR1 by D-133 (flat per-NIC carve, applied at Stage 4). There is nothing for it to do on a VR1 fleet." >&2 + exit 1 ;; esac lib_net_select_dc "$DC" lib_hosts_select_dc "$DC" fi diff --git a/scripts/lib-hosts.sh b/scripts/lib-hosts.sh index 46a25b6..1a1db14 100644 --- a/scripts/lib-hosts.sh +++ b/scripts/lib-hosts.sh @@ -51,38 +51,104 @@ # host_sysid : resolve the LIVE MAAS system_id by hostname (never # hardcode it). Echoes the system_id, or empty if the host is not enrolled. +# VR0 ONLY: VR0 hostnames are stable (hostname == domain name == power_id). +# For VR1 DC nodes the MAAS hostname is a RANDOM name RE-MINTED on every +# (re-)enlistment (measured: the 2026-07-21 pod-delete recovery renamed all 9) +# -- use host_sysid_by_bootmac() there instead. host_sysid() { local hn="$1" maas "${MAAS_PROFILE:-admin}" machines read 2>/dev/null \ | jq -r --arg h "$hn" '.[] | select(.hostname==$h) | .system_id' | head -1 } +# host_sysid_by_bootmac : resolve the LIVE MAAS system_id by boot-NIC MAC. +# THE VR1 resolver: VR1 node identity is the tofu node name -> its config-PINNED +# boot MAC (opentofu/vr1-dcN-substrate; pinned 2026-07-21/22); the MAC survives +# re-enlistment, the random hostname does not. +host_sysid_by_bootmac() { + local mac="$1" + maas "${MAAS_PROFILE:-admin}" machines read 2>/dev/null \ + | jq -r --arg m "$mac" '.[] | select(.boot_interface.mac_address==$m) | .system_id' | head -1 +} + # --- $DC selector convention (2026-07-09, tooling gap register #1) --- # # Mirrors lib-net.sh's lib_net_select_dc -- SAME region-qualified selectors -# (D-119), but the answer here is honestly different, not a parallel no-op: -# HOSTS / HOST_OCTET / HOST_BOOT_MAC above are VR0 DC0's real, measured, -# enrolled hosts. VR1 DC0's and VR1 DC1's node VMs are created by -# opentofu/modules/node-vm (Stage 3 / buildout-design Phase 2), which has not -# run -- their hostnames, boot MACs, and octets do not exist yet, and -# VIRSH_POWER_ADDRESS above is VR0 DC0's own vcloud host connection (the -# buildout design's VR1 vcloud host is a distinct, not-yet-measured endpoint -- -# see opentofu's root `libvirt_uri` variable, also unset). Selecting either VR1 -# DC FAILS LOUDLY rather than inventing placeholder host data or silently -# reusing VR0 DC0's real hosts for a different DC. +# (D-119). The VR1 arms were POPULATED 2026-07-23 (Stage 4 carve session) from +# MEASURED enrollment: both fleets commissioned Ready, identities cross-checked +# tofu-name -> pinned boot MAC -> live system_id 9/9 per DC +# (docs/audit/stage4-carve-verify-20260723.txt). Until then both VR1 arms +# failed loud by design; that era's asymmetry note is history (git). # -# DELIBERATE ASYMMETRY WITH lib-net.sh: `vr1-dc0` is a NO-OP there (D-101 says -# it inherits VR0 DC0's v4 layout) but FAILS here (it has no enrolled hosts). -# That is intentional and is asserted by tests/dc-selector. +# VR1 IDENTITY MODEL (differs from VR0 -- read before consuming these maps): +# - HOSTS entries are the TOFU NODE NAMES (config-of-record, stable), NOT +# MAAS hostnames -- MAAS names are random and re-minted on re-enlistment. +# - Resolve a live system_id via host_sysid_by_bootmac "${HOST_BOOT_MAC[$h]}" +# (never via host_sysid; never cache system_ids). +# - HOST_OCTET carries the D-134-AMENDED per-role bands (control .100-.102, +# compute .120-.121, storage .150-.153), same octet on all six planes +# (D-133 flat carve -- applied + verified live, capture above). +# - VIRSH_POWER_ADDRESS is the per-DC rack transit URI (ruled step-D shape, +# per-machine virsh power -- D-103/D-123 amendments; SEC-012/SEC-016 keys). +# - HOST_TAG is the region-qualified per-DC placement tag (ruled 2026-07-23). +# ONE SELECTION PER SHELL: cross-DC re-selection is REFUSED (stale-value +# guard, lib-net precedent); same-DC re-selection is a no-op. +_LIB_HOSTS_SELECTED="" lib_hosts_select_dc() { local dc="${1:?usage: lib_hosts_select_dc }" + if [ -n "$_LIB_HOSTS_SELECTED" ] && [ "$_LIB_HOSTS_SELECTED" != "$dc" ]; then + echo "FAIL: lib-hosts already selected '$_LIB_HOSTS_SELECTED' in this shell -- start a fresh shell to select '$dc' (stale cross-DC host values are the D-117 off-by-one class)" >&2 + return 1 + fi case "$dc" in vr0-dc0) : # already the sourced defaults above -- explicit no-op, not a guess. ;; - vr1-dc0|vr1-dc1) - echo "FAIL: $dc has no enrolled hosts yet -- Phase 2/3 node-VM creation (opentofu/modules/node-vm) and MAAS enrollment haven't run for it. Do not invent HOSTS/HOST_OCTET/HOST_BOOT_MAC/VIRSH_POWER_ADDRESS values; populate this file with a real \$dc block once those hosts are actually enrolled and measured." >&2 - return 1 + vr1-dc0) + # MEASURED 2026-07-23 (Stage 4): 9/9 Ready; MACs are the config pins + # (opentofu/vr1-dc0-substrate, pinned from virsh domiflist 2026-07-21); + # octets applied live per D-134 amendment; tag applied 9/9 same day. + HOSTS=( vr1-dc0-control-01 vr1-dc0-control-02 vr1-dc0-control-03 + vr1-dc0-compute-01 vr1-dc0-compute-02 + vr1-dc0-storage-01 vr1-dc0-storage-02 vr1-dc0-storage-03 vr1-dc0-storage-04 ) + HOST_OCTET=( [vr1-dc0-control-01]=100 [vr1-dc0-control-02]=101 [vr1-dc0-control-03]=102 + [vr1-dc0-compute-01]=120 [vr1-dc0-compute-02]=121 + [vr1-dc0-storage-01]=150 [vr1-dc0-storage-02]=151 + [vr1-dc0-storage-03]=152 [vr1-dc0-storage-04]=153 ) + HOST_BOOT_MAC=( [vr1-dc0-control-01]=52:54:00:be:69:c5 + [vr1-dc0-control-02]=52:54:00:02:ff:57 + [vr1-dc0-control-03]=52:54:00:4f:de:a9 + [vr1-dc0-compute-01]=52:54:00:1b:19:e6 + [vr1-dc0-compute-02]=52:54:00:18:ab:b4 + [vr1-dc0-storage-01]=52:54:00:5f:8d:42 + [vr1-dc0-storage-02]=52:54:00:48:86:2c + [vr1-dc0-storage-03]=52:54:00:b1:94:d1 + [vr1-dc0-storage-04]=52:54:00:2b:ed:ab ) + VIRSH_POWER_ADDRESS="qemu+ssh://jessea123@172.31.0.2/system" + HOST_TAG="openstack-vr1-dc0" + ;; + vr1-dc1) + # MEASURED 2026-07-23 (Stage 4): 9/9 Ready; MACs are the config pins + # (opentofu/vr1-dc1-substrate, 52:54:01:d1:NN:01 encodes the node index); + # octets applied live per D-134 amendment; tag applied 9/9 same day. + HOSTS=( vr1-dc1-control-01 vr1-dc1-control-02 vr1-dc1-control-03 + vr1-dc1-compute-01 vr1-dc1-compute-02 + vr1-dc1-storage-01 vr1-dc1-storage-02 vr1-dc1-storage-03 vr1-dc1-storage-04 ) + HOST_OCTET=( [vr1-dc1-control-01]=100 [vr1-dc1-control-02]=101 [vr1-dc1-control-03]=102 + [vr1-dc1-compute-01]=120 [vr1-dc1-compute-02]=121 + [vr1-dc1-storage-01]=150 [vr1-dc1-storage-02]=151 + [vr1-dc1-storage-03]=152 [vr1-dc1-storage-04]=153 ) + HOST_BOOT_MAC=( [vr1-dc1-control-01]=52:54:01:d1:01:01 + [vr1-dc1-control-02]=52:54:01:d1:02:01 + [vr1-dc1-control-03]=52:54:01:d1:03:01 + [vr1-dc1-compute-01]=52:54:01:d1:04:01 + [vr1-dc1-compute-02]=52:54:01:d1:05:01 + [vr1-dc1-storage-01]=52:54:01:d1:06:01 + [vr1-dc1-storage-02]=52:54:01:d1:07:01 + [vr1-dc1-storage-03]=52:54:01:d1:08:01 + [vr1-dc1-storage-04]=52:54:01:d1:09:01 ) + VIRSH_POWER_ADDRESS="qemu+ssh://jessea123@172.31.0.6/system" + HOST_TAG="openstack-vr1-dc1" ;; dc0|dc1|dc2) echo "FAIL: bare '$dc' is RETIRED (D-119). It was AMBIGUOUS ACROSS REGIONS: 'dc0' meant VR0's live DC0 here, but VR1's FIRST DC in the NetBox importer. Use the region-qualified selector: vr0-dc0 | vr1-dc0 | vr1-dc1" >&2 @@ -93,4 +159,5 @@ return 1 ;; esac + _LIB_HOSTS_SELECTED="$dc" } diff --git a/scripts/lib-net.sh b/scripts/lib-net.sh index 6195b74..341c560 100644 --- a/scripts/lib-net.sh +++ b/scripts/lib-net.sh @@ -116,6 +116,14 @@ # this arm diverges and vr0-dc0's must not. Folding the two arms together # would encode the coincidence and lose the reason -- and the reason is # the thing that changes (VR1 DC0 gains its own v6 per the family matrix). + # + # D-133 CAVEAT (2026-07-23): the inherited METAL_INTERNAL_VID=103 / + # METAL_INTERNAL_IFACE=br-internal are VR0 CARVE ARTIFACTS ONLY. VR1 DC + # nodes carve every plane FLAT on its dedicated NIC (D-133 ADOPTED; + # metal-internal = untagged on enp3s0). Do NOT consume these two values + # for any VR1 carve/netplan surface -- they survive here solely because + # the D-101 inheritance is value-for-value and VR0 tooling still reads + # them under the vr0-dc0 arm. ;; vr1-dc1) # VR1's SECOND DC -- literals RATIFIED by the D-124 AMENDMENT (2026-07-21) diff --git a/scripts/reenroll-hosts.sh b/scripts/reenroll-hosts.sh index 214601f..041e5ba 100644 --- a/scripts/reenroll-hosts.sh +++ b/scripts/reenroll-hosts.sh @@ -42,13 +42,17 @@ # dc-dc-phase3 runbook's Step 1: an explicit $DC env var, never an inferred # default. Unset/empty $DC changes NOTHING -- this script runs exactly as it # always has, implicitly against DC0/VR0's real, enrolled hosts (backward -# compatible by construction). Set DC=vr0-dc0|vr1-dc0|vr1-dc1 to select explicitly (D-119); the -# selectors' own fail-loud behavior becomes this script's own exit code via -# set -e (lib_hosts_select_dc fails for BOTH vr1-dc0 and vr1-dc1 today -- no -# per-DC host inventory exists yet for either). No second validation layer is -# added. +# compatible by construction). Set DC=vr0-dc0|vr1-dc0|vr1-dc1 to select +# explicitly (D-119). VR1 DCs are REFUSED here even though lib-hosts' VR1 arms +# are populated (2026-07-23): this script's re-enroll mechanics are the VR0 +# flow (hostname-keyed, auto-commission assumptions); the VR1-native flow is +# PXE re-enlist (pinned MACs) + scripts/maas-node-power.sh -- proven in the +# 2026-07-21 recovery. Do not point this script at a VR1 fleet. DC="${DC:-}" if [ -n "$DC" ]; then + case "$DC" in vr1-*) + echo "FAIL: $DC refused -- this is the VR0 re-enroll flow. VR1 re-enrollment is PXE re-enlist (config-pinned MACs) + scripts/maas-node-power.sh (D-103/D-123 amendments); see the 2026-07-21 recovery in appendix-A." >&2 + exit 1 ;; esac lib_net_select_dc "$DC" lib_hosts_select_dc "$DC" fi diff --git a/tests/carve-host-interfaces/run-tests.sh b/tests/carve-host-interfaces/run-tests.sh index 142174b..8bf066c 100644 --- a/tests/carve-host-interfaces/run-tests.sh +++ b/tests/carve-host-interfaces/run-tests.sh @@ -83,12 +83,12 @@ has 'WOULD: create br-ex \(OVS\) parent=enp1s0\(id=100\)' has 'Summary: 0 fatal' -run_dc 1 "DC=vr1-dc0: hosts selector fails loud (no per-DC host data)" vr1-dc0 -has 'no enrolled hosts yet' +run_dc 1 "DC=vr1-dc0: REFUSED (D-133 guard -- VR0 flow, VR1 arms populated 2026-07-23)" vr1-dc0 +has 'superseded for VR1 by D-133' absent 'DO:|WOULD:' # must exit before any MAAS interaction -run_dc 1 "DC=vr1-dc1: hosts selector fails loud (net arm ratified, D-124)" vr1-dc1 -has 'no enrolled hosts yet' +run_dc 1 "DC=vr1-dc1: REFUSED (D-133 guard -- VR0 flow, VR1 arms populated 2026-07-23)" vr1-dc1 +has 'superseded for VR1 by D-133' absent 'DO:|WOULD:' # must exit before any MAAS interaction run_dc 1 "DC=bogus: unknown token fails loud" bogus diff --git a/tests/dc-selector/run-tests.sh b/tests/dc-selector/run-tests.sh index 29e66c0..f079d3e 100644 --- a/tests/dc-selector/run-tests.sh +++ b/tests/dc-selector/run-tests.sh @@ -11,7 +11,7 @@ # 10.12.64.0/19) and UNSETS the not-yet-ruled OpenStack-layer values; # unknown/retired tokens fail loud; one selection per shell (the guard # added with the diverging dc1 arm). -# - lib_hosts_select_dc: vr0-dc0 no-op, BOTH VR1 DCs fail loud (no real +# - lib_hosts_select_dc: vr0-dc0 no-op, VR1 arms POPULATED 2026-07-23 (was: fail loud -- no real # per-DC host enrollment exists yet for either -- this is the documented # asymmetry vs. lib-net.sh). # - Neither function ever silently invents/reuses a value across DCs. @@ -94,25 +94,43 @@ VR1_VAL="$(lib_net_select_dc vr1-dc0 >/dev/null 2>&1; echo "${PLANE_CIDRS[0]}")" chk "net vr0-dc0/vr1-dc0 identical values (D-101 inheritance)" "$VR0_VAL" "$VR1_VAL" -# --- lib_hosts_select_dc: vr0-dc0 no-op, BOTH VR1 DCs fail loud --- +# --- lib_hosts_select_dc: vr0-dc0 no-op; VR1 arms POPULATED 2026-07-23 --- +# (Stage 4 carve session: both fleets enrolled+Ready, identities measured -- +# the historical fail-loud behavior and its asymmetry test are history.) ( lib_hosts_select_dc vr0-dc0 ); chk "hosts vr0-dc0 no-op rc" "$?" 0 -( lib_hosts_select_dc vr1-dc0 2>/dev/null ); chk "hosts vr1-dc0 fails-loud rc" "$?" 1 -( lib_hosts_select_dc vr1-dc1 2>/dev/null ); chk "hosts vr1-dc1 fails-loud rc" "$?" 1 -HOSTS_ERR1="$(lib_hosts_select_dc vr1-dc0 2>&1 1>/dev/null || true)" -grep -q "no enrolled hosts" <<<"$HOSTS_ERR1" && ok "hosts vr1-dc0 error cites no-enrollment" || no "hosts vr1-dc0 error cites no-enrollment" -HOSTS_ERR2="$(lib_hosts_select_dc vr1-dc1 2>&1 1>/dev/null || true)" -grep -q "no enrolled hosts" <<<"$HOSTS_ERR2" && ok "hosts vr1-dc1 error cites no-enrollment" || no "hosts vr1-dc1 error cites no-enrollment" +( lib_hosts_select_dc vr1-dc0 ); chk "hosts vr1-dc0 selects rc" "$?" 0 +( lib_hosts_select_dc vr1-dc1 ); chk "hosts vr1-dc1 selects rc" "$?" 0 ( lib_hosts_select_dc bogus 2>/dev/null ); chk "hosts unknown-token fails-loud rc" "$?" 1 ( lib_hosts_select_dc 2>/dev/null ); chk "hosts missing-arg fails-loud rc" "$?" 1 for retired in dc0 dc1 dc2; do ( lib_hosts_select_dc "$retired" 2>/dev/null ); chk "hosts RETIRED '$retired' rejected (D-119)" "$?" 1 done -# --- the documented asymmetry: vr1-dc0 differs between the two libs --- -# net no-ops (D-101 says it inherits VR0 DC0's v4), hosts FAILS (it has no -# enrolled hosts yet). Intentional; asserted so nobody "fixes" one to match. -( lib_net_select_dc vr1-dc0 ); NET_RC=$? -( lib_hosts_select_dc vr1-dc0 2>/dev/null ); HOSTS_RC=$? -[ "$NET_RC" = 0 ] && [ "$HOSTS_RC" = 1 ] && ok "vr1-dc0 asymmetry: net no-ops, hosts fails (documented, not a bug)" || no "vr1-dc0 asymmetry broken (net=$NET_RC hosts=$HOSTS_RC)" +# --- VR1 arms carry the MEASURED identity + D-134-amended octets --- +H0_OCT="$(lib_hosts_select_dc vr1-dc0 >/dev/null 2>&1; echo "${HOST_OCTET[vr1-dc0-control-01]}")" +chk "hosts vr1-dc0 control-01 octet .100 (D-134 amendment)" "$H0_OCT" "100" +H0_ST="$(lib_hosts_select_dc vr1-dc0 >/dev/null 2>&1; echo "${HOST_OCTET[vr1-dc0-storage-04]}")" +chk "hosts vr1-dc0 storage-04 octet .153" "$H0_ST" "153" +H0_MAC="$(lib_hosts_select_dc vr1-dc0 >/dev/null 2>&1; echo "${HOST_BOOT_MAC[vr1-dc0-compute-02]}")" +chk "hosts vr1-dc0 compute-02 pinned boot MAC" "$H0_MAC" "52:54:00:18:ab:b4" +H1_MAC="$(lib_hosts_select_dc vr1-dc1 >/dev/null 2>&1; echo "${HOST_BOOT_MAC[vr1-dc1-storage-04]}")" +chk "hosts vr1-dc1 storage-04 pinned boot MAC (index-encoded)" "$H1_MAC" "52:54:01:d1:09:01" +H0_TAG="$(lib_hosts_select_dc vr1-dc0 >/dev/null 2>&1; echo "$HOST_TAG")" +chk "hosts vr1-dc0 tag openstack-vr1-dc0 (ruled 2026-07-23)" "$H0_TAG" "openstack-vr1-dc0" +H1_PWR="$(lib_hosts_select_dc vr1-dc1 >/dev/null 2>&1; echo "$VIRSH_POWER_ADDRESS")" +chk "hosts vr1-dc1 power addr rack transit .6" "$H1_PWR" "qemu+ssh://jessea123@172.31.0.6/system" +H0_N="$(lib_hosts_select_dc vr1-dc0 >/dev/null 2>&1; echo "${#HOSTS[@]}")" +chk "hosts vr1-dc0 fleet count 9 (D-121 Option C)" "$H0_N" "9" + +# --- ONE SELECTION PER SHELL (stale cross-DC guard, lib-net precedent) --- +( lib_hosts_select_dc vr1-dc0 >/dev/null 2>&1; lib_hosts_select_dc vr1-dc1 2>/dev/null ); \ + chk "hosts dc0->dc1 re-select REFUSED (stale cross-DC values)" "$?" 1 +HSW_ERR="$(lib_hosts_select_dc vr1-dc0 >/dev/null 2>&1; lib_hosts_select_dc vr1-dc1 2>&1 1>/dev/null || true)" +grep -q "already selected" <<<"$HSW_ERR" && ok "hosts re-select error says already selected" || no "hosts re-select error says already selected" +( lib_hosts_select_dc vr1-dc1 >/dev/null 2>&1; lib_hosts_select_dc vr1-dc1 ); \ + chk "hosts same-DC re-select stays a no-op rc" "$?" 0 + +# --- the VR1 resolver exists: system_id by pinned boot MAC, never hostname --- +type host_sysid_by_bootmac >/dev/null 2>&1 && ok "host_sysid_by_bootmac defined (VR1 resolver)" || no "host_sysid_by_bootmac defined (VR1 resolver)" echo; [ "$F" = 0 ] && { echo "ALL PASS ($P checks)"; exit 0; } || { echo "FAILURES: $F"; exit 1; } diff --git a/tests/reenroll-hosts/run-tests.sh b/tests/reenroll-hosts/run-tests.sh index d04bcfe..c2e0e99 100644 --- a/tests/reenroll-hosts/run-tests.sh +++ b/tests/reenroll-hosts/run-tests.sh @@ -54,12 +54,12 @@ has 'NOT-ENROLLED' has 'read-only check mode; no changes made' -run_dc 1 "DC=vr1-dc0: hosts selector fails loud (no per-DC host data)" vr1-dc0 -has 'no enrolled hosts yet' +run_dc 1 "DC=vr1-dc0: REFUSED (D-133 guard -- VR0 flow, VR1 arms populated 2026-07-23)" vr1-dc0 +has 'VR1 re-enrollment is PXE re-enlist' absent 'Current host status' # must exit before report() ever runs -run_dc 1 "DC=vr1-dc1: hosts selector fails loud (net arm ratified, D-124)" vr1-dc1 -has 'no enrolled hosts yet' +run_dc 1 "DC=vr1-dc1: REFUSED (D-133 guard -- VR0 flow, VR1 arms populated 2026-07-23)" vr1-dc1 +has 'VR1 re-enrollment is PXE re-enlist' absent 'Current host status' # D-119 REGRESSION GUARD: the bare dcN tokens are RETIRED. 'dc0' used to mean