diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 05e8f45..d0d313d 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -124,6 +124,16 @@ ## 3. What is AUTHORED-BUT-NOT-APPLIED (in the tree, not in state / live) +- voffice1 transit NIC (2026-07-19): `module "voffice1"` gains the + mesh-vr1-dc0-office1 leg (appended second; enp1s0 preserved) + a + per-NIC-named seed netplan with the region end `172.31.0.1/30` from + the D-124 tfvars. Basis: measured this session -- the transit net is + ISOLATED with only vvr1-dc0 attached; NOTHING held 172.31.0.1, so + step B had no ssh path and no rack->region enrollment path. Apply is + gated (planned voffice1 restart window); the LIVE address lands via + an in-guest netplan drop-in at the same gated step (cloud-init does + not re-run). See section 5 + the session changelog. + - `module "vvr1_dc0"` (`opentofu/main.tf:360`) -- the DC0 containment VM (416 GiB / 108 vCPU, D-121/D-123 sizing) + its disk, seed volume, and cloudinit seed. 4 of the 5 committed DC0 creates in the plan capture. @@ -181,15 +191,20 @@ - pre-reboot gate was 5/0/6 (recorded at `docs/dc0-deploy-readiness.md:59`, `docs/session-ledger.md:278`). -The EXPECTED outer plan is **ZERO DIFF** ("no changes needed"), recorded -2026-07-19 post-step-A with its evidencing capture -(`docs/audit/outer-plan-20260719-postA-converged.txt`). WHY: step A -applied the saved 6/0/6 plan exactly (pre-apply capture -`outer-plan-20260719-preapply.txt` matched the recorded triple; G8 -same-session planes check passed first). History: 7/2/7 post-reboot -symptom -> 6/2/6 post-D-130 -> 6/0/6 post-G6-reconcile -> applied. -A future outer plan showing ANY diff is a STOP (investigate drift -before touching anything). +The EXPECTED outer plan is **1/1/1** (voffice1 transit-NIC change, +AUTHORED 2026-07-19, apply gated): + replace of +`module.voffice1.libvirt_cloudinit_disk.seed` (network_config change; +seed VOLUME suppressed per D-130) + in-place interface add on +`module.voffice1.libvirt_domain.vm` (mesh transit appended SECOND -- +office1-local stays index 0/enp1s0, verified from the plan JSON). +Capture: `docs/audit/outer-plan-20260719-voffice1-transit.txt`. +Mechanism operator-ruled 2026-07-19 ("voffice1 via IaC + restart" -- +session changelog, ruling record). After its gated apply the expected +plan returns to **ZERO DIFF**. History: 7/2/7 post-reboot symptom -> +6/2/6 post-D-130 -> 6/0/6 post-G6-reconcile -> applied 6/0/6 exact -> +zero diff converged (`outer-plan-20260719-postA-converged.txt`) -> +1/1/1 authored (this entry). Any OTHER diff is a STOP (investigate +drift before touching anything). ## 6. Open gates diff --git a/docs/audit/outer-plan-20260719-voffice1-transit.txt b/docs/audit/outer-plan-20260719-voffice1-transit.txt new file mode 100644 index 0000000..b14bb74 --- /dev/null +++ b/docs/audit/outer-plan-20260719-voffice1-transit.txt @@ -0,0 +1,81 @@ +module.vvr1_dc0.libvirt_cloudinit_disk.seed: Refreshing state... [id=4666ca51f035e4d8] +module.voffice1.libvirt_cloudinit_disk.seed: Refreshing state... [id=775234004c2669d9] +module.office1_network.libvirt_network.office1_local: Refreshing state... [id=8fdd2a97-417c-44d4-89e4-ae8d65594135] +module.mesh_vr1_dc0_vr1_dc1.libvirt_network.link: Refreshing state... [id=9cbc8589-9f40-48e6-872e-ef3abfe29a93] +module.vr1_dc0_storage.libvirt_pool.dc: Refreshing state... [id=7ce1101c-a89e-40ca-9263-5f572bee40a9] +module.vr1_dc1_storage.libvirt_pool.dc: Refreshing state... [id=4a1df114-ee04-4c80-9233-cc0c140c8556] +module.office1_storage.libvirt_pool.dc: Refreshing state... [id=5f94194c-69c1-4b04-a85f-c18d87303a03] +module.mesh_vr1_dc1_office1.libvirt_network.link: Refreshing state... [id=38a20d2d-cd91-4604-a5f4-8e2a6609633c] +module.mesh_vr1_dc0_office1.libvirt_network.link: Refreshing state... [id=8318548f-c3d6-4e06-bef4-fe3f11d68125] +module.vr1_dc0_uplink.libvirt_network.site_wan: Refreshing state... [id=f3500153-e4de-45f1-8854-9c92974a6094] +module.vvr1_dc0.libvirt_volume.seed: Refreshing state... [id=/var/lib/libvirt/vr1/vr1-dc0/vvr1-dc0-cloudinit.iso] +module.voffice1.libvirt_volume.seed: Refreshing state... [id=/var/lib/libvirt/vr1/office1/voffice1-cloudinit.iso] +module.office1_opnsense.libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1/office1/office1-opnsense-disk.qcow2] +module.ubuntu_noble_base.libvirt_volume.base: Refreshing state... [id=/var/lib/libvirt/vr1/office1/ubuntu-24.04-base.qcow2] +module.voffice1.libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1/office1/voffice1-disk.qcow2] +module.vvr1_dc0.libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1/vr1-dc0/vvr1-dc0-disk.qcow2] +module.office1_opnsense.libvirt_domain.vm: Refreshing state... [name=office1-opnsense] +module.voffice1.libvirt_domain.vm: Refreshing state... [name=voffice1] +module.vvr1_dc0.libvirt_domain.vm: Refreshing state... [name=vvr1-dc0] + +OpenTofu used the selected providers to generate the following execution +plan. Resource actions are indicated with the following symbols: + ~ update in-place (current -> planned) +-/+ destroy and then create replacement + +OpenTofu will perform the following actions: + + # module.voffice1.libvirt_cloudinit_disk.seed must be replaced +-/+ resource "libvirt_cloudinit_disk" "seed" { + ~ id = "775234004c2669d9" -> (known after apply) + name = "voffice1-cloudinit" + ~ network_config = <<-EOT # forces replacement + version: 2 + ethernets: + lan: + match: + - name: "en*" + + name: "enp1s0" + dhcp4: true + + transit: + + match: + + name: "enp2s0" + + addresses: ["172.31.0.1/30"] + EOT + ~ path = "/tmp/terraform-provider-libvirt-cloudinit/cloudinit-775234004c2669d9.iso" -> (known after apply) + ~ size = 45056 -> (known after apply) + # (2 unchanged attributes hidden) + } + + # module.voffice1.libvirt_domain.vm will be updated in-place + ~ resource "libvirt_domain" "vm" { + ~ devices = { + ~ interfaces = [ + + { + + model = { + + type = "virtio" + } + + source = { + + network = { + + network = "mesh-vr1-dc0-office1" + } + } + }, + # (1 unchanged element hidden) + ] + # (1 unchanged attribute hidden) + } + id = 1 + name = "voffice1" + # (10 unchanged attributes hidden) + } + +Plan: 1 to add, 1 to change, 1 to destroy. + +───────────────────────────────────────────────────────────────────────────── + +Saved the plan to: +/home/jessea123/openstack-caracal-dc-dc/opentofu/voffice1-transit.tfplan + +To perform exactly these actions, run the following command to apply: + tofu apply "/home/jessea123/openstack-caracal-dc-dc/opentofu/voffice1-transit.tfplan" diff --git a/docs/changelog-20260719-dc0-deploy-stepB.md b/docs/changelog-20260719-dc0-deploy-stepB.md new file mode 100644 index 0000000..dfeed0f --- /dev/null +++ b/docs/changelog-20260719-dc0-deploy-stepB.md @@ -0,0 +1,46 @@ +# Changelog -- 2026-07-19 session: DC0 deploy step B (transit reach + bootstrap) + +Session changelog (GA-R2/D1: ONE per session). Logged window: `dc0-deploy` +(same-day append, per the A/B-boundary handoff). Status lives ONLY in +`docs/CURRENT-STATE.md`; this file is narrative + reverts. + +## 1. RULING RECORD (GA-R5) -- Office1 transit-end mechanism (OPS, under D-124) + +- **Question as presented (2026-07-19):** "Who should hold 172.31.0.1 (the + Office1 end of the dc0 transit /30), and by what mechanism? This is the + prerequisite for all of step B -- both ssh reach to vvr1-dc0 and the MAAS + rack-to-region enrollment path." +- **Operator answer, exact utterance (option selected):** "voffice1 via IaC + + restart (Recommended)" -- add the mesh NIC + 172.31.0.1/30 netplan to + `module "voffice1"` in the outer root; apply in a planned voffice1 restart + window (graceful shutdown first; region/NetBox/Tailscale return via proven + autostart). +- Classification: OPS mechanism decision implementing D-124's already-ruled + addressing ("region/peer .1" -- `docs/design-decisions.md:3727`); no new + D-number (GA-R3, doubt resolves DOWN). +- **Measured basis (this session, logged):** mesh-vr1-dc0-office1 (virbr7) is + ISOLATED (no ``/``) with exactly ONE attached interface + (vvr1-dc0 vnet3); voffice1 + office1-opnsense have NO mesh NIC; nothing + holds 172.31.0.1; vvr1-dc0's applied transit leg = 172.31.0.2/30 with only + route `10.10.0.0/22 via 172.31.0.1` (d124-rack.auto.tfvars, consumed by + step A). Item-20 disposition: with voffice1 holding .1, vvr1-dc0 is reached + by `-J voffice1` (D-126 shape) -- NO vcloud host leg / site-baseleg DC row + needed for this net. + +## 2. Outer root: voffice1 grows the transit NIC (AUTHORED; apply gated) + +- **What:** `opentofu/main.tf` `module "voffice1"`: `network_names` gains + `module.mesh_vr1_dc0_office1.network_name` SECOND (office1-local stays + first -- PCI order preserves enp1s0); `network_config` rewritten from the + single-NIC "en*" glob to per-NIC name matches (lan=enp1s0 dhcp4; + transit=enp2s0 static `${vr1_dc0_rack_transit_peer_ip}/${prefix}` -- same + D-124 tfvars the rack consumes, no new literal). Naming-trap + no-cloud-init- + re-run caveats documented in-block: the seed netplan is REBUILD correctness; + the live transit config is an in-guest netplan drop-in at the gated attach + step. +- **Why:** section 1's measured gap -- step B (ssh reach + rack enrollment) + has no L3 path until the region end of the transit exists. +- **Revert:** restore `network_names = [module.office1_network.network_name]` + and the previous "en*" glob `network_config` (git revert of this commit); + if already applied, a follow-up plan shows the interface removal and the + in-guest drop-in `/etc/netplan/60-transit.yaml` is deleted. diff --git a/opentofu/main.tf b/opentofu/main.tf index 2dc56d6..b644feb 100644 --- a/opentofu/main.tf +++ b/opentofu/main.tf @@ -178,15 +178,24 @@ } module "voffice1" { - source = "./modules/cloudinit-vm" - vm_name = "voffice1" - autostart = true # D-127: foundational Office1 host (MAAS region + LXD service host) - vcpu = var.voffice1_vcpu - memory_mib = var.voffice1_memory_mib - disk_size_bytes = var.voffice1_disk_bytes - pool_name = module.office1_storage.pool_name - base_volume_path = module.ubuntu_noble_base.path - network_names = [module.office1_network.network_name] + source = "./modules/cloudinit-vm" + vm_name = "voffice1" + autostart = true # D-127: foundational Office1 host (MAAS region + LXD service host) + vcpu = var.voffice1_vcpu + memory_mib = var.voffice1_memory_mib + disk_size_bytes = var.voffice1_disk_bytes + pool_name = module.office1_storage.pool_name + base_volume_path = module.ubuntu_noble_base.path + # NIC order is load-bearing (PCI slot order = guest naming): office1-local stays FIRST + # (enp1s0, the live 10.10.0.20 lease + Kea reservation). The mesh transit leg is SECOND + # (D-124 Scheme A: voffice1 holds the region end of the office1<->dc0 /30 -- operator- + # ruled mechanism 2026-07-19, "voffice1 via IaC + restart"). The rack (vvr1-dc0) routes + # 10.10.0.0/22 via this address; region traffic terminates HERE (no forwarding needed + # for enrollment). + network_names = [ + module.office1_network.network_name, # NIC1 enp1s0 -> lan (office1-local, DHCP/Kea) + module.mesh_vr1_dc0_office1.network_name, # NIC2 -> transit (region end, static .1/30) + ] expose_nested_virt = true user_data = <<-EOT @@ -213,19 +222,27 @@ local-hostname: voffice1 EOT - # Match by GLOB, not a guessed interface name: the NIC's kernel name (ens3 / enp1s0 / - # ...) depends on the machine type and PCI topology and is NOT known until first boot. - # Naming it would be an inferred value. dhcp4 is a DELIBERATE choice here, not the - # module's refused default: office1-local genuinely has an authoritative DHCP server - # (Kea on the OPNsense edge, pool .100-.199). A Kea RESERVATION is added afterwards, - # via the OPNsense REST API, to make the address stable. + # Two NICs now, so the old "en*" glob is retired: it would put dhcp4 on the transit + # leg (which has no DHCP) and stall systemd-networkd-wait-online. Names are matched + # per-NIC. INTERFACE-NAMING TRAP (same as vvr1-dc0): enp1s0 is MEASURED on the live + # guest (2026-07-19); enp2s0 for the appended NIC is the expected q35 slot order and + # MUST be confirmed on first boot/attach before relying on this. dhcp4 on lan is the + # DELIBERATE choice (Kea on the edge, pool .100-.199 + reservation). The transit end + # is static, NetBox-assigned (D-124), via the same tfvars the rack consumes -- no + # literal here. NOTE: cloud-init does NOT re-run on the live guest (same instance-id); + # this block is REBUILD correctness. The live transit config is a netplan drop-in + # applied in-guest at the same gated step that attaches the NIC. network_config = <<-EOT version: 2 ethernets: lan: match: - name: "en*" + name: "enp1s0" dhcp4: true + transit: + match: + name: "enp2s0" + addresses: ["${var.vr1_dc0_rack_transit_peer_ip}/${var.vr1_dc0_rack_transit_prefix}"] EOT }