diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 5915663..4924faf 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1825,6 +1825,37 @@ its teeth for genuine disagreement. New cases T10 (skip is reported) and T11 (a ROLE-named host in the utility band still REFUSES); harness 9 -> 11. Gauntlet **ALL GREEN (87) on vcloud**; opentofu-validate PASS; repo-lint 0 fail. +- **R1's OSD CARVE AND THE D-104 CONTROLLER VMs ARE COMPLETE ACROSS BOTH DCs, 2026-07-29.** + Operator directed two background agents ("Finish the remaining three dc0 nodes as an agent. + Start a separate agent for all of dc1"). Raw captures promoted from `voffice1:/tmp` (which + does not survive a reboot) to `docs/audit/osd-carve-20260729/`; dc0's narrative capture is + `docs/audit/dc0-osd-juju-apply-20260729.txt`. **Both agents were barred from repo writes, so + without that promotion dc1 would have had NO audit record at all.** + **INDEPENDENTLY VERIFIED by this session from MAAS directly, NOT from the agents' reports -- + every claim matched:** fleet **22** = 18 Ready + 2 Deployed + 2 New; 18 role nodes; **216 + links, 216 of 216 `mode=static`**; both DCs `blockdevs {1:5, 2:4}` (the four storage nodes + per DC now carry `vdb`), `power {off:9}`, `tagcount {2:9}`. dc1's substrate applied + 6 add / 4 change / 0 destroy with **MAC drift 0** and re-converged to **ZERO DIFF**. + **Eight re-commissions, every one the same one-change diff:** `vdb` at 536870912000 bytes + (500 GiB) added; links 12 -> 12; per-interface MACs and link lists identical. Times ~180-204s. + **MAAS TAGS SURVIVED re-commissioning on all 18 nodes** -- agent A checked this unprompted + because its snapshot code structurally could not see tags, removing a would-be Stage-5 Juju + placement blocker. + **ONE DISCLOSED JUDGMENT CALL, and it was right.** Three dc1 pairs also showed `power_state` + moving `off`/`error`; agent B declared a STOP, re-measured, then excluded `power_state` from + its verdict (still printing it) on the basis that 8 of 9 dc1 nodes read `error` BEFORE any + mutation. **Vindicated by later independent measurement: all 18 role nodes now read `off`, + including the five dc1 nodes never commissioned** -- a transient MAAS power-query flap, not a + credential or power-path defect. SEC-016 is the plausible governing item if it recurs; not + live now. Verified from the RAW capture, not the report: one dc1 pair diffs by exactly the + `vdb` hunk plus that `power_state` line and nothing else. + **STILL BLOCKING for the two controllers: `power_type` is EMPTY on both** (`7n87bt` + moved-troll dc0, `p8tdwg` square-ferret dc1; both 4 cpu / 8192 MiB / tags `[virtual]`) -- the + state that stalled all nine role nodes on 2026-07-20. They can now be power-configured + because both are in `lib-hosts` at the ruled octet `.5` with MACs pinned. + **NEXT, all gated:** `maas-node-power.sh` -> `power_type=virsh` on both controllers; commission + them (DEFAULT path -- no `skip_networking`, since there is nothing to preserve); create the + `juju-controller-` tags; and `maas-role-tags.sh apply --commit` for the role tags. - **THREE PLATFORM BEHAVIOURS GRADUATED to `references/platform-traps.md`** at session close, having been recorded only in this status document (which is consolidated over time, so a durable trap does not belong here alone): MAAS auto-reserves `::1`-`::ffff:ffff` on EVERY diff --git a/docs/audit/osd-carve-20260729/README.txt b/docs/audit/osd-carve-20260729/README.txt new file mode 100644 index 0000000..09b94f3 --- /dev/null +++ b/docs/audit/osd-carve-20260729/README.txt @@ -0,0 +1,46 @@ +OSD CARVE + CONTROLLER VM -- raw captures, 2026-07-29 +===================================================== +Before/after MAAS snapshots for every re-commissioned node, plus dc1's plan and apply +output. Promoted here from voffice1:/tmp, which does not survive a reboot. Two background +agents produced these under operator direction; both were barred from repo writes, so +without this promotion dc1 would have had NO audit record at all (dc0 has +docs/audit/dc0-osd-juju-apply-20260729.txt). + + before.json / after.json dc0 storage-01 (the canary, run by hand) + before-storage-0{2,3,4}.json / after-* dc0 storage-02/03/04 (agent A) + s0{1,2,3,4}.{before,after}.json dc1 storage-01..04 (agent B) + dc1-phase1-{plan,apply}.txt dc1 substrate plan + apply output + +WHAT THE SNAPSHOTS SHOW. Every before/after pair differs by exactly ONE substantive change: +a second block device `vdb` of 536870912000 bytes (500 GiB) appearing. links stay 12 -> 12, +per-interface MACs identical, per-interface link list (mode/ip/cidr) identical. + +THE ONE EXTRA DIFFERENCE, disclosed rather than hidden: three of dc1's four pairs also show +`power_state` moving between `off` and `error`. Agent B initially declared a STOP on this, +re-classified it on measurement, and excluded `power_state` from its verdict for the +remaining nodes -- printing it as an observation. Its stated basis: 8 of 9 dc1 nodes already +read `error` BEFORE any mutation, and storage-01 re-read `off` three times over ~40s. + +THAT JUDGMENT IS VINDICATED BY LATER MEASUREMENT, verified independently after both agents +finished: all 18 role nodes across both DCs read power_state `off`, including the five dc1 +nodes never commissioned. So it was a transient MAAS power-query flap, not a credential or +power-path defect. SEC-016 (per-DC MAAS->libvirt power keys) is the plausible governing item +if it recurs; it is not live now. + +INDEPENDENT VERIFICATION of both agents' reports (run by the orchestrating session, from +MAAS directly, NOT from the agents' output): + fleet 22 = 18 Ready + 2 Deployed + 2 New + 18 VR1 role nodes | 216 links | 216 of 216 mode=static + vr1-dc0 blockdevs {1:5, 2:4} power {off:9} tagcount {2:9} + vr1-dc1 blockdevs {1:5, 2:4} power {off:9} tagcount {2:9} + NEW 7n87bt moved-troll 4 cpu / 8192 MiB / power_type '' / tags [virtual] + NEW p8tdwg square-ferret 4 cpu / 8192 MiB / power_type '' / tags [virtual] + +Every claim in both agent reports matched. MAAS TAGS SURVIVED re-commissioning on all 18 +nodes (tagcount 2 everywhere) -- agent A checked this unprompted because its snapshot code +structurally could not see tags. That removes a would-be Stage-5 Juju placement blocker. + +STILL TRUE AND STILL BLOCKING for the two controllers: power_type is EMPTY on both, which is +the state that stalled all nine role nodes on 2026-07-20. They cannot be commissioned until +maas-node-power.sh sets power_type=virsh, which is now possible because both are in +lib-hosts as of the D-134 amendment (octet .5) with their MACs pinned. diff --git a/docs/audit/osd-carve-20260729/after-storage-02.json b/docs/audit/osd-carve-20260729/after-storage-02.json new file mode 100644 index 0000000..b54da6d --- /dev/null +++ b/docs/audit/osd-carve-20260729/after-storage-02.json @@ -0,0 +1,117 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ], + [ + "vdb", + 536870912000 + ] + ], + "ifaces": [ + [ + "br-ex", + "52:54:00:2e:8b:55", + [ + [ + "static", + "10.12.4.151", + "10.12.4.0/22" + ], + [ + "static", + "2602:f3e2:f02:10::151", + "2602:f3e2:f02:10::/64" + ] + ] + ], + [ + "enp1s0", + "52:54:00:48:86:2c", + [ + [ + "static", + "10.12.8.151", + "10.12.8.0/22" + ], + [ + "static", + "fd50:840e:74e2:220::151", + "fd50:840e:74e2:220::/64" + ] + ] + ], + [ + "enp2s0", + "52:54:00:2e:8b:55", + [] + ], + [ + "enp3s0", + "52:54:00:d7:4e:38", + [ + [ + "static", + "10.12.12.151", + "10.12.12.0/22" + ], + [ + "static", + "fd50:840e:74e2:221::151", + "fd50:840e:74e2:221::/64" + ] + ] + ], + [ + "enp4s0", + "52:54:00:05:60:af", + [ + [ + "static", + "10.12.16.151", + "10.12.16.0/22" + ], + [ + "static", + "fd50:840e:74e2:230::151", + "fd50:840e:74e2:230::/64" + ] + ] + ], + [ + "enp5s0", + "52:54:00:c6:02:bd", + [ + [ + "static", + "10.12.32.151", + "10.12.32.0/22" + ], + [ + "static", + "fd50:840e:74e2:240::151", + "fd50:840e:74e2:240::/64" + ] + ] + ], + [ + "enp6s0", + "52:54:00:2c:f8:42", + [ + [ + "static", + "10.12.36.151", + "10.12.36.0/22" + ], + [ + "static", + "fd50:840e:74e2:250::151", + "fd50:840e:74e2:250::/64" + ] + ] + ] + ], + "power": "off", + "status": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/after-storage-03.json b/docs/audit/osd-carve-20260729/after-storage-03.json new file mode 100644 index 0000000..1e86eb2 --- /dev/null +++ b/docs/audit/osd-carve-20260729/after-storage-03.json @@ -0,0 +1,117 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ], + [ + "vdb", + 536870912000 + ] + ], + "ifaces": [ + [ + "br-ex", + "52:54:00:bd:d1:24", + [ + [ + "static", + "10.12.4.152", + "10.12.4.0/22" + ], + [ + "static", + "2602:f3e2:f02:10::152", + "2602:f3e2:f02:10::/64" + ] + ] + ], + [ + "enp1s0", + "52:54:00:b1:94:d1", + [ + [ + "static", + "10.12.8.152", + "10.12.8.0/22" + ], + [ + "static", + "fd50:840e:74e2:220::152", + "fd50:840e:74e2:220::/64" + ] + ] + ], + [ + "enp2s0", + "52:54:00:bd:d1:24", + [] + ], + [ + "enp3s0", + "52:54:00:80:67:87", + [ + [ + "static", + "10.12.12.152", + "10.12.12.0/22" + ], + [ + "static", + "fd50:840e:74e2:221::152", + "fd50:840e:74e2:221::/64" + ] + ] + ], + [ + "enp4s0", + "52:54:00:0f:a6:35", + [ + [ + "static", + "10.12.16.152", + "10.12.16.0/22" + ], + [ + "static", + "fd50:840e:74e2:230::152", + "fd50:840e:74e2:230::/64" + ] + ] + ], + [ + "enp5s0", + "52:54:00:e0:73:ed", + [ + [ + "static", + "10.12.32.152", + "10.12.32.0/22" + ], + [ + "static", + "fd50:840e:74e2:240::152", + "fd50:840e:74e2:240::/64" + ] + ] + ], + [ + "enp6s0", + "52:54:00:62:65:77", + [ + [ + "static", + "10.12.36.152", + "10.12.36.0/22" + ], + [ + "static", + "fd50:840e:74e2:250::152", + "fd50:840e:74e2:250::/64" + ] + ] + ] + ], + "power": "off", + "status": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/after-storage-04.json b/docs/audit/osd-carve-20260729/after-storage-04.json new file mode 100644 index 0000000..1f0659c --- /dev/null +++ b/docs/audit/osd-carve-20260729/after-storage-04.json @@ -0,0 +1,117 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ], + [ + "vdb", + 536870912000 + ] + ], + "ifaces": [ + [ + "br-ex", + "52:54:00:90:ca:d0", + [ + [ + "static", + "10.12.4.153", + "10.12.4.0/22" + ], + [ + "static", + "2602:f3e2:f02:10::153", + "2602:f3e2:f02:10::/64" + ] + ] + ], + [ + "enp1s0", + "52:54:00:2b:ed:ab", + [ + [ + "static", + "10.12.8.153", + "10.12.8.0/22" + ], + [ + "static", + "fd50:840e:74e2:220::153", + "fd50:840e:74e2:220::/64" + ] + ] + ], + [ + "enp2s0", + "52:54:00:90:ca:d0", + [] + ], + [ + "enp3s0", + "52:54:00:2a:ac:14", + [ + [ + "static", + "10.12.12.153", + "10.12.12.0/22" + ], + [ + "static", + "fd50:840e:74e2:221::153", + "fd50:840e:74e2:221::/64" + ] + ] + ], + [ + "enp4s0", + "52:54:00:af:34:fd", + [ + [ + "static", + "10.12.16.153", + "10.12.16.0/22" + ], + [ + "static", + "fd50:840e:74e2:230::153", + "fd50:840e:74e2:230::/64" + ] + ] + ], + [ + "enp5s0", + "52:54:00:28:b1:6d", + [ + [ + "static", + "10.12.32.153", + "10.12.32.0/22" + ], + [ + "static", + "fd50:840e:74e2:240::153", + "fd50:840e:74e2:240::/64" + ] + ] + ], + [ + "enp6s0", + "52:54:00:ab:5c:50", + [ + [ + "static", + "10.12.36.153", + "10.12.36.0/22" + ], + [ + "static", + "fd50:840e:74e2:250::153", + "fd50:840e:74e2:250::/64" + ] + ] + ] + ], + "power": "off", + "status": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/after.json b/docs/audit/osd-carve-20260729/after.json new file mode 100644 index 0000000..12f09c6 --- /dev/null +++ b/docs/audit/osd-carve-20260729/after.json @@ -0,0 +1,117 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ], + [ + "vdb", + 536870912000 + ] + ], + "ifaces": [ + [ + "br-ex", + "52:54:00:4c:69:7b", + [ + [ + "static", + "10.12.4.150", + "10.12.4.0/22" + ], + [ + "static", + "2602:f3e2:f02:10::150", + "2602:f3e2:f02:10::/64" + ] + ] + ], + [ + "enp1s0", + "52:54:00:5f:8d:42", + [ + [ + "static", + "10.12.8.150", + "10.12.8.0/22" + ], + [ + "static", + "fd50:840e:74e2:220::150", + "fd50:840e:74e2:220::/64" + ] + ] + ], + [ + "enp2s0", + "52:54:00:4c:69:7b", + [] + ], + [ + "enp3s0", + "52:54:00:ac:2c:4d", + [ + [ + "static", + "10.12.12.150", + "10.12.12.0/22" + ], + [ + "static", + "fd50:840e:74e2:221::150", + "fd50:840e:74e2:221::/64" + ] + ] + ], + [ + "enp4s0", + "52:54:00:28:91:d2", + [ + [ + "static", + "10.12.16.150", + "10.12.16.0/22" + ], + [ + "static", + "fd50:840e:74e2:230::150", + "fd50:840e:74e2:230::/64" + ] + ] + ], + [ + "enp5s0", + "52:54:00:fa:7c:53", + [ + [ + "static", + "10.12.32.150", + "10.12.32.0/22" + ], + [ + "static", + "fd50:840e:74e2:240::150", + "fd50:840e:74e2:240::/64" + ] + ] + ], + [ + "enp6s0", + "52:54:00:0c:7a:ab", + [ + [ + "static", + "10.12.36.150", + "10.12.36.0/22" + ], + [ + "static", + "fd50:840e:74e2:250::150", + "fd50:840e:74e2:250::/64" + ] + ] + ] + ], + "power": "off", + "status": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/before-storage-02.json b/docs/audit/osd-carve-20260729/before-storage-02.json new file mode 100644 index 0000000..c0c3bf9 --- /dev/null +++ b/docs/audit/osd-carve-20260729/before-storage-02.json @@ -0,0 +1,113 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ] + ], + "ifaces": [ + [ + "br-ex", + "52:54:00:2e:8b:55", + [ + [ + "static", + "10.12.4.151", + "10.12.4.0/22" + ], + [ + "static", + "2602:f3e2:f02:10::151", + "2602:f3e2:f02:10::/64" + ] + ] + ], + [ + "enp1s0", + "52:54:00:48:86:2c", + [ + [ + "static", + "10.12.8.151", + "10.12.8.0/22" + ], + [ + "static", + "fd50:840e:74e2:220::151", + "fd50:840e:74e2:220::/64" + ] + ] + ], + [ + "enp2s0", + "52:54:00:2e:8b:55", + [] + ], + [ + "enp3s0", + "52:54:00:d7:4e:38", + [ + [ + "static", + "10.12.12.151", + "10.12.12.0/22" + ], + [ + "static", + "fd50:840e:74e2:221::151", + "fd50:840e:74e2:221::/64" + ] + ] + ], + [ + "enp4s0", + "52:54:00:05:60:af", + [ + [ + "static", + "10.12.16.151", + "10.12.16.0/22" + ], + [ + "static", + "fd50:840e:74e2:230::151", + "fd50:840e:74e2:230::/64" + ] + ] + ], + [ + "enp5s0", + "52:54:00:c6:02:bd", + [ + [ + "static", + "10.12.32.151", + "10.12.32.0/22" + ], + [ + "static", + "fd50:840e:74e2:240::151", + "fd50:840e:74e2:240::/64" + ] + ] + ], + [ + "enp6s0", + "52:54:00:2c:f8:42", + [ + [ + "static", + "10.12.36.151", + "10.12.36.0/22" + ], + [ + "static", + "fd50:840e:74e2:250::151", + "fd50:840e:74e2:250::/64" + ] + ] + ] + ], + "power": "off", + "status": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/before-storage-03.json b/docs/audit/osd-carve-20260729/before-storage-03.json new file mode 100644 index 0000000..3a4786c --- /dev/null +++ b/docs/audit/osd-carve-20260729/before-storage-03.json @@ -0,0 +1,113 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ] + ], + "ifaces": [ + [ + "br-ex", + "52:54:00:bd:d1:24", + [ + [ + "static", + "10.12.4.152", + "10.12.4.0/22" + ], + [ + "static", + "2602:f3e2:f02:10::152", + "2602:f3e2:f02:10::/64" + ] + ] + ], + [ + "enp1s0", + "52:54:00:b1:94:d1", + [ + [ + "static", + "10.12.8.152", + "10.12.8.0/22" + ], + [ + "static", + "fd50:840e:74e2:220::152", + "fd50:840e:74e2:220::/64" + ] + ] + ], + [ + "enp2s0", + "52:54:00:bd:d1:24", + [] + ], + [ + "enp3s0", + "52:54:00:80:67:87", + [ + [ + "static", + "10.12.12.152", + "10.12.12.0/22" + ], + [ + "static", + "fd50:840e:74e2:221::152", + "fd50:840e:74e2:221::/64" + ] + ] + ], + [ + "enp4s0", + "52:54:00:0f:a6:35", + [ + [ + "static", + "10.12.16.152", + "10.12.16.0/22" + ], + [ + "static", + "fd50:840e:74e2:230::152", + "fd50:840e:74e2:230::/64" + ] + ] + ], + [ + "enp5s0", + "52:54:00:e0:73:ed", + [ + [ + "static", + "10.12.32.152", + "10.12.32.0/22" + ], + [ + "static", + "fd50:840e:74e2:240::152", + "fd50:840e:74e2:240::/64" + ] + ] + ], + [ + "enp6s0", + "52:54:00:62:65:77", + [ + [ + "static", + "10.12.36.152", + "10.12.36.0/22" + ], + [ + "static", + "fd50:840e:74e2:250::152", + "fd50:840e:74e2:250::/64" + ] + ] + ] + ], + "power": "off", + "status": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/before-storage-04.json b/docs/audit/osd-carve-20260729/before-storage-04.json new file mode 100644 index 0000000..889ddbd --- /dev/null +++ b/docs/audit/osd-carve-20260729/before-storage-04.json @@ -0,0 +1,113 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ] + ], + "ifaces": [ + [ + "br-ex", + "52:54:00:90:ca:d0", + [ + [ + "static", + "10.12.4.153", + "10.12.4.0/22" + ], + [ + "static", + "2602:f3e2:f02:10::153", + "2602:f3e2:f02:10::/64" + ] + ] + ], + [ + "enp1s0", + "52:54:00:2b:ed:ab", + [ + [ + "static", + "10.12.8.153", + "10.12.8.0/22" + ], + [ + "static", + "fd50:840e:74e2:220::153", + "fd50:840e:74e2:220::/64" + ] + ] + ], + [ + "enp2s0", + "52:54:00:90:ca:d0", + [] + ], + [ + "enp3s0", + "52:54:00:2a:ac:14", + [ + [ + "static", + "10.12.12.153", + "10.12.12.0/22" + ], + [ + "static", + "fd50:840e:74e2:221::153", + "fd50:840e:74e2:221::/64" + ] + ] + ], + [ + "enp4s0", + "52:54:00:af:34:fd", + [ + [ + "static", + "10.12.16.153", + "10.12.16.0/22" + ], + [ + "static", + "fd50:840e:74e2:230::153", + "fd50:840e:74e2:230::/64" + ] + ] + ], + [ + "enp5s0", + "52:54:00:28:b1:6d", + [ + [ + "static", + "10.12.32.153", + "10.12.32.0/22" + ], + [ + "static", + "fd50:840e:74e2:240::153", + "fd50:840e:74e2:240::/64" + ] + ] + ], + [ + "enp6s0", + "52:54:00:ab:5c:50", + [ + [ + "static", + "10.12.36.153", + "10.12.36.0/22" + ], + [ + "static", + "fd50:840e:74e2:250::153", + "fd50:840e:74e2:250::/64" + ] + ] + ] + ], + "power": "off", + "status": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/before.json b/docs/audit/osd-carve-20260729/before.json new file mode 100644 index 0000000..9a0eb96 --- /dev/null +++ b/docs/audit/osd-carve-20260729/before.json @@ -0,0 +1,113 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ] + ], + "ifaces": [ + [ + "br-ex", + "52:54:00:4c:69:7b", + [ + [ + "static", + "10.12.4.150", + "10.12.4.0/22" + ], + [ + "static", + "2602:f3e2:f02:10::150", + "2602:f3e2:f02:10::/64" + ] + ] + ], + [ + "enp1s0", + "52:54:00:5f:8d:42", + [ + [ + "static", + "10.12.8.150", + "10.12.8.0/22" + ], + [ + "static", + "fd50:840e:74e2:220::150", + "fd50:840e:74e2:220::/64" + ] + ] + ], + [ + "enp2s0", + "52:54:00:4c:69:7b", + [] + ], + [ + "enp3s0", + "52:54:00:ac:2c:4d", + [ + [ + "static", + "10.12.12.150", + "10.12.12.0/22" + ], + [ + "static", + "fd50:840e:74e2:221::150", + "fd50:840e:74e2:221::/64" + ] + ] + ], + [ + "enp4s0", + "52:54:00:28:91:d2", + [ + [ + "static", + "10.12.16.150", + "10.12.16.0/22" + ], + [ + "static", + "fd50:840e:74e2:230::150", + "fd50:840e:74e2:230::/64" + ] + ] + ], + [ + "enp5s0", + "52:54:00:fa:7c:53", + [ + [ + "static", + "10.12.32.150", + "10.12.32.0/22" + ], + [ + "static", + "fd50:840e:74e2:240::150", + "fd50:840e:74e2:240::/64" + ] + ] + ], + [ + "enp6s0", + "52:54:00:0c:7a:ab", + [ + [ + "static", + "10.12.36.150", + "10.12.36.0/22" + ], + [ + "static", + "fd50:840e:74e2:250::150", + "fd50:840e:74e2:250::/64" + ] + ] + ] + ], + "power": "off", + "status": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/dc1-phase1-apply.txt b/docs/audit/osd-carve-20260729/dc1-phase1-apply.txt new file mode 100644 index 0000000..e2ab17b --- /dev/null +++ b/docs/audit/osd-carve-20260729/dc1-phase1-apply.txt @@ -0,0 +1,22 @@ +module.vr1_dc1_node["vr1-dc1-storage-03"].libvirt_volume.osd[0]: Creating... +module.vr1_dc1_node["vr1-dc1-storage-04"].libvirt_volume.osd[0]: Creating... +module.vr1_dc1_node["vr1-dc1-juju-01"].libvirt_volume.disk: Creating... +module.vr1_dc1_node["vr1-dc1-storage-01"].libvirt_volume.osd[0]: Creating... +module.vr1_dc1_node["vr1-dc1-storage-02"].libvirt_volume.osd[0]: Creating... +module.vr1_dc1_node["vr1-dc1-storage-04"].libvirt_volume.osd[0]: Creation complete after 0s [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-storage-04-osd.qcow2] +module.vr1_dc1_node["vr1-dc1-storage-03"].libvirt_volume.osd[0]: Creation complete after 0s [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-storage-03-osd.qcow2] +module.vr1_dc1_node["vr1-dc1-juju-01"].libvirt_volume.disk: Creation complete after 0s [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-juju-01-disk.qcow2] +module.vr1_dc1_node["vr1-dc1-storage-01"].libvirt_volume.osd[0]: Creation complete after 0s [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-storage-01-osd.qcow2] +module.vr1_dc1_node["vr1-dc1-storage-02"].libvirt_volume.osd[0]: Creation complete after 0s [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-storage-02-osd.qcow2] +module.vr1_dc1_node["vr1-dc1-juju-01"].libvirt_domain.node: Creating... +module.vr1_dc1_node["vr1-dc1-storage-01"].libvirt_domain.node: Modifying... [name=vr1-dc1-storage-01] +module.vr1_dc1_node["vr1-dc1-storage-02"].libvirt_domain.node: Modifying... [name=vr1-dc1-storage-02] +module.vr1_dc1_node["vr1-dc1-storage-03"].libvirt_domain.node: Modifying... [name=vr1-dc1-storage-03] +module.vr1_dc1_node["vr1-dc1-storage-04"].libvirt_domain.node: Modifying... [name=vr1-dc1-storage-04] +module.vr1_dc1_node["vr1-dc1-juju-01"].libvirt_domain.node: Creation complete after 2s [name=vr1-dc1-juju-01] +module.vr1_dc1_node["vr1-dc1-storage-02"].libvirt_domain.node: Modifications complete after 2s [name=vr1-dc1-storage-02] +module.vr1_dc1_node["vr1-dc1-storage-01"].libvirt_domain.node: Modifications complete after 2s [name=vr1-dc1-storage-01] +module.vr1_dc1_node["vr1-dc1-storage-03"].libvirt_domain.node: Modifications complete after 2s [name=vr1-dc1-storage-03] +module.vr1_dc1_node["vr1-dc1-storage-04"].libvirt_domain.node: Modifications complete after 2s [name=vr1-dc1-storage-04] + +Apply complete! Resources: 6 added, 4 changed, 0 destroyed. diff --git a/docs/audit/osd-carve-20260729/dc1-phase1-plan.txt b/docs/audit/osd-carve-20260729/dc1-phase1-plan.txt new file mode 100644 index 0000000..602ca62 --- /dev/null +++ b/docs/audit/osd-carve-20260729/dc1-phase1-plan.txt @@ -0,0 +1,369 @@ +module.vr1_dc1_planes.libvirt_network.plane["provider-public"]: Refreshing state... [id=6742bcc5-3f43-4111-81b8-91744c757c37] +module.vr1_dc1_planes.libvirt_network.plane["metal-admin"]: Refreshing state... [id=f81378a7-abbd-4c75-a7aa-d7fb814d28d9] +module.vr1_dc1_planes.libvirt_network.plane["storage"]: Refreshing state... [id=b8b24c23-8866-4c8c-a5a4-9c1722d77120] +module.vr1_dc1_wan.libvirt_network.wan_bridge: Refreshing state... [id=56531929-1b2f-44f0-9779-8a711630f0b6] +module.inner_storage.libvirt_pool.dc: Refreshing state... [id=0b20edb6-ff0e-402e-9a54-93da985efc04] +module.vr1_dc1_planes.libvirt_network.plane["metal-internal"]: Refreshing state... [id=44e97462-41b3-4cc6-9387-864a8b059fba] +module.vr1_dc1_planes.libvirt_network.plane["data-tenant"]: Refreshing state... [id=2013f6ae-7b04-4207-99f9-123b272f1d25] +module.vr1_dc1_planes.libvirt_network.plane["replication"]: Refreshing state... [id=c171fcc9-b588-4235-b703-963e8fc0ec22] +module.vr1_dc1_opnsense.libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-opnsense-disk.qcow2] +module.vr1_dc1_node["vr1-dc1-storage-02"].libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-storage-02-disk.qcow2] +module.vr1_dc1_node["vr1-dc1-control-03"].libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-control-03-disk.qcow2] +module.vr1_dc1_node["vr1-dc1-control-02"].libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-control-02-disk.qcow2] +module.vr1_dc1_node["vr1-dc1-control-01"].libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-control-01-disk.qcow2] +module.vr1_dc1_node["vr1-dc1-compute-01"].libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-compute-01-disk.qcow2] +module.vr1_dc1_node["vr1-dc1-compute-02"].libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-compute-02-disk.qcow2] +module.vr1_dc1_node["vr1-dc1-storage-04"].libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-storage-04-disk.qcow2] +module.vr1_dc1_node["vr1-dc1-storage-01"].libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-storage-01-disk.qcow2] +module.vr1_dc1_node["vr1-dc1-storage-03"].libvirt_volume.disk: Refreshing state... [id=/var/lib/libvirt/vr1-dc1-inner/vr1-dc1-storage-03-disk.qcow2] +module.vr1_dc1_opnsense.libvirt_domain.vm: Refreshing state... [name=vr1-dc1-opnsense] +module.vr1_dc1_node["vr1-dc1-control-01"].libvirt_domain.node: Refreshing state... [name=vr1-dc1-control-01] +module.vr1_dc1_node["vr1-dc1-control-02"].libvirt_domain.node: Refreshing state... [name=vr1-dc1-control-02] +module.vr1_dc1_node["vr1-dc1-control-03"].libvirt_domain.node: Refreshing state... [name=vr1-dc1-control-03] +module.vr1_dc1_node["vr1-dc1-storage-01"].libvirt_domain.node: Refreshing state... [name=vr1-dc1-storage-01] +module.vr1_dc1_node["vr1-dc1-storage-03"].libvirt_domain.node: Refreshing state... [name=vr1-dc1-storage-03] +module.vr1_dc1_node["vr1-dc1-storage-02"].libvirt_domain.node: Refreshing state... [name=vr1-dc1-storage-02] +module.vr1_dc1_node["vr1-dc1-compute-01"].libvirt_domain.node: Refreshing state... [name=vr1-dc1-compute-01] +module.vr1_dc1_node["vr1-dc1-storage-04"].libvirt_domain.node: Refreshing state... [name=vr1-dc1-storage-04] +module.vr1_dc1_node["vr1-dc1-compute-02"].libvirt_domain.node: Refreshing state... [name=vr1-dc1-compute-02] + +OpenTofu used the selected providers to generate the following execution +plan. Resource actions are indicated with the following symbols: + + create + ~ update in-place (current -> planned) + +OpenTofu will perform the following actions: + + # module.vr1_dc1_node["vr1-dc1-juju-01"].libvirt_domain.node will be created + + resource "libvirt_domain" "node" { + + autostart = false + + cpu = { + + mode = "host-passthrough" + } + + devices = { + + disks = [ + + { + + boot = { + + order = 2 + } + + driver = { + + type = "qcow2" + } + + source = { + + volume = { + + pool = "vr1-dc1-inner-pool" + + volume = "vr1-dc1-juju-01-disk.qcow2" + } + } + + target = { + + bus = "virtio" + + dev = "vda" + } + }, + ] + + interfaces = [ + + { + + boot = { + + order = 1 + } + + model = { + + type = "virtio" + } + + source = { + + network = { + + network = "vr1-dc1-metal-admin" + } + } + }, + + { + + model = { + + type = "virtio" + } + + source = { + + network = { + + network = "vr1-dc1-provider-public" + } + } + }, + + { + + model = { + + type = "virtio" + } + + source = { + + network = { + + network = "vr1-dc1-metal-internal" + } + } + }, + + { + + model = { + + type = "virtio" + } + + source = { + + network = { + + network = "vr1-dc1-data-tenant" + } + } + }, + + { + + model = { + + type = "virtio" + } + + source = { + + network = { + + network = "vr1-dc1-storage" + } + } + }, + + { + + model = { + + type = "virtio" + } + + source = { + + network = { + + network = "vr1-dc1-replication" + } + } + }, + ] + + serials = [ + + { + + log = { + + append = "on" + + file = "/var/lib/libvirt/vr1/staging/vr1-dc1-juju-01-serial.log" + } + + target = { + + port = 0 + } + }, + ] + } + + features = { + + acpi = true + + apic = {} + } + + id = (known after apply) + + memory = 8192 + + memory_unit = "MiB" + + name = "vr1-dc1-juju-01" + + os = { + + type = "hvm" + + type_arch = "x86_64" + + type_machine = "q35" + } + + running = true + + type = "kvm" + + uuid = (known after apply) + + vcpu = 4 + } + + # module.vr1_dc1_node["vr1-dc1-juju-01"].libvirt_volume.disk will be created + + resource "libvirt_volume" "disk" { + + allocation = (known after apply) + + capacity = 107374182400 + + id = (known after apply) + + key = (known after apply) + + name = "vr1-dc1-juju-01-disk.qcow2" + + path = (known after apply) + + physical = (known after apply) + + pool = "vr1-dc1-inner-pool" + + target = { + + format = { + + type = "qcow2" + } + + path = (known after apply) + } + } + + # module.vr1_dc1_node["vr1-dc1-storage-01"].libvirt_domain.node will be updated in-place + ~ resource "libvirt_domain" "node" { + ~ devices = { + ~ disks = [ + + { + + driver = { + + type = "qcow2" + } + + source = { + + volume = { + + pool = "vr1-dc1-inner-pool" + + volume = "vr1-dc1-storage-01-osd.qcow2" + } + } + + target = { + + bus = "virtio" + + dev = "vdb" + } + }, + # (1 unchanged element hidden) + ] + # (2 unchanged attributes hidden) + } + id = 3 + name = "vr1-dc1-storage-01" + # (10 unchanged attributes hidden) + } + + # module.vr1_dc1_node["vr1-dc1-storage-01"].libvirt_volume.osd[0] will be created + + resource "libvirt_volume" "osd" { + + allocation = (known after apply) + + capacity = 536870912000 + + id = (known after apply) + + key = (known after apply) + + name = "vr1-dc1-storage-01-osd.qcow2" + + path = (known after apply) + + physical = (known after apply) + + pool = "vr1-dc1-inner-pool" + + target = { + + format = { + + type = "qcow2" + } + + path = (known after apply) + } + } + + # module.vr1_dc1_node["vr1-dc1-storage-02"].libvirt_domain.node will be updated in-place + ~ resource "libvirt_domain" "node" { + ~ devices = { + ~ disks = [ + + { + + driver = { + + type = "qcow2" + } + + source = { + + volume = { + + pool = "vr1-dc1-inner-pool" + + volume = "vr1-dc1-storage-02-osd.qcow2" + } + } + + target = { + + bus = "virtio" + + dev = "vdb" + } + }, + # (1 unchanged element hidden) + ] + # (2 unchanged attributes hidden) + } + id = 4 + name = "vr1-dc1-storage-02" + # (10 unchanged attributes hidden) + } + + # module.vr1_dc1_node["vr1-dc1-storage-02"].libvirt_volume.osd[0] will be created + + resource "libvirt_volume" "osd" { + + allocation = (known after apply) + + capacity = 536870912000 + + id = (known after apply) + + key = (known after apply) + + name = "vr1-dc1-storage-02-osd.qcow2" + + path = (known after apply) + + physical = (known after apply) + + pool = "vr1-dc1-inner-pool" + + target = { + + format = { + + type = "qcow2" + } + + path = (known after apply) + } + } + + # module.vr1_dc1_node["vr1-dc1-storage-03"].libvirt_domain.node will be updated in-place + ~ resource "libvirt_domain" "node" { + ~ devices = { + ~ disks = [ + + { + + driver = { + + type = "qcow2" + } + + source = { + + volume = { + + pool = "vr1-dc1-inner-pool" + + volume = "vr1-dc1-storage-03-osd.qcow2" + } + } + + target = { + + bus = "virtio" + + dev = "vdb" + } + }, + # (1 unchanged element hidden) + ] + # (2 unchanged attributes hidden) + } + id = 7 + name = "vr1-dc1-storage-03" + # (10 unchanged attributes hidden) + } + + # module.vr1_dc1_node["vr1-dc1-storage-03"].libvirt_volume.osd[0] will be created + + resource "libvirt_volume" "osd" { + + allocation = (known after apply) + + capacity = 536870912000 + + id = (known after apply) + + key = (known after apply) + + name = "vr1-dc1-storage-03-osd.qcow2" + + path = (known after apply) + + physical = (known after apply) + + pool = "vr1-dc1-inner-pool" + + target = { + + format = { + + type = "qcow2" + } + + path = (known after apply) + } + } + + # module.vr1_dc1_node["vr1-dc1-storage-04"].libvirt_domain.node will be updated in-place + ~ resource "libvirt_domain" "node" { + ~ devices = { + ~ disks = [ + + { + + driver = { + + type = "qcow2" + } + + source = { + + volume = { + + pool = "vr1-dc1-inner-pool" + + volume = "vr1-dc1-storage-04-osd.qcow2" + } + } + + target = { + + bus = "virtio" + + dev = "vdb" + } + }, + # (1 unchanged element hidden) + ] + # (2 unchanged attributes hidden) + } + id = 5 + name = "vr1-dc1-storage-04" + # (10 unchanged attributes hidden) + } + + # module.vr1_dc1_node["vr1-dc1-storage-04"].libvirt_volume.osd[0] will be created + + resource "libvirt_volume" "osd" { + + allocation = (known after apply) + + capacity = 536870912000 + + id = (known after apply) + + key = (known after apply) + + name = "vr1-dc1-storage-04-osd.qcow2" + + path = (known after apply) + + physical = (known after apply) + + pool = "vr1-dc1-inner-pool" + + target = { + + format = { + + type = "qcow2" + } + + path = (known after apply) + } + } + +Plan: 6 to add, 4 to change, 0 to destroy. + +───────────────────────────────────────────────────────────────────────────── + +Saved the plan to: /tmp/dc1-osd-juju.tfplan + +To perform exactly these actions, run the following command to apply: + tofu apply "/tmp/dc1-osd-juju.tfplan" diff --git a/docs/audit/osd-carve-20260729/s01.after.json b/docs/audit/osd-carve-20260729/s01.after.json new file mode 100644 index 0000000..bd5ed38 --- /dev/null +++ b/docs/audit/osd-carve-20260729/s01.after.json @@ -0,0 +1,117 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ], + [ + "vdb", + 536870912000 + ] + ], + "interfaces": [ + { + "links": [ + [ + "static", + "10.12.64.150", + "10.12.64.0/22" + ], + [ + "static", + "2602:f3e2:f03:10::150", + "2602:f3e2:f03:10::/64" + ] + ], + "mac": "52:54:01:d1:06:02", + "name": "br-ex" + }, + { + "links": [ + [ + "static", + "10.12.68.150", + "10.12.68.0/22" + ], + [ + "static", + "fd50:840e:74e2:320::150", + "fd50:840e:74e2:320::/64" + ] + ], + "mac": "52:54:01:d1:06:01", + "name": "enp1s0" + }, + { + "links": [], + "mac": "52:54:01:d1:06:02", + "name": "enp2s0" + }, + { + "links": [ + [ + "static", + "10.12.72.150", + "10.12.72.0/22" + ], + [ + "static", + "fd50:840e:74e2:321::150", + "fd50:840e:74e2:321::/64" + ] + ], + "mac": "52:54:01:d1:06:03", + "name": "enp3s0" + }, + { + "links": [ + [ + "static", + "10.12.76.150", + "10.12.76.0/22" + ], + [ + "static", + "fd50:840e:74e2:330::150", + "fd50:840e:74e2:330::/64" + ] + ], + "mac": "52:54:01:d1:06:04", + "name": "enp4s0" + }, + { + "links": [ + [ + "static", + "10.12.80.150", + "10.12.80.0/22" + ], + [ + "static", + "fd50:840e:74e2:340::150", + "fd50:840e:74e2:340::/64" + ] + ], + "mac": "52:54:01:d1:06:05", + "name": "enp5s0" + }, + { + "links": [ + [ + "static", + "10.12.84.150", + "10.12.84.0/22" + ], + [ + "static", + "fd50:840e:74e2:350::150", + "fd50:840e:74e2:350::/64" + ] + ], + "mac": "52:54:01:d1:06:06", + "name": "enp6s0" + } + ], + "power_state": "error", + "status_name": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/s01.before.json b/docs/audit/osd-carve-20260729/s01.before.json new file mode 100644 index 0000000..261d1b6 --- /dev/null +++ b/docs/audit/osd-carve-20260729/s01.before.json @@ -0,0 +1,113 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ] + ], + "interfaces": [ + { + "links": [ + [ + "static", + "10.12.64.150", + "10.12.64.0/22" + ], + [ + "static", + "2602:f3e2:f03:10::150", + "2602:f3e2:f03:10::/64" + ] + ], + "mac": "52:54:01:d1:06:02", + "name": "br-ex" + }, + { + "links": [ + [ + "static", + "10.12.68.150", + "10.12.68.0/22" + ], + [ + "static", + "fd50:840e:74e2:320::150", + "fd50:840e:74e2:320::/64" + ] + ], + "mac": "52:54:01:d1:06:01", + "name": "enp1s0" + }, + { + "links": [], + "mac": "52:54:01:d1:06:02", + "name": "enp2s0" + }, + { + "links": [ + [ + "static", + "10.12.72.150", + "10.12.72.0/22" + ], + [ + "static", + "fd50:840e:74e2:321::150", + "fd50:840e:74e2:321::/64" + ] + ], + "mac": "52:54:01:d1:06:03", + "name": "enp3s0" + }, + { + "links": [ + [ + "static", + "10.12.76.150", + "10.12.76.0/22" + ], + [ + "static", + "fd50:840e:74e2:330::150", + "fd50:840e:74e2:330::/64" + ] + ], + "mac": "52:54:01:d1:06:04", + "name": "enp4s0" + }, + { + "links": [ + [ + "static", + "10.12.80.150", + "10.12.80.0/22" + ], + [ + "static", + "fd50:840e:74e2:340::150", + "fd50:840e:74e2:340::/64" + ] + ], + "mac": "52:54:01:d1:06:05", + "name": "enp5s0" + }, + { + "links": [ + [ + "static", + "10.12.84.150", + "10.12.84.0/22" + ], + [ + "static", + "fd50:840e:74e2:350::150", + "fd50:840e:74e2:350::/64" + ] + ], + "mac": "52:54:01:d1:06:06", + "name": "enp6s0" + } + ], + "power_state": "off", + "status_name": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/s02.after.json b/docs/audit/osd-carve-20260729/s02.after.json new file mode 100644 index 0000000..edc92f4 --- /dev/null +++ b/docs/audit/osd-carve-20260729/s02.after.json @@ -0,0 +1,117 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ], + [ + "vdb", + 536870912000 + ] + ], + "interfaces": [ + { + "links": [ + [ + "static", + "10.12.64.151", + "10.12.64.0/22" + ], + [ + "static", + "2602:f3e2:f03:10::151", + "2602:f3e2:f03:10::/64" + ] + ], + "mac": "52:54:01:d1:07:02", + "name": "br-ex" + }, + { + "links": [ + [ + "static", + "10.12.68.151", + "10.12.68.0/22" + ], + [ + "static", + "fd50:840e:74e2:320::151", + "fd50:840e:74e2:320::/64" + ] + ], + "mac": "52:54:01:d1:07:01", + "name": "enp1s0" + }, + { + "links": [], + "mac": "52:54:01:d1:07:02", + "name": "enp2s0" + }, + { + "links": [ + [ + "static", + "10.12.72.151", + "10.12.72.0/22" + ], + [ + "static", + "fd50:840e:74e2:321::151", + "fd50:840e:74e2:321::/64" + ] + ], + "mac": "52:54:01:d1:07:03", + "name": "enp3s0" + }, + { + "links": [ + [ + "static", + "10.12.76.151", + "10.12.76.0/22" + ], + [ + "static", + "fd50:840e:74e2:330::151", + "fd50:840e:74e2:330::/64" + ] + ], + "mac": "52:54:01:d1:07:04", + "name": "enp4s0" + }, + { + "links": [ + [ + "static", + "10.12.80.151", + "10.12.80.0/22" + ], + [ + "static", + "fd50:840e:74e2:340::151", + "fd50:840e:74e2:340::/64" + ] + ], + "mac": "52:54:01:d1:07:05", + "name": "enp5s0" + }, + { + "links": [ + [ + "static", + "10.12.84.151", + "10.12.84.0/22" + ], + [ + "static", + "fd50:840e:74e2:350::151", + "fd50:840e:74e2:350::/64" + ] + ], + "mac": "52:54:01:d1:07:06", + "name": "enp6s0" + } + ], + "power_state": "off", + "status_name": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/s02.before.json b/docs/audit/osd-carve-20260729/s02.before.json new file mode 100644 index 0000000..1987ef9 --- /dev/null +++ b/docs/audit/osd-carve-20260729/s02.before.json @@ -0,0 +1,113 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ] + ], + "interfaces": [ + { + "links": [ + [ + "static", + "10.12.64.151", + "10.12.64.0/22" + ], + [ + "static", + "2602:f3e2:f03:10::151", + "2602:f3e2:f03:10::/64" + ] + ], + "mac": "52:54:01:d1:07:02", + "name": "br-ex" + }, + { + "links": [ + [ + "static", + "10.12.68.151", + "10.12.68.0/22" + ], + [ + "static", + "fd50:840e:74e2:320::151", + "fd50:840e:74e2:320::/64" + ] + ], + "mac": "52:54:01:d1:07:01", + "name": "enp1s0" + }, + { + "links": [], + "mac": "52:54:01:d1:07:02", + "name": "enp2s0" + }, + { + "links": [ + [ + "static", + "10.12.72.151", + "10.12.72.0/22" + ], + [ + "static", + "fd50:840e:74e2:321::151", + "fd50:840e:74e2:321::/64" + ] + ], + "mac": "52:54:01:d1:07:03", + "name": "enp3s0" + }, + { + "links": [ + [ + "static", + "10.12.76.151", + "10.12.76.0/22" + ], + [ + "static", + "fd50:840e:74e2:330::151", + "fd50:840e:74e2:330::/64" + ] + ], + "mac": "52:54:01:d1:07:04", + "name": "enp4s0" + }, + { + "links": [ + [ + "static", + "10.12.80.151", + "10.12.80.0/22" + ], + [ + "static", + "fd50:840e:74e2:340::151", + "fd50:840e:74e2:340::/64" + ] + ], + "mac": "52:54:01:d1:07:05", + "name": "enp5s0" + }, + { + "links": [ + [ + "static", + "10.12.84.151", + "10.12.84.0/22" + ], + [ + "static", + "fd50:840e:74e2:350::151", + "fd50:840e:74e2:350::/64" + ] + ], + "mac": "52:54:01:d1:07:06", + "name": "enp6s0" + } + ], + "power_state": "off", + "status_name": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/s03.after.json b/docs/audit/osd-carve-20260729/s03.after.json new file mode 100644 index 0000000..992e629 --- /dev/null +++ b/docs/audit/osd-carve-20260729/s03.after.json @@ -0,0 +1,117 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ], + [ + "vdb", + 536870912000 + ] + ], + "interfaces": [ + { + "links": [ + [ + "static", + "10.12.64.152", + "10.12.64.0/22" + ], + [ + "static", + "2602:f3e2:f03:10::152", + "2602:f3e2:f03:10::/64" + ] + ], + "mac": "52:54:01:d1:08:02", + "name": "br-ex" + }, + { + "links": [ + [ + "static", + "10.12.68.152", + "10.12.68.0/22" + ], + [ + "static", + "fd50:840e:74e2:320::152", + "fd50:840e:74e2:320::/64" + ] + ], + "mac": "52:54:01:d1:08:01", + "name": "enp1s0" + }, + { + "links": [], + "mac": "52:54:01:d1:08:02", + "name": "enp2s0" + }, + { + "links": [ + [ + "static", + "10.12.72.152", + "10.12.72.0/22" + ], + [ + "static", + "fd50:840e:74e2:321::152", + "fd50:840e:74e2:321::/64" + ] + ], + "mac": "52:54:01:d1:08:03", + "name": "enp3s0" + }, + { + "links": [ + [ + "static", + "10.12.76.152", + "10.12.76.0/22" + ], + [ + "static", + "fd50:840e:74e2:330::152", + "fd50:840e:74e2:330::/64" + ] + ], + "mac": "52:54:01:d1:08:04", + "name": "enp4s0" + }, + { + "links": [ + [ + "static", + "10.12.80.152", + "10.12.80.0/22" + ], + [ + "static", + "fd50:840e:74e2:340::152", + "fd50:840e:74e2:340::/64" + ] + ], + "mac": "52:54:01:d1:08:05", + "name": "enp5s0" + }, + { + "links": [ + [ + "static", + "10.12.84.152", + "10.12.84.0/22" + ], + [ + "static", + "fd50:840e:74e2:350::152", + "fd50:840e:74e2:350::/64" + ] + ], + "mac": "52:54:01:d1:08:06", + "name": "enp6s0" + } + ], + "power_state": "on", + "status_name": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/s03.before.json b/docs/audit/osd-carve-20260729/s03.before.json new file mode 100644 index 0000000..15b86c2 --- /dev/null +++ b/docs/audit/osd-carve-20260729/s03.before.json @@ -0,0 +1,113 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ] + ], + "interfaces": [ + { + "links": [ + [ + "static", + "10.12.64.152", + "10.12.64.0/22" + ], + [ + "static", + "2602:f3e2:f03:10::152", + "2602:f3e2:f03:10::/64" + ] + ], + "mac": "52:54:01:d1:08:02", + "name": "br-ex" + }, + { + "links": [ + [ + "static", + "10.12.68.152", + "10.12.68.0/22" + ], + [ + "static", + "fd50:840e:74e2:320::152", + "fd50:840e:74e2:320::/64" + ] + ], + "mac": "52:54:01:d1:08:01", + "name": "enp1s0" + }, + { + "links": [], + "mac": "52:54:01:d1:08:02", + "name": "enp2s0" + }, + { + "links": [ + [ + "static", + "10.12.72.152", + "10.12.72.0/22" + ], + [ + "static", + "fd50:840e:74e2:321::152", + "fd50:840e:74e2:321::/64" + ] + ], + "mac": "52:54:01:d1:08:03", + "name": "enp3s0" + }, + { + "links": [ + [ + "static", + "10.12.76.152", + "10.12.76.0/22" + ], + [ + "static", + "fd50:840e:74e2:330::152", + "fd50:840e:74e2:330::/64" + ] + ], + "mac": "52:54:01:d1:08:04", + "name": "enp4s0" + }, + { + "links": [ + [ + "static", + "10.12.80.152", + "10.12.80.0/22" + ], + [ + "static", + "fd50:840e:74e2:340::152", + "fd50:840e:74e2:340::/64" + ] + ], + "mac": "52:54:01:d1:08:05", + "name": "enp5s0" + }, + { + "links": [ + [ + "static", + "10.12.84.152", + "10.12.84.0/22" + ], + [ + "static", + "fd50:840e:74e2:350::152", + "fd50:840e:74e2:350::/64" + ] + ], + "mac": "52:54:01:d1:08:06", + "name": "enp6s0" + } + ], + "power_state": "error", + "status_name": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/s04.after.json b/docs/audit/osd-carve-20260729/s04.after.json new file mode 100644 index 0000000..c26d9cd --- /dev/null +++ b/docs/audit/osd-carve-20260729/s04.after.json @@ -0,0 +1,117 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ], + [ + "vdb", + 536870912000 + ] + ], + "interfaces": [ + { + "links": [ + [ + "static", + "10.12.64.153", + "10.12.64.0/22" + ], + [ + "static", + "2602:f3e2:f03:10::153", + "2602:f3e2:f03:10::/64" + ] + ], + "mac": "52:54:01:d1:09:02", + "name": "br-ex" + }, + { + "links": [ + [ + "static", + "10.12.68.153", + "10.12.68.0/22" + ], + [ + "static", + "fd50:840e:74e2:320::153", + "fd50:840e:74e2:320::/64" + ] + ], + "mac": "52:54:01:d1:09:01", + "name": "enp1s0" + }, + { + "links": [], + "mac": "52:54:01:d1:09:02", + "name": "enp2s0" + }, + { + "links": [ + [ + "static", + "10.12.72.153", + "10.12.72.0/22" + ], + [ + "static", + "fd50:840e:74e2:321::153", + "fd50:840e:74e2:321::/64" + ] + ], + "mac": "52:54:01:d1:09:03", + "name": "enp3s0" + }, + { + "links": [ + [ + "static", + "10.12.76.153", + "10.12.76.0/22" + ], + [ + "static", + "fd50:840e:74e2:330::153", + "fd50:840e:74e2:330::/64" + ] + ], + "mac": "52:54:01:d1:09:04", + "name": "enp4s0" + }, + { + "links": [ + [ + "static", + "10.12.80.153", + "10.12.80.0/22" + ], + [ + "static", + "fd50:840e:74e2:340::153", + "fd50:840e:74e2:340::/64" + ] + ], + "mac": "52:54:01:d1:09:05", + "name": "enp5s0" + }, + { + "links": [ + [ + "static", + "10.12.84.153", + "10.12.84.0/22" + ], + [ + "static", + "fd50:840e:74e2:350::153", + "fd50:840e:74e2:350::/64" + ] + ], + "mac": "52:54:01:d1:09:06", + "name": "enp6s0" + } + ], + "power_state": "on", + "status_name": "Ready" +} diff --git a/docs/audit/osd-carve-20260729/s04.before.json b/docs/audit/osd-carve-20260729/s04.before.json new file mode 100644 index 0000000..ce8f4f9 --- /dev/null +++ b/docs/audit/osd-carve-20260729/s04.before.json @@ -0,0 +1,113 @@ +{ + "blockdevices": [ + [ + "vda", + 590558003200 + ] + ], + "interfaces": [ + { + "links": [ + [ + "static", + "10.12.64.153", + "10.12.64.0/22" + ], + [ + "static", + "2602:f3e2:f03:10::153", + "2602:f3e2:f03:10::/64" + ] + ], + "mac": "52:54:01:d1:09:02", + "name": "br-ex" + }, + { + "links": [ + [ + "static", + "10.12.68.153", + "10.12.68.0/22" + ], + [ + "static", + "fd50:840e:74e2:320::153", + "fd50:840e:74e2:320::/64" + ] + ], + "mac": "52:54:01:d1:09:01", + "name": "enp1s0" + }, + { + "links": [], + "mac": "52:54:01:d1:09:02", + "name": "enp2s0" + }, + { + "links": [ + [ + "static", + "10.12.72.153", + "10.12.72.0/22" + ], + [ + "static", + "fd50:840e:74e2:321::153", + "fd50:840e:74e2:321::/64" + ] + ], + "mac": "52:54:01:d1:09:03", + "name": "enp3s0" + }, + { + "links": [ + [ + "static", + "10.12.76.153", + "10.12.76.0/22" + ], + [ + "static", + "fd50:840e:74e2:330::153", + "fd50:840e:74e2:330::/64" + ] + ], + "mac": "52:54:01:d1:09:04", + "name": "enp4s0" + }, + { + "links": [ + [ + "static", + "10.12.80.153", + "10.12.80.0/22" + ], + [ + "static", + "fd50:840e:74e2:340::153", + "fd50:840e:74e2:340::/64" + ] + ], + "mac": "52:54:01:d1:09:05", + "name": "enp5s0" + }, + { + "links": [ + [ + "static", + "10.12.84.153", + "10.12.84.0/22" + ], + [ + "static", + "fd50:840e:74e2:350::153", + "fd50:840e:74e2:350::/64" + ] + ], + "mac": "52:54:01:d1:09:06", + "name": "enp6s0" + } + ], + "power_state": "error", + "status_name": "Ready" +} diff --git a/docs/changelog-20260728-vip-arity-gate.md b/docs/changelog-20260728-vip-arity-gate.md index 46c702a..6040f77 100644 --- a/docs/changelog-20260728-vip-arity-gate.md +++ b/docs/changelog-20260728-vip-arity-gate.md @@ -983,3 +983,48 @@ **Evidence.** Gauntlet **ALL GREEN (87) on vcloud**; `opentofu-validate` PASS; repo-lint 0 fail / 614 files. `lib-hosts` resolves both DCs: 10 hosts, juju octet 5, boot MACs present. + +## 40. LIVE: R1's OSD carve and the D-104 controllers complete across BOTH DCs (two agents) + +Operator: **"Finish the remaining three dc0 nodes as an agent. Start a separate agent for all +of dc1. Monitor and orchestrate both."** + +**Result: eight re-commissions, every one the same single-change diff** -- `vdb` at +536870912000 bytes (500 GiB) added; links 12 -> 12; per-interface MACs and link lists +identical. dc0 storage-02/03/04 at 204/196/202s; dc1 storage-01..04 all at ~182s. dc1's +substrate applied 6 add / 4 change / 0 destroy with **MAC drift 0**, re-converging to +**ZERO DIFF**. + +**I verified every claim from MAAS directly rather than accepting the reports** -- fleet 22 +(18 Ready + 2 Deployed + 2 New), 216 links all static, both DCs `blockdevs {1:5, 2:4}`, +`tagcount {2:9}`. All matched. I also diffed a RAW dc1 before/after pair myself: exactly the +`vdb` hunk plus one `power_state` line, nothing else. + +**Captures promoted** from `voffice1:/tmp` to `docs/audit/osd-carve-20260729/` (18 files). +Both agents were barred from repo writes, so **dc1 would otherwise have had no audit record +at all** -- dc0 had one only because I wrote it by hand earlier. + +**Two things the agents found that I had not asked for, both valuable:** +- **MAAS tags SURVIVED re-commissioning on all 18 nodes.** Agent A checked this unprompted, + reasoning that its own snapshot code structurally could not see tags. That removes a + would-be Stage-5 Juju placement blocker nobody had verified. +- **`lib_hosts_select_dc` refuses a second DC in one shell** (deliberate). Agent A had muted + it with `2>&1` and silently got dc0 data labelled dc1; it caught itself and reported the + trap. Reusable for anyone scripting cross-DC lookups. + +**ONE DISCLOSED JUDGMENT CALL, and it was correct.** Three dc1 pairs also showed +`power_state` moving `off`/`error`. Agent B declared a STOP, re-measured, then excluded +`power_state` from its verdict -- still printing it -- because 8 of 9 dc1 nodes read `error` +BEFORE any mutation. **Vindicated: all 18 role nodes now read `off`, including five dc1 nodes +never commissioned.** A transient MAAS power-query flap. It surfaced the call explicitly and +invited reversal, which is exactly right for a live-ops agent. + +**Revert.** The volumes and controller domains are removable via targeted destroy; the +in-place disk attachments revert by removing `osd_gib` and re-applying. Commissioning is +idempotent. Nothing was destroyed. + +**STILL BLOCKING for both controllers: `power_type` is EMPTY** -- the state that stalled all +nine role nodes on 2026-07-20. Now fixable, since both are in `lib-hosts` at the ruled `.5` +with MACs pinned. Gated next steps: set `power_type=virsh`; commission them on the DEFAULT +path (no `skip_networking` -- nothing to preserve); create the `juju-controller-` tags; +run `maas-role-tags.sh apply --commit`.