diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 0010d8d..ba11311 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -309,8 +309,25 @@ `runbooks/dc-dc-phase3-maas-enlist-deploy.md:412,484`, `runbooks/dc-dc-phase4-juju-bundle-per-dc.md:26`) and needs a DOCFIX re-expressing it as MAAS-hierarchy time verification before it can be checked at all. - Also queued to this close and NOT yet done: carve residue cleanup (192.168.1.0/24, emptied - fabric-5, ~90 empty auto-created fabrics), the SEC-024 retention question on the two + - **CARVE RESIDUE CLEANED 2026-07-27** (capture + `docs/audit/stage4-carve-residue-cleanup-20260727.txt`): **108 fabrics -> 17**. Deleted + subnet id=8 `192.168.1.0/24` (the superseded OPNsense FACTORY LAN -- both edges were + re-addressed to 10.12.4.1 / 10.12.64.1), then fabric-5, then the 90 auto-created empties + (ids 6..95). The audit caught an ORDERING dependency the original flag did not state: the + 192.168.1.0/24 subnet was the ONLY occupant of fabric-5, so deleting the fabric first + would have cascaded the subnet away. Emptiness was PROVEN per fabric (zero subnets, zero + ipranges, zero node interfaces across all its VLANs) against a fresh occupancy snapshot + taken immediately before the batch, and the cascade check was re-run AFTER -- the inverse + of the 2026-07-21 pod-delete incident, where the association check ran too late and cost 9 + machine records. POST-STATE: 18 Ready + 2 Deployed office1 guests, all 18 still + `power_type=virsh`, 7 interfaces each (6 flat planes per D-133 + br-ex), **ZERO orphaned + interfaces**, placement tags 9+9 intact, both artifact paths re-verified PASS. The 17 + survivors are all load-bearing and enumerated in the capture (office1 base+GUA+compose, + both transits, both metal-admin/boot fabrics, libvirt default, and the 10 named plane + fabrics). **dc1's leftover nginx also removed** the same day (purged nginx + nginx-common, + `/etc/nginx` gone, nothing listening on :80, proxy still PASS on :3142; it had been a + MIRROR prereq only -- dc0 keeps its nginx and is unaffected). + Also queued to this close and NOT yet done: the SEC-024 retention question on the two `pre-*` state-surgery snapshots (irreversible, operator-deferred), the `set-interface-v4` reload amendment (an unruled OPTION), and the GA-R6 close-out set itself (GA-R2 consolidation, skill sweep + snapshot regeneration, GA-R7 memory review, final gauntlet, diff --git a/docs/audit/stage4-carve-residue-cleanup-20260727.txt b/docs/audit/stage4-carve-residue-cleanup-20260727.txt new file mode 100644 index 0000000..cad246d --- /dev/null +++ b/docs/audit/stage4-carve-residue-cleanup-20260727.txt @@ -0,0 +1,80 @@ +=== Stage 4 carve residue cleanup + dc1 nginx removal -- 2026-07-27 === +repo HEAD: 84951307a918a695e6f7f84ed4f8c992f248cd01 branch: dc-dc-stage4-phase3-maas-deploy + +SCOPE: the residue CURRENT-STATE flagged at the 2026-07-23 carve, deferred to stage close: +192.168.1.0/24 + emptied fabric-5 + ~90 empty auto-created fabrics. Plus the nginx install +left behind on dc1 by the removed mirror. + +METHOD: read-only audit FIRST, emptiness PROVEN per fabric, never assumed. Deleting a MAAS +fabric cascades to its VLANs, and this repo lost 9 machine records to exactly that class on +2026-07-21 because the association check ran AFTER the delete. Here a fresh occupancy +snapshot (fabrics + subnets + ipranges + every machine interface) was taken IMMEDIATELY +before the batch, and the cascade check was ALSO re-run after each delete. + +--- 1. pre-cleanup audit: 108 fabrics, 90 empty / 18 occupied --- +A fabric counted EMPTY only with zero subnets, zero ipranges and zero node interfaces +across all its VLANs. The audit also caught an ORDERING dependency the flag did not state: +192.168.1.0/24 (subnet id 8) was the ONLY occupant of fabric-5, so the subnet had to be +deleted first or the fabric delete would have cascaded it away. + +KEPT -- load-bearing, verified occupied (fabric id / name / cidr / node interfaces): + 0 fabric-0 10.10.0.0/24 + 2602:f3e2:f01:100::/64 office1 base + GUA + 1 fabric-1 10.10.1.0/24 office1 compose (2 LXD guests) + 2 fabric-2 172.31.0.0/30 dc0 transit + 3 fabric-3 192.168.122.0/24 libvirt default + 4 fabric-4 10.12.8.0/22 dc0 metal-admin/boot, 9 nodes + 141 fabric-141 172.31.0.4/30 dc1 transit + 142 fabric-142 10.12.68.0/22 dc1 metal-admin/boot, 9 nodes + 188-192 vr1-dc0-{provider-public,metal-internal,data-tenant,storage,replication} 9-18 ifaces each + 193-197 vr1-dc1-{provider-public,metal-internal,data-tenant,storage,replication} 9-18 ifaces each + +DELETED: + subnet id=8 192.168.1.0/24 the superseded OPNsense FACTORY LAN (both edges were + re-addressed to 10.12.4.1 / 10.12.64.1); 0 ipranges, 0 interfaces + fabric id=5 fabric-5 empty once subnet 8 was gone (held only its vid-0 default VLAN) + fabrics 6..95 90 auto-created empties, each re-verified against a fresh + occupancy snapshot taken immediately before the batch + +--- 2. post-cleanup fabric list (17, all accounted for above) --- +fabric count: 17 + 0 fabric-0 vlans=[(5001, 0)] + 1 fabric-1 vlans=[(5002, 0)] + 2 fabric-2 vlans=[(5003, 0)] + 3 fabric-3 vlans=[(5004, 0)] + 4 fabric-4 vlans=[(5005, 0)] + 141 fabric-141 vlans=[(5142, 0)] + 142 fabric-142 vlans=[(5143, 0)] + 188 vr1-dc0-provider-public vlans=[(5189, 0)] + 189 vr1-dc0-metal-internal vlans=[(5190, 0)] + 190 vr1-dc0-data-tenant vlans=[(5191, 0)] + 191 vr1-dc0-storage vlans=[(5192, 0)] + 192 vr1-dc0-replication vlans=[(5193, 0)] + 193 vr1-dc1-provider-public vlans=[(5194, 0)] + 194 vr1-dc1-metal-internal vlans=[(5195, 0)] + 195 vr1-dc1-data-tenant vlans=[(5196, 0)] + 196 vr1-dc1-storage vlans=[(5197, 0)] + 197 vr1-dc1-replication vlans=[(5198, 0)] + +--- 3. CASCADE CHECK (the 2026-07-21 lesson, run after) --- + machines: 20 {'Deployed': 2, 'Ready': 18} + power_type: {'lxd': 2, 'virsh': 18} + interfaces-per-machine: {1: 2, 7: 18} + interfaces ORPHANED by the deletes: 0 [] + placement tags: {'openstack-vr1-dc0': 9, 'openstack-vr1-dc1': 9} + READING: 18 Ready + 2 Deployed office1 guests, all 18 still power_type=virsh, 7 interfaces + each (6 flat planes per D-133 + br-ex), ZERO orphaned interfaces, tags 9+9 intact. + +--- 4. dc1 nginx install removed (it was a MIRROR prereq; the proxy uses :3142) --- + /etc/nginx purged + listening :80 -> 0 + listening :3142 -> 2 + purged nginx + nginx-common (nothing else depended on them; no nginx-core was installed); + autoremoved liblockfile-simple-perl + liblog-agent-perl, debmirror's orphaned perl deps. + +--- 5. both DCs' artifact paths re-verified AFTER all of the above --- + OK proxy serves http://archive.ubuntu.com/ubuntu/dists/jammy/Release (200) + OK proxy serves http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/jammy-updates/caracal/Release (200) +dc-cache-proxy check (dc1): PASS + OK mirror answers http://10.12.8.4/ (200) + OK last-sync: OK 2026-07-27T08:43:46Z ubuntu=0 uca=0 +dc-mirror check (dc0): PASS diff --git a/docs/changelog-20260727-creds-consolidation.md b/docs/changelog-20260727-creds-consolidation.md index 69b4d0e..e3bcd27 100644 --- a/docs/changelog-20260727-creds-consolidation.md +++ b/docs/changelog-20260727-creds-consolidation.md @@ -345,6 +345,46 @@ `git revert` of `scripts/dc-cache-proxy.sh` + its harness, then re-running `dc-mirror.sh install dc1` to restore the mirror-owned layer. +## 10. Carve residue cleaned (108 fabrics -> 17); dc1's leftover nginx removed + +Capture `docs/audit/stage4-carve-residue-cleanup-20260727.txt`. Both items were queued to the +stage close at the 2026-07-23 carve. + +**nginx on dc1.** Purged `nginx` + `nginx-common` (dry-run first: nothing else depended on them, +and no `nginx-core` was installed), plus autoremove of `liblockfile-simple-perl` and +`liblog-agent-perl` -- debmirror's orphaned perl deps. `/etc/nginx` gone, nothing listening on +:80, proxy still PASS on :3142. dc0 keeps its nginx and was re-verified unaffected (it serves +the mirror). + +**Carve residue: 108 fabrics -> 17.** Deleted subnet id=8 `192.168.1.0/24` (the superseded +OPNsense FACTORY LAN -- both edges were re-addressed to 10.12.4.1 / 10.12.64.1), then fabric-5, +then the 90 auto-created empties (ids 6..95). + +The audit caught an **ordering dependency the original flag did not state**: the +192.168.1.0/24 subnet was the ONLY occupant of fabric-5, so deleting the fabric first would +have cascaded the subnet away rather than removing it cleanly. Subnet first, then the fabric. + +METHOD, chosen because of this repo's own history: a MAAS fabric delete cascades to its VLANs, +and the 2026-07-21 pod-delete incident cost 9 machine records precisely because the association +check ran AFTER the delete. So emptiness was PROVEN per fabric -- zero subnets, zero ipranges, +zero node interfaces across all of its VLANs -- against a fresh occupancy snapshot (fabrics + +subnets + ipranges + every machine's `interface_set`) taken immediately before the batch, and +the cascade check was re-run afterwards as well. + +KEPT, all verified occupied and enumerated in the capture: office1 base + GUA (fabric-0), +office1 compose with its 2 LXD guests (fabric-1), both transits (fabric-2 / fabric-141), both +metal-admin/boot fabrics with 9 nodes each (fabric-4 / fabric-142), libvirt default (fabric-3), +and the 10 named plane fabrics (`vr1-dc0-*` / `vr1-dc1-*`, 9-18 interfaces each). + +POST-STATE: 18 Ready + 2 Deployed office1 guests, all 18 still `power_type=virsh`, 7 interfaces +each (6 flat planes per D-133 + br-ex), **ZERO orphaned interfaces**, placement tags 9+9 intact, +and both DCs' artifact paths re-verified PASS after everything. + +Revert: fabrics and subnets are MAAS records, not config -- there is no undo. Recreating them +would mean `maas admin fabrics create` + `subnets create`, but nothing needs them: the 90 were +auto-created empties, and 192.168.1.0/24 was a superseded factory address. The nginx purge +reverts with `apt-get install nginx` (dc1 has no use for it while it is the proxy DC). + --- ## Findings LOGGED, not actioned (hard rule 1)