diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 052aab8..93a0982 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1960,6 +1960,37 @@ a build mid-deploy. (B) keep the mirror, restructure the armored key behind a single anchor, apply via the Step-7 pattern -- fastest unblock, keeps dc0 diverged from the proven config. **I am not choosing between them.** + **>>> MODEL REBUILD RULED; DESTROY BLAST RADIUS VERIFIED BEFORE THE MUTATION. <<<** + Operator ruled a MODEL rebuild (not an environment rebuild) and asked for the command to + be verified first: *"We have had issues previously when a destroy has had a larger blast + radius then intended."* Well-founded -- the 2026-07-21 MAAS pod delete cascaded to its + linked machine records. Full verification: `docs/audit/destroy-model-blast-radius-20260802.txt`. + **FINDING A -- I PROPOSED A DESTRUCTIVE FLAG THAT DOES NOTHING HERE, AND IT IS DROPPED.** + `juju storage -m vr1-dc0` -> **"No storage to display."** The model has ZERO juju-managed + storage, so `--destroy-storage` destroys nothing. `ceph-osd`'s `osd-devices` are RAW BLOCK + DEVICES set by charm config, not juju storage instances -- which is exactly why the list is + empty; MAAS re-partitions them at re-image. Neither storage flag is needed (juju demands + one only when persistent storage exists; the `controller` model has none either). + **FINDING B -- NO CROSS-MODEL BLAST RADIUS.** `juju offers -m vr1-dc0` is EMPTY, so no + other model consumes a relation from it. `juju help destroy-model` states the scope in its + own summary: "Terminate all machines/containers and resources for a **NON-CONTROLLER + model**" -- the `controller` model is out of scope by design. `--no-prompt` VERIFIED to + exist (the help's Details prose still says `-y`, which is STALE; the flag list wins). + **FINDING C -- THE NODES COME BACK FAST.** `enable_disk_erasing_on_release` = **false** on + the dc0 region, so releasing the nine nodes triggers NO disk-erase cycle; they return to + `Ready` and are re-imaged by the redeploy. + **MEASURED CORRECTION a future session would otherwise guess wrong: on the dc0 REGION VM + the MAAS CLI profile is `dc0`, NOT `vr1-dc0-region`** -- the tool's own error enumerated + the valid choices. Profile names are PER HOST; assert, never assume. + **COMMAND AS VERIFIED:** `juju destroy-model vr1-dc0 --force --no-prompt`. Deliberately NOT + passed: `--destroy-storage` / `--release-storage` (no-ops), `--no-wait` (help says it + rushes steps), `--timeout` (help warns `--force` + `--timeout` abandons clean shutdown). + OUT OF SCOPE and untouched: the controller model and its VM, the MAAS region and database, + the nine machine RECORDS (released to `Ready`, not deleted), the rack, the edge, both + proxies, the tofu substrate, and dc1 entirely. **POST-CHECKS before recreating the model:** + `juju models` shows only `controller`; the MAAS machine COUNT is unchanged with the nine + role nodes `Ready` (**a DROP in count is the 2026-07-21 cascade signature -- STOP**); both + proxy checks still PASS. **ROOSEVELT DELTA WORTH A RULING LATER, not resolved here:** an airgapped DC that mirrors the UCA must carry the archive's TRUST as deliberately as its CONTENT. The 2026-07-31 ruling pointed the URL at the mirror, which is half the job. Whether the durable answer is diff --git a/docs/audit/destroy-model-blast-radius-20260802.txt b/docs/audit/destroy-model-blast-radius-20260802.txt new file mode 100644 index 0000000..fe76b68 --- /dev/null +++ b/docs/audit/destroy-model-blast-radius-20260802.txt @@ -0,0 +1,81 @@ +destroy-model-blast-radius-20260802.txt +======================================= +Pre-mutation verification of `juju destroy-model vr1-dc0`, requested by the operator: +"We have had issues previously when a destroy has had a larger blast radius then +intended. Review and confirm that the command will only destroy what is intended." + +Well-founded: this repo has a logged cascade (2026-07-21, a MAAS pod delete taking its +linked machine records with it). Everything below is MEASURED on the live controller +from the dc0 rack (D-138), not reasoned from the man page. + +-------------------------------------------------------------------------------------- +1. FLAG SEMANTICS -- `juju help destroy-model`, juju 3.6.27, THIS controller +-------------------------------------------------------------------------------------- +Summary line: "Terminate all machines/containers and resources for a NON-CONTROLLER +model." -> the controller model is out of scope BY DESIGN, not by our care. + --no-prompt EXISTS. (Confirmed -- the Details prose still says "-y", which is + STALE; the flag list is authoritative. Do not use -y.) + --destroy-storage "Destroy all storage instances in the model" + --release-storage "Release ... without destroying them" + --force "removes all units ... and, potentially, machines without giving + them the opportunity to shutdown cleanly" + --no-wait "Rush through model destruction without waiting for each step" + --timeout WARNING in help: --force WITH --timeout continues "without + consideration or respect for clean shutdown or resource cleanup" + +-------------------------------------------------------------------------------------- +2. WHAT IS ACTUALLY IN THE MODEL +-------------------------------------------------------------------------------------- + juju storage -m vr1-dc0 -> "No storage to display." ZERO storage instances + juju storage -m controller -> "No storage to display." + juju offers -m vr1-dc0 -> EMPTY (no cross-model offers) + juju models -> controller (1 machine) + vr1-dc0 (36 machines/45 units) + +>>> FINDING A -- `--destroy-storage` IS A NO-OP HERE AND IS BEING DROPPED. <<< +The model has NO juju-managed storage, so the flag destroys nothing. I proposed the +DESTRUCTIVE storage flag for a model with no storage. ceph-osd's `osd-devices` are RAW +BLOCK DEVICES set by charm config, not juju storage instances -- which is exactly why +`juju storage` is empty. They are re-partitioned by MAAS at re-image. Neither storage +flag is required: juju only demands one when persistent storage exists. + +>>> FINDING B -- NO CROSS-MODEL BLAST RADIUS. <<< +Zero offers means no other model consumes a relation from vr1-dc0, so nothing outside +this model loses an endpoint. The `controller` model is a separate model and is +explicitly out of destroy-model's scope. + +-------------------------------------------------------------------------------------- +3. WHAT HAPPENS TO THE NINE MAAS NODES +-------------------------------------------------------------------------------------- + maas dc0 maas get-config name=enable_disk_erasing_on_release -> false + +So release does NOT trigger a disk-erase cycle; the nodes return to `Ready` promptly and +are re-imaged by the redeploy. (`disk_erase_with_secure_erase=true` / +`disk_erase_with_quick_erase=false` are the modes that WOULD apply if erasing were on -- +they are inert while the master switch is false.) + +MEASURED CORRECTION, recorded because a future session would guess it wrong: on the dc0 +REGION VM the MAAS CLI profile is **`dc0`**, NOT `vr1-dc0-region`. The tool's own error +enumerated the valid choices. Profile names are PER HOST -- assert, never assume. + +-------------------------------------------------------------------------------------- +4. THE COMMAND, AS APPROVED FOR REVIEW +-------------------------------------------------------------------------------------- + juju destroy-model vr1-dc0 --force --no-prompt + +DELIBERATELY NOT PASSED: + --destroy-storage no-op (Finding A) and it is the destructive one + --release-storage also unnecessary -- there is no storage to release + --no-wait help says it rushes steps; let each step complete + --timeout help warns --force + --timeout abandons clean shutdown + +IN SCOPE: the vr1-dc0 model only -- 45 units, 36 machines/containers. +OUT OF SCOPE, and none of it is touched: the `controller` model and the juju controller +VM, the MAAS region and its database, the 9 MAAS machine RECORDS (released to Ready, not +deleted), the rack host, the OPNsense edge, the apt caching proxy, the snap proxy, the +tofu substrate, and dc1 in its entirety. + +POST-CHECKS (run immediately after, before recreating the model): + (a) `juju models` shows ONLY `controller` + (b) MAAS machine COUNT is unchanged and the 9 role nodes read `Ready` -- a DROP in + count is the cascade signature from the 2026-07-21 incident and means STOP + (c) `dc-cache-proxy.sh check dc0` and `dc-snap-proxy.sh check dc0` still PASS