READ THIS BEFORE TYPING AN OPERATIONAL COMMAND. Created 2026-08-03 on operator direction after a session in which an ad-hoc juju destroy-model was run instead of the D-061 teardown scripts, took the dc0 Juju controller down, and cost hours of research the repo had already done and saved.
Operator, verbatim, and it is the charter of this file:
"The teardown scripts and write up after the incident that preceded D061 was documented and artifacts were deliberately created because of the findings and conclusions from that incident. We have created and saved these commands and artifacts to allow them to be reused in the future, like right now, when problems come up and a tested path would save time, troubleshooting, and research. The entire point of the repo is to provide a fully tested set of instructions, tested code, and troubleshooting tools. This is supposed to allow for a fully error free deployment of services. This is supposed to allow for troubleshooting and systems management with documentation and tools that do not require a build."
THE RULE THIS FILE EXISTS TO ENFORCE: an operational action resolves to an EXISTING tested artifact FIRST. Typing a raw juju / maas / tofu / virsh command for an operation is the ERROR SIGNAL, not the starting point -- it means the lookup was skipped. If no artifact exists, say so explicitly and propose building one; do not silently improvise and do not derive a command from --help when a tested invocation is on disk.
Scale of what is here, measured 2026-08-03: 93 scripts, 28 runbooks, 98 test harnesses, 65 scripts with their own tests/<name>/ harness. This is not a small library you can hold in your head. GREP IT.
Fastest lookups:
ls scripts/ | grep -i <topic> # is there a tool? ls tests/ | grep -i <topic> # is it harnessed? grep -rn "<verb>" runbooks/ docs/design-decisions.md # is there a ruled procedure?
| I want to... | USE THIS | Never |
|---|---|---|
| Tear down the juju MODEL (VR0/openstack) | scripts/phase-00-teardown-destroy.sh (full redeploy spine) or scripts/phase-00-teardown-release.sh (KEEP machines) -- D-061, harness tests/phase-00-teardown-d061 |
a bare juju destroy-model |
| Tear down the juju MODEL (VR1 per-DC) | runbooks/dc-dc-teardown-rollback.md Path M, which now carries the measured --force hazard |
a bare juju destroy-model |
| Tear down / rebuild a juju CONTROLLER | runbooks/dc-dc-teardown-rollback.md Path C (C.2b is the API-down path) |
kill-controller alone -- MEASURED not to work when the API is down |
| Tear down the OpenTofu SUBSTRATE | runbooks/dc-dc-teardown-rollback.md Paths A / B |
ad-hoc tofu destroy |
| Reset MAAS after a teardown | runbooks/phase-00-teardown-maas-reset.md |
-- |
Operator standard, verbatim: "When we use a script for a task we need to make sure that it is agnostic as possible since these are supposed to be tools that can be ran on any deployment without rewrites." This is SKILL.md hard rule 3 (dynamic lookups over hardcoded literals; unavoidable literals tagged and CENTRALISED in lib-net.sh / lib-hosts.sh, keyed by stable identity). Reviewed against it, the D-061 pair is HALF agnostic:
MAAS_PROFILE="${MAAS_PROFILE:-admin}", MODEL="${OPENSTACK_MODEL:-openstack}".lib-hosts.sh: HOSTS=(openstack0 openstack1 openstack2 openstack3), ORPHANS=(capi-mgmt), PROTECTED=(juju lxd tailscale), and --canary pins openstack0. The header claims the roster is "resolved live from maas admin machines read" -- it is not; only the machines JSON is live, the roster itself is a literal array.THE DEFECT THIS CREATES, and it is a safety defect, not a portability nicety. A host that does not resolve is a note, not a fail: if [ -z "$s" ]; then note "$h: already absent from MAAS"; continue; fi. FATAL stays 0, the safety gate passes, and the script proceeds to destroy-model --destroy-storage --force --no-wait --no-prompt anyway. Pointed at VR1 (OPENSTACK_MODEL=vr1-dc0 MAAS_PROFILE=dc0) every one of the four VR0 hostnames resolves to nothing, so the entire host-safety audit silently no-ops while the destructive step still fires -- and "already absent from MAAS" reads like a reassuring, normal line. This is the checker-that-cannot-fail class located in the safety rail itself.
CONSEQUENCE FOR USE TODAY: the D-061 pair is the tested path for VR0 and MUST NOT be pointed at a VR1 DC until the roster is derived. VR1 model/controller teardown goes through dc-dc-teardown-rollback.md Paths M / C.
FIX WHEN IT IS THE CURRENT STEP (logged, not executed -- hard rule 1): derive the roster from STABLE IDENTITY rather than hostnames -- VR1 nodes carry openstack-<site> and role tags, and MAAS auto-generated hostnames do not match ruled names anyway -- or site-key it through lib-hosts.sh; and make a roster that resolves to ZERO machines a FATAL, not a note. A teardown whose host audit found nothing has not proven safety, it has failed to look.
D-061 is the governing decision and it names the failure mode by flag combination: the damage occurred "with --destroy-storage --force, and AGAIN with --force --no-wait and NO storage flag at all". Both tested scripts ALWAYS pass a storage flag -- --destroy-storage on the destroy path, --release-storage on the release path. A destroy-model with neither is not a form this repo has ever validated.
| Question | Tested artifact |
|---|---|
| Is it safe to deploy? | scripts/preflight.sh -- THE single entry, do not run sub-gates piecemeal |
| Is the cloud healthy? | scripts/cloud-assert.sh (--capture for the BOM) |
| Does the DC egress work? | scripts/dc-egress-check.sh (layered; first failure is the cause) |
| Are the bundle invariants intact? | scripts/provider-bundle-check.py |
| Are the charm channels pinned? | scripts/channel_assert.py |
| Are the juju spaces right? | scripts/juju-spaces-check.sh |
| Is the repo deliverable-clean? | scripts/repo-lint.sh + scripts/run-tests-all.sh |
| Is OpenTofu valid? | scripts/opentofu-validate.sh |
| Service | Tool | Note | |||
|---|---|---|---|---|---|
| apt caching proxy | `scripts/dc-cache-proxy.sh <check\ | install\ | node> <dc0\ | dc1>` | site token is bare dc0/dc1 |
| apt full mirror | scripts/dc-mirror.sh |
historical arm, D-135 amendment (b); install is a strategy REVERSAL |
|||
| snap forward proxy | scripts/dc-snap-proxy.sh |
consume via juju snap-http-proxy/snap-https-proxy |
|||
| rack network layer | scripts/dc-rack-net.sh install <site> |
reboot-persistent legs + node DNS | |||
| host->site base leg | `scripts/site-baseleg.sh check\ | apply ` | first thing after a vcloud reboot |
| Job | Tool |
|---|---|
| Prove WHICH REGION a profile resolves to | scripts/maas-profile-assert.sh -- by rack identity; a machine count is NOT proof |
| Per-machine power config | scripts/maas-node-power.sh (pods are REFUTED for VR1) |
| Role / controller tags | scripts/maas-role-tags.sh check <site> |
| Region power key | scripts/maas-region-power-key.sh |
| Node interface carve | scripts/carve-host-interfaces.sh, scripts/dc-node-carve.sh |
| IPv6 plane carve / verify | scripts/dc-node-v6-carve.py, scripts/dc-node-v6-verify.sh |
| Plane IPAM incl. ULA retirement | scripts/dc-plane-ipam.sh |
| Region topology | scripts/dc-region-topology.sh |
| Job | Tool | |
|---|---|---|
| Expected-state credential register | scripts/creds-matrix.py (tiers 1-3; BLOCKING preflight P5) |
|
| Declared-credential audit | scripts/creds-audit.sh -- declaration-based, NOT evidence of completeness |
|
| Octavia amphora PKI verify / ROTATE | `scripts/octavia-pki.sh verify\ | reissue ` -- runs on the headend |
| Vault KV health | scripts/vault-kv-health.sh, scripts/vault-kv-inner-probe.sh |
scripts/tenant-onboard.sh, tenant-offboard.sh, tenant-assert.sh, tenant-acceptance.sh; phase-03..07 scripts for the service walk.
scripts/run-logged.sh <label> (as-executed log), scripts/ledger-scan.sh (open-work ground truth), scripts/record-audit.py, scripts/cloud-snapshot.sh.
It is a POINTER index, deliberately not a description of behaviour -- behaviour lives in each script's header, which is the authority. When a tool is added or its job changes, add or fix the ROW. A row that names a script which no longer exists is a defect; regenerate the inventory with:
ls scripts/ ; ls -d tests/*/ # what exists, and what is harnessed
If you cannot find a tool for the job, that is a FINDING, not a licence to improvise. Say so, propose the tool, and get the operator's ruling -- the same standard every other gap in this repo is held to.