# Changelog 2026-07-10 -- VR1 DC-DC teardown/rollback runbook (DOCFIX-176) No live infrastructure touched -- this is a new runbook only; nothing in `opentofu/` has ever been applied against real infrastructure in any session that authored this repo clone. Closes gap register item #19 (found alongside DOCFIX-175 while reviewing what's missing from the buildout ahead of tomorrow's execution). ## The gap `runbooks/phase-00-teardown-release.sh` / `-destroy.sh` (D-061) cover tearing down VR0's juju/MAAS-machine layer. Nothing covered the DIFFERENT layer this buildout also needs to be able to tear down: the OpenTofu-provisioned libvirt/MAAS-vm_host resources for DC1, DC2, and Office1. If Stage 1 or later fails partway through a real execution, there was no documented path back to a clean slate for this specific layer -- and no guidance on whether to even reach for a teardown vs. fixing forward, which is usually the safer, more appropriate response to a partial `tofu apply` failure. ## Fix New `runbooks/dc-dc-teardown-rollback.md`: - **Relationship to D-061**, stated explicitly rather than assumed: the same PRINCIPLE (clean up the higher-layer record before destroying the lower-layer resource underneath it) carries over, but the MECHANISM differs, because this repo's OpenTofu layer uses the plain `maas_vm_host` resource (not MAAS pod-composition), so D-061's specific `--keep-instance`/decompose-on-release failure mode doesn't directly transfer. Flagged as a residual, untested gap (this repo has never reached Stage 4 for any VR1 DC, so no real MAAS-enrolled machine has ever existed under a VR1 `vm_host` to test a release procedure against) rather than inventing untested MAAS CLI commands. - **Two paths**: Path A (scoped single-site teardown via `tofu`'s `-target` flag, with HashiCorp's own documented targeting-risk warning quoted and a mandatory untargeted `tofu plan` afterward to catch drift) and Path B (full VR1 teardown, plain `tofu destroy`, letting OpenTofu's own dependency graph sequence everything). - **Explicit dependency ordering for Path A**, since `maas_vm_host` has no Terraform-expressible dependency on the node-VM domains it's registered against (confirmed by reading `modules/maas-vm-host/main.tf` and `modules/node-vm/main.tf` directly -- the resource only references `power_address`, not a domain resource) and so would NOT be caught by a `-target` scoped to the storage/plane modules alone. - **Mesh-link handling**: explicit warning never to destroy a D-100 mesh-link leg during a single-site teardown (shared infrastructure with the far end), only when both endpoints are permanently going away. Also confirmed (by reading `modules/netem-link/main.tf` directly) that its `destroy`-time `local-exec` provisioner already correctly runs `tc qdisc del` -- no extra manual step needed for that piece specifically. - **Rollback decision tree**: four scenarios (nothing created yet, partial apply failure, suspected state corruption/loss, resources created outside OpenTofu via Stage 2's Option B manual `virt-install` path) with the default bias toward fix-forward over destroy, matching OpenTofu's own idempotent-convergence design rather than treating "something went wrong" as an automatic teardown trigger. - **State backup as Step 1 of every path**, cross-referencing DOCFIX-175 -- a destroy mutates state exactly like an apply does, so the same pre-mutation backup discipline applies. Every `tofu`/`terraform` CLI flag cited (`-destroy`, `-target`, the targeting-risk warning text) was verified against HashiCorp's own current documentation before being written (fetched 2026-07-10), not asserted from memory -- same discipline as `opentofu/README.md`'s other research sections. Cross-referenced from `opentofu/README.md` (new "Teardown / rollback" section) and `docs/dc-dc-deployment-workflow.md` (gap register item #19, CLOSED). ## Self-inflicted issue found and fixed mid-delivery (unrelated to this runbook, but in the same working session) While preparing this delivery's DOCFIX number, `bash scripts/ledger-scan.sh` was re-run and caught that the DOCFIX-174 changelog and its `docs/session-ledger.md` mirror had REINTRODUCED the exact false-positive class DOCFIX-174 itself had just fixed -- both quoted the illustrative example number as a literal string while narrating the incident, which re-inflated `ledger-scan.sh`'s reported next-free value. Caught by re-running the scan before assigning this delivery's own number rather than trusting the earlier fix was airtight; fixed by rewording both (see `docs/changelog-20260710-ledger-scan-self-inflicted-falsepositive-fix.md`'s own updated "It recurred immediately" section for the full account). No separate DOCFIX assigned for this -- folded into the DOCFIX-174 changelog itself since it corrects that same delivery before commit, not a new one. **Also corrected mid-delivery:** DOCFIX-175's first draft wrongly assumed Stage 3 ran the first real `tofu apply`; reading the runbooks directly showed Stage 1 does. Fixed across all of DOCFIX-175's files before this teardown-runbook delivery, so the "current plan" this runbook's own cross-references describe is accurate. ## Verification `bash scripts/repo-lint.sh`: 0 fail, 1 documented legacy warn. No script logic changed -- one new runbook, two doc cross-references, one README section. No test harness applies. `bash scripts/ledger-scan.sh`: confirms next-free is D-111/DOCFIX-176/BUNDLEFIX-013 after this delivery (was verified clean of the self-inflicted false positive before this number was assigned). REVERT: `git checkout HEAD~ -- runbooks/dc-dc-teardown-rollback.md opentofu/README.md docs/dc-dc-deployment-workflow.md` plus `git rm runbooks/dc-dc-teardown-rollback.md` if reverting past its introduction commit (safe -- no live infrastructure depends on this, nothing here has ever been applied against real state).