openstack-caracal-dc-dc / runbooks /
..
README.md DOCFIX-177: make the skill and runbooks/README.md aware the VR1 DC-DC track exists 4 hours ago
appendix-A-troubleshooting.md addendum 39: devteam cluster-template 403 diagnosed (Public/Hidden flag -> clustertemplate:publish admin_api, BY DESIGN, no cloud change) -- DOCFIX-121 clientdocs template-creation guidance + DOCFIX-122 appendix-A symptom entry; L7 receipt re-recorded; ~/devteam-package re-rendered; stage5-for-devteam option logged (unruled) 2 days ago
appendix-B-asbuilt-version-lock.md clearing queue 5 days ago
appendix-C-identity-rbac.md Patches 7 days ago
appendix-D-magnum-trust-model.md Tenant file updates 8 days ago
d011-batch3-window-DRAFT.md DOCFIX-139..141: repo-rename sweep (ipv4->dc-dc) + repo-lint L9 self-reference guard 1 day ago
dc-dc-phase0-vcloud-prep.md DC-DC Stage 1 (dc-dc-phase0): first real execution + scaffold/tooling fixes 2 hours ago
dc-dc-phase1-office1-standup.md DOCFIX-163: Office1-local network module + OPNsense edge ownership (gaps #12+16) 8 hours ago
dc-dc-phase2-tofu-dc-substrate.md DOCFIX-175/176: OpenTofu state file security + VR1 teardown/rollback runbook 4 hours ago
dc-dc-phase3-maas-enlist-deploy.md DOCFIX-166: $DC parameterization for reenroll-hosts/carve-host-interfaces/phase-00-maas-standup (gap #15, CLI half) 8 hours ago
dc-dc-phase4-juju-bundle-per-dc.md DOCFIX-171: full-project sweep -- DC-DC runbook consistency fixes 6 hours ago
dc-dc-phase5-dr-failover-drill.md DOCFIX-171: full-project sweep -- DC-DC runbook consistency fixes 6 hours ago
dc-dc-phase6-designate-cos-magnum.md DOCFIX-167: bundle.yaml -- Designate reactivation + D-108 DR mechanism (gaps #6+14) 8 hours ago
dc-dc-teardown-rollback.md DOCFIX-175/176: OpenTofu state file security + VR1 teardown/rollback runbook 4 hours ago
ops-capi-recovery.md Post deploy changes 22 days ago
ops-restart-procedure.md Patches 7 days ago
ops-update-procedure.md DOCFIX-139..141: repo-rename sweep (ipv4->dc-dc) + repo-lint L9 self-reference guard 1 day ago
phase-00-teardown-maas-reset.md Patches 7 days ago
phase-01-bundle-deploy.md DOCFIX-169: fix stale bundle.yaml expectations in phase-01-bundle-deploy.md 7 hours ago
phase-02-vault-bringup.md Updated Runbook Files 22 days ago
phase-03-core-verify.md addendum 37: D-075 ADOPTED+EXECUTED -- dashboard VIP root redirect(/->302 /horizon)+no-autoindex via conf-enabled override (RedirectMatch backend-port leak caught in verify, replaced with per-dir rewrite); DOCFIX-118 phase-03 PER-REBUILD block + appendix-A "Index of /" entry 3 days ago
phase-04-network-carve.md Phase end updates 9 days ago
phase-05-octavia-enablement.md Phase end updates 9 days ago
phase-06-incloud-mgmt-cluster.md Script and file updates 9 days ago
phase-07-conductor-graft.md Patches 7 days ago
phase-08-workload-cluster-acceptance.md DOCFIX-170: full-project sweep -- VR0 runbook + OpenTofu doc fixes 6 hours ago
tenant-onboarding-v2-DRAFT.md DOCFIX-139..141: repo-rename sweep (ipv4->dc-dc) + repo-lint L9 self-reference guard 1 day ago
README.md

v1 Deploy Runbook -- VR0 DC0 Omega Cloud (Caracal 2024.1, IPv4)

Command-label convention

Every command block below is bracketed by bold labels, so a command line is never mistaken for surrounding prose (these render in GitBucket and read clearly in a raw editor):

  • RUN -- LOC -- the block CHANGES state; run it at LOC (e.g. jumphost, vault/0, jumphost -> magnum/0).
  • CHECK (read-only) -- LOC -- a read-only verification; safe to re-run.
  • GATE: -- a hard stop; do NOT proceed past the block unless the stated condition holds.
  • Expect: -- what a passing result looks like.
  • > CAUTION: -- marks a destructive, secret-handling, or irreversible step.

The deploy is a gated sequence: run phase-00 through phase-08 in order. Each phase ends in a hard gate (an explicit pass/fail check); do not start the next phase until the current gate passes. The two appendices are reference, not steps.

Conventions

  • RUN location. Every command block is tagged with where it runs: # RUN: jumphost (the vopenstack-jesse jumphost, with juju + the openstack CLI), # RUN: mgmt VM (the in-cloud CAPI management VM, reached over SSH), or a charm unit via juju ssh <unit> -- '...' </dev/null.
  • Gates. A line beginning GATE: or EXIT GATE is a stop-and-verify. Read-only verification precedes every mutation; destructive and secret-handling steps are individually gated, never batched.
  • Dynamic lookups. VIPs, project names, IDs, and version constellations are discovered at run time, not hardcoded. Site-specific literals are tagged ENV(...) for the eventual generalization pass toward Roosevelt.
  • Repo clone path. Never a literal (this repo has already been renamed/forked once -- DOCFIX-139/141). Set $REPO once per session to your clone's root (export REPO=$(git rev-parse --show-toplevel) if already cd'd into it, else export REPO=~/<wherever you cloned it>); every RUN block that needs the repo path below references $REPO, never a hardcoded name. repo-lint L9 fails a script/RUN-block/skill file that hardcodes the current repo's own directory name.
  • ASCII only. All runbook content is ASCII (a mod_wsgi UnicodeDecodeError lesson); keep it that way on edit.

Phases

# File Purpose Decisions
00 phase-00-teardown-maas-reset.md Destroy the model + reset MAAS to a clean rebuild KI-P3-001
01 phase-01-bundle-deploy.md Octavia PKI overlay + juju deploy + settle wait
02 phase-02-vault-bringup.md Vault init/unseal + cert cascade (PKI root) manual unseal = v1 std
03 phase-03-core-verify.md Settle, regenerate admin-openrc, verify Horizon
04 phase-04-network-carve.md Provider external network + IPAM reference
05 phase-05-octavia-enablement.md Enable Octavia (amphora) D-021
06 phase-06-incloud-mgmt-cluster.md In-cloud single-homed CAPI management cluster D-035
07 phase-07-conductor-graft.md Trustee domain-setup + graft the magnum-capi-helm driver D-031 / D-037 / D-042 / D-046 / D-047
08 phase-08-workload-cluster-acceptance.md End-to-end tenant cluster + acceptance bar D-011 (amended D-019)

VR1 DC-DC track (separate from the VR0 phases above)

The dc-dc-phase*.md runbooks below are a SEPARATE track from the VR0 phase-NN sequence above: they build the two-DC + Office1-headend buildout ("VR1"), per docs/dc-dc-buildout-design.md and D-100..D-110, with VR0's own testcloud host becoming DC1 (repurposed, not a second concurrent cloud). The master tracker -- current stage status, the full tooling gap register, exit gates -- is docs/dc-dc-deployment-workflow.md; read that first, not just this table. "Stage" numbering here matches that tracker, not this README's own VR0 phase numbers (they are two independent sequences that happen to share a repo).

Stage File Purpose Decisions
0 (no runbook -- docs/design-decisions.md) Decision ratification (D-100..D-110) D-100..D-110
1 dc-dc-phase0-vcloud-prep.md vcloud host prep + first real tofu apply (DC1 planes/Office1 pool/mesh links) D-101/D-102
2 dc-dc-phase1-office1-standup.md Office1 headend: MAAS region, OpenTofu, NetBox, GitBucket, Tailscale, Office1's own OPNsense edge D-103/D-106
3 dc-dc-phase2-tofu-dc-substrate.md OpenTofu builds each DC substrate (node-VM domains, OPNsense edge, MAAS vm_host) D-103
4 dc-dc-phase3-maas-enlist-deploy.md MAAS enlist / commission / deploy, per DC
5 dc-dc-phase4-juju-bundle-per-dc.md Juju controller + OpenStack bundle, per DC (high reuse from VR0's bundle.yaml) D-106/D-108
6 dc-dc-phase5-dr-failover-drill.md DR wiring (radosgw multisite, rbd-mirror) + failover drill D-108
7 dc-dc-phase6-designate-cos-magnum.md Designate, COS, Magnum/CAPI, per DC
-- dc-dc-teardown-rollback.md Teardown/rollback for this track's OpenTofu-provisioned infrastructure specifically -- NOT the same as phase-00-teardown-*.sh (D-061), which covers VR0's juju/MAAS-machine layer --

As of 2026-07-10: all seven Stage runbooks + the teardown/rollback runbook are WRITTEN, NONE EXECUTED. docs/dc-dc-deployment-workflow.md's tooling gap register tracks what's still open before each stage can run for real -- read it before starting Stage 1, not just this index.

Appendices

  • appendix-A-troubleshooting.md -- symptom -> cause -> fix index, keyed by D-NNN / DOCFIX-NNN / lesson.
  • appendix-B-asbuilt-version-lock.md -- charm channels, the CAPI version constellation, and the magnum-capi-helm driver pin.

History

This phase-NN set supersedes the earlier v1-do-doc-NN-* execution documents (and the older NN-*.md set and the deprecated/ folder), which were removed in the repo sanitation sweep. Git history preserves them.

  • ops-capi-recovery.md -- parking, restart, and LB repair for the CAPI/Magnum stack (post-deploy operations companion; not a deploy phase). Added 2026-06-10.
  • ops-restart-procedure.md -- full-cloud restart / recovery for planned windows and full power/network-loss events (DOCFIX-075). Indexed 2026-07-04 (was missing from this list since commit).
  • ops-update-procedure.md -- routine update window: Juju controller patch, model agents, in-channel charm refreshes (DOCFIX-086; policy: D-071). Added 2026-07-04.