# DC-DC Phase 6 -- Designate, COS, Magnum/CAPI (per DC) (Stage 7)

Complete the per-DC service set: reactivate Designate (cloud-internal DNS), stand
up COS observability, and bring up the Magnum/CAPI management + workload cluster
path -- once per DC, against that DC's OWN Juju model and Keystone. This is the
LAST per-DC build stage before the buildout is functionally complete in both
DCs; DR wiring (Stage 6 / Phase 5) is a peer stage this runbook does not
re-verify beyond confirming its gate passed.

This runbook is written PREP-ONLY: no DC1/DC2 cloud exists or is reachable this
session. Every command below is the procedure to run LATER, once each DC has a
live Charmed OpenStack deployment and its own Juju controller (Stage 4 / Phase 4
in `docs/dc-dc-buildout-design.md`). Every literal that depends on that future
live state is marked `MEASURE` and must be resolved from real output at
execution time -- nothing here is carried over from the VR0 DC0 as-built values
quoted for reference.

**Entry condition:** Stage 6's gate (DR wiring / failover drill) has passed for
the DC you are about to work on. This runbook does not re-verify Stage 6; it
only requires that gate closed before starting.

Decisions: **D-106** (Designate reactivation -- supersedes D-019, reactivates
D-008), **D-008** (original DNS architecture / bootstrap order, now reactivated
verbatim), **D-019** (the v1 Designate deferral D-106 supersedes; its root
cause is what this stage closes), **D-021** (the amphora-pipeline constraint
whose root cause is the D-019 no-DNS posture -- resolved here), **D-105** (CAPI
mgmt-cluster placement + COS/Magnum scope, per DC, carries D-035 and D-046),
**D-046** (Magnum trustee domain-setup re-run trap), **D-101** (IPv6 family
matrix -- gates whether AAAA records are populated), **D-110** (numbering; no
new D-NNN is assigned by this runbook).

Troubleshooting: `runbooks/appendix-A-troubleshooting.md` for anything charm/
driver-specific inherited from phase-06/07/08 below; this runbook does not
duplicate that index.

---

!!! D-046 TRAP -- READ BEFORE TOUCHING MAGNUM, EVERY TIME, PER DC:
    "the magnum charm action `domain-setup` is MANUAL, not automatic; magnum
    reports active / 'Unit is ready' regardless of whether it has run" (D-046).
    Per D-105: "The Magnum `domain-setup` action is re-run after every
    teardown / redeploy per DC (D-046 trap: magnum reports 'Unit is ready'
    regardless)." Magnum's unit status is NOT evidence the trustee domain
    exists. Re-run and re-assert Step 7.0 of `phase-07-conductor-graft.md`
    (reproduced as Step 6.3 below) after ANY teardown/redeploy of either DC's
    magnum unit -- do not skip it because the unit looks healthy.

!!! BUNDLE.YAML IS A BUILT SURFACE -- this runbook's Step 1 changes it.
    Per this repo's standing discipline ("grep design-decisions for the
    governing D-NNN before ANY change to a built surface"): D-106 is the
    governing decision and its Status line reads ADOPTED (2026-07-09
    operator ruling), so the edit below IS authorized in principle. That
    is NOT a license to wave the edit through unreviewed -- it is still a
    verify-before-mutate, individually-gated repo change like any other
    built-surface edit (repo-lint clean, `preflight.sh` PASS before any
    live apply, changelog entry with a revert). Do not fold it into a
    "just add designate" one-liner; read Step 1 in full before editing.

!!! COS MECHANISM IS UNDETERMINED -- do not invent it. D-105 rules the SCOPE
    (COS in-scope, per-DC, no Office1 roll-up) but no D-NNN or design doc in
    this repo specifies the COS deployment MECHANISM (bundle vs. separate
    Juju model, exact channel, cross-model-relation topology). Step 10 below
    presents the standard options for the operator to pick from at execution
    time -- it does not pre-select one. (Corrected 2026-07-29: this read "Step 5",
    which is the Designate step, not the COS step.)

---

## Sequence (run ONCE PER DC -- DC1 first, then DC2, independently; do not
## assume DC2 repeats DC1's literals -- each DC has its own Keystone)

```
0.  Entry gate + DC context            (read-only: confirm Stage 6 gate + which
                                         DC's controller/model you are targeting)
1.  bundle.yaml edit (Designate)       [REPO CHANGE: reviewed once, applies to
                                         both DCs -- gated behind D-106]
2.  Static /etc/hosts bootstrap        (D-008 step 1, per DC)
3.  os-public-hostname per API charm   [MUTATION: juju config, per DC]
4.  Vault issues FQDN-SAN certs        [MUTATION: cert cascade, per DC]
5.  Verify designate present + active,  (read-only assertion, then juju config
    then CONFIGURE it                    only -- NOT a deploy; designate ships
                                         in the bundle and Stage 5 installs it.
                                         D-106 ruling note 2026-07-27)
6.  Designate zone + A/AAAA records    [MUTATION: per DC, from measured values]
7.  Neutron default_dns_domain/        [MUTATION: per DC]
    dns_servers -> Designate
8.  Tenant subnets --dns-nameserver    [MUTATION: per DC, incl. capi-mgmt-subnet]
9.  Verify D-008/D-106 bootstrap       (read-only: resolution + cert validity)
10. COS observability                  [DECISION POINT + MUTATION, per DC]
11. CAPI mgmt cluster                  (reuse phase-06-incloud-mgmt-cluster.md,
                                         per DC, DC-parameterized)
12. Magnum conductor graft             (reuse phase-07-conductor-graft.md, per
                                         DC -- D-046 trap, Step 6.3 below)
13. Workload cluster acceptance        (reuse phase-08-workload-cluster-
                                         acceptance.md, per DC)
    -> GATE (this doc) -> Stage 7 exit; repeat 0-13 for the other DC if not
       already done
```

---

## Command-label convention
Every command block below is bracketed by bold labels, so a command line is never mistaken
for surrounding prose (these render in GitBucket and read clearly in a raw editor):
- **RUN -- LOC** -- the block CHANGES state; run it at LOC (e.g. `jumphost`, `vault/0`, `jumphost -> magnum/0`).
- **CHECK (read-only) -- LOC** -- a read-only verification; safe to re-run.
- **GATE:** -- a hard stop; do NOT proceed past the block unless the stated condition holds.
- **Expect:** -- what a passing result looks like.
- `> CAUTION:` -- marks a destructive, secret-handling, or irreversible step.

---

## Step 0 -- Entry gate + DC context (READ-ONLY)

MEASURE, never assume: which DC you are working, which Juju controller/model
fronts it, and which admin credentials scope to it. Stage 4 (`docs/dc-dc-
buildout-design.md` Phase 4) is where each DC's controller and Keystone came
into existence; this runbook does not know that stage's naming choices in
advance (they are not yet made this session) -- confirm them from that stage's
own as-built record before substituting `$DC`, `$DC_CONTROLLER`, `$DC_MODEL`
below.

**CHECK (read-only) -- jumphost / Office1 operator seat**
```bash
# Set once per DC-pass through this runbook -- MEASURE from Stage 4's as-built
# record, never guess a naming scheme:
#   DC=vr1-dc0|vr1-dc1            # the D-119 region-qualified selector. Bare
#                                 # dcN is REJECTED loudly by lib-net/lib-hosts.
#   DC_CONTROLLER=<this DC's own Juju controller name, from Stage 4>
#   DC_MODEL=<this DC's OpenStack model name, from Stage 4 -- may or may not be
#             literally "openstack"; VR0/DC0 used "openstack" but VR1 per-DC
#             naming was not decided in this session>
# DERIVE the two D-008 FQDN labels from the selector -- NEVER interpolate $DC
# itself into a name (DOCFIX-205). The D-119 SELECTOR token and the D-008 <dc>
# LABEL are different strings: "omega.${DC}.vr1..." expands to
# "omega.vr1-dc0.vr1..." and carries the region TWICE. Same shell parameter
# expansions octavia-pki.sh uses; verified for vr1-dc0, vr1-dc1 and vr0-dc0.
DC_REGION="${DC%%-*}"      # vr1-dc0 -> vr1   (D-008 <region>)
DC_DNS_LABEL="${DC#*-}"    # vr1-dc0 -> dc0   (D-008 <dc>, 0-indexed per D-117)
# <cloud> label + base domain from the ONE place they live: a rebuild that renames the
# estate edits scripts/lib-identity.sh only (2026-07-30).
source "$REPO/scripts/lib-identity.sh" || { echo "FAIL: cannot read scripts/lib-identity.sh"; exit 1; }
DC_ZONE="${CLOUD_NAME}.${DC_DNS_LABEL}.${DC_REGION}.${CLOUD_DOMAIN}"
echo "$DC_ZONE"            # EXPECT omega.dc0.vr1.cloud.neumatrix.local (vr1-dc0)

juju controllers                 # confirm DC_CONTROLLER is listed and reachable
juju switch "${DC_CONTROLLER}:${DC_MODEL}"
juju status -m "${DC_MODEL}" --format=short | grep -vE 'active|idle' || echo "all active/idle"
```
**GATE:** the DC's controller is reachable, its model shows all charms
`active/idle` (Stage 4/5/6 already delivered this), and you have confirmed
which physical DC you are about to mutate. Do not proceed on an ambiguous
target.

**CHECK (read-only)**
```bash
# Confirm Stage 6 (DR wiring / failover drill) closed for this DC -- this
# runbook's stated entry condition. Reference the Stage 6 runbook's own EXIT
# GATE for the exact check; it is not reproduced here (sibling-stage content).
```
**GATE:** Stage 6's own EXIT GATE has passed for this DC.

---

## Step 1 -- bundle.yaml edit: reactivate Designate (REPO CHANGE, gated behind D-106)

### 1.1 -- What is currently in the repo (measured, not paraphrased)

**CORRECTED 2026-07-29 per the D-106 RULING NOTE of 2026-07-27.** This section
previously ended "There is no `designate:` or `designate-bind:` application block
anywhere in `bundle.yaml` today", which CONTRADICTED section 1.2 below and the
bundle itself. The ruling names this defect directly: ":172 states 'There is no
`designate:` or `designate-bind:` application block anywhere in ...' while :178
states 'Designate is deployed in-bundle in each DC' ... Both cannot be true; the
bundle settles it, and DOCFIX-167 put designate there on 2026-07-10."

**MEASURED STATE OF `bundle.yaml` TODAY -- designate IS in the bundle.** Four
application blocks plus eight relations, all of whose peers Stage 5 creates:

| Application | Charm / channel | Placement | Where |
|---|---|---|---|
| `designate` | `designate` `2024.1/stable`, `num_units: 1` | `to: [lxd:0]` | `bundle.yaml:853` |
| `designate-mysql-router` | `mysql-router` `8.0/stable` | subordinate | `bundle.yaml:872` |
| `designate-bind` | `designate-bind` `2024.1/stable`, `num_units: 1` | `to: [lxd:2]` | `bundle.yaml:881` |
| `designate-hacluster` | `hacluster` `2.4/stable`, `cluster_count: 1` | subordinate | `bundle.yaml:963` |

(Line numbers are as of 2026-07-29; grep the app names, not the numbers, if the
bundle has moved since.)

Note `designate`'s provides-side binding is `dnsaas: provider-public`, not
`public` (see 1.5) -- which is why it does not join provider-bundle-check's
`EXPECT_PUBLIC_VIP` count of 11.

**HISTORICAL -- the pre-D-106 state, retained because it is what the decision
reverses.** Before DOCFIX-167 (2026-07-10), `bundle.yaml` recorded the D-019
deferral in its description block (`D-019 (supersedes D-008) Designate deferred to
v2`) and carried the subordinate as a comment rather than a live entry
(`# v2-deferred (D-019): designate-hacluster: ...`);
`runbooks/phase-01-bundle-deploy.md` still states the VR0 dry-run expectation
`NO vault:ha, NO designate (D-019)`. That phase-01 line is VR0/testcloud scope and
is NOT a claim about this bundle's current VR1 content -- do not read it as one.

**CONSEQUENCE FOR THIS RUNBOOK, RULED:** designate lands as INERT INFRASTRUCTURE
at Stage 5 -- it deploys with the bundle and does nothing, because
`os-public-hostname` is set in no deploy artifact. Stage 7 (this runbook) does not
install it. Step 5 below is a CONFIGURE step, not a deploy step.

### 1.2 -- The governing decision, quoted

D-106 (Status: ADOPTED 2026-07-09) states:
> Designate is deployed in-bundle in each DC, backend `designate-bind` (bind9,
> the Charmed-OpenStack-native backend, lowest delta).

and the reactivated bootstrap order, quoted verbatim (this is D-008's original
text, reactivated by D-106 without change):
> static /etc/hosts bootstrap -> `os-public-hostname` per API charm -> Vault
> issues FQDN-SAN certs -> Designate zone plus A/AAAA -> Neutron
> `default_dns_domain` and `dns_servers` -> tenant subnets with
> `--dns-nameserver`.

D-106 also carries a naming instantiation, not a new invention -- this is the
one place a "zone name" appears in this runbook, and it is cited from the
decision text itself, not invented:
> Naming per D-008 instantiated with the VR1 labels:
> `<service>.omega.dc1.vr1.cloud.neumatrix.local` and `...dc2.vr1...`.
> Per-DC INDEPENDENT zones (independent Keystones), served with A and AAAA
> records (AAAA for the v6-enabled planes per D-101).

**THE QUOTED LABELS ABOVE ARE SUPERSEDED -- do not paste them (DOCFIX-205).** The
quote is D-106's ratified 2026-07-09 text and is reproduced unchanged, but D-117
(ADOPTED 2026-07-13) retired the repo's `dc1`/`dc2` labels in favour of
`dc0`/`dc1`, naming this supersession in its own Status line. The VR1 zones are
therefore `omega.dc0.vr1.cloud.neumatrix.local` (substrate `vr1-dc0`) and
`omega.dc1.vr1.cloud.neumatrix.local` (substrate `vr1-dc1`). Use the derived
`$DC_ZONE` set in Step 1 rather than typing either. D-106 now carries a D-117
annotation recording this.

### 1.3 -- The reversal is bigger than "add one application" -- flag this honestly

D-106's bootstrap order includes `os-public-hostname` per API charm as its
SECOND step. `bundle.yaml`'s current header records the opposite posture,
under its own B5 label (not a separate D-NNN -- a 2026-06-01 bundle-cleanup
item):
```
# Endpoints:        IP-ONLY -- os-public-hostname dropped on all API charms; the dual VIPs ARE the
#                   catalog endpoints (public 10.12.4.N / internal+admin 10.12.8.N). Vault issues
#                   per-VIP IP-SAN certs. No control-plane DNS dependency. (B5)
```
D-106's own text explicitly names `os-public-hostname` per API charm as part
of the order it reactivates, so re-adding it is NOT an inferred scope
addition -- it is in the ratified decision text itself. But it IS a second,
larger surface than "add designate": every API charm's `os-public-hostname`
option goes from unset back to a real FQDN, and Vault's cert issuance moves
from per-VIP IP-SAN back to FQDN-SAN. Treat this as two coupled changes under
one D-106 umbrella, not one:
1. Add `designate` + `designate-bind` (+ `designate-hacluster`) applications
   and their relations.
2. Re-add `os-public-hostname: <fqdn>` to every clustered API charm's
   `options:` block (the same charm list D-020 already dual-VIPs: keystone,
   glance, neutron-api, nova-cloud-controller, placement, cinder, octavia,
   barbican, magnum, openstack-dashboard, ceph-radosgw).

If, on review, the operator wants item 2 scoped or sequenced differently from
item 1 (e.g. reactivate DNS-serving without yet flipping every charm's
hostname), that is a real design fork -- log it as a proposal against D-106
rather than deciding it inside this runbook.

### 1.4 -- Keep bundle.yaml DC-agnostic; push per-DC literals to an overlay

`bundle.yaml` is ONE shared file used to deploy both DC1 and DC2 (D-101: DC1
inherits DC0's v4 layout unchanged; DC2 gets its own NetBox-assigned
supernet, still pending -- gap #3 in `docs/dc-dc-deployment-workflow.md`'s
tooling gap register). The new `os-public-hostname` values are NOT DC-agnostic
-- they embed `dc1` or `dc2` per D-106's naming convention. Do not hardcode a
DC-specific FQDN into the shared `bundle.yaml`. Follow the existing overlay
pattern already established for secrets (`overlays/${DC}-octavia-pki.yaml` --
itself made PER-DC by the D-109 amendment, gitignored by glob; Step 1.0 of
`phase-01-bundle-deploy.md`) and add one per-DC hostnames overlay:
`overlays/vr1-dc0-hostnames.yaml`, `overlays/vr1-dc1-hostnames.yaml` (region-
qualified: the `dc1`/`dc2` spelling is retired -- item 3.1 measured "DC1" meaning
`vr1-dc0` on one surface and `vr1-dc1` on another), each setting
only the `os-public-hostname` options for that DC, applied at deploy time
with an additional `--overlay` alongside whichever overlays that DC's deploy
already uses. This is a proposed pattern, not yet reviewed -- present it at
the same review gate as the rest of Step 1, do not silently adopt it.

### 1.5 -- The application/relation shape (done 2026-07-10 -- bundle.yaml already carries this)

**This step is CLOSED, not a template anymore.** `bundle.yaml` now carries
real `designate`/`designate-bind`/`designate-mysql-router`/`designate-
hacluster` application blocks + 8 relations, researched directly against
`charm-designate`'s real `metadata.yaml` (not the standard-pattern guess
this runbook originally sketched here). Two things this runbook's first
draft got wrong were caught and fixed during that research, worth knowing
before you run `--dry-run`:
- Designate's real provides-side endpoint is `dnsaas` (interface
  `designate`), **not** `public` like the other API charms in this bundle.
- `dns-backend` (interface `bind-rndc`), `ha` (interface `hacluster`), and
  `coordinator-memcached` (interface `memcache`, relating to the existing
  `memcached` principal) were all confirmed from the charm's own source.
  `shared-db`/`identity-service`/`amqp`/`certificates` still follow the
  standard pattern (very high confidence, matches 10 other charms) but
  were NOT individually re-fetched for designate -- confirm with
  `juju deploy --dry-run` before treating this as fully final.

See `docs/archive/changelogs/changelog-20260709-designate-cinderbackup-rbdmirror.md` for the
full record. What's still genuinely open: the `os-public-hostname` overlay
(Step 1.4 above) -- `bundle.yaml` deliberately stays DC-agnostic, so that
overlay is still this runbook's own job at real execution time, not
pre-built.

**CHECK (read-only) -- jumphost, once bundle.yaml + overlays are edited**
```bash
bash scripts/repo-lint.sh          # expect 0 fail
bash scripts/preflight.sh          # expect PREFLIGHT: PASS (incl. Charmhub channel assert)
```
**GATE:** repo-lint clean; preflight PASS. Only after this passes do the live
per-DC steps become eligible. (Corrected 2026-07-29: this read "does Step 5 (the
live per-DC apply) become eligible" -- Step 5 is no longer an apply, see the
D-106 ruling note; the applies live in Steps 3/6/7/8.) Deliver this edit with a changelog
entry (what changed, why -- D-106, how to revert) per this repo's standing
delivery rule, same as any other script/runbook/bundle change.

---

## Step 2 -- Static /etc/hosts bootstrap (D-008 step 1, per DC)

MEASURE this DC's own hostnames and IPs from its live `juju status` /
`maas admin machines read` output -- do not carry over DC0's `/etc/hosts`
entries or VR0's IP ranges. Per D-101, DC1's v4 layout equals DC0's
(deliberate, ratified reuse -- not an assumption), so DC1's measured values
may look identical to DC0's on paper; measure them live anyway. DC2's values
are, per gap #3, not yet assigned in NetBox as of this writing -- if DC2's
literals still are not populated by the time you reach this step, STOP and
resolve that gap first (do not invent a DC2 supernet here).

**CHECK (read-only) -- this DC's jumphost-equivalent seat**
```bash
juju status -m "${DC_MODEL}" --format=yaml | grep -E 'public-address|hostname'
```
**RUN -- every host + LXD container in this DC's model (mirrors phase-00's
Pattern A hosts, D-008 step 1's "jumphost + all openstack hosts + all LXD
containers" scope, but for THIS DC)**
```bash
# Populate /etc/hosts on each unit with THIS DC's own measured
# <fqdn> <ip> pairs, one line per API charm, using D-106's naming:
#   <service>.${DC_ZONE}   -- derived in Step 1; NEVER "omega.${DC}.vr1..."
# Do not template a value you have not just measured.
```
**GATE:** every host/LXD container in this DC resolves every other API
charm's FQDN via `/etc/hosts` (the D-008 bootstrap floor, before Designate
itself exists).

---

## Step 3 -- os-public-hostname per API charm (D-008 step 2, per DC)

**RUN -- jumphost, per charm (repeat for every clustered API charm listed in Step 1.3)**
```bash
juju config -m "${DC_MODEL}" keystone os-public-hostname="keystone.${DC_ZONE}"
# ... repeat per charm: glance, neutron-api, nova-cloud-controller, placement,
#     cinder, octavia, barbican, magnum, openstack-dashboard, ceph-radosgw
```
**CHECK (read-only) -- jumphost**
```bash
juju status -m "${DC_MODEL}" --format=short | grep -vE 'active|idle' || echo "all active/idle"
```
**GATE:** all charms return to `active/idle` after the config change settles
(a hostname-only config change is non-disruptive on this charm family, but
confirm rather than assume for this DC).

---

## Step 4 -- Vault issues FQDN-SAN certs (D-008 step 3, per DC)

This DC's Vault is an INDEPENDENT root (D-109) issuing its own certs -- do not
reuse a cert or CA from the other DC or from VR0 DC0. Trigger the certificates
relation to re-issue now that `os-public-hostname` is set; the charm-native
cert cascade (Vault's `certificates` interface) picks up the new SAN
automatically once the relation re-fires -- no manual CSR step is expected on
this charm family (confirm this is still true for the designate charm's own
certificate -- its `certificates` relation ships in the bundle and is already up
by the time you reach here, so there is nothing to relate at Step 5; corrected
2026-07-29, this read "once Step 5 relates it").

**CHECK (read-only) -- jumphost, per charm**
```bash
juju ssh -m "${DC_MODEL}" keystone/leader \
  "sudo openssl x509 -in /etc/apache2/ssl/keystone_1/cert -noout -text | grep -A1 'Subject Alternative Name'" </dev/null
# expect the FQDN now present in the SAN list, alongside the existing VIP IP-SANs
# (D-020 dual-VIP posture is not being reversed -- IP-SANs stay, FQDN-SAN is added)
```
**GATE:** every clustered API charm's live cert carries the DC's own FQDN in
its SAN. Do not proceed to Designate zone population (Step 6) until this
holds -- Designate resolving a name to a VIP whose cert does not cover that
name reproduces the exact D-019/D-021 failure mode this stage exists to close.

---

## Step 5 -- Verify designate is present and active, then configure it (NOT a deploy step)

**REWRITTEN 2026-07-29 UNDER THE D-106 RULING NOTE OF 2026-07-27 (GA-R5).**
Question put to the operator, and its selected option, verbatim: **"Accept at
Stage 5; rewrite Stage 7 Step 5 to configure-not-deploy (Recommended)"**. The
ruled shape, quoted: "Designate lands as INERT INFRASTRUCTURE at Stage 5 ...
Step 5 is rewritten from 'deploy' to 'verify present and active, then
configure'. Its diff-gate is replaced by a presence-and-state assertion."

**DO NOT RUN `juju deploy` IN THIS STEP.** The four designate applications and
their eight relations ship in `bundle.yaml` (see 1.1) and every relation peer is
created at Stage 5, so Stage 5's own bundle deploy already installed them. They
arrive FUNCTIONALLY INERT because `os-public-hostname` is set in no deploy
artifact -- which is the state this stage exists to resolve, and the work is
Steps 3/4/6/7/8, not a second deploy here.

What was here before, and why it could not work:
- The step was written as a deploy with a gate requiring "the diff shows ONLY
  the new designate/... applications and their relations being added". That is
  UNSATISFIABLE against a model that already has them -- the ruling names it as
  the surface defect.
- Both commands passed `--overlay "overlays/${DC}-machines.yaml"` UNGUARDED, and
  the second was the real apply against a LIVE model. `overlays/vr1-dc0-machines.yaml`
  did not exist, so a dc0 run would have errored out at `juju deploy` while
  `preflight.sh`'s P2 -- which guards the same path with `[ -f ]` -- went green.
  That gap is now closed from BOTH sides: the deploy commands are gone from this
  step, and `overlays/vr1-dc0-machines.yaml` was created 2026-07-29 (item 3.8), so
  the P2 guard no longer silently skips a file the deploy requires.
- It named `overlays/octavia-pki.yaml`, renamed PER-DC to
  `overlays/${DC}-octavia-pki.yaml` by the D-109 amendment.

The per-DC deploy command itself belongs to Stage 5
(`runbooks/dc-dc-phase4-juju-bundle-per-dc.md`), not to this runbook. Do not
reintroduce a copy of it here.

### 5.1 -- Presence-and-state assertion (replaces the old diff gate)

!!! COMMANDS BELOW ARE UNVERIFIED FROM THE AUTHORING HOST. `juju` is ABSENT on
    vcloud (measured 2026-07-27, register row U9: `juju` at `/snap/bin/juju` on
    voffice1, absent on vcloud), so nothing in this step could be executed while
    it was written. Run them from the Plane-2 host (voffice1) per D-128, and
    treat any flag that errors as an authoring defect to fix here, not as a
    reason to improvise a different command.

**CHECK (read-only) -- from the Office1 headend (voffice1)**
```bash
# All four applications must ALREADY exist in this DC's live model.
# One `juju status --format=yaml` carries applications AND their relations, so it
# answers all three questions below without relying on a separate flag.
juju status -m "${DC_MODEL}" --format=yaml > /tmp/designate-presence.yaml

# 1. presence:
grep -E '^\s+(designate|designate-bind|designate-mysql-router|designate-hacluster):' \
  /tmp/designate-presence.yaml

# 2. workload/agent state, human-readable:
juju status -m "${DC_MODEL}" designate designate-bind

# 3. the in-bundle relations, designate side:
grep -i -A3 designate /tmp/designate-presence.yaml | grep -i 'relation\|endpoint'
```
**GATE (presence AND state -- both, not either):**
1. `designate`, `designate-bind`, `designate-mysql-router` and
   `designate-hacluster` are all PRESENT in the model. If any is MISSING, that
   is a Stage-5 deploy defect, not work for this step -- STOP and reconcile
   against Stage 5's own gate; do not deploy it from here.
2. All four are `active/idle`.
3. `designate:dns-backend - designate-bind:dns-backend` and
   `designate:identity-service - keystone:identity-service` are up.

An `active/idle` designate at this point means the CHARM is healthy. It does NOT
mean DNS works -- the feature is inert until the configuration below and Steps
6/7/8 land. Do not read unit status as evidence of function (same class of trap
as the D-046 magnum `domain-setup` warning at the top of this runbook).

### 5.2 -- Configure designate

Steps 2/3/4 have by now set `os-public-hostname` per API charm and re-issued
Vault FQDN-SAN certs; Steps 6/7/8 own the zone + A/AAAA records, Neutron's
`default_dns_domain`/`dns_servers`, and tenant subnet `--dns-nameserver`. What
remains to THIS step is designate's own charm configuration -- the settings that
make the already-running charm serve this DC's zone.

**MEASURE THE OPTION NAMES, DO NOT CITE THEM FROM HERE.** This runbook
deliberately names no designate charm config key: no such key is recorded in any
ratified decision or measured capture in this repo, and 1.5 already records that
several designate endpoints were confirmed only by reading the charm's own
source. Read the real option set at execution time and configure from that:

```bash
# THE authoritative read: for a deployed application this returns the full option
# set -- real key names, current values, defaults and descriptions. It is
# sufficient on its own; nothing else is needed to drive this step.
juju config -m "${DC_MODEL}" designate
```
If you want the charm's raw `config.yaml` as well, the unit-local path is
`/var/lib/juju/agents/unit-designate-*/charm/config.yaml` -- UNVERIFIED, recorded
as a fallback only. If it is not there, do not go hunting: `juju config` above is
the supported surface and already answers the question.
**GATE:** designate returns to `active/idle` after any `juju config` change, and
every key you set was read from the charm's own `config.yaml` in this session --
not carried from memory or from another cloud. If a value you need is a per-DC
literal (a nameserver address, an FQDN), MEASURE it live per this runbook's
standing rule; the zone NAME is the one literal cited from decision text (D-106),
and it is cited in Step 6 where the zone is created.

If nothing in the charm's config set needs to change for this DC, say so
explicitly in the as-executed log and proceed -- an empty configure is a valid
outcome of this step and is not a reason to invent one.

---

## Step 6 -- Designate zone + A/AAAA records (D-008 step 4, per DC)

Per-DC INDEPENDENT zone (D-106). The zone name itself is the one place this
runbook cites a literal beyond a live measurement, and it is cited from D-106's
own ratified naming text as corrected by D-117, not invented: `$DC_ZONE` (derived
in Step 1 = `omega.dc0.vr1.cloud.neumatrix.local` for `vr1-dc0`).
The A/AAAA record VALUES are NOT cited from any decision text -- they are this
DC's real, live, measured VIP addresses. Populate them from what Step 3/4 just
proved live, never from a planning number.

**RUN -- jumphost (designate-scoped or admin, per this DC's own admin-openrc)**
```bash
openstack zone create --email hostmaster@neumatrix.local \
  "${DC_ZONE}."

# per API charm, MEASURE the live provider VIP (A) and, if this DC's GUA plane
# is up (D-101), the live provider-public GUA address (AAAA) -- do not invent
# either:
KEYSTONE_VIP_V4=$(openstack ... )   # MEASURE: this DC's live keystone provider VIP
openstack recordset create "${DC_ZONE}." \
  keystone --type A --records "$KEYSTONE_VIP_V4"

# AAAA only if D-101's v6-enabled planes are confirmed up for this DC -- gate
# on that confirmation, do not assume:
KEYSTONE_VIP_V6=$(openstack ... )   # MEASURE: this DC's live GUA provider address
openstack recordset create "${DC_ZONE}." \
  keystone --type AAAA --records "$KEYSTONE_VIP_V6"
# repeat per API charm
```
**GATE:** every API charm has an A record resolving to its own live provider
VIP; AAAA records exist wherever D-101's GUA/dual-stack planes are confirmed
up for this DC (not before -- an AAAA record pointing at an address that
is not yet live is worse than no record).

---

## Step 7 -- Neutron default_dns_domain / dns_servers -> Designate (D-008 step 5, per DC)

**RUN -- jumphost**
```bash
DESIGNATE_VIP=$( ... )   # MEASURE: this DC's live designate-bind / designate API VIP
juju config -m "${DC_MODEL}" neutron-api \
  dns-domain="${DC_ZONE}." \
  dns-servers="${DESIGNATE_VIP}"
```
**GATE:** `neutron-api` returns to `active/idle`; `openstack network show
<ext-net>` (or the relevant config surface) reflects the new dns_domain.

---

## Step 8 -- Tenant subnets --dns-nameserver (D-008 step 6, per DC)

Every existing tenant subnet in this DC created BEFORE Designate existed (most
notably `capi-mgmt-subnet` from phase-06 Step 6.1, which set
`--dns-nameserver 1.1.1.1 --dns-nameserver 1.0.0.1` under the D-019 "no
cloud DNS, tenants use public resolvers" posture) needs updating, and every
subnet created AFTER this point should use the Designate VIP from the start.

**RUN -- jumphost, per existing tenant subnet**
```bash
DESIGNATE_VIP=$( ... )   # MEASURE: same value as Step 7
openstack subnet set --dns-nameserver "$DESIGNATE_VIP" capi-mgmt-subnet
# clear the old public-resolver entries only if the subnet update replaces
# rather than appends -- confirm openstack-client's actual semantics for
# --dns-nameserver on `subnet set` live, do not assume append vs. replace
```
**GATE:** `openstack subnet show capi-mgmt-subnet` shows the Designate VIP as
the (or a) DNS nameserver.

---

## Step 9 -- Verify the D-008/D-106 bootstrap end to end (read-only)

**CHECK (read-only) -- jumphost -> a tenant VM in this DC (e.g. capi-mgmt-v2)**
```bash
source ~/capi-mgmt-net.env   # this DC's own env file, per phase-06 Step 6.2 -- confirm it is THIS DC's, not the other DC's stale file
ssh ... ubuntu@"$MGMT_FIP" bash -s <<'REOF'
getent hosts keystone.${DC_ZONE}   # substitute the real DC label
dig +short AAAA keystone.${DC_ZONE}  # if AAAA populated
curl -sk https://keystone.${DC_ZONE}:5000/v3 -o /dev/null -w '%{http_code}\n'
REOF
```
**GATE:** the FQDN resolves (A, and AAAA where populated); the HTTPS call
against the FQDN succeeds with a cert the client trusts (Vault's CA is this
DC's own root, per D-109 -- the tenant VM/OS trust store must carry it, same
as any other charm's client already does). This is the concrete proof the
D-019/D-021 root cause (metal-only charms pulling a public FQDN endpoint that
does not resolve/route, behind a cert with no FQDN SAN) is closed for this DC.

---

## Step 10 -- COS observability (DECISION POINT + MUTATION, per DC)

No D-NNN or design doc in this repo specifies the COS deployment mechanism --
only its scope (D-105: per-DC, in scope, no Office1 roll-up). Do not pick one
silently. Present the standard options to the operator at execution time:

!!! **Edge scrape is a SEPARATE scope decision (D-129, pinned 2026-07-18).** D-105 scopes COS to the
    cloud (per-DC, no Office1 roll-up). The DC OPNsense **edge sits OUTSIDE** that failure domain, so
    "should COS scrape the boundary edge?" is its own sub-decision -- NOT ruled by D-105. IF the operator
    rules the edge in-scope, install **`os-node_exporter`** on the DC edge (D-113 firmware API) and add
    it as a Prometheus scrape target; the edge LAN faces provider-public (D-100), so reachability from
    the in-cloud COS is a segmentation call to make explicitly. If out-of-scope, do nothing here.

- **Option A -- `cos-lite` bundle in a dedicated per-DC `cos` Juju model**
  (Canonical's standard COS Lite: Prometheus, Grafana, Loki, Alertmanager,
  Traefik ingress), related to this DC's `openstack` model applications via
  Juju cross-model relations (`juju offer` / `juju consume` +
  `cos-configuration-k8s` or the `grafana-agent` subordinate pattern for
  metrics/log forwarding). Needs a Kubernetes substrate for the `cos`
  model (microk8s or equivalent) -- confirm this DC has one before picking
  this option.
- **Option B -- machine-charm-native observability** (`prometheus2` /
  `grafana` / `telegraf` charms in the SAME model as the OpenStack bundle,
  no k8s substrate needed) -- lower delta if no k8s host is available in
  this DC for a `cos` model, but is not "COS" in the strict Canonical-bundle
  sense; confirm this is an acceptable substitution for the D-105 "COS"
  scope before treating it as satisfying the gate.

**CHECK (read-only) -- confirm before choosing**
```bash
# Does this DC have a k8s substrate available for a cos-lite model?
# (microk8s, or another already-enrolled k8s -- MEASURE, do not assume)
```
**GATE (decision, log it):** record which option was chosen and why in the
session ledger / a changelog entry -- this is a real architectural choice,
not a detail to bury in a command block.

Once chosen, the deployment steps are option-specific and are NOT detailed
further here (this is genuinely new tooling territory for this repo -- author
the concrete steps as their own change when the option is picked, with its own
harness per this repo's delivery rule, rather than improvising them live).

**GATE (Stage 7's COS criterion):** COS scraping -- at minimum, this DC's
OpenStack API charms' metrics/logs and the CAPI mgmt cluster + workload
cluster (Steps 11-13) are visible in whichever COS surface was chosen.

---

## Step 11 -- CAPI management cluster (reuse `phase-06-incloud-mgmt-cluster.md`, per DC)

Run `runbooks/phase-06-incloud-mgmt-cluster.md` Steps 6.0-BOOT through 6.6f
IN FULL, once per DC, against this DC's own project/Keystone/model. Do not
duplicate its content here -- point at it by filename. DC-parameterization
notes (found by tracing what that runbook's canonical scripts actually
resolve dynamically vs. hardcode):

- `scripts/phase-06-k8s-bootstrap.sh` sources `scripts/lib-net.sh` and reads
  `KEYSTONE_VIP_DEFAULT` from it (DOCFIX-138 centralized this). Before running
  it for a DC other than DC0, call `lib_net_select_dc "$DC"` (DOCFIX-151)
  FIRST: for `dc1` this is currently a documented no-op (D-101: DC1 inherits
  DC0's v4 layout unchanged -- a ratified reuse, not a stale default), for
  `dc2` it currently FAILS LOUDLY (`lib-net.sh`'s dc2 literals are not yet
  assigned -- tooling gap register #3). If dc2's literals are still unassigned
  by the time you reach this stage, STOP; do not bypass the fail-loud guard.
- `scripts/phase-06-kubeconfig-gate.sh` -- same `lib-net.sh` dependency; same
  `lib_net_select_dc "$DC"` call required first.
- Every other literal in phase-06 (`ENV(project)`, `ENV(ext-net)`,
  `ENV(image)`, `ENV(mgmt-fip)`, project/image/network UUIDs) is ALREADY
  written as "resolve by name" / "resolve dynamically" in that runbook -- it
  re-discovers these fresh on every run, so it needs no further DC-
  parameterization beyond pointing it at this DC's own `admin-openrc` and
  Keystone. Confirm you have sourced THIS DC's own admin-openrc (naming
  convention for per-DC openrc files was not established this session --
  confirm it from Stage 4's as-built record, do not assume a filename).
- `ENV(mgmt-cidr) 10.20.0.0/24` is a non-IPAM tenant overlay CIDR -- safe to
  reuse identically in both DCs (they are independent clouds; no collision
  risk). Do not route this through NetBox.
- `scripts/lib-hosts.sh`'s `lib_hosts_select_dc()` is NOT used by phase-06 (it
  governs MAAS host enrollment, a Stage 3 concern, not the tenant-VM layer
  phase-06 operates at) -- no action needed here, noted so it is not
  mistakenly assumed required.

**GATE (this runbook, restating phase-06's EXIT GATE):** per DC, GATE 1
(VIP-OK) and GATE 2 (agnhost pod-egress `Completed`) both pass; the CAPI
provider stack is up with no crash-looping CAPO.

---

## Step 12 -- Magnum conductor graft (reuse `phase-07-conductor-graft.md`, per DC)

Run `runbooks/phase-07-conductor-graft.md` in full, once per DC, against this
DC's own `magnum/0` and this DC's phase-06 kubeconfig/FIP. Point at it by
filename; do not duplicate its content.

**!!! D-046 GATE -- REPRODUCED HERE BECAUSE IT IS THE MOST-MISSED STEP IN
THIS DECISION-DERIVED CHECKLIST, restated verbatim from D-105/D-046:**
> The magnum charm action `domain-setup` is MANUAL, not automatic; magnum
> reports active / "Unit is ready" regardless of whether it has run.

Concretely, this means: for THIS DC, run phase-07's Step 7.0 (`juju run
magnum/leader domain-setup`, then assert `openstack domain show magnum` and
`openstack user show magnum_domain_admin --domain magnum` both succeed, then
gate `openstack coe service list` with no 403) EVERY time this DC's magnum
unit is torn down and redeployed -- including if this whole Stage 7 pass is
itself a redo after an earlier Stage 4-6 rebuild of this DC. Do not treat a
green `juju status` for magnum as evidence this step already happened.

DC-parameterization: phase-07's `ENV(conductor-unit)`, `ENV(mgmt-fip)`,
`ENV(magnum-ns)` are all already written as "resolve by name" / per-rebuild --
re-resolve them fresh against THIS DC's model; do not carry over the other
DC's or DC0's recorded values (the as-built reference blocks in that runbook
are VR0/DC0 audit trail, not this DC's expected values).

**GATE (this runbook, restating phase-07's EXIT GATE):** Step 7.0's domain-
setup assertion passes; conductor reaches the mgmt apiserver; magnum-capi-helm
1.4.0 (or whatever the then-current contract-coherent pin is -- confirm
against D-042/D-034 rather than assuming 1.4.0 is still current) installed;
`health_status` gate discharged per that runbook's FRESH DEPLOY routing (i.e.
deferred to Step 13 if no cluster exists yet in this DC).

---

## Step 13 -- Workload-cluster acceptance (reuse `phase-08-workload-cluster-acceptance.md`, per DC)

Run `runbooks/phase-08-workload-cluster-acceptance.md` in full, once per DC.
Point at it by filename; do not duplicate its content. One correction to
apply when running it under D-106 (rather than the D-019-amended acceptance
bar that runbook currently documents): D-011 item 8 ("Designate zones + tenant
hostname resolution") is NO LONGER deferred for VR1 -- Step 9 above is what
discharges it. Do not mark it DEFERRED per the old D-019 wording when running
this under D-106; record it PASS/FAIL against Step 9's actual result instead.

DC-parameterization: same shape as Steps 11-12 -- every `ENV(...)` in that
runbook already resolves by name; point it at this DC's own project/model,
re-seed this DC's own workload image via stage-and-verify (D-048) rather than
assuming the image already exists in a fresh DC.

**GATE (this runbook, restating phase-08's EXIT GATE, D-106-corrected):**
`capi-test-1`-equivalent cluster reaches `CREATE_COMPLETE`; 3 (or as-templated)
nodes Ready; CNI/CCM/CSI/CoreDNS Running; D-011 items 1-6 PASS; item 7
(snapshot) per this DC's own baseline policy; item 8 PASS (not deferred) via
Step 9's resolution proof.

---

## GATE -- Stage 7 exit condition (per DC; both DC1 and DC2 must pass independently)

Restating `docs/dc-dc-buildout-design.md` Phase 6 / `docs/dc-dc-deployment-
workflow.md` Stage 7's gate:

- Designate resolving A + AAAA for this DC's own zone (Step 6/9; AAAA only
  where D-101's v6 planes are confirmed up).
- FQDN-SAN certs valid (Step 4/9) -- the D-019/D-021 root cause (metal-only
  charms pulling an unresolvable/unroutable public FQDN behind an IP-only
  cert) is CLOSED for this DC.
- COS scraping this DC's OpenStack + CAPI mgmt + workload cluster (Step 10).
- A CAPI workload cluster comes up in this DC (Step 13, `CREATE_COMPLETE`).

Do not mark Stage 7 complete for the DC-DC buildout as a whole until BOTH DCs
independently clear all four bullets -- one DC passing does not imply the
other DC's independent Keystone/Vault/Designate zone/CAPI stack also passes.

---

## Delivery checklist

- [ ] NOTE: the designate half of Step 1 is ALREADY DONE in the repo (DOCFIX-167,
      2026-07-10) -- `bundle.yaml` carries all four designate applications and
      their eight relations, and Stage 5 deploys them. Do NOT re-add them. The
      only Step 1 repo work still owed is the per-DC `os-public-hostname`
      overlay proposed in 1.4, which is NOT AUTHORED
      (`overlays/vr1-dc0-hostnames.yaml` / `overlays/vr1-dc1-hostnames.yaml` do
      not exist -- and neither does the `overlays/${DC}-hostnames.yaml` spelling
      other surfaces used to reference; never pass a name from this bullet to
      `--overlay` before authoring the file, juju errors out on a missing
      overlay). It is a PROPOSED pattern per 1.4: present it at the review gate,
      do not silently adopt it.
- [ ] Once that overlay is authored: `scripts/repo-lint.sh` 0 fail;
      `scripts/preflight.sh` PASS (incl. Charmhub channel assert).
- [ ] Changelog entry for the overlay change: what changed
      (os-public-hostname re-add per DC), why (D-106, superseding D-019), how to
      revert (drop the hostnames overlay, re-run preflight to confirm the revert
      is clean). Reverting designate itself is NOT part of this stage -- it is
      Stage 5 bundle content under the D-106 ruling note.
- [ ] `docs/session-ledger.md` updated with this runbook's authoring and,
      once executed, each DC's Stage 7 pass/fail state.
- [ ] `docs/dc-dc-deployment-workflow.md` Stage 7 row's `**State:**` line
      updated from NOT STARTED once execution begins/completes.
- [ ] Per DC: Steps 0-9 (Designate reactivation) executed and Step 9's gate
      passed BEFORE Step 10 (COS) is attempted -- do not build COS scraping
      against a DNS layer that has not yet proven itself.
- [ ] Per DC: Step 10's COS mechanism choice recorded (which option, why).
- [ ] Per DC: phase-06/07/08 run to their own EXIT GATES with THIS DC's own
      resolved values (never a copy-pasted literal from DC0 or the other DC).
- [ ] D-046 domain-setup re-run + assertion done for THIS pass, not assumed
      from a prior one.
- [ ] This runbook's own GATE section cleared for BOTH DCs before Stage 7 is
      marked complete in the deployment workflow tracker.

## Next

Once both DCs clear this runbook's GATE, the DC-DC buildout's per-DC service
set is complete. Remaining work is whatever the deployment-workflow doc's
"Where things stand today" table still lists as outstanding at that point
(consult it fresh -- it is a living tracker, not reproduced here).
