# DC-DC (VR1) NetBox Buildout -- Planning & Decision Scope **Status:** planning/scope reference for the Code stream. NOT a decision record -- it consolidates ADOPTED decisions and flags the sub-decisions still needing operator ratification (candidate D-111+; assign via `ledger-scan` at adoption). **Authority order:** repo `design-decisions.md` (ADOPTED) > this doc > live NetBox (a reconciliation target, not a source) > design-exploration chats. NetBox is the IPAM apex (D-010, D-103) -- everything below it consumes it; it is never read back as a planning input. **Scope:** what must exist in `netbox.baldurkeep.com` before Phase 1 completes so the VR1 two-DC buildout can consume it. Reconciled against a live read of NetBox on 2026-07-10 (v4.5.8). --- ## 1. Authoritative address/family model (D-101, D-100, D-108) Six planes per DC, family follows reachability. Verbatim from D-101 / `dc-dc-buildout-design.md`: | Plane | v4 (DC1 = DC0 template) | v6 family | Notes | |---|---|---|---| | provider-public | `10.12.4.0/22` | **GUA** (dual) | public API VIPs + FIP/ext_net; GUA via owned OPNsense edge (D-015 gate N/A in VR1) | | metal-admin | `10.12.8.0/22` | **ULA** (dual) | admin-API ULA leg (2026-07-09 amendment); PXE stays v4-first | | metal-internal | `10.12.12.0/22` | **ULA** (dual) | API endpoints dual-stack; MySQL/RabbitMQ stay v4 | | data-tenant | `10.12.16.0/22` | **ULA-only** | geneve overlay; tenant GUA delegated separately | | storage | `10.12.32.0/22` | **ULA-only** | Ceph public; `ms_bind_ipv6` | | replication | `10.12.36.0/22` | **ULA-only** | Ceph cluster; the cross-DC leg rides this ULA plane (D-108) | Also ULA-only: **Octavia lb-mgmt** and **P2P links**. IPv4-retained: **MAAS PXE** (v4-first) and **datastore east-west** (MySQL/RabbitMQ). Tenant GUA: **native routing, no NAT66**, SLAAC default, **`/60` per tenant** (~4080/pool; `/56` on request) -- delegated from a per-DC tenant GUA pool, distinct from the infra planes. **Per-DC v4 (D-101):** DC1 **inherits DC0's six-plane CIDRs unchanged** (the six above). DC2 gets a **distinct, non-overlapping** parallel supernet, same six-plane structure, non-colliding with DC1 + operator infra + client VPN (D-074). **Cross-DC replication carrier = IPv6-only ULA** `/64` (D-101/D-108; supersedes the DR-seed `10.12.36.0/22` v4 line). **Three literals are NetBox-assigned, NOT hardcoded in any decision (D-101 open item / tooling-gap #3):** the org ULA `/48`, the per-DC GUA carve out of the `/36`, and DC2's v4 supernet. The import tool takes them as required env vars with no defaults. --- ## 2. Live NetBox state (2026-07-10) -- reconciliation `netbox.baldurkeep.com` v4.5.8. Regions: US West -> Eugene (prod); **Virtual Regions -> VR0, VR1**. Sites incl. **VR1 DC0 (`vr1-dc0`), VR1 DC1 (`vr1-dc1`)**. 90 prefixes (77 v6 / 13 v4), 56 VLANs, 2 VRFs (Roosevelt, Willamette), 1 RIR (ARIN). What is already right, and what is missing/divergent -- **Code must close these before the import runs**: | # | Finding | State | Action | |---|---|---|---| | G1 | **Six-plane IPAM roles** | Only `storage` exists; `provider-public`, `metal-admin`, `metal-internal`, `data-tenant`, `replication` **MISSING** | Create the 5 roles out-of-band (import script does NOT create roles) | | G2 | **Aggregates** | **0 aggregates** despite ARIN RIR present | Create aggregates (section 4b) -- IPAM apex should declare its allocations | | G3 | **Org ULA `/48`** | Not present (only `fc00::/8`, `fd00::/8` containers) | Assign + create (section 4c); the D-101 literal | | G4 | **VR1 VLAN groups** | VR1 DC0/DC1 have **no VLAN groups** (VR0 DC0 + prod sites do) | Create per-DC VLAN groups + VLANs | | G5 | **DC naming** | NetBox = `vr1-dc0`/`vr1-dc1`; decisions + import tool = **DC1/DC2** (`--dc dc1` targets `vr1-dc1`, DC2 -> `vr1-dc2` which does not exist) | Operator IPAM-hygiene call: rename to `vr1-dc1`/`vr1-dc2`, or re-point the tool. **Blocks the import** until resolved | | G6 | **VR0 DC0 / Willamette prefixes** | Fully built on the **OLD** model: single roles (`provider`/`metal`/`data`/`storage`/`repl`/`lbaas-mgmt`), **all GUA**, no ULA | This is the v1/DC0-era model, **not** the VR1 template. Leave as-is or retire (hygiene call, D-101 site-collision note) | --- ## 3. Structure to build on (RECONCILED template) The deployed carve and D-101 agree on a byte-aligned layout; use it for VR1. **Template `2602:f3e2:0RSS:CCNN`** where nibble-1 = `0` (region-0 = the held `/36`; varies once expanded to the `/32`), nibble-2 = the region/metro selector (Eugene=`1`, VR0=`e`, **VR1=`f`**), `SS` = site byte (`/48`), `CC` = cloud byte (`/56`), `NN` = net byte (`/64`). Net-byte role mnemonic already in use: `10` provider, `11` provider-VIP, `20` metal, `30` data, `40` storage, `50` replication, `80` lbaas, `e0` VPN, `f0` OOB. VR1 already occupies `2602:f3e2:f00::/40` with DC0=`f02::/48`, DC1=`f03::/48` as sites. **The per-DC GUA carve (D-101 literal) should be these existing `/48` site blocks** (`DC_GUA_PREFIX` = `2602:f3e2:f02::/48` for DC1, `2602:f3e2:f03::/48` for DC2), aligned to G5's naming decision -- not the README's placeholder `2602:f3e2:1000::/40`. > **RECONCILIATION NOTE (design-chat vs repo -- for the operator, so nothing surprises you):** > The parallel IPAM-design chat produced a carve (v3 / dual-region bundle) that diverged from what > is deployed and adopted. Those artifacts are **superseded for the DC-DC buildout**; the deployed > pattern + D-101 govern. Specifically: > - **Region label:** chat used `RMSS` (region=`/36`, a "metro" at `/40`); deployed + D-101 use > region-0=`/36` with the `/40` as the region/metro selector (`0RSS`). Addresses match for > production; **only VR0/VR1 placement differed** -- the chat bundle put them at nibble-1 > (`2602:f3e2:e000::/36`, `f000::/36`); NetBox has them at nibble-2 (`0e00`/`0f00::/40`). > **Deployed nibble-2 placement wins.** > - **Tenant size:** chat concluded `/64`-per-tenant (assuming tenants share the infra cloud > `/56`); D-101 says **`/60`**, because tenant GUA is a **separate delegation pool**, not the > infra `/56`. **D-101 `/60` wins.** > - **Roles:** chat used the old 5-6 role set; VR1 uses the **six-plane** set. **Six-plane wins.** --- ## 4. Buildout steps for Code (ordered; all idempotent, dry-run first) **4a. IPAM roles (G1).** Create the 5 missing six-plane roles (`provider-public`, `metal-admin`, `metal-internal`, `data-tenant`, `replication`; `storage` exists). Slugs must match `scripts/lib-net.sh` `SPACES6` exactly -- the import tool references them by slug. Created by `netbox/roles-aggregates-import.py` (this delivery). **4b. Aggregates (G2).** Under RIR ARIN: **`2602:f3e2::/36`** (the *held* allocation = region-0 of the planned `/32`; update to `/32` post-expand -- declaring a `/32` you don't hold is not honest for an IPAM apex), `23.157.124.0/24`. Under a new RFC-4193 RIR: the org ULA `/48` (4c). Under a new RFC-1918 RIR: `10.0.0.0/8`, `172.16.0.0/12`. All created idempotently by `netbox/roles-aggregates-import.py` (this delivery). **4c. Org ULA `/48` (G3, D-101 literal).** Assign one RFC-4193 pseudo-random `/48`. **Recommended value: `fd50:840e:74e2::/48`** -- CSPRNG-generated (Global ID `0x50840e74e2`), collision-checked against the Tailscale `fd7a:115c:a1e0::/48` already in NetBox (distinct). This is a *recommendation for the operator to ratify as the literal*, not an invented fact; if adopted, it becomes `ORG_ULA_48` for the import tool. **4d. Per-DC prefixes via `netbox/dc-dc-prefixes-import.py` (D-101).** Run per DC with the three literals. Recommended literals (pending G5 naming + section 5 ratification): ``` # DC1 (v4 inherited; only v6 top-level blocks required) ORG_ULA_48=fd50:840e:74e2::/48 DC_GUA_PREFIX=2602:f3e2:f02::/48 --dc dc1 # DC2 (distinct v4 supernet ALSO required -- >= /19; tool re-packs 6 sequential /22s, see 5.3) ORG_ULA_48=fd50:840e:74e2::/48 DC_GUA_PREFIX=2602:f3e2:f03::/48 DC2_V4_SUPERNET=10.13.0.0/19 --dc dc2 ``` Run `--verify-only` first. The tool creates the six-plane v4 + the D-101 family-matrix v6 legs. **4e. VLAN groups + VLANs (G4).** Per-DC VLAN groups scoped to `vr1-dc1`/`vr1-dc2`, VLANs per the plane->VID scheme (mirror `docs/maas-as-built-reference.md`; note metal-internal VID 103 per the buildout doc). VIDs reuse per DC (separate fabrics). **4f. Tenant GUA delegation pool.** Per D-101, a per-DC tenant GUA pool (from the DC's GUA carve or a dedicated block) from which `/60`s are delegated (SLAAC, native-routed). Model the pool; per- tenant `/60`s are Neutron-managed (D-016/D-074), not enumerated in NetBox. --- ## 5. Sub-decisions requiring operator ratification (candidate D-111+) None of these block *reading* NetBox; they block *trusting the exact values written*. Each is a grep-before-assign D-NNN when ruled. 1. **Per-plane subcarve (the big one).** `dc-dc-prefixes-import.py`'s ULA/GUA subdivision is the script's OWN proposal, explicitly un-ratified: ULA = a `/56`-per-DC out of the org `/48` (dc1 `::0/56`, dc2 `::1:00/56`), then a `/64` per ULA-bearing plane in fixed order; GUA = provider-public gets the whole `DC_GUA_PREFIX` as one `/64`. **Two conflicts to resolve:** (a) this does NOT use the deployed `NN` role mnemonic (provider at `NN=10`, etc.) -- decide whether to align the tool to the deployed mnemonic or accept the tool's index scheme; (b) the ULA `/56`-per-DC index is independent of the GUA geography -- confirm that's acceptable (it matches the "internal `/48` reused across the domain" intent). **Recommend:** align the tool's per-plane `/64` offsets to the `NN` mnemonic so GUA and ULA read consistently, then ratify. 2. **DC naming (G5).** `vr1-dc0`/`vr1-dc1` (NetBox) vs DC1/DC2 (decisions/tool). **Recommend:** rename sites to `vr1-dc1`/`vr1-dc2` to match the adopted DC1/DC2 language and the tool's `--dc dc2 -> vr1-dc2` expectation; retire or repurpose the now-orphaned `vr1-dc0`. 3. **DC2 v4 supernet value + size.** D-101 leaves it NetBox-assigned. The import tool (`DC2_MIN_SUPERNET_PREFIXLEN = 19`) carves **six SEQUENTIAL `/22`s** from the supernet base and FAILS LOUD below `/19` (a `/19` = 8 `/22`s; six used, two spare). So a **`/19` is correct** (e.g. `10.13.0.0/19`); DC2's planes land at sequential offsets (`.0/.4/.8/.12/.16/.20`), NOT mirroring DC1's `.4/.8/.12/.16/.32/.36` -- fine per D-101 (same six-plane *structure*, not same offsets). Just pick a non-colliding `/19` (per D-074). 4. **Tenant size confirmation.** D-101 says `/60`. Confirm the design-chat `/64` is dropped (it is, per section 3) so Code builds the `/60` delegation model. 5. **VR0 DC0 old-model retirement.** Keep for history vs retire once VR1 supersedes the rehearsal (D-101 site-collision note leaves this to the operator). --- ## 6. Cross-references - **Decisions:** D-100 (fabric/topology), **D-101** (address plan + family matrix; folds D-102 tenant/MTU), D-103 (IaC seam; NetBox apex), D-104 (controllers), D-106 (Designate A/AAAA), D-107 (per-DC mirror), D-108 (replication carrier = ULA), D-109 (Vault v6-SAN). D-010 (NetBox-upstream), D-052/D-053 (six-plane), D-016/D-074 (tenant CIDR). - **Tooling:** `netbox/dc-dc-prefixes-import.py` (+ `README.md`), `netbox/ipv6-mark-reserved.py`, `scripts/lib-net.sh` (`SPACES6`/`PLANE_CIDRS`), `docs/dc-dc-buildout-design.md` (Phase 1 gate), `docs/maas-as-built-reference.md` (VLANs/fabrics), `docs/dc-dc-replication-DR-seed.md`. - **Phase:** this is the **Phase 1** NetBox-population gate ("NetBox authoritative and populated: planes, per-DC v4, ULA/GUA") in `dc-dc-buildout-design.md`. ## 7. Open literals summary (hand to whoever assigns them) | Literal | Status | Recommendation | |---|---|---| | `ORG_ULA_48` | unassigned | `fd50:840e:74e2::/48` (ratify) | | `DC_GUA_PREFIX` dc1 | ambiguous | `2602:f3e2:f02::/48` (existing VR1 DC0 site block, renamed dc1) | | `DC_GUA_PREFIX` dc2 | ambiguous | `2602:f3e2:f03::/48` (existing VR1 DC1 site block, renamed dc2) | | `DC2_V4_SUPERNET` | unassigned | `10.13.0.0/19` (distinct; tool needs >= `/19`, re-packs 6 sequential `/22`s -- see 5.3) |