DC-DC (VR1) NetBox Buildout -- Planning & Decision Scope

Status: planning/scope reference for the Code stream. NOT a decision record -- it consolidates ADOPTED decisions and flags the sub-decisions still needing operator ratification (candidate D-111+; assign via ledger-scan at adoption). Authority order: repo design-decisions.md (ADOPTED) > this doc > live NetBox (a reconciliation target, not a source) > design-exploration chats. NetBox is the IPAM apex (D-010, D-103) -- everything below it consumes it; it is never read back as a planning input. Scope: what must exist in netbox.baldurkeep.com before Phase 1 completes so the VR1 two-DC buildout can consume it. Reconciled against a live read of NetBox on 2026-07-10 (v4.5.8).

1. Authoritative address/family model (D-101, D-100, D-108)

Six planes per DC, family follows reachability. Verbatim from D-101 / dc-dc-buildout-design.md:

Also ULA-only: Octavia lb-mgmt and P2P links. IPv4-retained: MAAS PXE (v4-first) and datastore east-west (MySQL/RabbitMQ). Tenant GUA: native routing, no NAT66, SLAAC default, /60 per tenant (~4080/pool; /56 on request) -- delegated from a per-DC tenant GUA pool, distinct from the infra planes.

Per-DC v4 (D-101): DC1 inherits DC0's six-plane CIDRs unchanged (the six above). DC2 gets a distinct, non-overlapping parallel supernet, same six-plane structure, non-colliding with DC1 + operator infra + client VPN (D-074). Cross-DC replication carrier = IPv6-only ULA /64 (D-101/D-108; supersedes the DR-seed 10.12.36.0/22 v4 line).

**Three literals are NetBox-assigned, NOT hardcoded in any decision (D-101 open item / tooling-gap

#3):** the org ULA /48, the per-DC GUA carve out of the /36, and DC2's v4 supernet. The import tool takes them as required env vars with no defaults.

2. Live NetBox state (2026-07-10) -- reconciliation

netbox.baldurkeep.com v4.5.8. Regions: US West -> Eugene (prod); Virtual Regions -> VR0, VR1. Sites incl. VR1 DC0 (vr1-dc0), VR1 DC1 (vr1-dc1). 90 prefixes (77 v6 / 13 v4), 56 VLANs, 2 VRFs (Roosevelt, Willamette), 1 RIR (ARIN).

What is already right, and what is missing/divergent -- Code must close these before the import runs:

3. Structure to build on (RECONCILED template)

The deployed carve and D-101 agree on a byte-aligned layout; use it for VR1. Template 2602:f3e2:0RSS:CCNN where nibble-1 = 0 (region-0 = the held /36; varies once expanded to the /32), nibble-2 = the region/metro selector (Eugene=1, VR0=e, VR1=f), SS = site byte (/48), CC = cloud byte (/56), NN = net byte (/64). Net-byte role mnemonic already in use: 10 provider, 11 provider-VIP, 20 metal, 30 data, 40 storage, 50 replication, 80 lbaas, e0 VPN, f0 OOB.

VR1 already occupies 2602:f3e2:f00::/40 with DC0=f02::/48, DC1=f03::/48 as sites. The per-DC GUA carve (D-101 literal) should be these existing /48 site blocks (DC_GUA_PREFIX = 2602:f3e2:f02::/48 for DC1, 2602:f3e2:f03::/48 for DC2), aligned to G5's naming decision -- not the README's placeholder 2602:f3e2:1000::/40.

RECONCILIATION NOTE (design-chat vs repo -- for the operator, so nothing surprises you): The parallel IPAM-design chat produced a carve (v3 / dual-region bundle) that diverged from what is deployed and adopted. Those artifacts are superseded for the DC-DC buildout; the deployed pattern + D-101 govern. Specifically:

• Region label: chat used RMSS (region=/36, a "metro" at /40); deployed + D-101 use region-0=/36 with the /40 as the region/metro selector (0RSS). Addresses match for production; only VR0/VR1 placement differed -- the chat bundle put them at nibble-1 (2602:f3e2:e000::/36, f000::/36); NetBox has them at nibble-2 (0e00/0f00::/40). Deployed nibble-2 placement wins.
• Tenant size: chat concluded /64-per-tenant (assuming tenants share the infra cloud /56); D-101 says /60, because tenant GUA is a separate delegation pool, not the infra /56. D-101 /60 wins.
• Roles: chat used the old 5-6 role set; VR1 uses the six-plane set. Six-plane wins.

4. Buildout steps for Code (ordered; all idempotent, dry-run first)

4a. IPAM roles (G1). Create the 5 missing six-plane roles (provider-public, metal-admin, metal-internal, data-tenant, replication; storage exists). Slugs must match scripts/lib-net.sh SPACES6 exactly -- the import tool references them by slug. Created by netbox/roles-aggregates-import.py (this delivery).

4b. Aggregates (G2). Under RIR ARIN: 2602:f3e2::/36 (the held allocation = region-0 of the planned /32; update to /32 post-expand -- declaring a /32 you don't hold is not honest for an IPAM apex), 23.157.124.0/24. Under a new RFC-4193 RIR: the org ULA /48 (4c). Under a new RFC-1918 RIR: 10.0.0.0/8, 172.16.0.0/12. All created idempotently by netbox/roles-aggregates-import.py (this delivery).

4c. Org ULA /48 (G3, D-101 literal). Assign one RFC-4193 pseudo-random /48. Recommended value: fd50:840e:74e2::/48 -- CSPRNG-generated (Global ID 0x50840e74e2), collision-checked against the Tailscale fd7a:115c:a1e0::/48 already in NetBox (distinct). This is a recommendation for the operator to ratify as the literal, not an invented fact; if adopted, it becomes ORG_ULA_48 for the import tool.

4d. Per-DC prefixes via netbox/dc-dc-prefixes-import.py (D-101). Run per DC with the three literals. Recommended literals (pending G5 naming + section 5 ratification):

# DC1 (v4 inherited; only v6 top-level blocks required)
ORG_ULA_48=fd50:840e:74e2::/48  DC_GUA_PREFIX=2602:f3e2:f02::/48  --dc dc0   # D-117: VR1 DC0
# DC2 (distinct v4 supernet ALSO required -- >= /19; tool re-packs 6 sequential /22s, see 5.3)
ORG_ULA_48=fd50:840e:74e2::/48  DC_GUA_PREFIX=2602:f3e2:f03::/48  DC1_V4_SUPERNET=10.12.64.0/19 --dc dc1  # D-117: VR1 DC1

Run --verify-only first. The tool creates the six-plane v4 + the D-101 family-matrix v6 legs.

4e. VLAN groups + VLANs (G4). Per-DC VLAN groups scoped to vr1-dc1/vr1-dc2, VLANs per the plane->VID scheme (mirror docs/maas-as-built-reference.md; note metal-internal VID 103 per the buildout doc). VIDs reuse per DC (separate fabrics).

4f. Tenant GUA delegation pool. Per D-101, a per-DC tenant GUA pool (from the DC's GUA carve or a dedicated block) from which /60s are delegated (SLAAC, native-routed). Model the pool; per- tenant /60s are Neutron-managed (D-016/D-074), not enumerated in NetBox.

5. Sub-decisions requiring operator ratification (candidate D-111+)

None of these block reading NetBox; they block trusting the exact values written. Each is a grep-before-assign D-NNN when ruled.

1. Per-plane subcarve (the big one). dc-dc-prefixes-import.py's ULA/GUA subdivision is the script's OWN proposal, explicitly un-ratified: ULA = a /56-per-DC out of the org /48 (dc1 ::0/56, dc2 ::1:00/56), then a /64 per ULA-bearing plane in fixed order; GUA = provider-public gets the whole DC_GUA_PREFIX as one /64. Two conflicts to resolve: (a) this does NOT use the deployed NN role mnemonic (provider at NN=10, etc.) -- decide whether to align the tool to the deployed mnemonic or accept the tool's index scheme; (b) the ULA /56-per-DC index is independent of the GUA geography -- confirm that's acceptable (it matches the "internal /48 reused across the domain" intent). Recommend: align the tool's per-plane /64 offsets to the NN mnemonic so GUA and ULA read consistently, then ratify. RATIFIED as D-111 (2026-07-11) + implemented in carve_v6 (DOCFIX-182): NN offsets, /60+/64 per plane, metal /60 shared by admin :20 / internal :21, ULA /56 indexed to the GUA nibble so the 4th hextet reads DC.NN in both families.

2. DC naming (G5).SUPERSEDED -- do NOT follow this. This recommended renaming the APEX to vr1-dc1/vr1-dc2. D-117 (ADOPTED 2026-07-13) ruled the opposite: the repo moves to the apex's names (dc0/dc1), because NetBox is the IPAM apex and is not bent to fit the repo. The import tool now REJECTS --dc dc2 and the DC2_V4_SUPERNET env var by name.

3. DC2 v4 supernet value + size. D-101 leaves it NetBox-assigned. The import tool (DC2_MIN_SUPERNET_PREFIXLEN = 19) carves six SEQUENTIAL /22s from the supernet base and FAILS LOUD below /19 (a /19 = 8 /22s; six used, two spare). So a /19 is correct (10.12.64.0/19 per D-115; was 10.13.0.0/19); DC2's planes land at sequential offsets (.0/.4/.8/.12/.16/.20), NOT mirroring DC1's .4/.8/.12/.16/.32/.36 -- fine per D-101 (same six-plane structure, not same offsets). Just pick a non-colliding /19 (per D-074).

4. Tenant size confirmation. D-101 says /60. Confirm the design-chat /64 is dropped (it is, per section 3) so Code builds the /60 delegation model.

5. VR0 DC0 old-model retirement. Keep for history vs retire once VR1 supersedes the rehearsal (D-101 site-collision note leaves this to the operator).

6. Cross-references

• Decisions: D-100 (fabric/topology), D-101 (address plan + family matrix; folds D-102 tenant/MTU), D-103 (IaC seam; NetBox apex), D-104 (controllers), D-106 (Designate A/AAAA), D-107 (per-DC mirror), D-108 (replication carrier = ULA), D-109 (Vault v6-SAN). D-010 (NetBox-upstream), D-052/D-053 (six-plane), D-016/D-074 (tenant CIDR).
• Tooling: netbox/dc-dc-prefixes-import.py (+ README.md), netbox/ipv6-mark-reserved.py, scripts/lib-net.sh (SPACES6/PLANE_CIDRS), docs/dc-dc-buildout-design.md (Phase 1 gate), docs/maas-as-built-reference.md (VLANs/fabrics), docs/dc-dc-replication-DR-seed.md.
• Phase: this is the Phase 1 NetBox-population gate ("NetBox authoritative and populated: planes, per-DC v4, ULA/GUA") in dc-dc-buildout-design.md.

7. Open literals summary (hand to whoever assigns them)

8. The NetBox sandbox loop (C4)

8.1 Why a sandbox exists at all

NetBox netbox.baldurkeep.com is the IPAM apex and source of truth (D-010, D-103). The section-4 buildout mutates it (roles, aggregates, org ULA /48, two DCs of six-plane prefixes). We do NOT rehearse those mutations on the apex. A SECOND NetBox -- office1-netbox, 10.10.1.201:8000, NetBox 4.6.4 on the Office1 VM (docs/changelog-20260713-office1-netbox-deployed.md) -- is the SANDBOX. The upstream draft is copied in, the planned changes are applied and proven there, and only then is the validated delta fed back upstream as a separate, operator-gated write.

upstream (draft source, READ-ONLY)  --[prod-draft-dump.py]-->  netbox/draft/vr1-draft.json
vr1-draft.json                      --[sandbox-seed.py]------>  Office1 sandbox
...apply the planned delta in the sandbox (the section-4 importers)...
...prove sandbox == upstream draft + EXACTLY the planned delta (fidelity check)...
validated refinements               --[operator-gated write]->  upstream

The sim NEVER --commits upstream as a side effect. Feeding the apex is always a deliberate operator act (8.4, 8.6).

8.2 Why a FIDELITY CHECK, and not just counts

Counts and idempotency CANNOT prove fidelity. On 2026-07-13 the seeder silently dropped the scope on 17 of 90 prefixes -- it mapped only dcim.site, and most of the VR1 draft is dcim.region-scoped -- and every count still matched and re-running stayed idempotent. It was caught by luck on a spot-check printing site None. netbox/sandbox-fidelity-check.py is the check that catches it on purpose: it diffs FIELD BY FIELD on every shared object, so a dropped or mangled field is a hard failure.

8.3 The 2026-07-14 hardening -- do NOT trust a pre-hardening "faithful" verdict

Until 2026-07-14 the fidelity check could return a FALSE GREEN. It only proved the sandbox delta was a SUBSET of the plan (unexpected = extra - expected); it NEVER proved the plan was CARRIED OUT. With ZERO of the 36 DC prefixes created, extra and unexpected were both empty, and it printed "Nothing lost, nothing stray" and exited 0. EXPECTED_NEW was an upper bound masquerading as an assertion. Fixed 2026-07-14 (docs/changelog-20260714-netbox-write-path-hardening.md): EXPECTED_NEW is now BOTH bounds (EXPECTED-BUT-ABSENT is a hard failure), and delta prefixes are checked for scope and role. A NEW harness tests/sandbox-fidelity-check/run-tests.sh (10/10; T3 IS the false-green scenario) ships with it -- the checker had shipped with NO harness, which is why the false green survived.

CONSEQUENCE: any "sandbox is faithful" claim dated before 2026-07-14 was made by a checker that could not see an unapplied or under-applied plan. Re-run the current checker before trusting fidelity (this is why the ledger flags a sandbox re-verify owed before C2).

Two blind spots remain OPEN (logged, not fixed): SAME-DUMPER (both sides use the same field list, so a field the dumper omits is invisible on both), and DUPLICATE-CIDR collapse (prefix-keyed dicts collapse duplicates; latent -- none upstream today). "Faithful" means faithful within the dumped field list, not absolutely.

8.4 Write-guard architecture -- FIVE tools, and how each is kept off the apex

"The sim never writes upstream" is enforced differently per tool. As of 2026-07-14 the three write-capable importers ALL gate an upstream --commit, but by two different mechanisms:

Before 2026-07-14 the two pynetbox importers had NO target guard at all; the only thing stopping an accidental upstream --commit was the WAF 403 (8.5) -- and fixing the WAF (below) removed that accidental safety, which is exactly why the explicit --yes-write-upstream gate was added in the same change (docs/changelog-20260714-netbox-importers-waf-useragent.md). The whole-plan preflights are the OTHER layer: both importers had previously died mid-loop and left the production IPAM half-populated (docs/changelog-20260714-netbox-write-path-hardening.md).

8.5 The upstream WAF trap (FIXED 2026-07-14)

Upstream NetBox sits behind a User-Agent-filtering WAF: urllib/requests/pynetbox get a 403 that looks EXACTLY like an auth failure, while the SAME token works from curl (measured 2026-07-13, re-measured 2026-07-14: curl -> 200, urllib/pynetbox -> 403; references/platform-traps.md). Do NOT diagnose it as a token problem and re-mint credentials -- confirm with curl first.

The three stdlib tools always sent an accepted UA (curl/8.5.0). The two pynetbox importers did NOT until 2026-07-14 -- meaning they could NEVER have written to the apex, only to the WAF-less sandbox. get_nb() in both now sets nb.http_session.headers["User-Agent"] = "curl/8.5.0" (docs/changelog-20260714-netbox-importers-waf-useragent.md). NOTE: pynetbox is NOT installed on vcloud -- the two pynetbox importers must run on office1-netbox (pynetbox 7.0.0), which reaches upstream through the edge.

8.6 Feeding the delta upstream (C2) -- STILL PENDING

C2 is a production IPAM write and is NOT part of the sim. Preconditions before it runs: (a) re-run the HARDENED fidelity check against the sandbox -- the "faithful" verdict predates the hardening (8.3); (b) run the two pynetbox importers from office1-netbox (pynetbox + upstream reach); (c) each importer needs --yes-write-upstream on top of --commit for the non-sandbox target; (d) present and individually approve each mutation -- never batched. As of 2026-07-14 C2 has NOT been run; the upstream apex still lacks the six-plane roles, the RIRs/aggregates, the vr1-off1 carve, and the 36 DC prefixes (measured read-only).