DC-DC (VR1) NetBox Buildout -- Planning & Decision Scope

Status: planning/scope reference for the Code stream. NOT a decision record -- it consolidates ADOPTED decisions and flags the sub-decisions still needing operator ratification (candidate D-111+; assign via ledger-scan at adoption). Authority order: repo design-decisions.md (ADOPTED) > this doc > live NetBox (a reconciliation target, not a source) > design-exploration chats. NetBox is the IPAM apex (D-010, D-103) -- everything below it consumes it; it is never read back as a planning input. Scope: what must exist in netbox.baldurkeep.com before Phase 1 completes so the VR1 two-DC buildout can consume it. Reconciled against a live read of NetBox on 2026-07-10 (v4.5.8).

1. Authoritative address/family model (D-101, D-100, D-108)

Six planes per DC, family follows reachability. Verbatim from D-101 / dc-dc-buildout-design.md:

Also ULA-only: Octavia lb-mgmt and P2P links. IPv4-retained: MAAS PXE (v4-first) and datastore east-west (MySQL/RabbitMQ). Tenant GUA: native routing, no NAT66, SLAAC default, /60 per tenant (~4080/pool; /56 on request) -- delegated from a per-DC tenant GUA pool, distinct from the infra planes.

Per-DC v4 (D-101): DC1 inherits DC0's six-plane CIDRs unchanged (the six above). DC2 gets a distinct, non-overlapping parallel supernet, same six-plane structure, non-colliding with DC1 + operator infra + client VPN (D-074). Cross-DC replication carrier = IPv6-only ULA /64 (D-101/D-108; supersedes the DR-seed 10.12.36.0/22 v4 line).

**Three literals are NetBox-assigned, NOT hardcoded in any decision (D-101 open item / tooling-gap

#3):** the org ULA /48, the per-DC GUA carve out of the /36, and DC2's v4 supernet. The import tool takes them as required env vars with no defaults.

2. Live NetBox state (2026-07-10) -- reconciliation

netbox.baldurkeep.com v4.5.8. Regions: US West -> Eugene (prod); Virtual Regions -> VR0, VR1. Sites incl. VR1 DC0 (vr1-dc0), VR1 DC1 (vr1-dc1). 90 prefixes (77 v6 / 13 v4), 56 VLANs, 2 VRFs (Roosevelt, Willamette), 1 RIR (ARIN).

What is already right, and what is missing/divergent -- Code must close these before the import runs:

3. Structure to build on (RECONCILED template)

The deployed carve and D-101 agree on a byte-aligned layout; use it for VR1. Template 2602:f3e2:0RSS:CCNN where nibble-1 = 0 (region-0 = the held /36; varies once expanded to the /32), nibble-2 = the region/metro selector (Eugene=1, VR0=e, VR1=f), SS = site byte (/48), CC = cloud byte (/56), NN = net byte (/64). Net-byte role mnemonic already in use: 10 provider, 11 provider-VIP, 20 metal, 30 data, 40 storage, 50 replication, 80 lbaas, e0 VPN, f0 OOB.

VR1 already occupies 2602:f3e2:f00::/40 with DC0=f02::/48, DC1=f03::/48 as sites. The per-DC GUA carve (D-101 literal) should be these existing /48 site blocks (DC_GUA_PREFIX = 2602:f3e2:f02::/48 for DC1, 2602:f3e2:f03::/48 for DC2), aligned to G5's naming decision -- not the README's placeholder 2602:f3e2:1000::/40.

RECONCILIATION NOTE (design-chat vs repo -- for the operator, so nothing surprises you): The parallel IPAM-design chat produced a carve (v3 / dual-region bundle) that diverged from what is deployed and adopted. Those artifacts are superseded for the DC-DC buildout; the deployed pattern + D-101 govern. Specifically:

• Region label: chat used RMSS (region=/36, a "metro" at /40); deployed + D-101 use region-0=/36 with the /40 as the region/metro selector (0RSS). Addresses match for production; only VR0/VR1 placement differed -- the chat bundle put them at nibble-1 (2602:f3e2:e000::/36, f000::/36); NetBox has them at nibble-2 (0e00/0f00::/40). Deployed nibble-2 placement wins.
• Tenant size: chat concluded /64-per-tenant (assuming tenants share the infra cloud /56); D-101 says /60, because tenant GUA is a separate delegation pool, not the infra /56. D-101 /60 wins.
• Roles: chat used the old 5-6 role set; VR1 uses the six-plane set. Six-plane wins.

4. Buildout steps for Code (ordered; all idempotent, dry-run first)

4a. IPAM roles (G1). Create the 5 missing six-plane roles (provider-public, metal-admin, metal-internal, data-tenant, replication; storage exists). Slugs must match scripts/lib-net.sh SPACES6 exactly -- the import tool references them by slug. Created by netbox/roles-aggregates-import.py (this delivery).

4b. Aggregates (G2). Under RIR ARIN: 2602:f3e2::/36 (the held allocation = region-0 of the planned /32; update to /32 post-expand -- declaring a /32 you don't hold is not honest for an IPAM apex), 23.157.124.0/24. Under a new RFC-4193 RIR: the org ULA /48 (4c). Under a new RFC-1918 RIR: 10.0.0.0/8, 172.16.0.0/12. All created idempotently by netbox/roles-aggregates-import.py (this delivery).

4c. Org ULA /48 (G3, D-101 literal). Assign one RFC-4193 pseudo-random /48. Recommended value: fd50:840e:74e2::/48 -- CSPRNG-generated (Global ID 0x50840e74e2), collision-checked against the Tailscale fd7a:115c:a1e0::/48 already in NetBox (distinct). This is a recommendation for the operator to ratify as the literal, not an invented fact; if adopted, it becomes ORG_ULA_48 for the import tool.

4d. Per-DC prefixes via netbox/dc-dc-prefixes-import.py (D-101). Run per DC with the three literals. Recommended literals (pending G5 naming + section 5 ratification):

# DC1 (v4 inherited; only v6 top-level blocks required)
ORG_ULA_48=fd50:840e:74e2::/48  DC_GUA_PREFIX=2602:f3e2:f02::/48  --dc dc0   # D-117: VR1 DC0
# DC2 (distinct v4 supernet ALSO required -- >= /19; tool re-packs 6 sequential /22s, see 5.3)
ORG_ULA_48=fd50:840e:74e2::/48  DC_GUA_PREFIX=2602:f3e2:f03::/48  DC1_V4_SUPERNET=10.12.64.0/19 --dc dc1  # D-117: VR1 DC1

Run --verify-only first. The tool creates the six-plane v4 + the D-101 family-matrix v6 legs.

4e. VLAN groups + VLANs (G4). Per-DC VLAN groups scoped to vr1-dc1/vr1-dc2, VLANs per the plane->VID scheme (mirror docs/maas-as-built-reference.md; note metal-internal VID 103 per the buildout doc). VIDs reuse per DC (separate fabrics).

4f. Tenant GUA delegation pool. Per D-101, a per-DC tenant GUA pool (from the DC's GUA carve or a dedicated block) from which /60s are delegated (SLAAC, native-routed). Model the pool; per- tenant /60s are Neutron-managed (D-016/D-074), not enumerated in NetBox.

5. Sub-decisions requiring operator ratification (candidate D-111+)

None of these block reading NetBox; they block trusting the exact values written. Each is a grep-before-assign D-NNN when ruled.

1. Per-plane subcarve (the big one). dc-dc-prefixes-import.py's ULA/GUA subdivision is the script's OWN proposal, explicitly un-ratified: ULA = a /56-per-DC out of the org /48 (dc1 ::0/56, dc2 ::1:00/56), then a /64 per ULA-bearing plane in fixed order; GUA = provider-public gets the whole DC_GUA_PREFIX as one /64. Two conflicts to resolve: (a) this does NOT use the deployed NN role mnemonic (provider at NN=10, etc.) -- decide whether to align the tool to the deployed mnemonic or accept the tool's index scheme; (b) the ULA /56-per-DC index is independent of the GUA geography -- confirm that's acceptable (it matches the "internal /48 reused across the domain" intent). Recommend: align the tool's per-plane /64 offsets to the NN mnemonic so GUA and ULA read consistently, then ratify. RATIFIED as D-111 (2026-07-11) + implemented in carve_v6 (DOCFIX-182): NN offsets, /60+/64 per plane, metal /60 shared by admin :20 / internal :21, ULA /56 indexed to the GUA nibble so the 4th hextet reads DC.NN in both families.

2. DC naming (G5).SUPERSEDED -- do NOT follow this. This recommended renaming the APEX to vr1-dc1/vr1-dc2. D-117 (ADOPTED 2026-07-13) ruled the opposite: the repo moves to the apex's names (dc0/dc1), because NetBox is the IPAM apex and is not bent to fit the repo. The import tool now REJECTS --dc dc2 and the DC2_V4_SUPERNET env var by name.

3. DC2 v4 supernet value + size. D-101 leaves it NetBox-assigned. The import tool (DC2_MIN_SUPERNET_PREFIXLEN = 19) carves six SEQUENTIAL /22s from the supernet base and FAILS LOUD below /19 (a /19 = 8 /22s; six used, two spare). So a /19 is correct (10.12.64.0/19 per D-115; was 10.13.0.0/19); DC2's planes land at sequential offsets (.0/.4/.8/.12/.16/.20), NOT mirroring DC1's .4/.8/.12/.16/.32/.36 -- fine per D-101 (same six-plane structure, not same offsets). Just pick a non-colliding /19 (per D-074).

4. Tenant size confirmation. D-101 says /60. Confirm the design-chat /64 is dropped (it is, per section 3) so Code builds the /60 delegation model.

5. VR0 DC0 old-model retirement. Keep for history vs retire once VR1 supersedes the rehearsal (D-101 site-collision note leaves this to the operator).

6. Cross-references

• Decisions: D-100 (fabric/topology), D-101 (address plan + family matrix; folds D-102 tenant/MTU), D-103 (IaC seam; NetBox apex), D-104 (controllers), D-106 (Designate A/AAAA), D-107 (per-DC mirror), D-108 (replication carrier = ULA), D-109 (Vault v6-SAN). D-010 (NetBox-upstream), D-052/D-053 (six-plane), D-016/D-074 (tenant CIDR).
• Tooling: netbox/dc-dc-prefixes-import.py (+ README.md), netbox/ipv6-mark-reserved.py, scripts/lib-net.sh (SPACES6/PLANE_CIDRS), docs/dc-dc-buildout-design.md (Phase 1 gate), docs/maas-as-built-reference.md (VLANs/fabrics), docs/dc-dc-replication-DR-seed.md.
• Phase: this is the Phase 1 NetBox-population gate ("NetBox authoritative and populated: planes, per-DC v4, ULA/GUA") in dc-dc-buildout-design.md.

7. Open literals summary (hand to whoever assigns them)

8. The NetBox sandbox loop (C4)

8.1 Why a second NetBox exists

DOCFIX-195 (operator ruling 2026-07-15) -- READ THIS FIRST; it INVERTS the roles the rest of this section was originally written around. During VR1 the VR1 IPAM APEX is office1-netbox (10.10.1.10:8000, NetBox 4.6.4 on the Office1 VM, docs/changelog-20260713-office1-netbox-deployed.md): it takes ALL VR1 writes (the section-4 buildout, the D-120 additions, any further refinement) and every consuming system (OpenTofu, MAAS, overlays, the Juju bundle) derives its NetBox values from it, not from netbox.baldurkeep.com. It is a working draft -- edited as the buildout proceeds -- but it is the authoritative NetBox. netbox.baldurkeep.com is held as a READ-ONLY v1 REFERENCE DRAFT for the duration -- it gets NO write during VR1. Merging the validated office1-netbox dataset back into netbox.baldurkeep.com is a DEFERRED, end-of-deployment MAYBE (8.6), not a Stage 2 gate. This DEFERS the apex write; it does NOT revoke D-010 -- netbox.baldurkeep.com remains the eventual IPAM source of truth.

The loop that builds and validates office1-netbox from the draft is unchanged; only the FINAL step (feeding the apex) moved OUT of Stage 2:

upstream draft (READ-ONLY, v1 reference)  --[prod-draft-dump.py]-->  netbox/draft/vr1-draft.json
vr1-draft.json                            --[sandbox-seed.py]------>  office1-netbox (the RECORD)
...apply the planned delta in office1-netbox (the section-4 importers + the D-120 additions)...
...prove office1-netbox == draft + EXACTLY the planned delta (fidelity check)...
validated dataset                         --[DEFERRED, end-of-deployment, operator-gated]-> apex

The importers NEVER --commit the apex as a side effect (8.4). Writing office1-netbox is normal and unguarded (it is in SANDBOX_HOSTS); writing netbox.baldurkeep.com stays behind --yes-write-upstream and is out of scope until the deferred merge-back.

8.2 Why a FIDELITY CHECK, and not just counts

Counts and idempotency CANNOT prove fidelity. On 2026-07-13 the seeder silently dropped the scope on 17 of 90 prefixes -- it mapped only dcim.site, and most of the VR1 draft is dcim.region-scoped -- and every count still matched and re-running stayed idempotent. It was caught by luck on a spot-check printing site None. netbox/sandbox-fidelity-check.py is the check that catches it on purpose: it diffs FIELD BY FIELD on every shared object, so a dropped or mangled field is a hard failure.

8.3 The 2026-07-14 hardening -- do NOT trust a pre-hardening "faithful" verdict

Until 2026-07-14 the fidelity check could return a FALSE GREEN. It only proved the sandbox delta was a SUBSET of the plan (unexpected = extra - expected); it NEVER proved the plan was CARRIED OUT. With ZERO of the 36 DC prefixes created, extra and unexpected were both empty, and it printed "Nothing lost, nothing stray" and exited 0. EXPECTED_NEW was an upper bound masquerading as an assertion. Fixed 2026-07-14 (docs/changelog-20260714-netbox-write-path-hardening.md): EXPECTED_NEW is now BOTH bounds (EXPECTED-BUT-ABSENT is a hard failure), and delta prefixes are checked for scope and role. A NEW harness tests/sandbox-fidelity-check/run-tests.sh (10/10; T3 IS the false-green scenario) ships with it -- the checker had shipped with NO harness, which is why the false green survived.

CONSEQUENCE: any "sandbox is faithful" claim dated before 2026-07-14 was made by a checker that could not see an unapplied or under-applied plan. Re-run the current checker before trusting fidelity (this is why the ledger flags a sandbox re-verify owed before C2).

Two blind spots remain OPEN (logged, not fixed): SAME-DUMPER (both sides use the same field list, so a field the dumper omits is invisible on both), and DUPLICATE-CIDR collapse (prefix-keyed dicts collapse duplicates; latent -- none upstream today). "Faithful" means faithful within the dumped field list, not absolutely.

8.4 Write-guard architecture -- FIVE tools, and how each is kept off the apex

"The sim never writes upstream" is enforced differently per tool. As of 2026-07-14 the three write-capable importers ALL gate an upstream --commit, but by two different mechanisms:

Before 2026-07-14 the two pynetbox importers had NO target guard at all; the only thing stopping an accidental upstream --commit was the WAF 403 (8.5) -- and fixing the WAF (below) removed that accidental safety, which is exactly why the explicit --yes-write-upstream gate was added in the same change (docs/changelog-20260714-netbox-importers-waf-useragent.md). The whole-plan preflights are the OTHER layer: both importers had previously died mid-loop and left the production IPAM half-populated (docs/changelog-20260714-netbox-write-path-hardening.md).

8.5 The upstream WAF trap (FIXED 2026-07-14)

Upstream NetBox sits behind a User-Agent-filtering WAF: urllib/requests/pynetbox get a 403 that looks EXACTLY like an auth failure, while the SAME token works from curl (measured 2026-07-13, re-measured 2026-07-14: curl -> 200, urllib/pynetbox -> 403; references/platform-traps.md). Do NOT diagnose it as a token problem and re-mint credentials -- confirm with curl first.

The three stdlib tools always sent an accepted UA (curl/8.5.0). The two pynetbox importers did NOT until 2026-07-14 -- meaning they could NEVER have written to the apex, only to the WAF-less sandbox. get_nb() in both now sets nb.http_session.headers["User-Agent"] = "curl/8.5.0" (docs/changelog-20260714-netbox-importers-waf-useragent.md). NOTE: pynetbox is NOT installed on vcloud -- the two pynetbox importers must run on office1-netbox (pynetbox 7.0.0), which reaches upstream through the edge.

8.6 C2 -- office1-netbox holds the validated dataset (apex write-back DEFERRED)

DOCFIX-195 (operator ruling 2026-07-15) REDEFINED C2. C2 is NO LONGER "feed the carve upstream" -- there is NO netbox.baldurkeep.com write in Stage 2 at all. C2 is met when office1-netbox (the VR1 IPAM apex) holds the COMPLETE validated dataset AND it is VERIFIED. Two sub-tasks remain:

(i) Re-run the HARDENED fidelity check against office1-netbox for the shared carve/roles/DCs -- the "faithful" verdict predates the 2026-07-14 hardening (8.3), so it must be re-established. (ii) Load the D-120 additions into office1-netbox -- 10.10.1.0/24's child ranges (ip-ranges: static .2-.49 / dynamic .100-.200 / node .201-.254) and the two service IP assignments (ip-addresses: office1-netbox 10.10.1.10, office1-tailscale 10.10.1.11) -- and verify them.

TOOLING GAP (logged DOCFIX-195, NOT fixed here). The D-120 objects are ip-ranges/ip-addresses -- object classes the sandbox loop does NOT cover: prod-draft-dump.py's ENDPOINTS and sandbox-fidelity-check.py's KEY/EXPECTED_NEW both stop at rirs/roles/regions/sites/aggregates/ prefixes. So the hardened fidelity check CANNOT see the D-120 load. Verifying sub-task (ii) requires EITHER extending both tools (add the two endpoints + KEY entries + EXPECTED_NEW bounds + delta field checks, each with its harness) OR a recorded manual read-back against office1-netbox. That choice is an open C2-execution sub-question, to be ruled when C2 runs.

The DEFERRED merge-back (end-of-deployment, MAYBE). If/when the operator elects to seed netbox.baldurkeep.com from the validated office1-netbox dataset, it is a production IPAM write and NOT part of the sim: (a) run the two pynetbox importers from office1-netbox (pynetbox + apex reach); (b) each needs --yes-write-upstream on top of --commit for the non-sandbox target; (c) the NetBox write-path bugs (session-ledger "NetBox WRITE-PATH BUGS") gate THAT write, not Stage 2 close; (d) present and individually approve each mutation -- never batched. As of 2026-07-15 the apex is untouched and still holds only the v1 draft (measured read-only).