Newer
Older
openstack-caracal-dc-dc / docs / changelog-20260709-netbox-dc-dc-pipeline.md

Changelog 2026-07-09 -- NetBox multi-DC/dual-stack import pipeline (DOCFIX-152)

No live infrastructure touched -- new script + test harness only. Closes tooling gap register item #3 in docs/dc-dc-deployment-workflow.md.

Item

1. DOCFIX-152 -- netbox/dc-dc-prefixes-import.py

FILES: netbox/dc-dc-prefixes-import.py (new), netbox/README.md (new VR1 section), tests/dc-dc-prefixes-import/ (new: run-tests.sh, test_logic.py, fake_pynetbox.py, stub_syspath/pynetbox.py).

WHAT: extends the v1 single-site IPv4-only NetBox import pattern (netbox/ipv4-prefixes-import.py) to VR1's two-DC, dual-stack model per D-101. Per DC (--dc dc1/--dc dc2), creates the six-plane IPv4 structure plus the D-101 family-matrix IPv6 legs (ULA for metal-admin/metal-internal/data-tenant/storage/replication, GUA for provider-public).

Nothing invented, consistent with this repo's hard rule: D-101 names three literals as NetBox-assigned, explicitly "NOT hardcoded in this decision" -- the org ULA /48, the per-DC GUA carve out of the real ARIN 2602:f3e2::/32 block, and DC2's v4 supernet. All three are REQUIRED environment variables (ORG_ULA_48, DC_GUA_PREFIX, and, for dc2 only, DC2_V4_SUPERNET) with NO defaults -- the script validates format and containment and FAILS LOUD if any is missing, malformed, out of range, or (for DC2's supernet) overlapping DC1. DC1's six v4 planes ARE hardcoded, deliberately: D-101 rules DC1 inherits DC0's layout UNCHANGED, so those CIDRs are the ADOPTED decision text itself, not an inference.

The exact per-plane subdivision of the ULA /48 and the GUA carve (which /64 goes to which plane) is this script's OWN PROPOSED scheme, documented in the module docstring as NOT itself an operator-ratified D-NNN sub-decision -- flagged for review before trusting real NetBox writes, the same posture D-102 had before its own ratification pass folded it into D-101.

FOUND AND FIXED while writing the test harness: the first draft's example GUA addresses (2602:f3e2:0:100::/40 style) had host bits set beyond the /40 boundary (the fourth hextet exceeds bit 40) -- Python's ipaddress.ip_network(..., strict=True) correctly rejected them. Corrected to 2602:f3e2:1000::/40 / 2602:f3e2:2000::/40 in both the script's own docstring examples and the test data.

Also found and fixed a real test-harness bug (not a script bug): the end-to-end test counted the substring "CREATED" in captured stdout to verify prefix-creation counts, but the script's own site-creation message also contains the word "CREATED" ('... -- CREATED.' when a new NetBox site is created), inflating the count by one on first run. Tightened the assertion to count "CREATED prefix" / "EXISTS prefix" specifically.

FLAGGED, not fixed (out of scope): netbox/ipv4-prefixes-import.py's existing IPAM role slugs (provider/metal/lbaas-management/ openstack-tenant) predate the D-052/D-053 six-plane cutover that scripts/lib-net.sh reflects. This new script uses NEW role slugs matching the CURRENT six-plane names -- a real, pre-existing staleness in the v1 script, called out in both the new script's docstring and netbox/README.md rather than silently perpetuated or silently fixed mid-step in an unrelated v1 artifact.

FLAGGED as a genuine open call, not silently decided: DC1's import may create prefixes numerically identical to the existing vr0-dc0 NetBox site's entries. NetBox permits duplicate global prefixes (soft warning, not a hard block) but whether to retire the vr0-dc0 entries once DC1 supersedes that rehearsal environment, or keep both for historical record, is an operator IPAM-hygiene decision -- the script prints a note and does not decide it.

VERIFIED, not just asserted:

  • New harness tests/dc-dc-prefixes-import/: 37/37 PASS. Pure-logic tests for every validator (validate_ula_48, validate_gua_prefix, validate_dc2_supernet) and carve function (carve_dc2_v4, carve_ula, carve_gua), plus a full fake-NetBox end-to-end run of main() for both DCs covering: happy path, idempotency (re-run creates nothing, reports all EXISTS), DC2's supernet is genuinely required (fails loud when unset), and argparse's --dc requirement/choice validation. The fake NetBox client (fake_pynetbox.py) implements only the specific pynetbox calls this script makes -- not a general mock.
  • bash scripts/repo-lint.sh: 0 fail, 1 documented legacy WARN. Caught and fixed a real CRLF regression during authoring: a Python-based find/replace used during test-data cleanup wrote the file back in Windows text mode (\r\n), which repo-lint's L1 check correctly flagged; fixed by rewriting the file in binary mode with LF-only line endings.
  • bash scripts/run-tests-all.sh: full gauntlet 23/45 FAILED -- the SAME 23 pre-existing environment-gap failures already documented in docs/changelog-20260709-dc-selector-convention.md (jq missing on this workstation, or live juju/cloud-dependent suites); the new dc-dc-prefixes-import suite itself is clean (37/37), confirming zero regressions.

UNVALIDATED regardless of the above: no live NetBox / pynetbox-reachable instance exists this session (operator's workstation, not the vopenstack-jesse jumphost) to run this end-to-end against a real server. The fake-NetBox tests confirm the script's OWN logic is internally consistent; they cannot confirm pynetbox's real API surface matches what this script assumes (which mirrors the same assumption the existing v1 scripts already make, uncontested, in this same directory).

REVERT: rm -f netbox/dc-dc-prefixes-import.py && rm -rf tests/dc-dc-prefixes-import && git checkout HEAD~ -- netbox/README.md (safe -- no other script calls this one yet).

Next actionable step

Tooling gap register #3 CLOSED (mechanism). Before a real run: the operator must generate the org ULA /48 (RFC 4193 random allocation) and obtain the per-DC GUA carve from whoever manages 2602:f3e2::/32 region-0 /36 -- neither exists yet (this was already true; this script doesn't create that gap, it surfaces it explicitly via fail-loud env-var requirements instead of a silent default). Stage 2/3 runbooks (not yet authored) are the first real consumers of this pipeline's output.