Newer
Older
openstack-caracal-dc-dc / netbox / dc-dc-prefixes-import.py
#!/usr/bin/env python3
"""
NetBox multi-DC / dual-stack prefix import for VR1 DC-DC (Omega Cloud).

Closes tooling gap register item #3 in docs/dc-dc-deployment-workflow.md:
extends the v1 (netbox/ipv4-prefixes-import.py) single-site IPv4-only import
to the VR1 two-DC, dual-stack model ratified in D-101 (docs/design-decisions.md).

Per DC (dc1 or dc2, selected via --dc, mirroring the scripts/lib-net.sh
$DC selector convention, DOCFIX-151), creates the six-plane structure with
the D-101 family matrix:

  Plane           v4                  v6 family
  provider-public  (see below)         GUA (dual-stack)
  metal-admin      (see below)         ULA (dual-stack, 2026-07-09 amendment)
  metal-internal   (see below)         ULA (dual-stack)
  data-tenant      (see below)         ULA-only
  storage          (see below)         ULA-only
  replication      (see below)         ULA-only

DC1 v4: INHERITS the DC0 six-plane CIDRs unchanged (D-101 -- "DC1 equals the
validated template"), i.e. the SAME literals as netbox/ipv4-prefixes-import.py,
hardcoded here for that reason (this is not an inferred value; it is the
explicit text of an ADOPTED decision). Scoped to a DISTINCT NetBox site
(vr1-dc1), not the existing vr0-dc0 site.

DC2 v4: D-101 explicitly states DC2's supernet is "NetBox-assigned... not
hardcoded." No default is provided here -- DC2_V4_SUPERNET is a REQUIRED
environment variable with no fallback. Running with --dc dc2 and no
DC2_V4_SUPERNET set FAILS LOUD rather than guessing or reusing DC1's range.

IPv6 (both DCs): D-101 names two literals as NetBox-assigned and "NOT
hardcoded in this decision": the org ULA /48, and the per-DC GUA carve out
of the already-real ARIN 2602:f3e2::/32 region-0 /36 (the /32 itself IS a
real literal quoted in D-101's own text -- only its per-DC subdivision is
still open). Both are REQUIRED environment variables with no defaults:
ORG_ULA_48 and DC_GUA_PREFIX.

IMPORTANT -- what this script does NOT do: it does not invent the org ULA
/48 or the GUA carve. The operator must generate/obtain those (an RFC 4193
random ULA /48, and the actual per-DC GUA assignment from whoever manages
2602:f3e2::/32) and pass them in. This script only validates their FORMAT
and then applies a DETERMINISTIC, DOCUMENTED SUBDIVISION into per-plane
prefixes (see SUBCARVE SCHEME below). That subdivision scheme is this
script's own proposal -- it is NOT itself a ratified D-NNN sub-decision.
Review it before trusting the values this script would write to a real
NetBox. (Precedent: D-102's MTU sub-policy was folded into D-101 only after
explicit operator ratification; this carve scheme has not had that pass.)

SUBCARVE SCHEME (proposed, unratified -- review before real use):
  - ULA: each DC gets a /56 out of the org /48 (dc1 = ::0/56, dc2 = ::1:00/56,
    i.e. the 8 bits immediately after the /48 index the DC -- room for up to
    256 DCs, far beyond VR1's two). Within a DC's /56, each of the three
    ULA-family planes (data-tenant, storage, replication) plus the two
    dual-stack planes' ULA leg (metal-admin, metal-internal) gets a /64,
    indexed in the fixed plane order below (provider-public has no ULA leg,
    so its index is skipped -- the /64 offsets are NOT contiguous with the
    plane list index, they are assigned only to the five ULA-bearing planes).
  - GUA: each DC's already-carved GUA prefix (DC_GUA_PREFIX, e.g. a /40 or
    /44 -- length is read from what's actually passed in, not assumed) is
    subdivided the same way: provider-public gets the whole thing as a
    single /64 (VR1 has no other GUA-bearing plane in the matrix above;
    tenant-facing GUA delegation is a separate, later, per-tenant carve out
    of this same block -- not this script's job, see netbox/README.md).

Idempotent: re-running is safe (existing prefixes are detected and skipped
unless --update). NetBox version: 4.x (scope_type/scope_id, not legacy site=).

Usage:
    # DC1 (v4 inherited, only v6 top-level blocks required):
    NETBOX_URL=https://netbox.baldurkeep.com NETBOX_TOKEN=<token> \\
        ORG_ULA_48=fd00:1234:5678::/48 DC_GUA_PREFIX=2602:f3e2:1000::/40 \\
        python3 dc-dc-prefixes-import.py --dc dc1

    # DC2 (v4 supernet ALSO required, nothing defaulted):
    NETBOX_URL=... NETBOX_TOKEN=... \\
        ORG_ULA_48=fd00:1234:5678::/48 DC_GUA_PREFIX=2602:f3e2:2000::/40 \\
        DC2_V4_SUPERNET=10.13.0.0/19 \\
        python3 dc-dc-prefixes-import.py --dc dc2

    # Preview only, no writes:
    ... python3 dc-dc-prefixes-import.py --dc dc1 --verify-only

KNOWN PRE-EXISTING STALENESS (flagged, not fixed here): netbox/ipv4-prefixes-
import.py's role slugs (provider/metal/lbaas-management/openstack-tenant)
predate the D-052/D-053 six-plane cutover that scripts/lib-net.sh reflects.
This script uses NEW role slugs matching the CURRENT six-plane names
(provider-public/metal-admin/metal-internal/data-tenant/storage/replication)
since VR1 is being built against the current model, not the v1-era one.
These roles must exist in NetBox before running (create out of band, same
assumption netbox/ipv4-prefixes-import.py already makes for its own roles --
this script does not create IPAM roles, only prefixes/sites use them).

UNVALIDATED: no NETBOX_URL/live NetBox and no `pynetbox`-reachable instance
were available this session (workstation, not the vopenstack-jesse jumphost)
to run this end-to-end. Reviewed against the pynetbox API shape used
successfully by the existing v1 scripts in this same directory, not executed.
"""

from __future__ import annotations

import argparse
import ipaddress
import os
import sys

try:
    import pynetbox
except ImportError:
    sys.stderr.write("ERROR: pynetbox not installed. pip install pynetbox\n")
    sys.exit(1)

# -----------------------------------------------------------------------------
# DC1/DC2 site identity. Slugs are new -- no prior convention existed in this
# repo (grepped opentofu/*.tf and the buildout design; none found) -- chosen
# to match the D-106 Designate naming already ratified (dc1/dc2, vr1 label).
# -----------------------------------------------------------------------------

SITES = {
    "dc1": {"slug": "vr1-dc1", "name": "VR1 DC1"},
    "dc2": {"slug": "vr1-dc2", "name": "VR1 DC2"},
}

# Fixed plane order -- must match scripts/lib-net.sh's PLANE_NAME exactly
# (CIDR-keyed there; name-keyed here, same six planes, same order).
PLANE_ORDER = [
    "provider-public",
    "metal-admin",
    "metal-internal",
    "data-tenant",
    "storage",
    "replication",
]

# D-101 family matrix: which v6 family (if any) each plane carries.
# "gua" = dual-stack v4+GUA; "ula" = dual-stack v4+ULA; "ula-only" = no v4
# plane object created for the v6 leg (data-tenant/storage/replication ARE
# dual-stack-capable per D-102's Phase-A rollback posture text, but the
# ADOPTED default is IPv6-only for those three -- see D-101/D-102 body).
PLANE_V6_FAMILY = {
    "provider-public": "gua",
    "metal-admin": "ula",
    "metal-internal": "ula",
    "data-tenant": "ula-only",
    "storage": "ula-only",
    "replication": "ula-only",
}

# DC1 v4: the DC0 six-plane CIDRs, UNCHANGED, per D-101 ("DC1 equals the
# validated template"). Identical to netbox/ipv4-prefixes-import.py's
# historical single-plane set is NOT the comparison here -- these are the
# CURRENT six-plane values from scripts/lib-net.sh's PLANE_CIDRS, which is
# this repo's single source of truth for that layout.
DC1_V4_PREFIXES = {
    "provider-public": "10.12.4.0/22",
    "metal-admin": "10.12.8.0/22",
    "metal-internal": "10.12.12.0/22",
    "data-tenant": "10.12.16.0/22",
    "storage": "10.12.32.0/22",
    "replication": "10.12.36.0/22",
}

# DC2 v4: NO default. Must come from DC2_V4_SUPERNET (env var), carved into
# six sequential /22s in PLANE_ORDER, exactly mirroring DC1's /22-per-plane
# structure. The supernet must be large enough to hold six non-overlapping
# /22s (a /19 holds exactly eight /22s -- six used, two spare; anything
# smaller than /19 FAILS LOUD).
DC2_MIN_SUPERNET_PREFIXLEN = 19  # a /19 = exactly 8x /22, room for the six planes


def die(msg: str, code: int = 1) -> None:
    sys.stderr.write(f"ERROR: {msg}\n")
    sys.exit(code)


def get_nb() -> "pynetbox.api":
    url = os.environ.get("NETBOX_URL")
    token = os.environ.get("NETBOX_TOKEN")
    if not url:
        die("NETBOX_URL environment variable not set")
    if not token:
        die("NETBOX_TOKEN environment variable not set")
    nb = pynetbox.api(url, token=token)
    try:
        _ = nb.status()
    except Exception as exc:  # noqa: BLE001
        die(f"Could not reach NetBox at {url}: {exc}")
    return nb


def find_or_create_site(nb, slug: str, name: str):
    site = nb.dcim.sites.get(slug=slug)
    if site is not None:
        return site, False
    # Minimal site record -- status defaults to NetBox's own default (active).
    created = nb.dcim.sites.create(name=name, slug=slug)
    return created, True


def find_role(nb, slug: str):
    role = nb.ipam.roles.get(slug=slug)
    if role is None:
        die(
            f"IPAM role with slug '{slug}' not found in NetBox. Create it first "
            f"(out of band) -- this script does not create roles, matching "
            f"netbox/ipv4-prefixes-import.py's existing assumption."
        )
    return role


# -----------------------------------------------------------------------------
# Required-literal validation. Each of these dies with a specific, actionable
# message rather than silently defaulting -- the whole point of this pipeline
# closing gap #3 is that these values are NOT invented here.
# -----------------------------------------------------------------------------


def require_env(name: str, why: str) -> str:
    val = os.environ.get(name)
    if not val:
        die(f"{name} environment variable is required and not set -- {why}")
    return val


def validate_ula_48(cidr: str) -> ipaddress.IPv6Network:
    try:
        net = ipaddress.ip_network(cidr, strict=True)
    except ValueError as exc:
        die(f"ORG_ULA_48='{cidr}' is not a valid CIDR: {exc}")
    if net.version != 6:
        die(f"ORG_ULA_48='{cidr}' is not an IPv6 prefix")
    if net.prefixlen != 48:
        die(f"ORG_ULA_48='{cidr}' must be a /48 (got /{net.prefixlen})")
    # RFC 4193 ULA range is fc00::/7; the locally-assigned form sets the L bit
    # (fd00::/8). Reject anything outside fc00::/7 -- a GUA or other range
    # here would be a real misconfiguration, not a style nit.
    if not net.subnet_of(ipaddress.ip_network("fc00::/7")):
        die(f"ORG_ULA_48='{cidr}' is not within the ULA range fc00::/7 (RFC 4193)")
    return net


def validate_gua_prefix(cidr: str) -> ipaddress.IPv6Network:
    try:
        net = ipaddress.ip_network(cidr, strict=True)
    except ValueError as exc:
        die(f"DC_GUA_PREFIX='{cidr}' is not a valid CIDR: {exc}")
    if net.version != 6:
        die(f"DC_GUA_PREFIX='{cidr}' is not an IPv6 prefix")
    parent = ipaddress.ip_network("2602:f3e2::/32")
    if not net.subnet_of(parent):
        die(
            f"DC_GUA_PREFIX='{cidr}' is not within the D-101-cited ARIN block "
            f"{parent} -- if this is a genuinely different GUA allocation, "
            f"that is itself a design-decision-worthy change (grep D-101 "
            f"first), not something this script should silently accept."
        )
    if net.prefixlen >= 64:
        die(f"DC_GUA_PREFIX='{cidr}' must be shorter than /64 (need room to carve a /64 for provider-public)")
    return net


def validate_dc2_supernet(cidr: str) -> ipaddress.IPv4Network:
    try:
        net = ipaddress.ip_network(cidr, strict=True)
    except ValueError as exc:
        die(f"DC2_V4_SUPERNET='{cidr}' is not a valid CIDR: {exc}")
    if net.version != 4:
        die(f"DC2_V4_SUPERNET='{cidr}' is not an IPv4 prefix")
    if net.prefixlen > DC2_MIN_SUPERNET_PREFIXLEN:
        die(
            f"DC2_V4_SUPERNET='{cidr}' (/{net.prefixlen}) is too small to hold "
            f"six /22 planes -- need at least a /{DC2_MIN_SUPERNET_PREFIXLEN}"
        )
    dc1_nets = [ipaddress.ip_network(c) for c in DC1_V4_PREFIXES.values()]
    for d1 in dc1_nets:
        if net.overlaps(d1):
            die(
                f"DC2_V4_SUPERNET='{cidr}' overlaps DC1's inherited range {d1} -- "
                f"D-101 requires DC2 to be non-overlapping. Choose a different supernet."
            )
    return net


def carve_dc2_v4(supernet: ipaddress.IPv4Network) -> dict:
    subnets = list(supernet.subnets(new_prefix=22))
    if len(subnets) < len(PLANE_ORDER):
        die(
            f"DC2_V4_SUPERNET yields only {len(subnets)} /22 subnet(s), "
            f"need {len(PLANE_ORDER)}"
        )
    return {plane: str(subnets[i]) for i, plane in enumerate(PLANE_ORDER)}


def carve_ula(org_ula_48: ipaddress.IPv6Network, dc: str) -> dict:
    dc_index = 0 if dc == "dc1" else 1
    # /48 -> per-DC /56 (8-bit DC index). See SUBCARVE SCHEME in module docstring.
    dc_56_candidates = list(org_ula_48.subnets(new_prefix=56))
    dc_56 = dc_56_candidates[dc_index]
    ula_planes = [p for p in PLANE_ORDER if PLANE_V6_FAMILY[p] in ("ula", "ula-only")]
    dc_64_candidates = list(dc_56.subnets(new_prefix=64))
    return {plane: str(dc_64_candidates[i]) for i, plane in enumerate(ula_planes)}


def carve_gua(dc_gua_prefix: ipaddress.IPv6Network) -> dict:
    # Only provider-public carries GUA in the current family matrix.
    gua_64_candidates = list(dc_gua_prefix.subnets(new_prefix=64))
    return {"provider-public": str(gua_64_candidates[0])}


# -----------------------------------------------------------------------------
# NetBox writes
# -----------------------------------------------------------------------------


def create_or_report_prefix(nb, cidr: str, role_slug: str, site, description: str, update: bool) -> None:
    role = find_role(nb, role_slug)
    existing = nb.ipam.prefixes.get(prefix=cidr)
    payload = {
        "prefix": cidr,
        "role": role.id,
        "description": description,
        "scope_type": "dcim.site",
        "scope_id": site.id,
    }
    if existing is None:
        created = nb.ipam.prefixes.create(**payload)
        print(f"  CREATED prefix {cidr} (id={created.id}) role={role_slug}")
    elif update:
        existing.update(payload)
        print(f"  UPDATED prefix {cidr} (id={existing.id}) role={role_slug}")
    else:
        print(f"  EXISTS  prefix {cidr} (id={existing.id}) -- skipped (use --update to overwrite)")


def verify(nb, site, v4: dict, v6_ula: dict, v6_gua: dict) -> None:
    print()
    print("=" * 72)
    print(f"Verification -- final state for site {site.name} (id={site.id})")
    print("=" * 72)
    print("\nIPv4 (per plane):")
    for plane in PLANE_ORDER:
        cidr = v4[plane]
        p = nb.ipam.prefixes.get(prefix=cidr)
        status = "MISSING" if p is None else f"OK (id={p.id})"
        print(f"  {plane.ljust(16)} {cidr.ljust(18)} {status}")
    print("\nIPv6 ULA (per plane, where applicable):")
    for plane, cidr in v6_ula.items():
        p = nb.ipam.prefixes.get(prefix=cidr)
        status = "MISSING" if p is None else f"OK (id={p.id})"
        print(f"  {plane.ljust(16)} {cidr.ljust(24)} {status}")
    print("\nIPv6 GUA (per plane, where applicable):")
    for plane, cidr in v6_gua.items():
        p = nb.ipam.prefixes.get(prefix=cidr)
        status = "MISSING" if p is None else f"OK (id={p.id})"
        print(f"  {plane.ljust(16)} {cidr.ljust(24)} {status}")


def main() -> int:
    parser = argparse.ArgumentParser(description=__doc__.split("\n\n", 1)[0])
    parser.add_argument("--dc", required=True, choices=["dc1", "dc2"], help="Which DC to import (no default -- explicit, measured selection per the $DC convention)")
    parser.add_argument("--update", action="store_true", help="Update existing prefixes in place")
    parser.add_argument("--verify-only", action="store_true", help="Skip writes; only print verification block")
    args = parser.parse_args()

    dc = args.dc
    site_cfg = SITES[dc]

    org_ula_48 = validate_ula_48(require_env("ORG_ULA_48", "D-101's org ULA /48 -- not defaulted, must be a real RFC 4193 /48 you generated"))
    dc_gua_prefix = validate_gua_prefix(require_env("DC_GUA_PREFIX", f"the {dc} carve out of ARIN 2602:f3e2::/32 region-0 /36 cited in D-101 -- not defaulted"))

    if dc == "dc1":
        v4 = dict(DC1_V4_PREFIXES)
    else:
        dc2_supernet = validate_dc2_supernet(require_env("DC2_V4_SUPERNET", "D-101 explicitly requires DC2's v4 supernet be NetBox-assigned, not hardcoded"))
        v4 = carve_dc2_v4(dc2_supernet)

    v6_ula = carve_ula(org_ula_48, dc)
    v6_gua = carve_gua(dc_gua_prefix)

    nb = get_nb()
    site, site_created = find_or_create_site(nb, site_cfg["slug"], site_cfg["name"])
    print(f"Connected. Site '{site_cfg['name']}' (id={site.id}){' -- CREATED' if site_created else ''}.")

    if dc == "dc1":
        print(
            "\nNOTE: this may create prefixes identical to the existing 'vr0-dc0' "
            "site's entries (D-101: DC1 inherits DC0's v4 layout unchanged). "
            "NetBox permits duplicate global prefixes (soft-warn, not a hard "
            "block) but this is a real IPAM-hygiene decision for the operator: "
            "either retire/delete the vr0-dc0 entries once DC1 supersedes that "
            "rehearsal environment, or knowingly keep both for historical "
            "record. This script does not decide that for you."
        )

    if not args.verify_only:
        print("\nIPv4 prefixes:")
        for plane in PLANE_ORDER:
            create_or_report_prefix(
                nb, v4[plane], plane, site,
                f"VR1 {dc.upper()} {plane} (v4; D-101)", args.update,
            )
        print("\nIPv6 ULA prefixes:")
        for plane, cidr in v6_ula.items():
            create_or_report_prefix(
                nb, cidr, plane, site,
                f"VR1 {dc.upper()} {plane} (ULA leg; D-101, proposed subcarve -- see script docstring)",
                args.update,
            )
        print("\nIPv6 GUA prefixes:")
        for plane, cidr in v6_gua.items():
            create_or_report_prefix(
                nb, cidr, plane, site,
                f"VR1 {dc.upper()} {plane} (GUA leg; D-101, proposed subcarve -- see script docstring)",
                args.update,
            )

    verify(nb, site, v4, v6_ula, v6_gua)
    print("\nDone.")
    return 0


if __name__ == "__main__":
    sys.exit(main())