Newer
Older
openstack-caracal-dc-dc / docs / changelog-20260717-netbox-apex-recovery-edge-wan-import.md

2026-07-17 -- office1-netbox apex recovery + restart policy + full D-124/edge-WAN import (Blocks A + B)

Context

Asked to run the D-124/edge-WAN NetBox imports while access was up, I found the IPAM apex was DOWN: the netbox-docker stack (NetBox 4.6, /opt/netbox-docker) had cleanly exited (0) ~3h earlier and was not auto-restarting (its instance was stopped/rebooted). Operator approved bringing it up + adding a restart policy.

What changed (on office1-netbox, 10.10.1.10 -- host state, NOT repo-tracked)

  • Apex brought up: cd /opt/netbox-docker && sudo docker compose up -d -> all 5 containers healthy; NetBox answers HTTP 302 on :8000.
  • Restart policy added: docker-compose.override.yml now sets restart: unless-stopped on all five services (netbox, netbox-worker, postgres, redis, redis-cache) -- verified via docker inspect. So the apex survives its instance's reboots (incl. the upcoming bookend). Original override backed up on the host as docker-compose.override.yml.bak-20260717. (Config docker compose config-validated before apply.)

Apex writes (Block A -- gated, dry-run-verified then committed)

  • d115-office-carve.py (dry-run): all 11 objects already present (office+edge roles, site vr1-off1, prefixes) -- no-op; edge-WAN precondition satisfied.
  • dc-edge-wan-import.py --commit: CREATED 172.30.2.0/24 (id=135, role=edge, scope=vr1-dc0) + 172.30.3.0/24 (id=136, role=edge, scope=vr1-dc1). Re-run dry-run confirms both EXIST (0 to create). Token read server-side (sudo cat /root/netbox-secrets/api.token ON the apex host) -- never entered agent context. Sandbox target (localhost) -- no upstream write; netbox.baldurkeep.com untouched.

Block B (D-124 rack transit + rack IP) -- COMPLETE (operator chose: write the seeder)

dc-rack-mgmt-import.py was die-if-absent on role 'transit' + the 172.31.0.0/24 container, which had no seeder. Operator ruled: write one.

  • netbox/d124-transit-seed.py (NEW) + tests/d124-transit-seed/run-tests.sh (21/21) -- mirrors d115-office-carve (NB client, sandbox gate, WAF UA, dry-by-default, --yes-write-upstream). Seeds ONLY the transit role + the 172.31.0.0/24 UNSCOPED container (mirrors edge's /16: the container is shared, each DC's link is scoped by dc-rack-mgmt). The harness includes a cross-file consistency check so the seeded role/container can never drift from dc-rack-mgmt-import's required ROLE_SLUG/CONTAINER.
  • d124-transit-seed.py --commit: CREATED role transit (id=30) + 172.31.0.0/24 (id=137, container).
  • dc-rack-mgmt-import.py --transit-cidr 172.31.0.0/30 --rack-ip 10.12.8.2 --commit: CREATED 172.31.0.0/30 (id=138, role=transit, scope=vr1-dc0) + 10.12.8.2/22 (id=3, dns=vvr1-dc0). Matches the D-124 ruled values + the vr1_dc0_rack_* tfvars. Re-run dry-run confirms both EXIST (0 to create).

So the full NetBox D-124/edge-WAN import is COMPLETE in the apex. Copied importers cleaned from the apex /tmp. Sandbox-only throughout (netbox.baldurkeep.com untouched); token read server-side (never in agent context).

Revert

  • Apex objects (delete via NetBox UI/API -- the importers have no un-import): edge /24s id=135,136; transit /30 id=138; rack IP id=3; container 172.31.0.0/24 id=137; role transit id=30.
  • Tooling: git rm netbox/d124-transit-seed.py tests/d124-transit-seed/.
  • Restart policy: restore docker-compose.override.yml.bak-20260717 + docker compose up -d.
  • Apex up/down is operational state, not a repo change.