Newer
Older
openstack-caracal-ipv4 / runbooks / phase-04-network-carve.md

Phase 04 -- Network Carve (provider external network + IPAM reference)

Create the Neutron external provider network that sources floating IPs and tenant router gateways (the FIP/ext_net leg of Option B), on top of the MAAS address carve done pre-deploy in phase-00. Also the IPAM reference for where addresses live.

Decisions: D-003 (provider shared-L2: public API VIPs + FIP/ext_net, Option B; FIP pool 10.12.5.0-10.12.7.254), the IPv4 provider/internal carve (front-loaded VIP /26), KI-P3-001 (VIP/primary collision -> the reserved-range fix). Troubleshooting: appendix-A -- KI-P3-001.

NOTE on what the "carve" is split across:

  • The MAAS ADDRESS carve (delete stale iprange 2; reserve the front-loaded VIP /26 on provider + metal) runs POST-TEARDOWN / PRE-REDEPLOY -- it is in phase-00, because you must reserve the VIP block before deploying onto it.
  • The bundle's vip: values come from that reserved block -- phase-01.
  • THIS phase creates the Neutron EXTERNAL provider network on top of the carve -- the only post-deploy network mutation.

IPAM carve reference (design; full detail in the carve doc + design-decisions D-003)

Provider 10.12.4.0/22 (role Provider; shared-L2, Option B):

  • 10.12.4.1 provider gateway
  • 10.12.4.2 - 10.12.4.63 public API HA VIPs (front-loaded /26) -- MAAS RESERVED; EXCLUDED from the
                           neutron allocation_pool. Every bundle public `vip:` is from here.
  • 10.12.4.64 - 10.12.4.254 host + container primaries (MAAS auto-static)
  • 10.12.5.0 - 10.12.7.254 FIP pool / ext_net allocation_pool (this phase's subnet) -- MAAS RESERVED

Metal 10.12.8.0/22 (role Metal; charm control plane + internal VIPs):

  • 10.12.8.2 - 10.12.8.63 internal/admin API HA VIPs (front-loaded /26) -- MAAS RESERVED
  • 10.12.8.64 - 10.12.8.254 host + container primaries (incl single-unit svc endpoints, e.g. radosgw)
  • 10.12.9.0 - 10.12.11.254 MAAS PXE/enlistment DHCP (dynamic; iprange id 1)

KI-P3-001 invariant: on every space carrying juju VIPs (provider AND metal), the VIP block is MAAS-reserved and DISTINCT from the primary range and any neutron allocation_pool, so a MAAS auto-static primary can never land on a configured VIP. (Root cause of the original collision: provider had NO VIP reservation, so MAAS auto-assigned container primaries .225/.226/.227 onto the .224-.236 VIP block.)

Prerequisites (must be true entering phase-04)

  • phase-01/02/03 done (deploy + vault + core verify); charms active/idle.
  • phase-00 MAAS carve applied: FIP pool 10.12.5.0-10.12.7.254 RESERVED on the provider subnet (iprange id 3), and the front-loaded VIP /26 reservations present.
  • Provider segment is FLAT on physnet1 (bundle ovn-bridge-mappings physnet1:br-ex; flat-network-providers=physnet1). The provider /22 is untagged L2 (not vlan).

Constants and env-literals (TAG: confirm per site on rebuild)

  • ENV(physnet) physnet1
  • ENV(ext-net) provider-ext ENV(ext-subnet) provider-ext-fip
  • ENV(ext-cidr) 10.12.4.0/22 (full provider /22 so .1 gateway is in-subnet + FIP ARP spans the L2)
  • ENV(fip-pool) 10.12.5.0 - 10.12.7.254 (D-003 Option-A; ~765 FIPs; full pool, not a slice)
  • gateway 10.12.4.1 -- READ DYNAMICALLY from MAAS, never hardcoded.

Run-location legend

  • # RUN: jumphost -- vopenstack-jesse as jessea123, admin-openrc sourced; openstack + maas admin.

Step 4.1 -- Create the external provider network (B29; idempotent)

# RUN: jumphost --external but NOT --share (usable as router gateway + FIP source, but tenants cannot attach instance ports to the provider segment -- Option B isolation). --no-dhcp (MAAS owns DHCP on this segment; FIPs are NAT'd). The subnet is the FULL provider /22 with the FIP pool as the allocation_pool; the VIP block and primaries are MAAS-reserved so neutron never allocates them.

Read-only pre-check first (verify the FIP pool is MAAS-reserved so neutron can own it):

# RUN: jumphost (MAAS profile is 'admin'; never run 'maas list' -- it prints the API key, DOCFIX-016)
maas admin ipranges read | jq -r '.[] | select(.type=="reserved") | "\(.start_ip)-\(.end_ip) subnet=\(.subnet.id) [\(.comment)]"'
# expect a reserved 10.12.5.0-10.12.7.254 on subnet id 1 (provider); + the front-loaded VIP /26 reservations.

Create (idempotent ( set -e ); dynamic gateway; tags applied via set, not an inline --tag flag):

source ~/admin-openrc
( set -e
  PHYSNET=physnet1; EXT_NET=provider-ext; EXT_SUBNET=provider-ext-fip
  EXT_CIDR=10.12.4.0/22; FIP_START=10.12.5.0; FIP_END=10.12.7.254
  GW=$(maas admin subnet read 1 | jq -r '.gateway_ip')                 # dynamic; never hardcode .1
  [ "$GW" = "10.12.4.1" ] || { echo "GATE FAIL: MAAS provider gateway='$GW' (expected 10.12.4.1)"; exit 1; }
  echo "[OK] gateway $GW"
  if openstack network show "$EXT_NET" -f value -c id >/dev/null 2>&1; then
    echo "[SKIP] network $EXT_NET exists"
  else
    openstack network create --external --provider-network-type flat \
      --provider-physical-network "$PHYSNET" "$EXT_NET" -f value -c id
    openstack network set --tag role=provider "$EXT_NET"
    echo "[OK] network $EXT_NET created + tagged"
  fi
  if openstack subnet show "$EXT_SUBNET" -f value -c id >/dev/null 2>&1; then
    echo "[SKIP] subnet $EXT_SUBNET exists"
  else
    openstack subnet create --network "$EXT_NET" --subnet-range "$EXT_CIDR" \
      --gateway "$GW" --no-dhcp --allocation-pool start="$FIP_START",end="$FIP_END" \
      "$EXT_SUBNET" -f value -c id
    openstack subnet set --tag role=provider --tag "netbox-iprange=${FIP_START}-${FIP_END}" "$EXT_SUBNET"
    echo "[OK] subnet $EXT_SUBNET created + tagged"
  fi
  echo "=== CONFIRM ==="
  openstack network show "$EXT_NET" -f json | jq -c '{name, external: ."router:external", type: ."provider:network_type", physnet: ."provider:physical_network", shared, tags}'
  openstack subnet show "$EXT_SUBNET" -f json | jq -c '{name, cidr, gateway_ip, enable_dhcp, allocation_pools, tags}'
)

GATE: provider-ext external=true, type=flat, physnet=physnet1, shared=false; provider-ext-fip cidr=10.12.4.0/22, gateway 10.12.4.1, enable_dhcp=false, allocation_pools=[10.12.5.0-10.12.7.254].


EXIT GATE (phase-04 complete)

  • provider-ext (external, flat/physnet1, not shared) + provider-ext-fip (full /22, FIP allocation pool, no-dhcp) present and tagged role=provider.
  • FIP allocation + tenant router gateways are now possible (needed by phase-06 mgmt VM FIP, phase-08 cluster FIPs + LB validation).

As-built reference (2026-06-03 run -- audit trail)

  • network provider-ext = 70b34bb2-3afb-4b43-96d3-f520dbcbf9a8 (external, flat, physnet1, shared=false, role=provider)
  • subnet provider-ext-fip = e3afcbae-ec34-4125-9007-2bfa51851422 (cidr 10.12.4.0/22, gateway 10.12.4.1, enable_dhcp=false, alloc 10.12.5.0-10.12.7.254, tags role=provider + netbox-iprange=10.12.5.0-10.12.7.254)
  • Transitional note: MAAS already carried the front-loaded VIP reservations (.2-.63 provider + .8.2-.63 metal; old D-020 .8.224-.254 gone) ahead of the bundle's interim .50-.60 VIPs -- harmless (a reserved range blocks future auto-assign, does not evict live VIPs). NetBox modeling DEFERRED (allocate after a clean deploy).

Next

phase-05 -- octavia enablement.