Newer
Older
openstack-caracal-dc-dc / docs / network-space-binding-reference.md

Network-space binding & placement reference (VR1 per-DC)

Purpose. A grounded, cross-checkable reference for the six network planes, what each carries, and where every application's Juju bindings land -- so a future session can VERIFY a binding against intent without re-researching or re-engineering it. Built 2026-08-05 from the live bundle + relation topology + the 2026-08-03 plane-purpose sweep, verified against dataflow.

Authority. This is a DERIVED reference, not a new authority. The governing decision is D-052 (docs/design-decisions.md) and its amendments; plane CIDRs come from scripts/lib-net.sh; deviations are ruled in their own D-NNN. Where this doc and D-052 ever disagree, D-052 wins and this doc is corrected. Regenerate the matrix from bundle.yaml when bindings change (the generator is trivial: invert applications[*].bindings by space).


1. The six planes

CIDRs are per-DC (dc0 / dc1). metal-admin and metal-internal are the two "metal" planes that split the legacy single metal space (D-052).

Plane dc0 CIDR dc1 CIDR Encap Routes? Role RHOSP analog
provider-public 10.12.4.0/22 10.12.64.0/22 untagged YES (edge .1) Public API VIPs + floating IPs External
metal-admin 10.12.8.0/22 10.12.68.0/22 untagged no (flat L2; controller lives here) Management/OAM: MAAS/PXE, juju agent<->controller, operator, monitoring, admin API, and the '' default of every app. DC-LOCAL Management + Provisioning
metal-internal 10.12.12.0/22 10.12.72.0/22 tagged VLAN no (isolated, D-125) All service-to-service control: internal API, shared-db, amqp, certificates/secrets, cache, cluster peers, ovsdb*, identity. MAY cross DC fiber Internal API
data-tenant 10.12.16.0/22 10.12.76.0/22 tagged no Tenant geneve overlay Tenant
storage 10.12.32.0/22 10.12.80.0/22 tagged no Ceph public (RBD client <-> mon/OSD) Storage
replication 10.12.36.0/22 10.12.84.0/22 tagged no Ceph OSD cluster replication + RBD mirror Storage-Management

Reachability mechanics (why the '' default matters): metal-admin does not route in VR1 (the .1 gateway was removed 2026-07-29). Nodes reach the juju controller (.5 on metal-admin) by L2 adjacency on the flat /22. A unit with no metal-admin leg cannot reach the controller at all -- it cannot download agent binaries and its agent never starts. This is the single most important operational fact behind the binding rules below.


2. The binding rule (the one that governs everything)

Every app binds '': metal-admin (its management leg), and binds each explicit endpoint to the plane matching that endpoint's TRAFFIC TYPE.

The Juju '' (default) binding has dual significance: (a) the fallback space for any unbound endpoint, and (b) the unit's primary NIC -- its private-address and its path to the controller. It is the MANAGEMENT binding, not a "what kind of service is this" tag. A pure storage service (ceph-osd) still binds '': metal-admin; a pure DB service (mysql-innodb-cluster) still binds '': metal-admin. Their data rides storage / metal-internal via EXPLICIT endpoint bindings; their management stays on metal-admin. Binding '' to an isolated data plane strips the management leg -- this is exactly the ovn-central regression that the 2026-08-03 amendment introduced and the 2026-08-05 re-amendment reverted (Section 5).

Endpoint -> plane, by traffic type:

Traffic type Plane Example endpoints
Public API / FIPs provider-public :public, designate:dnsaas (DNS-native VIP)
Internal API, shared-db, amqp, certificates, identity, cache, HA/cluster peers, ovsdb* metal-internal :shared-db, :amqp, :certificates, :identity-service, :ha, :ovsdb*
Ceph client + mon + ceph control storage :ceph, ceph-mon:{client,osd,mon,radosgw,...}, ceph-osd:public
Ceph OSD replication + RBD mirror replication ceph-osd:cluster, ceph-mon:cluster, ceph-rbd-mirror:{ceph-local,ceph-remote}
Tenant geneve overlay data-tenant nova-compute:neutron-plugin, ovn-chassis:data, ovn-chassis-octavia:data, octavia:ovsdb-cms
Management/OAM (fallback) metal-admin '' default (all apps)

3. Per-app binding matrix (56 apps)

Cell = count of that app's endpoints on the plane (blank = 0). '' default is called out separately -- it is metal-admin for all 56 apps (the rule in Section 2). Generated from bundle.yaml.

App '' default prov-pub m-internal storage repl data m-admin (non-default)
barbican metal-admin 1 8
barbican-hacluster metal-admin 4
barbican-mysql-router metal-admin 3
barbican-vault metal-admin 3
ceph-mon metal-admin 7 2
ceph-osd metal-admin 1 2 1
ceph-radosgw metal-admin 1 9 1
ceph-radosgw-hacluster metal-admin 4
ceph-rbd-mirror metal-admin 1 2
cinder metal-admin 1 12 1
cinder-backup metal-admin 1
cinder-ceph metal-admin 1 2
cinder-hacluster metal-admin 4
cinder-mysql-router metal-admin 3
dashboard-mysql-router metal-admin 3
designate metal-admin 1 6
designate-bind metal-admin 1
designate-hacluster metal-admin 4
designate-mysql-router metal-admin 3
glance metal-admin 1 11 1
glance-hacluster metal-admin 4
glance-mysql-router metal-admin 3
glance-simplestreams-sync metal-admin 4
keystone metal-admin 1 13
keystone-hacluster metal-admin 4
keystone-mysql-router metal-admin 3
magnum metal-admin 1 7
magnum-dashboard metal-admin 1
magnum-hacluster metal-admin 4
magnum-mysql-router metal-admin 3
memcached metal-admin 2
mysql-innodb-cluster metal-admin 5
ncc-mysql-router metal-admin 3
neutron-api metal-admin 1 10
neutron-api-hacluster metal-admin 4
neutron-api-mysql-router metal-admin 3
neutron-api-plugin-ovn metal-admin 3
nova-cloud-controller metal-admin 1 18
nova-cloud-controller-hacluster metal-admin 4
nova-compute metal-admin 9 2 1
octavia metal-admin 1 10 1
octavia-dashboard metal-admin 2
octavia-diskimage-retrofit metal-admin 2
octavia-hacluster metal-admin 4
octavia-mysql-router metal-admin 3
openstack-dashboard metal-admin 1 10 1
openstack-dashboard-hacluster metal-admin 4
ovn-central metal-admin 6
ovn-chassis metal-admin 4 1
ovn-chassis-octavia metal-admin 4 1
placement metal-admin 1 8
placement-hacluster metal-admin 4
placement-mysql-router metal-admin 3
rabbitmq-server metal-admin 4
vault metal-admin 7
vault-mysql-router metal-admin 3

4. Dataflow confirmation (108 relations, spot-verified)

Every relation's plane is consistent with its endpoints' traffic type:

  • Service-to-service -> metal-internal: all :certificates <-> vault:certificates, all :shared-db <-> *-mysql-router, all :amqp <-> rabbitmq-server, all :identity-service <-> keystone, all :ha <-> *-hacluster.
  • Ceph client -> storage; OSD replication -> replication: cinder-ceph/nova-compute/glance:ceph <-> ceph-mon:client (storage); ceph-osd:cluster, ceph-rbd-mirror:ceph-local <-> ceph-mon:rbd-mirror (replication).
  • Overlay -> data-tenant: ovn-chassis:ovsdb <-> ovn-central:ovsdb control on metal-internal; the geneve data legs on data-tenant.
  • Public -> provider-public: only the :public / designate:dnsaas VIPs.

5. Ruled exceptions & their governing decisions

Do NOT "correct" these against the generic rule -- each is a ruled fix. (CLAUDE.md: grep the governing D-NNN before touching a built binding. The 2026-08-03 sweep flagged #1 as a deviation and had to self-correct -- the lesson is baked in here.)

Binding Looks like Actually Governing
openstack-dashboard cluster: metal-admin should be metal-internal horizon renders haproxy's 443 backend on the cluster address but only makes apache SSL vhosts for default+public -> cluster on metal-internal = dashboard HTTPS VIP dead D-072 / BUNDLEFIX-011
designate dnsaas: provider-public svc-to-svc? designate's public DNS-native VIP (dual-VIP pattern), a public-facing endpoint D-106
octavia ovsdb-cms: data-tenant mis-placed D-052 lists it under data-tenant; endpoint is inert (octavia reaches OVN via the chassis subordinate) D-052
ceph-radosgw public/object-store/cluster Ceph nets? radosgw is a gateway: public=S3/Swift API, object-store=svc API, cluster=radosgw HA -- none are Ceph OSD/public networks D-052
ovn-central '': metal-admin (see below) RE-AMENDMENT 2026-08-05 -- reverts the 08-03 '' -> metal-internal which cut controller reachability; cert CN now handled by dc-node-etchosts.sh D-052 re-amendment

The ovn-central story (the reference case for "management leg != service category"): the 2026-08-03 amendment moved ovn-central's '' to metal-internal to fix a cert-resolution bug (LP #2044324). It never fixed the cert (its own clarification admits this), and when first deployed (2026-08-04) it left all three ovn-central containers single-legged on the isolated metal-internal plane, unable to reach the controller -> agent never started. The 2026-08-05 re-amendment restores '': metal-admin (the universal pattern); ovn-central's functional endpoints (certificates, ovsdb*) stay metal-internal (its data IS a metal-internal service); the cert CN is fixed independently by the /etc/hosts postruncmd (scripts/dc-node-etchosts.sh, phase-01 Step 1.2b). See D-052 RE-AMENDMENT 2026-08-05 and docs/CURRENT-STATE.md.


6. Cross-references

  • D-052 (+ 2026-08-03 amendment, 2026-08-05 re-amendment) -- the governing decision.
  • D-072 / BUNDLEFIX-011 -- dashboard cluster:metal-admin exception.
  • D-106 -- designate reactivation (dnsaas dual-VIP).
  • D-125 -- metal-internal egress isolation.
  • scripts/lib-net.sh -- plane CIDRs / names (per-DC).
  • docs/audit/binding-plane-purpose-sweep-20260803.txt -- the full 446-endpoint classification.
  • scripts/dc-node-etchosts.sh + tests/dc-node-etchosts -- the ovn-central cert-CN fix.

Roosevelt deltas (from D-052): dedicated redundant corosync ring (heartbeat off the service plane); dedicated live-migration plane + QEMU-native TLS; revisit whether cross-DC service-to-service on metal-internal warrants its own inter-DC plane.