SUPERSEDED IN PART BY D-058 (full plane renumber, 2026-06-29): provider-vip is now 10.12.8.0/22 (gateway 10.12.8.1), not 10.12.24.0/22. metal-admin->.12, metal-internal->.16, data-tenant->.20, oob->.60. The .24/.8/.12 values below are the original D-057 instantiation, kept for history; the durable framework (separate tagged routed VIP plane) is unchanged. See D-058-renumber.md for the authoritative scheme.

D-057: Provider NIC L3-on-OVS-uplink -- split the public API VIP plane onto its own routed VLAN (2026-06-27)

Status: DECIDED. Root cause CONFIRMED live (this session) and direction CONFIRMED by operator. Supersedes the standalone PROPOSED/OPEN note D-057-provider-uplink-l3-separation.md and resolves its one open question (the D-003B interaction) in favor of option (a)/(c). Implementation pending: MAAS plane add -> carve-host-interfaces.sh delta -> bundle delta -> teardown/redeploy -> D-011 re-validate. grep-before-assign confirmed D-057 free (max prior D-053; D-054/055/056 are DOCFIX).

Symptom: every provider-ext floating IP (pool 10.12.5.0-10.12.7.254) unreachable cloud-wide; the .4.x public API VIPs answered. Blocked phase-06 Step 6.3 (SSH to capi-mgmt-v2 via FIP 10.12.7.107).

Root cause (CONFIRMED, not inferred -- measured on all three ovn-chassis hosts this session): a two-consumer collision on the untagged provider NIC enp1s0. (1) ovn-chassis is configured bridge-interface-mappings: br-ex:<enp1s0-MAC> and wants enp1s0 as an OVS br-ex port. (2) ~11 public: provider-public API charms are LXD containers; Juju attaches a container to a subnet by bridging the host NIC, which created the Linux bridge br-enp1s0 and captured enp1s0. A single untagged physical NIC cannot be both a Linux-bridge member and an OVS br-ex port: the Linux bridge won, br-ex was starved (no-carrier on all three), FIPs dark while the containerized .4.x VIPs kept answering. Evidence: br-ex operstate=down / carrier=Invalid-argument on openstack1/2/3; enp1s0 master br-enp1s0 on all three; br-enp1s0 ports = enp1s0 + container veth* taps (1/3/5 taps); host static (.41/.42/.43) riding br-enp1s0; no public VIP on any host (all in containers); host default route via metal-admin 10.12.8.1, provider 10.12.4.1 only on-link. The host has NO provider-routing role -- only its containers do.

Why option (b) was rejected: "remove the host provider static" addresses consumer (1)'s L3 but NOT consumer (2). br-enp1s0 exists for container attach; removing the host static cannot release enp1s0. Refuted by the measured veth taps.

Decision (a)/(c): relocate the container public attach off the untagged uplink onto a tagged sub-interface, freeing untagged enp1s0 for OVS br-ex. This is the Canonical shared-NIC pattern and is the exact mirror of the existing metal-internal tagged-secondary stack (br-metal.103 -> br-internal). Because a single CIDR cannot span two L2s, the public API VIPs re-IP onto a new subnet on the tagged VLAN -- which removes D-003B's same-L2 property and replaces it with L3 routing (see amendment below).

Target interface tree (per host; octet N = .40-.43 by HOST_OCTET):

• enp1s0 -- RAW, untagged on fabric 1_provider, NO L3 / NO subnet link / NO bridge. ovn-chassis MAC-enslaves it into OVS br-ex. (The old host static 10.12.4.N is removed FROM THIS UPLINK -- that removal is the fix.)
• enp1s0.104 (VLAN, VID 104, on fabric 1_provider) -> br-prov-api (standard bridge) -> STATIC 10.12.24.N. Carries the new provider-vip space. Container public endpoints bind here. The host's provider-plane presence MOVES from enp1s0 to br-prov-api (tagged, OVS-free, no br-ex competition -> zero D-057-class risk), mirroring metal-internal's br-internal host static -- the proven container-attach pattern on this cloud. (L3-less br-prov-api considered and rejected: unproven for Juju attach here; not worth gambling the redeploy.)

New plane (framework, not final subnetting): provider-vip -- 10.12.24.0/22, VID 104, fabric 1_provider, routed (gateway 10.12.24.1), VIP reserve 10.12.24.2-.100 (VIPs .50-.60). The subnet/VID values are this deployment's instantiation; the DURABLE decision is the framework: the public API VIP plane is its own tagged, routed plane distinct from the provider ext_net/FIP plane. The DC-DC byte-aligned plan adopts the same split (its provider VLAN 240 -> 240 ext_net + 241 provider-vip; its already-present NN-11 role is the home).

provider-public after the split: keeps only FIP/ext_net (10.12.5.0-10.12.7.254) + the OVN gateway SNAT + mgmt reserve; untagged enp1s0 -> OVS br-ex; NO container attach, NO host L3. Neutron stays flat physnet1 (no retype) -- the OVN/Neutron layer, confirmed correct this session, is untouched.

D-003B AMENDMENT: D-003B deliberately co-located public API VIPs and FIPs on one provider L2 ("tenant->API by construction") -- a Bobcat-era convenience that this session proved unworkable on the NIC-limited host (the OVS-vs-container collision). It is hereby amended: the public API VIP plane is a separate, routed VLAN. tenant->API is preserved via L3 routing (tenant SNAT egress on provider 10.12.4.0/22 -> gateway -> provider-vip 10.12.24.0/22), and re-validated per D-011 #3. This is also better for the commercial hard-isolation goal (API VIPs out of the FIP broadcast domain; an L3 policy enforcement point) and matches the Roosevelt NIC-limited reality.

ROUTED-GATEWAY PREREQUISITE (gating, highest risk): today only provider-public and metal-admin route (as-built: all other planes gateway: none). provider-vip must be routed: gateway 10.12.24.1 must exist and route to/from 10.12.4.0/22 on whatever owns 10.12.4.1 (jumphost/libvirt), with the return path 10.12.24.1 -> 10.12.4.0/22. Without it the redeploy passes D-011 #1/#2 and silently fails #3. Confirm/establish before redeploy -- do not discover at validation.

bridge-interface-mappings: trim openstack0's provider MAC 52:54:00:3d:fd:54 (measured: the only one of the four with no ovn-chassis unit; nova-compute/ovn-chassis run on machines 9/10/11 = openstack1/2/3). Keep 9d:63:77/89:7f:ce/99:fc:c2. Settles openstack0 as control+storage role (feeds Roosevelt node-role split). The mapping still targets the untagged enp1s0 MAC (now free).

bundle changes (summary): on the 11 public:-bound API charms, rebind public: provider-public -> public: provider-vip, and re-IP each vip: first token 10.12.4.X -> 10.12.24.X (X=50-60; metal-admin .8.X and metal-internal .12.X tokens unchanged). Trim the openstack0 MAC. No Neutron/OVN config change.

Roosevelt relevance: the provider-vip tagged-routed-plane split joins metal-internal VID-103 as a per-host interface-tree region-invariant (maas-as-built reference). The DC-DC byte-aligned plan inherits the split.

Related: root-causes phase-06 6.3; amends D-003B; extends D-052/D-053 plane model with the host-interface-layer rule the rebuild exposed; the committed docs/maas-as-built-reference.md carve table needs the follow-on correction (enp1s0 raw/L3-less + the enp1s0.104->br-prov-api stack).