> SUPERSEDED IN PART BY D-058 (full plane renumber, 2026-06-29): provider-vip is now > **10.12.8.0/22** (gateway 10.12.8.1), not 10.12.24.0/22. metal-admin->.12, metal-internal->.16, > data-tenant->.20, oob->.60. The .24/.8/.12 values below are the original D-057 instantiation, > kept for history; the durable framework (separate tagged routed VIP plane) is unchanged. > See D-058-renumber.md for the authoritative scheme. ## D-057: Provider NIC L3-on-OVS-uplink -- split the public API VIP plane onto its own routed VLAN (2026-06-27) **Status:** DECIDED. Root cause CONFIRMED live (this session) and direction CONFIRMED by operator. Supersedes the standalone PROPOSED/OPEN note `D-057-provider-uplink-l3-separation.md` and resolves its one open question (the D-003B interaction) in favor of option (a)/(c). Implementation pending: MAAS plane add -> `carve-host-interfaces.sh` delta -> bundle delta -> teardown/redeploy -> D-011 re-validate. grep-before-assign confirmed D-057 free (max prior D-053; D-054/055/056 are DOCFIX). **Symptom:** every provider-ext floating IP (pool 10.12.5.0-10.12.7.254) unreachable cloud-wide; the .4.x public API VIPs answered. Blocked phase-06 Step 6.3 (SSH to capi-mgmt-v2 via FIP 10.12.7.107). **Root cause (CONFIRMED, not inferred -- measured on all three ovn-chassis hosts this session):** a two-consumer collision on the untagged provider NIC `enp1s0`. (1) ovn-chassis is configured `bridge-interface-mappings: br-ex:` and wants `enp1s0` as an OVS `br-ex` port. (2) ~11 `public: provider-public` API charms are LXD containers; Juju attaches a container to a subnet by bridging the host NIC, which created the Linux bridge `br-enp1s0` and captured `enp1s0`. A single untagged physical NIC cannot be both a Linux-bridge member and an OVS `br-ex` port: the Linux bridge won, `br-ex` was starved (no-carrier on all three), FIPs dark while the containerized .4.x VIPs kept answering. Evidence: `br-ex` operstate=down / carrier=Invalid-argument on openstack1/2/3; `enp1s0 master br-enp1s0` on all three; `br-enp1s0` ports = `enp1s0` + container `veth*` taps (1/3/5 taps); host static (.41/.42/.43) riding `br-enp1s0`; no public VIP on any host (all in containers); host default route via metal-admin `10.12.8.1`, provider `10.12.4.1` only on-link. The host has NO provider-routing role -- only its containers do. **Why option (b) was rejected:** "remove the host provider static" addresses consumer (1)'s L3 but NOT consumer (2). `br-enp1s0` exists for container attach; removing the host static cannot release `enp1s0`. Refuted by the measured `veth` taps. **Decision (a)/(c):** relocate the container `public` attach off the untagged uplink onto a tagged sub-interface, freeing untagged `enp1s0` for OVS `br-ex`. This is the Canonical shared-NIC pattern and is the exact mirror of the existing metal-internal tagged-secondary stack (`br-metal.103 -> br-internal`). Because a single CIDR cannot span two L2s, the public API VIPs re-IP onto a new subnet on the tagged VLAN -- which removes D-003B's same-L2 property and replaces it with L3 routing (see amendment below). **Target interface tree (per host; octet N = .40-.43 by HOST_OCTET):** - `enp1s0` -- RAW, untagged on fabric 1_provider, **NO L3 / NO subnet link / NO bridge**. ovn-chassis MAC-enslaves it into OVS `br-ex`. (The old host static 10.12.4.N is removed FROM THIS UPLINK -- that removal is the fix.) - `enp1s0.104` (VLAN, VID 104, on fabric 1_provider) -> `br-prov-api` (standard bridge) -> **STATIC 10.12.24.N**. Carries the new `provider-vip` space. Container `public` endpoints bind here. The host's provider-plane presence MOVES from `enp1s0` to `br-prov-api` (tagged, OVS-free, no `br-ex` competition -> zero D-057-class risk), mirroring metal-internal's `br-internal` host static -- the proven container-attach pattern on this cloud. (L3-less `br-prov-api` considered and rejected: unproven for Juju attach here; not worth gambling the redeploy.) **New plane (framework, not final subnetting):** `provider-vip` -- 10.12.24.0/22, VID 104, fabric 1_provider, **routed** (gateway 10.12.24.1), VIP reserve 10.12.24.2-.100 (VIPs .50-.60). The subnet/VID values are this deployment's instantiation; the DURABLE decision is the framework: the public API VIP plane is its own tagged, routed plane distinct from the provider ext_net/FIP plane. The DC-DC byte-aligned plan adopts the same split (its provider VLAN 240 -> 240 ext_net + 241 provider-vip; its already-present NN-11 role is the home). **provider-public after the split:** keeps only FIP/ext_net (10.12.5.0-10.12.7.254) + the OVN gateway SNAT + mgmt reserve; untagged enp1s0 -> OVS br-ex; NO container attach, NO host L3. Neutron stays flat physnet1 (no retype) -- the OVN/Neutron layer, confirmed correct this session, is untouched. **D-003B AMENDMENT:** D-003B deliberately co-located public API VIPs and FIPs on one provider L2 ("tenant->API by construction") -- a Bobcat-era convenience that this session proved unworkable on the NIC-limited host (the OVS-vs-container collision). It is hereby amended: the public API VIP plane is a separate, routed VLAN. tenant->API is preserved via L3 routing (tenant SNAT egress on provider 10.12.4.0/22 -> gateway -> provider-vip 10.12.24.0/22), and re-validated per D-011 #3. This is also better for the commercial hard-isolation goal (API VIPs out of the FIP broadcast domain; an L3 policy enforcement point) and matches the Roosevelt NIC-limited reality. **ROUTED-GATEWAY PREREQUISITE (gating, highest risk):** today only provider-public and metal-admin route (as-built: all other planes `gateway: none`). provider-vip must be routed: gateway 10.12.24.1 must exist and route to/from 10.12.4.0/22 on whatever owns 10.12.4.1 (jumphost/libvirt), with the return path 10.12.24.1 -> 10.12.4.0/22. Without it the redeploy passes D-011 #1/#2 and silently fails #3. Confirm/establish before redeploy -- do not discover at validation. **bridge-interface-mappings:** trim openstack0's provider MAC `52:54:00:3d:fd:54` (measured: the only one of the four with no ovn-chassis unit; nova-compute/ovn-chassis run on machines 9/10/11 = openstack1/2/3). Keep `9d:63:77`/`89:7f:ce`/`99:fc:c2`. Settles openstack0 as control+storage role (feeds Roosevelt node-role split). The mapping still targets the untagged `enp1s0` MAC (now free). **bundle changes (summary):** on the 11 `public:`-bound API charms, rebind `public: provider-public -> public: provider-vip`, and re-IP each `vip:` first token `10.12.4.X -> 10.12.24.X` (X=50-60; metal-admin .8.X and metal-internal .12.X tokens unchanged). Trim the openstack0 MAC. No Neutron/OVN config change. **Roosevelt relevance:** the provider-vip tagged-routed-plane split joins metal-internal VID-103 as a per-host interface-tree region-invariant (maas-as-built reference). The DC-DC byte-aligned plan inherits the split. **Related:** root-causes phase-06 6.3; amends D-003B; extends D-052/D-053 plane model with the host-interface-layer rule the rebuild exposed; the committed `docs/maas-as-built-reference.md` carve table needs the follow-on correction (enp1s0 raw/L3-less + the enp1s0.104->br-prov-api stack).