diff --git a/docs/tenant-cidr-overlap-correction-PLAN.md b/docs/tenant-cidr-overlap-correction-PLAN.md new file mode 100644 index 0000000..ee6e7a0 --- /dev/null +++ b/docs/tenant-cidr-overlap-correction-PLAN.md @@ -0,0 +1,154 @@ +# Tenant CIDR overlap correction -- work order / change plan + +**Status:** DRAFT for the Code (jumphost) stream + operator ruling. Authored by the +main-chat stream at repo HEAD `59a7c73` (2026-07-06). Read-only analysis; no live +mutation and no committed-surface edit performed by main-chat. + +**Proposes:** decision **D-074** (PROPOSED; amends D-016). D-074 is next-free per +`bash scripts/ledger-scan.sh` at this HEAD. NOTE: D-073 and DOCFIX up to 105 were +consumed the same day by the jumphost stream -- **Code must re-run the scan and +re-confirm next-free before inserting the decision header.** Do not hand-type it. + +--- + +## 1. Finding (what is actually wrong) + +The stage-4 tenant CIDR guard in `scripts/tenant-onboard.sh` **over-enforces relative +to the onboarding contract it is supposed to implement**, and separately has an +overlap-detection hardening gap. + +- **Contract intent** (`docs/tenant-onboarding-contract.md` sec. 3, item 4): a tenant + CIDR must be RFC1918 and non-colliding with **(a) our allocations** and **(b) the + client's own on-prem/VPN ranges they may later interconnect**. The contract does + **not** require non-collision with *other tenants*. +- **Guard behaviour** (`tenant-onboard.sh` stage 4 -- the `grep -qw "$TENANT_CIDR"` + check run under `admin_env` against a cloud-wide `openstack subnet list`): it dies on + **any** exact subnet-string match cloud-wide, which includes other tenants' subnets. + So it forbids tenant-vs-tenant exact overlap that the contract never intended to + forbid. +- **Independent hardening gap:** the guard is exact-string (`grep -qw`), so it MISSES + partial/containing overlaps (e.g. a requested `10.20.20.128/25` against an existing + `10.20.20.0/24`). It is therefore simultaneously *too strict* on exact tenant matches + and *too loose* on partial overlaps against ranges we must actually protect. + +**Origin of the uniqueness behaviour (and a correction of the record):** it is an IPAM +artifact of **D-016**, which models a single `10.20.0.0/16` tenant pool and carves +non-overlapping /24s from it. It is **not** a dataplane requirement. An earlier +main-chat claim that the uniqueness rule was needed for "CAPI/Magnum reachability" was +wrong and is retracted here: per **D-035**, `capi-mgmt-v2` is single-homed on its own +`10.20.0.0/24` and reaches workload clusters via their API-LB floating IPs and OpenStack +via the API VIPs -- it never routes into other tenants' `10.20.x` space, so overlapping +tenant CIDRs cannot touch the management path. + +## 2. Proposed decision -- D-074 (PROPOSED; amends D-016) + +Tenant private (overlay) CIDRs are **tenant-chosen and MAY overlap across tenants.** +Non-collision is enforced **only** against a reserved-ranges registry (operator/infra +allocations, section 3). NetBox (the IPAM apex) tracks the reserved/routed space; tenant +overlay space leaves NetBox uniqueness tracking. + +Rationale: this aligns the implementation with the onboarding contract's already-stated +intent, and with the hard-isolation (SCS Domain Manager) persona. Overlap-allowed tenant +CIDRs is the standard isolated-multi-tenant model -- it is why Neutron ships +`allow_overlapping_ips=true` by default and why public-cloud VPCs let tenants pick +arbitrary RFC1918. Globally-unique-per-tenant is the enterprise-private-cloud pattern +(shared routed fabric, interoperating internal units), which is the opposite of what this +cloud is. + +Forward-only: existing tenants are **not** re-CIDR'd (northwind stays `10.20.20.0/24`; +design-decisions forbids transient-overlap re-CIDR). + +**PROPOSED means the operator has not ruled.** Nothing in sections 3-4 is implemented +until D-074 is ADOPTED. + +## 3. Reserved-ranges registry (the ONLY thing the guard rejects) + +Non-collision is enforced against these, using real subnet-overlap math (containment in +either direction), not string equality: + +| plane / allocation | CIDR (confirm live before wiring) | +|---|---| +| provider-public (incl. API VIPs + `provider-ext` FIP pool) | `10.12.4.0/22` | +| metal-admin | `10.12.8.0/22` | +| metal-internal | `10.12.12.0/22` | +| data-tenant | `10.12.16.0/22` | +| storage | `10.12.32.0/22` | +| replication | `10.12.36.0/22` | +| capi-mgmt tenant network (D-035) | `10.20.0.0/24` -- OPERATOR DECISION: keep reserved? (recommend YES) | +| metadata service | `169.254.169.254/32` (link-local; not RFC1918; low concern) | + +**These values were read from `docs/maas-as-built-reference.md` by main-chat.** Code MUST +re-derive them from the authoritative live/modeled source (pre-flight `PLANE_CIDRS` / +`scripts/lib-net.sh` / NetBox) before wiring the guard -- do not hardcode from this doc +(hard rules 2 and 3: dynamic lookup, no inferred values). Centralize the list in +`lib-net.sh` keyed by plane name, not as scattered literals. + +## 4. Gated correction procedure (harness-first; each phase gates the next) + +### Phase 0 -- read-only verification (MUST pass before any mutation) +- **0.1 Confirm `allow_overlapping_ips`.** Read it live (`juju config neutron-api ...` + or `neutron.conf` on a unit). If it is `false`, **STOP** -- flipping it is a wider + blast radius that needs its own ruling; do not proceed on the assumption it is `true`. +- **0.2 Overlap+Magnum proof on a FOIL tenant.** Onboard a foil tenant with a CIDR that + deliberately overlaps an existing tenant's /24; confirm (a) the subnet creates and (b) + a Magnum cluster reaches ACTIVE with a working API-LB. Capture evidence to + `~/openstack-baseline/`. (This foil doubles as the `d011-05` P3 isolation foil.) + +### Phase 1 -- guard change (Code's lane; Code owns `tenant-onboard.sh`) +- Replace the exact-string cloud-wide guard with subnet-overlap math (Python + `ipaddress`) against the reserved-ranges registry **only**. Drop the tenant-vs-tenant + check. +- Extend `tests/tenant-onboard/` with fixtures covering: exact tenant-vs-tenant overlap + (now **ALLOWED**), partial overlap against a reserved range (**REJECTED**), exact match + against a reserved range (**REJECTED**), and a disjoint range (**ALLOWED**). No script + change ships without its harness. +- `bash scripts/repo-lint.sh` + the harness green before commit. + +### Phase 2 -- doc alignment +- `tenant-onboarding-contract.md` sec. 3.4: make explicit that tenant-vs-tenant overlap + is permitted (the wording is already close; remove any implication of cloud-wide + uniqueness). +- Amend **D-016** with a forward-pointer to D-074. +- `runbooks/tenant-onboarding-runbook.md`: replace the "carve a non-colliding /24 from + `10.20.0.0/16`" guidance with "tenant CIDR is client-choice or the standard default; + only reserved ranges are rejected." +- `clientdocs/` intake field: reword "requested internal IP address" -> + "requested internal network range (CIDR) -- optional; overlap-safe; default per policy" + (also fixes the IP-vs-range wording mismatch). + +### Phase 3 -- create the devops tenant on the corrected cloud +- `bash scripts/tenant-onboard.sh ` with `TENANT_CIDR` per the default + policy ruled in section 6. Verify with `scripts/tenant-assert.sh`. + +## 5. Lose / gain + +**Gain:** implementation matches the contract and the hard-isolation persona; onboarding +drops the "find a free /24" step; the intake field goes optional; clients can align to +their own on-prem/VPN ranges; NetBox's uniqueness job shrinks to the routed space that +genuinely must be unique (cleaner, and no global per-tenant IPAM registry to maintain +across DCs for Roosevelt); the 256-/24 ceiling of one /16 stops being a scaling limit. + +**Lose / cost:** debuggability -- an IP no longer uniquely identifies a tenant, so +captures/triage need net+tenant context (mitigate with disciplined naming); no +tenant-private-IP routing between tenants (an anti-pattern under hard isolation anyway); +IPAM discipline narrows to reserved ranges rather than disappearing; the guard change +touches a file Code is actively editing (coordination cost); D-016's `/16` pool becomes +vestigial (not reclaimed, just no longer enforced forward). + +## 6. Open operator decisions (nothing proceeds until ruled) + +1. **Rule D-074** -- adopt the overlap-allowed model? +2. **Default tenant CIDR policy** -- (a) uniform default (e.g. `10.0.0.0/24`) for all, or + (b) client-choice with a fallback default. Recommend (b). This sets what the devops + tenant receives. +3. **Keep `capi-mgmt` `10.20.0.0/24` reserved?** Recommend YES. +4. **`allow_overlapping_ips` confirmed `true`?** (Phase 0.1 gates everything.) + +## 7. Coordination / lane notes + +- The jumphost stream is actively editing `tenant-onboard.sh` and the contract today + (changelog addenda 21 / 23 / 26). The Phase-1 guard edit and all live steps are the + jumphost stream's lane; main-chat authored this plan only. **Code: pull and read this + before resuming onboard-script work.** +- D-074 is next-free per the scan at HEAD `59a7c73`; re-confirm before inserting. +- This is a new file with no edits to any shared/fenced file, to minimize collision.