Fork: 0
OpenStack / openstack-caracal-dc-dc
Transfer to URL with SHA Find file
Newer
Older
openstack-caracal-dc-dc / docs / changelog-20260728-vip-arity-gate.md
@JANeumatrix JANeumatrix 22 hours ago 9 KB Exercise the dc1 arm of the apex lookup; confirm P2's rc=2 classification
Raw Blame History

Session changelog 2026-07-28 -- the VIP arity gap + R11's ruled gate changes

ONE changelog per session (GA-R2/D1). Under blanket approval this is the review surface: every item states WHAT, WHY (evidence), and HOW TO REVERT.

No live mutation in this session. Everything below is a repo change; the only live access was READ-ONLY (a NetBox apex poll and a pre-flight-checks run). No logged window was opened because nothing was mutated.

Status authority for everything here is docs/CURRENT-STATE.md (GA-R1). This file is session-scoped scratch and is NOT citable as status or decision authority.

1. provider-bundle-check.py: VIP arity, family coupling, and the apex-derived v6 bands

What. The vip validator now accepts a v4 TRIPLE (3 addresses) or a dual-family SEXTET (6), validates the three v6 legs against the per-DC v6 /64s, requires the v6 host part to MIRROR the v4 octet textually, and couples prefer-ipv6 to the arity in both directions. The v6 bands are READ FROM THE NETBOX APEX RECORD, not hardcoded. The standalone octet-uniqueness block (old item 6) folded into the same loop.

Why. Measured at :137: if len(parts) != 3 fails EVERY application once R2's ruled dual-stack lands, and :149's p.split(".")[-1] returns the whole address on a v6 literal, so every v6 leg was trivially "unique" and unchecked. The apex now holds the dual-family set a renderer will emit (verified live this session: 160 ip-addresses, 78 IPv6, 156 VIP objects), so the only gate that validates overlays could not validate what is about to be produced.

The prefer-ipv6 coupling is the substance, not a bonus. prefer-ipv6 makes HAProxy bind :::port in ADDITION to *:port; the measured L3-9 finding is that the overlay merge order which keeps prefer-ipv6 while dropping the v6 VIP legs is the one that EXITS 0 -- the dangerous order is the green one. Both directions now fail.

Authority. The arity gap was carried in docs/CURRENT-STATE.md as an open question, explicitly NOT covered by R11. Operator direction 2026-07-28, exact utterance: "Fix the arity gap first, then start the renderer", with the scope fork (arity alone vs arity + R11's three ruled changes) put separately and answered "Arity gap + R11's three ruled changes (Recommended)". OPS under GA-R3 -- a script change making an already-RULED end state expressible; doubt resolves DOWN, no D-number assigned.

Prior art followed, not reinvented. The (role, kind) apex keying and the "provider leg lives in the DEDICATED GUA VIP /64, admin/internal in their plane /64s" rule are copied from scripts/dc-plane-ipam.sh:102-131 and netbox/dc-plane-apex-import.py. Both live-API and repo-dump shapes are handled (nested scope.slug vs flat scope_site) -- writing against one and matching zero objects on the other is a trap this repo already hit once.

Refuses rather than passes. A dual-family vip with an unreadable or absent apex record exits 2, never 0: "could not look" is never "nothing there". A v4-only bundle needs no apex at all, so this introduces no new dependency for the current tree (harness T25/T26 lock both halves).

Revert. git revert this commit. The checker returns to triple-only; nothing live depends on it.

2. R11's three ruled gate changes (D-020 AMENDMENT, RULED 2026-07-27)

What.

  • provider-bundle-check.py: OCTET_LO, OCTET_HI 50-60 -> 50-99.
  • scripts/lib-net.sh: VIP_OCTET_MAX 60 -> 99, VIP_COUNT_EXPECT 11 -> 13.
  • provider-bundle-check.py: NEW invariant 8 -- an hacluster principal with no vip is a FAIL.
  • EXPECT_PUBLIC_VIP deliberately STAYS 11.

Why. R11 ruled vault .61 and designate .62 as full dual-family triples and ruled the gate hardening IN SCOPE, not deferred. cluster_count is asserted NOWHERE in scripts/ or tests/, so a 3->1 rewrite of all 20 values produces a byte-identical PASS -- decorative HA was found by audit rather than by gate.

EXPECT_PUBLIC_VIP stays 11 because -- measured -- NEITHER vault nor designate carries a public binding, so they do not join that count. Bumping both constants together breaks the gate. The two band constants are SEPARATELY NAMED in two files; this is a two-file change by construction, and both sites carry a comment saying so.

MEASURED CONSEQUENCE, stated precisely. Two sub-checks flip PASS -> FAIL:

  • provider-bundle-check on the base bundle now FAILS with hacluster relation but no vip: designate.
  • pre-flight-checks CHECK 1 now reports VIP validation OK=11 bad=0 (want OK=13).

preflight.sh was ALREADY exit 1 before this change (octavia-pki overlay absent, MAAS unreachable from the jumphost, P5's 7 credential findings). Measured after: still exit 1, now 3 fatal, 2 warning. This did NOT flip preflight from pass to fail and no deploy path that was open is closed by it -- the same precedent this document records for P5. The reds are the ruled gate reporting real work owed: R11's .61/.62 VIPs are RULED-BUT-NOT-BUILT, and per R6 they land BEFORE the HA overlay.

Revert. Revert this commit; the constants and the invariant-8 block are contiguous and independent of item 1.

3. Harness: 15 -> 30 cases, with three cases RE-POINTED rather than deleted

What. tests/provider-bundle-check/run-tests.sh gains 15 cases. The fixture base splits: pristine.yaml is the repo bundle untouched; good.yaml is that bundle PLUS designate's ruled .62 VIP.

Why the split. Invariant 8 legitimately fails the real repo bundle. Left alone, every rc=0 case (T1/T9/T14) would have been asserting "fails for the reason we already know" instead of its own invariant. Per the standing rule -- when remediating a finding turns the harness red, REPLACE the assertion with the new invariant and say so, never delete it to go green -- the real tree's state is kept honest by a NEW case (T16) that asserts pristine.yaml DOES trip it. When .62 lands in bundle.yaml, T16 must be re-pointed, not deleted.

Proven able to fail AND to pass -- the complement this project recorded on 2026-07-27 after building three gates whose every live run failed:

  • T16 the pristine bundle FAILS on designate; T18 the same check goes GREEN once the ruled VIP lands.
  • T17 vault FAILS under dc-ha-scaleup.yaml -- R6's "VIPs before the overlay" ordering, made executable.
  • T19 a valid dual-family VIP PASSES and is counted as dual; T20/T21 both directions of the prefer-ipv6 coupling FAIL; T22 a v6 leg in the node /64 instead of the VIP /64 FAILS; T23 the ::32 hex trap FAILS; T24 a 2-address vip FAILS; T25 an unreadable apex REFUSES at rc=2; T26 a v4-only bundle needs no apex; T27/T28 the band is widened to 99 but still bounded.

T29/T30 exercise the dc1 arm of the apex lookup, which T19-T23 never reached (they all run at the default --dc vr1-dc0): T29 resolves dc1's f03:11::/64 / :320::/64 / :321::/64 and passes, T30 proves a dc0 v6 leg under --dc vr1-dc1 FAILS, so a cross-DC copy-paste of a rendered overlay cannot pass. The renderer's first real output is per-DC overlays, and a gate observed in only one DC mode is the same untrustworthiness T16/T18 exist to rule out.

T8's assertion string moved outside 50-60 -> outside 50-99 (the same re-pointing rule; the case still guards the band, at its ruled bounds).

Evidence. Harness 30/30 (was 15). Gauntlet ALL GREEN (84 harnesses) on vcloud -- the host is named because the gauntlet is measured HOST-DEPENDENT (2/81 fail on voffice1 on an identical commit). repo-lint 0 fail / 1 standing warn / 605 files scanned. No harness was added or renamed, so tests/HARNESS-MANIFEST is unchanged and needed no re-record.

Revert. Revert this commit; the harness returns to 15 cases.

Findings LOGGED, NOT ACTIONED (hard rule 1)

  1. pre-flight-checks.sh CHECK 1 is triple-shaped. Its awk parse assumes three columns, so it will mis-read a dual-family sextet when those artifacts land. Same class as the arity gap just fixed, one gate over. Not touched -- it interlocks with BLOCKER-1 (the ruling-3 VIP extraction makes CHECK 1 see 0 VIPs).
  2. cluster_count coherence is still unasserted. Invariant 8 catches an hacluster principal with NO vip, which is what R11 ruled. It does NOT check that cluster_count matches the principal's num_units -- a 3->1 rewrite of all 20 values still produces a PASS. Out of ruled scope; worth its own exchange.
  3. NetBox instance/target drift (raised by the operator this session). Four netbox/*.py tools carry in-file usage examples pointing at https://netbox.baldurkeep.com -- the FROZEN v1 reference. Two of them (ipv6-mark-reserved.py, ipv4-prefixes-import.py) have NO SANDBOX_HOSTS write guard at all while carrying write paths (update_status, create_or_report_prefix, create_or_report_iprange). The guarded tools also expose a deliberate --yes-write-upstream escape hatch, so "writes only reach 10.10.1.10" is a convention with an override, not a hard invariant.
  4. Credential env-file naming is inverted. ~/vr1-office1-creds/vr1-netbox.env points at netbox.baldurkeep.com (the read-only v1 reference) while vr1-netbox-sandbox.env points at the LIVE VR1 apex http://10.10.1.10:8000. The filename says the opposite of what each is.
  5. The NetBox ?site= API filter silently does not filter. Measured this session: /api/ipam/ip-addresses/?site=vr1-dc0 and ?site=vr1-dc1 each returned the full 160. A per-site apex count taken that way would be wrong and look right.