diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index b5a865d..091690e 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1113,6 +1113,26 @@ no-op. Type judgement is deliberately conservative: a mismatch is reported only where juju's own coercion cannot save it; a scalar where a STRING is declared is a NOTE; an UNRECOGNISED declared type REFUSES. + **A DEFECT IN MY OWN FIRST BUILD, FOUND BY A REPO-WIDE GREP AND CORRECTED THE SAME + SESSION -- and it would NOT have shown up in attempt 2.** Two faults, one cause: I measured + and gated only the THIRTEEN VIP charms. (1) `PREFER_IPV6_CHARMS` was WRONG -- re-measured + across ALL 33 charms in `bundle.yaml`, **TWELVE declare the option, not seven**: the seven + VIP ones plus `ceph-mon` 491, `ceph-osd` 953 (squid/stable), `nova-compute` 894, + `hacluster` 166 (2.4/stable) and `mysql-innodb-cluster` 164 (8.0/stable). (2) Invariant 9a + was written INSIDE the VIP loop, so an application with no `vip` was outside it entirely -- + and the repo ALREADY has that case: **`overlays/dc-dc-ipv6-family-matrix.yaml` sets + `prefer-ipv6` on `ceph-mon`, which carries no VIP.** That overlay is a LATER deploy step + (`runbooks/dc-dc-phase4-juju-bundle-per-dc.md:728/737`), so the miss would have passed + attempt 2 cleanly and surfaced at the step after, looking like a new fault. ceph-mon's own + in-file claim ("CONFIRMED real option, charm-ceph-mon config.yaml") is now INDEPENDENTLY + VERIFIED from the artifact rather than taken from the comment. 9a now runs over EVERY + application; T47 (a non-VIP app whose charm lacks the option FAILS) and T48 (ceph-mon, a + non-VIP app whose charm DOES declare it, PASSES) are both mutation-proven -- re-scoping 9a + back to the VIP loop kills T47, dropping ceph-mon from the list kills T48. Harness + **50/50**; gauntlet ALL GREEN (93); repo-lint 0 fail. **The generalisable lesson, and it is + the same one this repo keeps paying for: I measured the population the QUESTION named (the + 13 VIP charms) rather than the population the INVARIANT covers (every application juju + validates).** **WHAT REMAINS BEFORE ATTEMPT 2, none of it a decision:** the dc0 rack's `~/repo-stage` (the D-138 client host's deploy input) is still at the PRE-ruling overlays and must be refreshed from the branch; then `preflight.sh` for the DC (expected RED on P5 only, diff --git a/docs/changelog-20260731-prefer-ipv6-research.md b/docs/changelog-20260731-prefer-ipv6-research.md index de7f2d0..ce81326 100644 --- a/docs/changelog-20260731-prefer-ipv6-research.md +++ b/docs/changelog-20260731-prefer-ipv6-research.md @@ -185,8 +185,43 @@ **Verification.** Gauntlet ALL GREEN (93); repo-lint 0 fail; the fixed dc0 deploy input now sweeps CLEAN on option names (75 assignments, 20 schemas, 0 unknown, 0 unresolvable). -**Revert.** `git revert` this commit. It restores the previous renderer behaviour, the -previous invariant 9 and the previous overlays together -- they must move as one, since +## Item 7 -- A DEFECT IN ITEM 6, found by a repo-wide grep and corrected the same session + +**What was wrong.** Two faults with one cause: item 6 measured and gated only the +THIRTEEN VIP charms. + +1. **`PREFER_IPV6_CHARMS` was WRONG.** Re-measured across ALL 33 charms in + `bundle.yaml`: **twelve declare the option, not seven** -- the seven VIP ones plus + `ceph-mon` 491, `ceph-osd` 953 (squid/stable), `nova-compute` 894, `hacluster` 166 + (2.4/stable) and `mysql-innodb-cluster` 164 (8.0/stable). +2. **Invariant 9a was written INSIDE the VIP loop**, so an application with no `vip` + never reached it. + +**Why it mattered, and why no test would have caught it.** The repo ALREADY has the +missed case: `overlays/dc-dc-ipv6-family-matrix.yaml` sets `prefer-ipv6` on `ceph-mon`, +which carries no VIP. That overlay is a LATER deploy step +(`runbooks/dc-dc-phase4-juju-bundle-per-dc.md:728/737`), not part of the swept attempt-2 +input -- **so the miss would have passed attempt 2 cleanly and surfaced at the step +after, looking like a new fault.** ceph-mon happens to declare the option, so nothing +would have broken today; the gate was simply blind there. + +**Fixed.** 9a now runs over EVERY application, before the VIP loop, and the VIP loop +skips an app 9a already flagged so the diagnosis is not doubled. `ceph-mon`'s in-file +claim ("CONFIRMED real option, charm-ceph-mon config.yaml") is now INDEPENDENTLY +VERIFIED from the artifact rather than taken from the comment. Harness 48 -> **50/50**: +T47 (a non-VIP app whose charm lacks the option FAILS) and T48 (ceph-mon PASSES), both +mutation-proven -- re-scoping 9a back to the VIP loop kills T47; dropping ceph-mon from +the list kills T48. Gauntlet ALL GREEN (93); repo-lint 0 fail. + +**The generalisable lesson, and it is one this repo keeps paying for:** I measured the +population the QUESTION named (the 13 VIP charms) rather than the population the +INVARIANT covers (every application juju validates). The grep that found it took two +minutes and was not prompted by any gate. + +**Revert.** Folded into item 6's revert. + +**Revert (item 6).** `git revert` this commit. It restores the previous renderer +behaviour, the previous invariant 9 and the previous overlays together -- they must move as one, since a re-rendered overlay without the matching renderer fails `tests/render-drift`. --- diff --git a/scripts/provider-bundle-check.py b/scripts/provider-bundle-check.py index f54a4b4..e2f57c0 100644 --- a/scripts/provider-bundle-check.py +++ b/scripts/provider-bundle-check.py @@ -111,15 +111,28 @@ # the deployer's choice and only the charm has a schema. # # MEASURED 2026-07-31 from the charm ARTIFACT at each pinned channel, downloaded for -# amd64/ubuntu@22.04 and read with `grep -c prefer-ipv6 /config.yaml`. NOT from -# Charmhub's config-yaml API alone, and NOT from a repo comment. Capture: +# amd64/ubuntu@22.04 and read out of `config.yaml`. NOT from Charmhub's config-yaml API +# alone, and NOT from a repo comment. Capture: # docs/audit/stage5-prefer-ipv6-charm-research-20260731.txt. # -# DECLARE (7) ceph-radosgw 600 (squid/stable), cinder 820, glance 681, -# keystone 857, neutron-api 710, nova-cloud-controller 823, -# openstack-dashboard 750 (2024.1/stable unless noted) -# DO NOT (6) barbican 265, designate 418, magnum 96, octavia 571, -# placement 154, vault 724 (1.8/stable) +# SCOPE: ALL 33 charms in bundle.yaml, not only the thirteen that carry a VIP. The +# first pass measured the VIP charms alone and would have left this list WRONG -- five +# non-VIP charms declare the option, and `overlays/dc-dc-ipv6-family-matrix.yaml` +# already sets it on ceph-mon, an application with no VIP at all. +# +# DECLARE (12) ceph-mon 491, ceph-osd 953, ceph-radosgw 600 (all squid/stable); +# cinder 820, glance 681, keystone 857, neutron-api 710, +# nova-cloud-controller 823, nova-compute 894, +# openstack-dashboard 750 (all 2024.1/stable); +# hacluster 166 (2.4/stable); mysql-innodb-cluster 164 (8.0/stable) +# DO NOT (21) barbican 265, barbican-vault 99, cinder-backup 94, cinder-ceph 568, +# designate 418, designate-bind 266, glance-simplestreams-sync 152, +# magnum 96, magnum-dashboard 122, octavia 571, octavia-dashboard 168, +# octavia-diskimage-retrofit 257, placement 154 (all 2024.1/stable); +# ceph-rbd-mirror 62 (squid/stable); memcached 39 (latest/stable); +# mysql-router 1154 (8.0/stable); neutron-api-plugin-ovn 215; +# ovn-central 311, ovn-chassis 396 (24.03/stable); +# rabbitmq-server 295 (3.9/stable); vault 724 (1.8/stable) # # barbican returns NO at 2023.2/stable, 2023.1/stable AND ussuri/stable, so this is a # SUBSET of the OpenStack charms rather than an option newer charms dropped. @@ -129,12 +142,17 @@ # scripts/render-dc-overlays.py READS this tuple with `ast` rather than restating it -- # a second copy is a second thing to drift (same pattern as APP_OCTET). PREFER_IPV6_CHARMS = ( + "ceph-mon", + "ceph-osd", "ceph-radosgw", "cinder", "glance", + "hacluster", "keystone", + "mysql-innodb-cluster", "neutron-api", "nova-cloud-controller", + "nova-compute", "openstack-dashboard", ) @@ -328,6 +346,28 @@ v6_bands, v6_refusal = None, None vip_ok = vip_dual = 0 octet_owner = {} + + # 9a runs over EVERY application, not only the VIP-carrying ones. juju's `unknown + # option` rejection is fatal for any application, and it aborts the WHOLE bundle + # atomically. Written inside the VIP loop first, which would have missed the case + # the repo ALREADY has: overlays/dc-dc-ipv6-family-matrix.yaml sets prefer-ipv6 on + # ceph-mon, an application with no vip -- outside every check in this file. That + # overlay is a LATER deploy step, so the miss would not have shown up in attempt 2; + # it would have shown up at the step after, looking like a new fault. + # Asserted on PRESENCE, not truthiness: `prefer-ipv6: false` is the same fatal + # `unknown option` to juju as `true`. + for n in sorted(apps): + spec = apps[n] or {} + if "prefer-ipv6" not in ((spec.get("options") or {})): + continue + ch = str(spec.get("charm") or "") + if ch not in PREFER_IPV6_CHARMS: + fails.append("%s sets prefer-ipv6 but its charm %r does NOT declare that " + "option -- juju rejects the whole bundle (2026-07-31 attempt 1). " + "The v6 VIP legs do NOT need it: the ':::port' bind is gated on " + "the kernel disable_ipv6 sysctl, not on this option (D-101 " + "RULING NOTE 2026-07-31)" % (n, ch)) + for n in sorted(apps): opts = ((apps[n] or {}).get("options") or {}) vip = opts.get("vip") @@ -339,19 +379,12 @@ fails.append("%s vip is neither a v4 triple (3) nor a dual-family sextet (6) " "-- got %d address(es): %r" % (n, len(parts), vip)); continue dual = (len(parts) == 6) - declared = "prefer-ipv6" in opts charm = str((apps[n] or {}).get("charm") or "") - # 9a -- the option is only legal on a charm that DECLARES it. juju rejects the - # WHOLE bundle atomically on an unknown option and --dry-run does not check - # option names, so this is the only thing in the repo that can catch it. - # Asserted on PRESENCE, not truthiness: `prefer-ipv6: false` on barbican is the - # same fatal `unknown option` to juju as `true`. - if declared and charm not in PREFER_IPV6_CHARMS: - fails.append("%s sets prefer-ipv6 but its charm %r does NOT declare that " - "option -- juju rejects the whole bundle (2026-07-31 attempt 1). " - "The v6 VIP legs do NOT need it: the ':::port' bind is gated on " - "the kernel disable_ipv6 sysctl, not on this option (D-101 " - "RULING NOTE 2026-07-31)" % (n, charm)); continue + # 9a is asserted above, over every application. If it already flagged this app, + # do not also run 9b on it -- 9b would report the coupling as the cause, sending + # the reader to add v6 legs when the fix is to remove the option. + if "prefer-ipv6" in opts and charm not in PREFER_IPV6_CHARMS: + continue # 9b -- where the option IS legal, it stays coupled to the v6 legs. Measured # (L3-9): the overlay merge order that keeps prefer-ipv6 while dropping the v6 # legs is the one that exits 0, so the dangerous order is the GREEN one. diff --git a/tests/provider-bundle-check/run-tests.sh b/tests/provider-bundle-check/run-tests.sh index f4cb1c0..042d911 100644 --- a/tests/provider-bundle-check/run-tests.sh +++ b/tests/provider-bundle-check/run-tests.sh @@ -457,6 +457,23 @@ echo "$t46out" | sed 's/^/ /' | head -6; FAIL=$((FAIL+1)) fi +# T47 9a must cover applications that carry NO vip. Written inside the VIP loop first, +# which would have missed the case the repo ALREADY has: +# overlays/dc-dc-ipv6-family-matrix.yaml sets prefer-ipv6 on ceph-mon, which has no +# vip and is outside every other check in provider-bundle-check. That overlay is a +# LATER deploy step, so the miss would not have shown up in attempt 2 -- it would +# have shown up at the step after, looking like a new fault. +mutate t47.yaml 'b["applications"]["ovn-central"].setdefault("options",{})["prefer-ipv6"]=True' +run 1 'ovn-central sets prefer-ipv6' \ + "T47 prefer-ipv6 on a NON-VIP app whose charm lacks it FAILS" "$TMP/t47.yaml" + +# T48 the positive control for T47: ceph-mon has no vip and DOES declare the option +# (measured, charm-ceph-mon squid/stable rev 491). 9a must not over-fire on it, or +# the real dc-dc-ipv6-family-matrix overlay becomes un-gateable. +mutate t48.yaml 'b["applications"]["ceph-mon"].setdefault("options",{})["prefer-ipv6"]=True' +run 0 'PASS: Pattern A' \ + "T48 prefer-ipv6 on a NON-VIP app whose charm DOES declare it PASSES" "$TMP/t48.yaml" + echo echo "RESULT: PASS=$PASS FAIL=$FAIL" [[ "$FAIL" -eq 0 ]] && { echo "ALL PASS"; exit 0; } || exit 1