diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 6394765..e8c4a46 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2696,6 +2696,39 @@ 200, snap proxy LISTENING, juju controller connected. Every one was true; none tested egress. **A listener is not a path, and an HTTP 200 from a LOCAL service says nothing about upstream.** The check set was incomplete and the summary outran its evidence. + **>>> TWO RULINGS 2026-08-02 (GA-R5), taken in separate exchanges. <<<** + **(1) D-135 AMENDED -- dc0 CONVERGES ON THE PROXY AT ITS NEXT REBUILD.** Operator, exact + utterance: **"if we have to rebuild in dc0 for any reason we will be using a proxy rather than + a full mirror rebuild."** The 953 GB dc0 mirror is NOT reconstructed; on any rebuild the + artifact service comes back as an apt caching proxy on the dc1 pattern (`dc-cache-proxy.sh`, + `.4:3142`, consumed via `juju apt-http-proxy`). **THIS QUALIFIES A STANDING TERM OF D-135, + which reads "there is no fallback and no later convergence"** -- there is none while the + mirror stands, but a REBUILD converges rather than reinstates. **NOTHING CHANGES TODAY:** the + running dc0 mirror is verified intact (236/236 Ubuntu + 2/2 UCA indices, 2026-08-02) and stays. + **OWED AT THE REBUILD, or the experiment produced nothing:** D-135 existed to COMPARE the two + strategies, so the mirror arm's outcome must be captured before it is dismantled -- storage + cost, sync-failure modes, the 2026-07-27 currency-gate defect class, and the two arms' + behaviour under the 2026-08-01 outage (the mirror kept SERVING stale-but-valid content with + its upstream gone; a proxy fails closed on a miss). **CONSEQUENCE FOR THE FOLD: the phase + runbooks can carry ONE artifact strategy instead of branching per DC**, with the mirror + recorded as the superseded arm; `dc-mirror.sh install dc0` becomes a deliberate strategy + REVERSAL, never a repair. + **(2) D-140 PINNED -- OpenTofu management of the Juju layer, AFTER a hardened, tested dc0 + deployment.** Operator, exact utterance: **"I accept the opentofu plan. Hardened dc0 + deployment then opentofu management with a successful tested deployment method from dc0. + Something to pin for the end of deployment, review and add opentofu management to the + remaining tasks."** Order ruled: dc0 deploys and is hardened -> that method is TESTED -> the + proven method is the INPUT to the translation -> reviewed at end-of-deployment and added to + the remaining-task set. **NOT converted now, and the reason is recorded so it is not + re-litigated:** the bundle deploy has never completed end to end, and changing the mechanism + before it succeeds means debugging two unknowns at once. **WHERE THE VALUE MEASURABLY IS:** + nearly every failure this stage has been in the MODEL/CONFIG layer (`add-model`, spaces, + `apt-mirror`, snap proxy, `prefer-ipv6`, the VIP overlay), not application topology -- so the + strongest candidate is that layer with `bundle.yaml` retained, not a wholesale replacement. + **COST TO PRICE AT REVIEW:** the provider is RESOURCE-shaped, not bundle-shaped, and the + measured plan is **56 apps / 108 relations**; `docs/dc-dc-buildout-design.md:12` records the + repo was seeded at `8813efc` specifically so `bundle.yaml` + `phase-01..08` could be reused. + **Provider capabilities must be READ (`references/opentofu-provider-docs.md`), never recalled.** **STILL OWED BEFORE THE DEPLOY, in ruled order:** ~~D-139 step 1~~ DONE (apex CREATE-only push, `netbox/d139-gua-carve.py --dc vr1-dc0 --commit`; tool built, independently reviewed, dry-run byte-identical, `--commit` never yet passed), step 2 (MAAS GUA `/64`s alongside the diff --git a/docs/design-decisions.md b/docs/design-decisions.md index 12f1806..3a85374 100644 --- a/docs/design-decisions.md +++ b/docs/design-decisions.md @@ -7276,3 +7276,87 @@ **TRIGGER:** Roosevelt VPN/site-to-site design. Present alongside D-132 (per-DC MAAS topology) and D-131 sub-4 (pinned DNS review), which are pinned to the same design window. + +### AMENDMENT 2026-08-02 -- D-135: dc0 CONVERGES ON THE PROXY AT ITS NEXT REBUILD + +**Operator ruling, exact utterance: "if we have to rebuild in dc0 for any reason we will be +using a proxy rather than a full mirror rebuild."** + +**RULED.** The dc0 full mirror is NOT reconstructed. Should dc0's artifact service be rebuilt +for any reason, it comes back as an **apt caching proxy** on the dc1 pattern +(`scripts/dc-cache-proxy.sh`, apt-cacher-ng on the D-134 utility `.4:3142`, consumed via +`juju apt-http-proxy`). Both DCs then run the same strategy. + +**THIS CHANGES A STANDING TERM OF D-135 AND SAYS SO PLAINLY.** D-135's amended text describes +the dc0-mirror / dc1-proxy split as "the EXPERIMENT, not an interim state -- there is no +fallback and no later convergence". **That clause is now qualified: there is no convergence +while the mirror stands, but a rebuild converges rather than reinstates.** The experiment is +not abandoned -- it ran, and its arms are compared below. + +**WHY THIS IS THE RIGHT CALL ON THE EVIDENCE ALREADY IN HAND.** The mirror arm cost 953 GB of +rack storage, a nightly `debmirror` window, an upstream dependency that fails independently of +the DC (2026-08-02: `500 Can't connect to ubuntu-cloud.archive.canonical.com`), and a checker +defect that read CLEAN over a FAIL status until it was fixed on 2026-07-27. The proxy arm +carries none of that: no bulk storage, no sync window, no attested-currency problem, and a +cache miss degrades to a fetch rather than to stale content. + +**WHAT MUST BE CAPTURED BEFORE THE MIRROR GOES, or the experiment produced nothing.** D-135 +existed to compare the two strategies. If the mirror is dismantled without recording its +outcome, the comparison is lost and Roosevelt inherits a preference with no evidence behind it. +Owed at the rebuild: the measured storage cost, the sync-failure modes seen, the currency-gate +defect class, and the observed behaviour of each arm under the 2026-08-01 outage (the mirror +kept SERVING stale-but-valid content with its upstream gone; a proxy would have failed closed +on a miss). That comparison is the experiment's deliverable. + +**CONSEQUENCE FOR THE RUNBOOK FOLD:** the phase runbooks can now carry ONE artifact strategy +(the proxy) with the mirror recorded as the superseded experiment arm, instead of branching +per DC. `dc-mirror.sh` is retained for the historical arm and for the currency lesson its +`check` encodes; `dc-mirror.sh install dc0` becomes a deliberate strategy REVERSAL and never a +repair -- the same warning dc1's row already carries. + +**UNCHANGED:** dc1 is unaffected (it was already the proxy arm). Nothing here touches the +currently-running dc0 mirror, which is verified intact (236/236 + 2/2 indices, 2026-08-02). + +## D-140: PINNED -- OpenTofu management of the Juju layer, after a hardened and tested dc0 deployment [ARCH] + +**Status: PINNED 2026-08-02 to the end-of-deployment review.** Operator ruling, exact +utterance: **"I accept the opentofu plan. Hardened dc0 deployment then opentofu management +with a successful tested deployment method from dc0. Something to pin for the end of +deployment, review and add opentofu management to the remaining tasks."** + +**RULED -- the SEQUENCING, which is the substance.** The Juju layer is NOT converted to +OpenTofu now. Order: (1) dc0 deploys and is HARDENED; (2) that deployment method is TESTED and +demonstrated to work; (3) the proven method becomes the INPUT to the OpenTofu translation; +(4) the conversion is reviewed at end-of-deployment and enters the remaining-task set. + +**WHY NOT NOW, recorded so the sequencing is not re-litigated:** the bundle deploy has never +completed end to end at VR1. Changing the deploy MECHANISM before the deploy SUCCEEDS means +debugging two unknowns at once, and the failures to date (2026-07-31: `unknown option +prefer-ipv6`, a missing `policies/overrides.zip`, `default-base` deployable exactly once; +2026-08-01/02: model/spaces/`apt-mirror`/snap-proxy/VIP-overlay work) would have been harder to +localise, not easier, with a translation layer in between. + +**WHERE THE VALUE ACTUALLY IS, measured rather than assumed.** Almost every failure in this +stage has been in the MODEL and CONFIG layer -- `add-model`, spaces, `apt-mirror`, snap proxy, +`prefer-ipv6`, the VIP overlay -- and NOT in the application topology. Preflight gate **P8** +(2026-08-01) demonstrated the value of declarative drift detection by finding substrate drift +that had been sitting unseen. So the strongest candidate is the model/spaces/model-config +layer, with `bundle.yaml` retained for topology, rather than a wholesale replacement. + +**THE COST THAT MUST BE PRICED AT REVIEW TIME, not hand-waved.** The Juju provider is +RESOURCE-shaped, not bundle-shaped: it declares applications and integrations, not a +`bundle.yaml`. The measured 2026-07-31 plan is **56 applications / 108 relations**, so a +wholesale translation is a substantial deliverable in its own right. `docs/dc-dc-buildout- +design.md:12` also records that this repo was seeded from `openstack-caracal-ipv4` at commit +`8813efc` SPECIFICALLY so `bundle.yaml` and `runbooks/phase-01..08` could be reused; abandoning +bundles diverges from that lineage and from the Charmed OpenStack reference material. + +**OWED BEFORE THE REVIEW CAN RULE:** provider capabilities must be READ, not recalled -- +`references/opentofu-provider-docs.md` gives the sources and the fetch method, and this repo +treats an unverified option name or version as the cardinal sin. Options to present: (a) +model/config layer only, bundle retained; (b) full translation; (c) decline and keep the +scripted path with gates. No option is pre-selected here. + +**A1 TEST (GA-R3):** admitted as a D-number because a Roosevelt build session would grep this +before choosing how to manage its Juju layer, and because it decides what the deliverable +runbooks look like.