diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 57c87ed..a7bef2f 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1899,6 +1899,43 @@ permission layer in three command shapes; no workaround was attempted, and in hindsight the refusal blocked a mechanism that was wrong anyway. The errored units retry harmlessly meanwhile; nothing is lost by the delay. + **>>> AS-BUILT COMPARISON, OPERATOR-DIRECTED, AND IT REFRAMES THE WHOLE ITEM. <<<** + Full table: `docs/audit/uca-origin-asbuilt-comparison-20260802.txt`. Operator: *"You have + the past deployment yaml values to refer to ... review what has worked successfully + previously to compare against the current yaml file values."* Sources are all in-repo: + `asbuilt/20260706-224851/bundle-exported.yaml` (a `juju export-bundle` of the VR0 + testcloud that DEPLOYED SUCCESSFULLY; the five other `asbuilt/` captures agree exactly). + **(1) THE ONLY CONFIGURATION THIS PROJECT HAS EVER DEPLOYED SUCCESSFULLY IS + `cloud:jammy-caracal` ON SIX APPS** -- barbican/magnum/octavia (`openstack-origin`) and + ceph-mon/ceph-osd/ceph-radosgw (`source`). **ZERO raw `deb` lines and ZERO `key:` options + appear in ANY as-built capture.** `bundle.yaml` encodes exactly that, defining the value + ONCE via YAML anchors (`&openstack-origin`, `&ceph-source`). + **(2) THE 07-31 REPOINT CHANGED TWO THINGS AT ONCE AND NEITHER HAS EVER DEPLOYED:** app + count 6 -> 15 (SOUND -- under the airgap the other nine inherit the charm default + `caracal`, resolving to the same unreachable pocket) AND the value FORM `cloud:` -> raw + `deb`. **The form change is what dropped the key**, since the `cloud:` path installs + `ubuntu-cloud-keyring` as a side effect and a raw `deb` line does not. The count change is + right; the form change cost the deploy. + **(3) THE PROXY ARM KEEPS THE PROVEN CONFIG AND NEEDS NO OVERRIDE AT ALL** -- dc1 carries + no origin block because apt-cacher-ng forwards whatever URL it is handed. **This sharpens + the D-135 experiment result: the MIRROR arm is what forces divergence from the only + known-good configuration, and that divergence has now cost one failed deploy plus standing + key-rotation surface.** + **(4) MY OVERLAY EDIT VIOLATES THE BASE BUNDLE'S OWN CONVENTION** -- `bundle.yaml` defines + its origin ONCE behind an anchor; I duplicated a 29-line armored key FIFTEEN times. Anchors + demonstrably work here (the base uses them), so even if raw-deb is kept, the key belongs + behind ONE anchor. Fifteen copies is fifteen places to miss at the next rotation. + **(5) THE OPERATOR HAS ALREADY RULED THE DIRECTION THAT DISSOLVES THIS** -- the D-135 + AMENDMENT 2026-08-02, *"if we have to rebuild in dc0 for any reason we will be using a + proxy rather than a full mirror rebuild"*. Under the proxy arm the origin block is DELETED, + not fixed. Its stated trigger is a REBUILD of dc0's artifact service and the mirror + currently stands, so whether this fires that trigger is the OPERATOR'S call. + **THE FORK IS OPEN AND UNRULED (GA-R5 shape).** (A) converge dc0 on the proxy now + (`dc-cache-proxy.sh`, built and proven on dc1), delete the origin block, return to the + as-built config -- eliminates the key problem rather than working around it, at the cost of + a build mid-deploy. (B) keep the mirror, restructure the armored key behind a single anchor, + apply via the Step-7 pattern -- fastest unblock, keeps dc0 diverged from the proven config. + **I am not choosing between them.** **ROOSEVELT DELTA WORTH A RULING LATER, not resolved here:** an airgapped DC that mirrors the UCA must carry the archive's TRUST as deliberately as its CONTENT. The 2026-07-31 ruling pointed the URL at the mirror, which is half the job. Whether the durable answer is diff --git a/docs/audit/uca-origin-asbuilt-comparison-20260802.txt b/docs/audit/uca-origin-asbuilt-comparison-20260802.txt new file mode 100644 index 0000000..38b16a1 --- /dev/null +++ b/docs/audit/uca-origin-asbuilt-comparison-20260802.txt @@ -0,0 +1,80 @@ +uca-origin-asbuilt-comparison-20260802.txt +========================================== +Comparison requested by the operator: "You have the past deployment yaml values to +refer to ... review what has worked successfully previously to compare against the +current yaml file values." + +SOURCES (all in-repo, no inference): + asbuilt/20260706-224851/bundle-exported.yaml -- `juju export-bundle` of the VR0 + testcloud that DEPLOYED SUCCESSFULLY (five other asbuilt/ captures agree exactly) + bundle.yaml -- the VR1 base deployed 2026-08-02 + overlays/vr1-dc0-machines.yaml -- the 2026-07-31 UCA repoint + overlays/vr1-dc1-machines.yaml -- the proxy DC's overlay + +NOTE ON INSTRUMENT: the as-built export cannot be PyYAML-parsed -- juju writes `**` for +redacted values, which the parser reads as an alias. Extracted textually with awk instead. + +-------------------------------------------------------------------------------------- + apps setting value form key copies of + an origin option key material +-------------------------------------------------------------------------------------- +AS-BUILT (VR0, 6 cloud:jammy-caracal none 0 + deployed OK) +bundle.yaml (VR1) 6 cloud:jammy-caracal none 0 + defined ONCE via YAML anchors + &openstack-origin / *ceph-source +vr1-dc1 overlay 0 (inherits base; proxy forwards none 0 + (PROXY arm) whatever URL it is handed) +vr1-dc0 overlay 15 raw `deb http://10.12.8.4/...` 3 ceph 15 x 29 lines + (MIRROR arm, mine) +-------------------------------------------------------------------------------------- + +FINDING 1 -- THE ONLY CONFIGURATION THIS PROJECT HAS EVER DEPLOYED SUCCESSFULLY IS +`cloud:jammy-caracal` ON SIX APPS. Exactly barbican / magnum / octavia +(`openstack-origin`) and ceph-mon / ceph-osd / ceph-radosgw (`source`). ZERO raw `deb` +lines and ZERO `key:` options appear anywhere in any as-built capture. + +FINDING 2 -- THE 2026-07-31 REPOINT CHANGED TWO THINGS AT ONCE, AND NEITHER HAS EVER +DEPLOYED. It moved the app count 6 -> 15 (justified: under the D-107 airgap the other +nine inherit the charm default `caracal`, which resolves to the same unreachable upstream +pocket) AND changed the value FORM from `cloud:` to a raw `deb` line. The form change is +what dropped the signing key: the `cloud:` path installs `ubuntu-cloud-keyring` as a side +effect, a raw `deb` line does not. The app-count change is sound; the form change is what +cost the deploy. + +FINDING 3 -- THE PROXY ARM KEEPS THE PROVEN CONFIGURATION AND NEEDS NO OVERRIDE AT ALL. +dc1 carries no origin block because apt-cacher-ng forwards whatever URL it is handed, so +`cloud:jammy-caracal` -- the as-built value -- keeps working there unmodified. This is a +GENUINE D-135 EXPERIMENT RESULT and it is now sharper than when it was recorded: the +MIRROR arm is what forces divergence from the only known-good configuration, and that +divergence has now cost one failed deploy and introduced permanent key-rotation surface. + +FINDING 4 -- MY OVERLAY EDIT VIOLATES THE BASE BUNDLE'S OWN CONVENTION. `bundle.yaml` +deliberately defines its origin ONCE (`&openstack-origin`, `&ceph-source`) and aliases it. +I duplicated a 29-line armored key FIFTEEN times. YAML anchors demonstrably work in this +bundle -- the base uses them -- so even if the raw-deb direction is kept, the key belongs +behind a single anchor, not in fifteen copies. Fifteen copies is fifteen places to miss at +the next key rotation. + +FINDING 5 -- THE OPERATOR HAS ALREADY RULED THE DIRECTION THAT DISSOLVES THIS. +D-135 AMENDMENT 2026-08-02, exact utterance: "if we have to rebuild in dc0 for any reason +we will be using a proxy rather than a full mirror rebuild." Both DCs then run the same +strategy. Under the proxy arm the origin block is DELETED, not fixed, and dc0 returns to +the as-built-proven `cloud:jammy-caracal`. The amendment's stated trigger is a REBUILD of +dc0's artifact service and the mirror currently stands, so whether this situation fires +that trigger is the operator's call, not mine. + +-------------------------------------------------------------------------------------- +THE FORK, for a GA-R5 ruling. Both are real; I am not choosing. +-------------------------------------------------------------------------------------- +(A) CONVERGE dc0 ON THE PROXY NOW. Stand up `scripts/dc-cache-proxy.sh` on dc0's D-134 + utility `.4` (built, and PROVEN on dc1), set `juju model-config apt-http-proxy`, + and DELETE the overlay origin block entirely -- all 15 apps return to the as-built + configuration. ELIMINATES the signing-key problem rather than working around it, + removes 15 copies of key material, retires a 953 GB mirror and its nightly window, + and executes the direction already ruled. COST: it is a build, not a config edit, + and it pauses the deploy while it is stood up. +(B) KEEP THE MIRROR, FIX THE KEY. Restructure my overlay edit to the base bundle's + anchor convention so the armored key appears ONCE, then apply via the Step-7 + pattern. Fastest path to an unblocked deploy. COST: keeps dc0 diverged from the only + proven configuration, and keeps per-DC key rotation as standing surface. diff --git a/docs/changelog-20260802-deploy-input.md b/docs/changelog-20260802-deploy-input.md index dc515dd..cb6d9ad 100644 --- a/docs/changelog-20260802-deploy-input.md +++ b/docs/changelog-20260802-deploy-input.md @@ -416,3 +416,48 @@ set. Live mutation, awaiting the operator. - **Revert:** `git revert`; the overlay returns to bare `deb` lines with no key. + +## Item 10 -- as-built comparison (operator-directed): the proven config is NOT what we deployed + +Operator: *"You have the past deployment yaml values to refer to ... review what has +worked successfully previously to compare against the current yaml file values."* + +Full table: `docs/audit/uca-origin-asbuilt-comparison-20260802.txt`. Source is +`asbuilt/20260706-224851/bundle-exported.yaml`, a `juju export-bundle` of the VR0 +testcloud that **deployed successfully** (the five other `asbuilt/` captures agree). + +| | apps w/ origin | value form | key option | key copies | +|---|---|---|---|---| +| as-built (VR0, deployed OK) | 6 | `cloud:jammy-caracal` | none | 0 | +| `bundle.yaml` (VR1 base) | 6 | same, defined ONCE via YAML anchors | none | 0 | +| `vr1-dc1` overlay (PROXY) | 0 | inherits base | none | 0 | +| `vr1-dc0` overlay (MIRROR, mine) | 15 | raw `deb http://10.12.8.4/...` | 3 ceph | 15 x 29 lines | + +1. **The only configuration this project has ever deployed successfully is + `cloud:jammy-caracal` on six apps.** Zero raw `deb` lines and zero `key:` options in + any as-built capture. +2. **The 07-31 repoint changed two things at once and neither has ever deployed.** App + count 6 -> 15 is SOUND (under the airgap the other nine inherit `caracal`, resolving + to the same unreachable pocket). The value form `cloud:` -> raw `deb` is what dropped + the key: the `cloud:` path installs `ubuntu-cloud-keyring` as a side effect; a raw + `deb` line does not. +3. **The proxy arm keeps the proven config and needs no override.** This sharpens the + D-135 experiment result: the MIRROR arm is what forces divergence from the only + known-good configuration, and that divergence has now cost a failed deploy plus + standing key-rotation surface. +4. **My overlay edit violates the base bundle's own convention.** `bundle.yaml` defines + its origin once behind an anchor; I wrote fifteen copies of a 29-line key. Anchors + demonstrably work here. Fifteen copies is fifteen places to miss at rotation. +5. **The operator already ruled the direction that dissolves this** -- D-135 amendment + 2026-08-02, *"if we have to rebuild in dc0 for any reason we will be using a proxy + rather than a full mirror rebuild."* Under the proxy arm the origin block is DELETED, + not fixed. Its trigger is a REBUILD and the mirror stands, so whether this fires it is + the operator's call. + +**INSTRUMENT NOTE:** the as-built export cannot be PyYAML-parsed -- juju writes `**` for +redacted values and the parser reads it as an alias. Extracted textually with awk. + +**FORK LEFT OPEN (GA-R5).** (A) converge dc0 on the proxy now and delete the origin block; +(B) keep the mirror and restructure the key behind one anchor. Not choosing. + +- **Revert:** n/a -- this item adds a comparison record only.