diff --git a/docs/changelog-20260808-dc0-activation.md b/docs/changelog-20260808-dc0-activation.md index 7438f7a..58c515f 100644 --- a/docs/changelog-20260808-dc0-activation.md +++ b/docs/changelog-20260808-dc0-activation.md @@ -197,3 +197,41 @@ **NEXT (in flight):** Step 5.2 amphora image pipeline (phase-05-amphora-pipeline.sh, staged + sha-verified) fired in background. + +--- + +## Item 7 -- INCIDENT: amphora retrofit build FAILS (octavia-diskimage-retrofit exit 1) + +**Symptom (verbatim).** phase-05-amphora-pipeline.sh Step 5.2 got through the config gate, +downloaded + sha256-verified the jammy base, uploaded it to glance (base id +`04c982c2-8906-48b9-8ddc-2febce82c9ef`), then the retrofit action FAILED: +`Action id 72 failed: Command '['octavia-diskimage-retrofit', '-O', 'raw', +'/var/snap/octavia-diskimage-retrofit/common/tmp/tmpsqda9ukv', +'/var/snap/.../tmpw76vcx4e']' returned non-zero exit status 1.` -> CONFIRM FAIL: no ACTIVE +image tagged octavia-amphora. (First run also failed earlier -- MODEL defaulted to `openstack` +(VR0); fixed by MODEL=vr1-dc0; this is the SECOND, model-correct run.) + +**Also fixed en route (DOCFIX candidate for the rebuild):** the amphora pipeline script +defaults `MODEL=openstack` (VR0 model name). VR1 requires `MODEL=vr1-dc0`. Not a per-DC +tunable the runbook flags prominently -- the rebuild tooling should derive the model or +default it per-site. + +**What was checked (read-only):** octavia-diskimage-retrofit/0 unit is active/idle "Unit is +ready" (the UNIT is healthy; the ACTION failed). appendix-A has no matching symptom. juju +debug-log for the unit shows only update-status hooks -- the dib stderr is in the action +output, not surfaced beyond the wrapper line. + +**Hypothesis (NOT yet confirmed -- needs sudo on the unit):** the classic dib-in-LXD-container +failure -- `octavia-diskimage-retrofit` runs diskimage-builder, which needs privileged +loop/mount/kpartx that an unprivileged LXD container restricts. This is DC-substrate-relevant +(would recur on the 10.13 rebuild) -> worth real diagnosis, not a throwaway-only quirk. + +**OWED diagnosis (gated -- sudo on octavia-diskimage-retrofit/0):** capture the actual dib +stderr (`juju exec --unit octavia-diskimage-retrofit/0 -- sudo octavia-diskimage-retrofit ...` +or the dib log under `/var/snap/octavia-diskimage-retrofit/common/`), check loop-device / +`/dev/kvm` availability + disk space in the snap common dir. + +**State left:** octavia CORE is ACTIVATED (octavia active, lb-mgmt net + o-hm0 up, +configure-resources done) -- only the amphora IMAGE is missing, so the "1 test LB" smoke test +(F6) is BLOCKED until retrofit succeeds. Leftover: the jammy base image `04c982c2...` in glance +(harmless; torn down with the cloud). Task #3 CORE-done / image-blocked.