dc0 deploy blocked: mirrored UCA has no signing key on fresh nodes; root-caused ...

All 9 machines reached started. Nine units then failed the install hook across
unrelated charms -- one common cause:

  W: GPG error: http://10.12.8.4/cloud-archive jammy-updates/caracal InRelease:
     NO_PUBKEY 5EDB1B62EC4926EA
  E: The repository is not signed.
  apt-get update returned exit 100

The mirrored content is reachable and its upstream signatures are intact; the
key to verify them is absent.

This refutes a claim overlays/vr1-dc0-machines.yaml carried as MEASURED -- that
the UCA key is already on the nodes. Measured on ceph-osd/0: trusted.gpg.d holds
only the cdimage and 2018-archive keyrings. The cloud-archive keyring ships in
the ubuntu-cloud-keyring package, installed as a side effect of the
cloud:<series>-<release> origin path; a raw deb line bypasses it. The claim was
almost certainly measured on a node from the 07-31 deploy, which used
cloud:jammy-caracal and had the keyring. True when taken, false on a fresh node.

Fix is charm-documented, quoted from the charms: openstack charms take
openstack-origin "deb <URL>|<PUB KEY>"; the three ceph charms take a separate
key option accepting ASCII armor. Armored block, never a keyid -- a keyid sends
charm-helpers to keyserver.ubuntu.com, unreachable under the D-107 airgap.

Key provenance measured, not downloaded: the mirror serves ubuntu-cloud-keyring
2021.03.26; URI from apt-get download --print-uris on a node (source package is
ubuntu-keyring, not ubuntu-cloud-keyring -- I guessed and got a 404); sha512
matched apt's declared digest; keyring holds exactly 5EDB1B62EC4926EA. Armored
export committed as public key material.

PREPARED, NOT APPLIED: config for all 15 apps generated from that one verified
key and staged on the rack. juju config --file was refused by the permission
layer in three shapes; it is a live mutation across 15 apps and belongs to the
operator. No workaround attempted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf