Newer
Older
openstack-caracal-dc-dc / docs / changelog-20260804-ovn-central-cert-fix.md

Changelog -- 2026-08-04: ovn-central cert root-caused, fix PROVEN, and WIRED for redeploy

Session scope: finish re-evaluating the ovn-central "awaiting server certificate data" failure (the prior diagnosis was wrong), prove the real fix on the live dc0 model, and wire it into the deploy for the clean redeploy. Stage 5 remains OPEN; nothing here opens/closes a stage. Live mutations were reversible tests, all reverted (model left at its captured before-state).

Item 1 -- root cause corrected (measured), binding approach refuted live

The committed rdns_mode/binding framing was wrong. MEASURED: rdns_mode=2 on all planes and the PTR exists at the region BIND; the metal-internal plane is ISOLATED (no gateway/route to the resolver), so get_hostname(metal-internal-addr) returns None -> empty cert CN. Three binding configs tested live (''->metal-admin; certificates->metal-admin; ''->metal-admin+bounce) -- all left the CN empty, because charm-ovn-central derives the CN from its metal-internal address regardless of which endpoint is rebound. So the app STAYS on metal-internal (D-052-correct for its OVSDB/certificates data type). Records corrected: CORRECTION/RESOLUTION blocks in docs/audit/ovn-central-cert-reeval-20260803.md, VOID banner in the remediation-plan doc, sweep F1 correction, CURRENT-STATE.

Item 2 -- fix PROVEN end-to-end (controlled live test)

An /etc/hosts entry 10.12.12.122 <name> on ovn-central/0 made the charm's own get_hostname("10.12.12.122") resolve, the charm published common_name, vault issued ovn-central_0.server.cert, and /etc/ovn/{cert_host,key_host,ovn-central.crt} were written -- while control units /1 and /2 (no entry) stayed broken. The entry is the sole cause of the fix. OVN imposes no CN-content rule (2026-08-04 research) and vault signs any non-empty CN, so the name only needs to be non-empty and stable.

Item 3 -- NEW: scripts/dc-node-etchosts.sh + tests/dc-node-etchosts (harness 9/9)

Renders a per-DC cloudinit-userdata (modes render/apply/check) that, at machine provision, adds each node's metal-internal address -> <hostname>.maas <hostname> to /etc/hosts, scoped to THIS DC's metal-internal CIDR derived from lib-net.sh (dc0 10.12.12.0/22, dc1 10.12.72.0/22 -- never typed; the harness caught an author-guessed dc1 CIDR). Deterministic + idempotent (no race-prone DNS guard, per review); a python3 CIDR-membership test handles the /22 exactly. The rendered runcmd was EXECUTED live and correctly added only the metal-internal entry (skipping the metal-admin address) and made get_hostname resolve. Harness grades: valid cloud-init YAML, per-DC CIDR, runcmd behaviour (scoping + idempotency + non-empty name), refuse on bad site/mode.

  • Changed: NEW scripts/dc-node-etchosts.sh, NEW tests/dc-node-etchosts/run-tests.sh.
  • Verified: harness 9/9 ALL PASS; render parses as cloud-init YAML; live runcmd behaviour confirmed on ovn-central/0 then cleaned up.
  • Revert: delete both files and Step 1.2b (below); no live state depends on it (unapplied).

Item 4 -- runbook wiring: Step 1.2b (gated, pre-deploy)

runbooks/phase-01-bundle-deploy.md gains Step 1.2b between add-model (1.2) and deploy (1.3): dc-node-etchosts.sh apply <DC> + check <DC> gate. VR1 multi-space DCs only (the script REFUSES a site with no metal-internal plane, so VR0 is a no-op). Notes the DELIVERY is verify-at-provision.

  • Revert: remove the Step 1.2b block.

NOTE -- this is a NEW mechanism, not a D-008 extension

It borrows D-008's static-/etc/hosts SHAPE, but D-008's /etc/hosts is FQDN->VIP for os-public-hostname (unset in VR1) and no /etc/hosts script existed before this. A future session grepping D-008 for this will not find it -- it is dc-node-etchosts.sh.