PREFLIGHT TARGET: DC=vr1-dc0 (override: DC=vr1-dc1 bash scripts/preflight.sh)
Every gate below is run against THIS DC; the verdict line repeats it.
================ P1: repo lint ================
[WARN] L1 docs/design-decisions.md: 239 non-ASCII byte(s) (legacy D-001..018 carve-out; NEW entries must be ASCII)
WARN: repo lint (0 fail, 1 warn, 640 files scanned)
================ P2: bundle invariants ================
validating the MERGED vr1-dc0 deploy input: bundle.yaml --overlay overlays/vr1-dc0-vips.yaml --overlay overlays/vr1-dc0-machines.yaml --overlay overlays/vr1-dc0-octavia-pki.yaml --dc vr1-dc0
[ok] 11 charms bind public->provider-public; none on provider-vip
[ok] 13 clustered VIP(s) are provider/admin/internal, octet 50-99 (13 dual-family)
[ok] ovn-chassis bridge-interface-mappings: 2 well-formed MAC(s) (role-sep; VR0 set N/A)
[ok] 108 relations well-formed (explicit endpoints, all apps exist)
[ok] mysql-innodb-cluster num_units=3 (D-062)
[ok] 12 hacluster principal(s) all carry a VIP (R11)
[ok] 12 hacluster subordinate(s) declare cluster_count == principal num_units
[ok] keystone policyd-override wired in-bundle; zip content matches source (DOCFIX-071)
[ok] machines block: all 9 machine(s) tagged openstack-vr1-dc0, matching --dc vr1-dc0
[ok] placement: role-separated (3 control/2 compute/4 storage); anti-affinity + role placement + counts OK
PASS: Pattern A / D-052-D-053 bundle invariants (bundle.yaml)
================ P3: channel assert (charmhub) ================
[ok] barbican 2024.1/stable (barbican)
[ok] barbican-vault 2024.1/stable (barbican-vault)
[ok] ceph-mon squid/stable (ceph-mon)
[ok] ceph-osd squid/stable (ceph-osd)
[ok] ceph-radosgw squid/stable (ceph-radosgw)
[ok] ceph-rbd-mirror squid/stable (ceph-rbd-mirror)
[ok] cinder 2024.1/stable (cinder)
[ok] cinder-backup 2024.1/stable (cinder-backup)
[ok] cinder-ceph 2024.1/stable (cinder-ceph)
[ok] designate 2024.1/stable (designate)
[ok] designate-bind 2024.1/stable (designate-bind)
[ok] glance 2024.1/stable (glance)
[ok] glance-simplestreams-sync 2024.1/stable (glance-simplestreams-sync)
[ok] hacluster 2.4/stable (keystone-hacluster, glance-hacluster, neutron-api-hacluster, nova-cloud-controller-hacluster, placement-hacluster, openstack-dashboard-hacluster, cinder-hacluster, octavia-hacluster, barbican-hacluster, magnum-hacluster, ceph-radosgw-hacluster, designate-hacluster)
[ok] keystone 2024.1/stable (keystone)
[ok] magnum 2024.1/stable (magnum)
[ok] magnum-dashboard 2024.1/stable (magnum-dashboard)
[ok] memcached latest/stable (memcached)
[ok] mysql-innodb-cluster 8.0/stable (mysql-innodb-cluster)
[ok] mysql-router 8.0/stable (vault-mysql-router, keystone-mysql-router, glance-mysql-router, ncc-mysql-router, placement-mysql-router, neutron-api-mysql-router, cinder-mysql-router, dashboard-mysql-router, octavia-mysql-router, barbican-mysql-router, magnum-mysql-router, designate-mysql-router)
[ok] neutron-api 2024.1/stable (neutron-api)
[ok] neutron-api-plugin-ovn 2024.1/stable (neutron-api-plugin-ovn)
[ok] nova-cloud-controller 2024.1/stable (nova-cloud-controller)
[ok] nova-compute 2024.1/stable (nova-compute)
[ok] octavia 2024.1/stable (octavia)
[ok] octavia-dashboard 2024.1/stable (octavia-dashboard)
[ok] octavia-diskimage-retrofit 2024.1/stable (octavia-diskimage-retrofit)
[ok] openstack-dashboard 2024.1/stable (openstack-dashboard)
[ok] ovn-central 24.03/stable (ovn-central)
[ok] ovn-chassis 24.03/stable (ovn-chassis, ovn-chassis-octavia)
[ok] placement 2024.1/stable (placement)
[ok] rabbitmq-server 3.9/stable (rabbitmq-server)
[ok] vault 1.8/stable (vault)
PASS: channel assert (33 pins, 0 fail, 0 warn)
================ P4: live pre-flight (MAAS/overlay/nodes) ================
=== DC selection ===
PASS: gating DC=vr1-dc0 (planes 10.12.4.0/22 .. 10.12.36.0/22; 10 node(s))
=== Repo (informational) ===
NOTE: REPO=/home/jessea123/openstack-caracal-dc-dc
NOTE: HEAD: fbe7b31 Build the 2026-07-31 ruling: renderer + invariant 9 replaced, overlays re-rendered
NOTE: working tree clean
=== CHECK 0: per-DC octavia-pki overlay (no key material printed) ===
PASS: overlay present with 5 lb-mgmt-* keys
PASS: overlay ASCII clean
=== CHECK 1: bundle VIPs -- v4 triple or R2 dual-family sextet, .50-.99 (provider/admin/internal) ===
PASS: vip: line count = 13 (from overlays/vr1-dc0-vips.yaml)
PASS: aligned VIPs OK=13 bad=0 (DC=vr1-dc0 bands 10.12.4/10.12.8/10.12.12)
=== MAAS reachability gate (read-only) ===
PASS: MAAS reachable (profile=vr1-dc0-region)
=== CHECK 3: six planes resolved BY CIDR (id/vid/gw/dns) ===
provider-public 10.12.4.0/22 id=3 vid=0 gw=10.12.4.1 dns=[]
metal-admin 10.12.8.0/22 id=1 vid=0 gw=none dns=["10.12.8.6"]
metal-internal 10.12.12.0/22 id=4 vid=0 gw=none dns=[]
data-tenant 10.12.16.0/22 id=5 vid=0 gw=none dns=[]
storage 10.12.32.0/22 id=6 vid=0 gw=none dns=[]
replication 10.12.36.0/22 id=7 vid=0 gw=none dns=[]
PASS: all six planes present (by CIDR)
PASS: metal-internal is UNTAGGED (vid 0) -- D-133 flat carve
NOTE: stale-NAME check is juju-side (run scripts/juju-spaces-check.sh after add-model)
=== CHECK 2: data/storage NIC links BY CIDR (DC=vr1-dc0 role nodes; octet per D-134 band) ===
== vr1-dc0-control-01 (677cta, octet .100) ==
enp3s0 -> 10.12.12.0/22 10.12.12.100 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.100 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.100 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.100 type=physical
== vr1-dc0-control-02 (wpftxc, octet .101) ==
enp3s0 -> 10.12.12.0/22 10.12.12.101 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.101 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.101 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.101 type=physical
== vr1-dc0-control-03 (6p8pbx, octet .102) ==
enp3s0 -> 10.12.12.0/22 10.12.12.102 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.102 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.102 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.102 type=physical
== vr1-dc0-compute-01 (m3cgc7, octet .120) ==
enp3s0 -> 10.12.12.0/22 10.12.12.120 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.120 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.120 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.120 type=physical
== vr1-dc0-compute-02 (t6kpe3, octet .121) ==
enp3s0 -> 10.12.12.0/22 10.12.12.121 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.121 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.121 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.121 type=physical
== vr1-dc0-storage-01 (t7ymp6, octet .150) ==
enp3s0 -> 10.12.12.0/22 10.12.12.150 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.150 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.150 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.150 type=physical
== vr1-dc0-storage-02 (fg6gxm, octet .151) ==
enp3s0 -> 10.12.12.0/22 10.12.12.151 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.151 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.151 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.151 type=physical
== vr1-dc0-storage-03 (yws74f, octet .152) ==
enp3s0 -> 10.12.12.0/22 10.12.12.152 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.152 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.152 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.152 type=physical
== vr1-dc0-storage-04 (6q4syf, octet .153) ==
enp3s0 -> 10.12.12.0/22 10.12.12.153 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.153 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.153 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.153 type=physical
NOTE: vr1-dc0-juju-01 (arfr7p, octet .5) is D-134 utility-band infrastructure, not an OpenStack role node -- data-plane carve not asserted
=== CHECK 4: DC=vr1-dc0 OpenStack role nodes -- status / power ===
vr1-dc0-control-01 -> mint-roughy Ready power=off
PASS: vr1-dc0-control-01 Ready
vr1-dc0-control-02 -> gentle-raven Ready power=off
PASS: vr1-dc0-control-02 Ready
vr1-dc0-control-03 -> square-insect Ready power=off
PASS: vr1-dc0-control-03 Ready
vr1-dc0-compute-01 -> pure-condor Ready power=off
PASS: vr1-dc0-compute-01 Ready
vr1-dc0-compute-02 -> able-puma Ready power=off
PASS: vr1-dc0-compute-02 Ready
vr1-dc0-storage-01 -> amused-corgi Ready power=off
PASS: vr1-dc0-storage-01 Ready
vr1-dc0-storage-02 -> known-alpaca Ready power=off
PASS: vr1-dc0-storage-02 Ready
vr1-dc0-storage-03 -> loyal-oyster Ready power=off
PASS: vr1-dc0-storage-03 Ready
vr1-dc0-storage-04 -> civil-bug Ready power=off
PASS: vr1-dc0-storage-04 Ready
Summary: 0 fatal, 0 warning
================ P5: credential matrix (D-137 tier 1 + tier 2 local) ================
=== creds-matrix: tier 1 (STATIC) ===
=== creds-matrix: tier 2 (EXISTENCE) ===
(host: voffice1)
[ok] S1 schema: 121 rows, all enums valid, site-keys region-qualified, no duplicate (id,site,host-role,filename)
[ok] S3 render: 5 source field(s) SKIPPED -- rendering them needs the declared path from creds-manifests/vm-secret-locations (ruling 3); the list now EXISTS but the source-field derivation is not wired
[ok] S3 render drift: rendered row fields (mode, source) match checked-in; header prose and non-jumphost rows are OUT OF SCOPE of this compare
[ok] S4 mint-ref: every script:/runbook: reference resolves to a real location
[ok] S4 provenance debt: 40 row(s) are mint-ref=operator-terminal -- NOT reproducible from the repo (research FINDING 1). Admitted by design; converting them is remediation, not a checker fix.
[ok] S7 notes: 39 note key(s) referenced, all resolve, none orphaned
[ok] E0 jumphost location '~/vr1-office1-creds/*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '~/vr1-dc0-creds/*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '~/vr1-dc1-creds/*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '~/vault-init/*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '~/tenant-*/*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '/home/jessea123/openstack-caracal-dc-dc/opentofu/terraform.tfstate' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '/home/jessea123/openstack-caracal-dc-dc/opentofu/terraform.tfstate.backup' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '/home/jessea123/openstack-caracal-dc-dc/opentofu/terraform.tfstate.pre-*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '~/admin-openrc' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 18 remote location(s) SKIPPED -- rerun with --remote to include the headend shadow stores (SEC-022) and the region secrets dir (SEC-020)
[ok] E1 91 expected artifact(s) NOT JUDGED -- their role (headend/-, headend/vr1-dc0, headend/vr1-dc1, headend/vr1-office1, jumphost/-, jumphost/vr1-dc0, jumphost/vr1-dc1, jumphost/vr1-office1, netbox/vr1-office1, rack/vr1-dc0, rack/vr1-dc1, region/vr1-dc0, region/vr1-dc1) has at least one location that could not be probed, so absence cannot be asserted over it
[ok] E1/E3 existence: every expected artifact present and nothing undeclared, across 0 fully-probed role(s)
[ok] tier 3 (VALIDITY) NOT RUN -- pass --tier3 (with --tier2) to compare cross-copy sha256 provenance
[FAIL] S2 vr1-dc0 EXPECTED-BUT-ABSENT: 'opnsense-api.txt' (id dc0-edge-api, SEC-021) is expected by the matrix but NOT declared in the manifest -- the credential is either missing or undeclared
[FAIL] S2 vr1-dc1 EXPECTED-BUT-ABSENT: 'maas-juju-api-key.txt' (id dc1-juju-apikey, SEC-028) is expected by the matrix but NOT declared in the manifest -- the credential is either missing or undeclared
[FAIL] S2 vr1-dc1 EXPECTED-BUT-ABSENT: 'maas-juju-user-password' (id dc1-juju-user, SEC-028) is expected by the matrix but NOT declared in the manifest -- the credential is either missing or undeclared
[FAIL] S2 vr1-dc1 EXPECTED-BUT-ABSENT: 'maas-region-admin-password' (id dc1-region-admin, SEC-027) is expected by the matrix but NOT declared in the manifest -- the credential is either missing or undeclared
[FAIL] S2 vr1-dc1 EXPECTED-BUT-ABSENT: 'maas-region-api-key.txt' (id dc1-region-apikey, SEC-027) is expected by the matrix but NOT declared in the manifest -- the credential is either missing or undeclared
[FAIL] S2 vr1-dc1 EXPECTED-BUT-ABSENT: 'maas-region-db-password' (id dc1-region-db, SEC-027) is expected by the matrix but NOT declared in the manifest -- the credential is either missing or undeclared
[FAIL] S5 ASYMMETRY: vr1-dc0 declares id=dcN-maas-power-key file=id_ed25519 on headend (custody=off-manifest-known) with no counterpart in vr1-dc1 -- a per-DC credential must exist at BOTH DCs in the same shape
[FAIL] S5 ASYMMETRY: vr1-dc0 declares id=dcN-maas-power-key file=maas-virsh_ed25519 on headend (custody=off-manifest-known) with no counterpart in vr1-dc1 -- a per-DC credential must exist at BOTH DCs in the same shape
[FAIL] S5 ASYMMETRY: vr1-dc1 declares id=dcN-maas-power-key file=id_dcN_power on headend (custody=off-manifest-known) with no counterpart in vr1-dc0 -- a per-DC credential must exist at BOTH DCs in the same shape
[FAIL] S6 IDENTITY CONFLATION: id 'maas-region-admin' serves 2 principal types (human via gui; service via api, cli-profile) -- ruling 5 requires one identity to serve one principal type
[FAIL] E4 UNCHECKABLE: 2 row(s) have no declared location for their (role, site) and can never be verified -- add a location row or correct the matrix: capi-mgmt-kubeconfig 'config' (cloud/-); rbd-mirror-peer-token 'rbd-mirror-bootstrap-token' (unit/-)
FAIL: creds-matrix tier 1 -- 121 row(s), 19 check group(s) clean, 11 finding(s)
================ P7: Octavia amphora PKI ================
=== octavia-pki verify: vr1-dc0 ===
ok host: 'voffice1' is the declared headend, so its filesystem is the right one to measure
(expect CA label 'VR1 DC0'; provider VIP v4=10.12.4.57 v6=2602:f3e2:f02:11::57)
ok A1 workspace present: ~/octavia-pki/vr1-dc0
ok A2 all 10 expected artifacts present
ok A3 private issuing-ca/passphrase.txt is 0600
ok A3 private issuing-ca/issuing-ca.key.enc is 0600
ok A3 private controller-ca/passphrase.txt is 0600
ok A3 private controller-ca/controller-ca.key.enc is 0600
ok A3 private controller/controller.key is 0600
ok A3 private controller/controller.bundle.pem is 0600
ok A3 cert issuing-ca/issuing-ca.cert.pem is 600 -- not group/world writable
ok A3 cert controller-ca/controller-ca.cert.pem is 600 -- not group/world writable
ok A3 cert controller-ca/controller-ca.cert.srl is 600 -- not group/world writable
ok A3 cert controller/controller.cert.pem is 600 -- not group/world writable
ok A4 issuing CA subject names this DC: contains 'VR1 DC0 Omega Cloud Octavia Issuing CA'
ok A5 controller CA subject names this DC: contains 'VR1 DC0 Omega Cloud Octavia Controller CA'
ok A6 issuing CA self-signature verifies
ok A7 controller CA self-signature verifies
ok A8 controller cert verifies against the CONTROLLER CA
ok A8 controller cert correctly does NOT verify against the issuing CA
ok A9 SAN carries 2 DNS names
ok A9 SAN carries this DC's provider v4 VIP (10.12.4.57)
ok A9 SAN carries this DC's provider v6 VIP (2602:f3e2:f02:11::57)
ok A12 DNS SANs are INERT -- os-public-hostname is set in no deploy artifact (B5 IP-only), so nothing resolves them; this assertion ARMS ITSELF when D-106 sets it
ok A12 DNS SANs are all in this DC's expected zone 'omega.dc0.vr1.cloud.neumatrix.local'
ok A13 controller cert CN is 'octavia-controller.omega.dc0.vr1.cloud.neumatrix.local'
ok A14 controller.key and controller.cert.pem carry the SAME public key (they are a pair)
ok A14 bundle carries exactly one CERTIFICATE block and one PRIVATE KEY block
ok A14 the bundle's CERTIFICATE block is byte-identical to controller.cert.pem
ok A14 the bundle's PRIVATE KEY block is byte-identical to controller.key
ok A15 controller cert carries keyUsage (critical: digitalSignature, keyEncipherment) and EKU (clientAuth, serverAuth)
ok A16 controller cert is valid and not expiring within 30 days
ok A10 overlay is 0600
ok A10 overlay declares 5 lb-mgmt-* keys
ok A10 overlay is ASCII clean
ok A10 overlay is gitignored (F4)
ok A17 the overlay's controller cert and CA values decode byte-identically to this workspace's bundle and controller CA
ok A11 all 3 compared artifacts differ from vr1-dc1 -- per-DC independence holds
octavia-pki verify (vr1-dc0): PASS -- 37 assertion(s), 0 failed
[ok] P7 octavia PKI verified for vr1-dc0, and its DNS SANs are in this DC's own zone
================ P6: stage-2 reminders (NOT run here) ================
- after 'juju add-model': bash scripts/juju-spaces-check.sh
- with sudo: bash scripts/osd-blank-check.sh
- phase-01 Step 1.2: juju deploy --dry-run (plan: 50 apps / 97 relations)
PREFLIGHT: FAIL (DC=vr1-dc0) -- do NOT deploy
================================================================================
P5 FINDING DIFF vs THE 2026-07-30 OPERATOR ACCEPTANCE (added by the capturing session)
================================================================================
The 2026-07-30 ruling accepted SIX findings, ENUMERATED, and says in terms that it
covers "these six, enumerated, and nothing else". This run reports ELEVEN. The diff:
COVERED BY THE ACCEPTANCE (6 of 6 still present, none resolved):
1 S2 vr1-dc0 'opnsense-api.txt' (dc0-edge-api, SEC-021)
2 S5 asymmetry vr1-dc0 id_ed25519 (dcN-maas-power-key)
3 S5 asymmetry vr1-dc0 maas-virsh_ed25519(dcN-maas-power-key)
4 S5 asymmetry vr1-dc1 id_dcN_power (dcN-maas-power-key)
5 S6 identity conflation 'maas-region-admin' (SEC-020)
6 E4 two uncheckable rows (capi-mgmt-kubeconfig, rbd-mirror-peer-token)
NOT COVERED -- FIVE NEW, all vr1-dc1, all opened AFTER the acceptance:
7 S2 vr1-dc1 'maas-region-db-password' (dc1-region-db, SEC-027)
8 S2 vr1-dc1 'maas-region-admin-password' (dc1-region-admin, SEC-027)
9 S2 vr1-dc1 'maas-region-api-key.txt' (dc1-region-apikey, SEC-027)
10 S2 vr1-dc1 'maas-juju-api-key.txt' (dc1-juju-apikey, SEC-028)
11 S2 vr1-dc1 'maas-juju-user-password' (dc1-juju-user, SEC-028)
WHAT THE FIVE ARE. Every one is the D-137 FORWARD REGISTER working as designed: dc0
was given its own MAAS region and its own juju service credential, the matrix was
extended to EXPECT the same at BOTH DCs, and dc1's half does not exist yet because
dc1's region VM is authored but NOT applied. The finding is "dc1 has not been built",
stated by a register that can see an absence. Deleting the rows to go green is the
one thing the standing rules forbid.
THIS DIFF IS NOT A RULING. It is the enumeration the acceptance requires before the
five can be carried, and it is put to the operator as its own GA-R5 exchange.
