PREFLIGHT TARGET: DC=vr1-dc0 (override: DC=vr1-dc1 bash scripts/preflight.sh)
Every gate below is run against THIS DC; the verdict line repeats it.
================ P1: repo lint ================
[WARN] L1 docs/design-decisions.md: 239 non-ASCII byte(s) (legacy D-001..018 carve-out; NEW entries must be ASCII)
WARN: repo lint (0 fail, 1 warn, 627 files scanned)
================ P2: bundle invariants ================
validating the MERGED vr1-dc0 deploy input: bundle.yaml --overlay overlays/vr1-dc0-vips.yaml --overlay overlays/vr1-dc0-machines.yaml --overlay overlays/vr1-dc0-octavia-pki.yaml --dc vr1-dc0
[ok] 11 charms bind public->provider-public; none on provider-vip
[ok] 13 clustered VIP(s) are provider/admin/internal, octet 50-99 (13 dual-family)
[ok] ovn-chassis bridge-interface-mappings: 2 well-formed MAC(s) (role-sep; VR0 set N/A)
[ok] 108 relations well-formed (explicit endpoints, all apps exist)
[ok] mysql-innodb-cluster num_units=3 (D-062)
[ok] 12 hacluster principal(s) all carry a VIP (R11)
[ok] 12 hacluster subordinate(s) declare cluster_count == principal num_units
[ok] keystone policyd-override wired in-bundle; zip content matches source (DOCFIX-071)
[ok] machines block: all 9 machine(s) tagged openstack-vr1-dc0, matching --dc vr1-dc0
[ok] placement: role-separated (3 control/2 compute/4 storage); anti-affinity + role placement + counts OK
PASS: Pattern A / D-052-D-053 bundle invariants (bundle.yaml)
================ P3: channel assert (charmhub) ================
[ok] barbican 2024.1/stable (barbican)
[ok] barbican-vault 2024.1/stable (barbican-vault)
[ok] ceph-mon squid/stable (ceph-mon)
[ok] ceph-osd squid/stable (ceph-osd)
[ok] ceph-radosgw squid/stable (ceph-radosgw)
[ok] ceph-rbd-mirror squid/stable (ceph-rbd-mirror)
[ok] cinder 2024.1/stable (cinder)
[ok] cinder-backup 2024.1/stable (cinder-backup)
[ok] cinder-ceph 2024.1/stable (cinder-ceph)
[ok] designate 2024.1/stable (designate)
[ok] designate-bind 2024.1/stable (designate-bind)
[ok] glance 2024.1/stable (glance)
[ok] glance-simplestreams-sync 2024.1/stable (glance-simplestreams-sync)
[ok] hacluster 2.4/stable (keystone-hacluster, glance-hacluster, neutron-api-hacluster, nova-cloud-controller-hacluster, placement-hacluster, openstack-dashboard-hacluster, cinder-hacluster, octavia-hacluster, barbican-hacluster, magnum-hacluster, ceph-radosgw-hacluster, designate-hacluster)
[ok] keystone 2024.1/stable (keystone)
[ok] magnum 2024.1/stable (magnum)
[ok] magnum-dashboard 2024.1/stable (magnum-dashboard)
[ok] memcached latest/stable (memcached)
[ok] mysql-innodb-cluster 8.0/stable (mysql-innodb-cluster)
[ok] mysql-router 8.0/stable (vault-mysql-router, keystone-mysql-router, glance-mysql-router, ncc-mysql-router, placement-mysql-router, neutron-api-mysql-router, cinder-mysql-router, dashboard-mysql-router, octavia-mysql-router, barbican-mysql-router, magnum-mysql-router, designate-mysql-router)
[ok] neutron-api 2024.1/stable (neutron-api)
[ok] neutron-api-plugin-ovn 2024.1/stable (neutron-api-plugin-ovn)
[ok] nova-cloud-controller 2024.1/stable (nova-cloud-controller)
[ok] nova-compute 2024.1/stable (nova-compute)
[ok] octavia 2024.1/stable (octavia)
[ok] octavia-dashboard 2024.1/stable (octavia-dashboard)
[ok] octavia-diskimage-retrofit 2024.1/stable (octavia-diskimage-retrofit)
[ok] openstack-dashboard 2024.1/stable (openstack-dashboard)
[ok] ovn-central 24.03/stable (ovn-central)
[ok] ovn-chassis 24.03/stable (ovn-chassis, ovn-chassis-octavia)
[ok] placement 2024.1/stable (placement)
[ok] rabbitmq-server 3.9/stable (rabbitmq-server)
[ok] vault 1.8/stable (vault)
PASS: channel assert (33 pins, 0 fail, 0 warn)
================ P4: live pre-flight (MAAS/overlay/nodes) ================
=== DC selection ===
PASS: gating DC=vr1-dc0 (planes 10.12.4.0/22 .. 10.12.36.0/22; 10 node(s))
=== Repo (informational) ===
NOTE: REPO=/home/jessea123/openstack-caracal-dc-dc
NOTE: HEAD: c58bf95 GA-R4 session close: F9 closed live, the reissue tool, P7, lib-identity
NOTE: working tree clean
=== CHECK 0: per-DC octavia-pki overlay (no key material printed) ===
PASS: overlay present with 5 lb-mgmt-* keys
PASS: overlay ASCII clean
=== CHECK 1: bundle VIPs -- v4 triple or R2 dual-family sextet, .50-.99 (provider/admin/internal) ===
PASS: vip: line count = 13 (from overlays/vr1-dc0-vips.yaml)
PASS: aligned VIPs OK=13 bad=0 (DC=vr1-dc0 bands 10.12.4/10.12.8/10.12.12)
=== MAAS reachability gate (read-only) ===
PASS: MAAS reachable (profile=admin)
=== CHECK 3: six planes resolved BY CIDR (id/vid/gw/dns) ===
provider-public 10.12.4.0/22 id=7 vid=0 gw=10.12.4.1 dns=[]
metal-admin 10.12.8.0/22 id=6 vid=0 gw=none dns=["10.12.8.3"]
metal-internal 10.12.12.0/22 id=12 vid=0 gw=none dns=[]
data-tenant 10.12.16.0/22 id=13 vid=0 gw=none dns=[]
storage 10.12.32.0/22 id=14 vid=0 gw=none dns=[]
replication 10.12.36.0/22 id=15 vid=0 gw=none dns=[]
PASS: all six planes present (by CIDR)
PASS: metal-internal is UNTAGGED (vid 0) -- D-133 flat carve
NOTE: stale-NAME check is juju-side (run scripts/juju-spaces-check.sh after add-model)
=== CHECK 2: data/storage NIC links BY CIDR (DC=vr1-dc0 role nodes; octet per D-134 band) ===
== vr1-dc0-control-01 (nhg3nf, octet .100) ==
enp3s0 -> 10.12.12.0/22 10.12.12.100 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.100 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.100 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.100 type=physical
== vr1-dc0-control-02 (ssyexn, octet .101) ==
enp3s0 -> 10.12.12.0/22 10.12.12.101 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.101 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.101 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.101 type=physical
== vr1-dc0-control-03 (sk8c4d, octet .102) ==
enp3s0 -> 10.12.12.0/22 10.12.12.102 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.102 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.102 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.102 type=physical
== vr1-dc0-compute-01 (dbcq8h, octet .120) ==
enp3s0 -> 10.12.12.0/22 10.12.12.120 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.120 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.120 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.120 type=physical
== vr1-dc0-compute-02 (sgwfnb, octet .121) ==
enp3s0 -> 10.12.12.0/22 10.12.12.121 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.121 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.121 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.121 type=physical
== vr1-dc0-storage-01 (kghggm, octet .150) ==
enp3s0 -> 10.12.12.0/22 10.12.12.150 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.150 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.150 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.150 type=physical
== vr1-dc0-storage-02 (8mtpxq, octet .151) ==
enp3s0 -> 10.12.12.0/22 10.12.12.151 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.151 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.151 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.151 type=physical
== vr1-dc0-storage-03 (sn6qda, octet .152) ==
enp3s0 -> 10.12.12.0/22 10.12.12.152 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.152 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.152 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.152 type=physical
== vr1-dc0-storage-04 (xqqwdq, octet .153) ==
enp3s0 -> 10.12.12.0/22 10.12.12.153 type=physical
enp4s0 -> 10.12.16.0/22 10.12.16.153 type=physical
enp5s0 -> 10.12.32.0/22 10.12.32.153 type=physical
enp6s0 -> 10.12.36.0/22 10.12.36.153 type=physical
NOTE: vr1-dc0-juju-01 (7n87bt, octet .5) is D-134 utility-band infrastructure, not an OpenStack role node -- data-plane carve not asserted
=== CHECK 4: DC=vr1-dc0 OpenStack role nodes -- status / power ===
vr1-dc0-control-01 -> wired-thrush Ready power=off
PASS: vr1-dc0-control-01 Ready
vr1-dc0-control-02 -> ace-robin Ready power=off
PASS: vr1-dc0-control-02 Ready
vr1-dc0-control-03 -> real-filly Ready power=off
PASS: vr1-dc0-control-03 Ready
vr1-dc0-compute-01 -> keen-dove Ready power=off
PASS: vr1-dc0-compute-01 Ready
vr1-dc0-compute-02 -> superb-piglet Ready power=off
PASS: vr1-dc0-compute-02 Ready
vr1-dc0-storage-01 -> first-oryx Ready power=off
PASS: vr1-dc0-storage-01 Ready
vr1-dc0-storage-02 -> wise-stud Ready power=off
PASS: vr1-dc0-storage-02 Ready
vr1-dc0-storage-03 -> alert-cub Ready power=off
PASS: vr1-dc0-storage-03 Ready
vr1-dc0-storage-04 -> moral-salmon Ready power=off
PASS: vr1-dc0-storage-04 Ready
Summary: 0 fatal, 0 warning
================ P5: credential matrix (D-137 tier 1 + tier 2 local) ================
=== creds-matrix: tier 1 (STATIC) ===
=== creds-matrix: tier 2 (EXISTENCE) ===
(host: voffice1)
[ok] S1 schema: 101 rows, all enums valid, site-keys region-qualified, no duplicate (id,site,host-role,filename)
[ok] S3 render: 3 source field(s) SKIPPED -- rendering them needs the declared path from creds-manifests/vm-secret-locations (ruling 3); the list now EXISTS but the source-field derivation is not wired
[ok] S3 render drift: rendered row fields (mode, source) match checked-in; header prose and non-jumphost rows are OUT OF SCOPE of this compare
[ok] S4 mint-ref: every script:/runbook: reference resolves to a real location
[ok] S4 provenance debt: 30 row(s) are mint-ref=operator-terminal -- NOT reproducible from the repo (research FINDING 1). Admitted by design; converting them is remediation, not a checker fix.
[ok] S7 notes: 37 note key(s) referenced, all resolve, none orphaned
[ok] E0 jumphost location '~/vr1-office1-creds/*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '~/vr1-dc0-creds/*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '~/vr1-dc1-creds/*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '~/vault-init/*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '~/tenant-*/*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '/home/jessea123/openstack-caracal-dc-dc/opentofu/terraform.tfstate' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '/home/jessea123/openstack-caracal-dc-dc/opentofu/terraform.tfstate.backup' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '/home/jessea123/openstack-caracal-dc-dc/opentofu/terraform.tfstate.pre-*' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 jumphost location '~/admin-openrc' NOT PROBED -- role 'jumphost' lives on 'vcloud' and this run is on 'voffice1'; probing it here would measure a different host's filesystem, so absence is not asserted over it
[ok] E0 8 remote location(s) SKIPPED -- rerun with --remote to include the headend shadow stores (SEC-022) and the region secrets dir (SEC-020)
[ok] E1 71 expected artifact(s) NOT JUDGED -- their role (headend/-, headend/vr1-dc0, headend/vr1-dc1, headend/vr1-office1, jumphost/-, jumphost/vr1-dc0, jumphost/vr1-dc1, jumphost/vr1-office1, netbox/vr1-office1) has at least one location that could not be probed, so absence cannot be asserted over it
[ok] E1/E3 existence: every expected artifact present and nothing undeclared, across 0 fully-probed role(s)
[ok] tier 3 (VALIDITY) NOT RUN -- pass --tier3 (with --tier2) to compare cross-copy sha256 provenance
[FAIL] S2 vr1-dc0 EXPECTED-BUT-ABSENT: 'opnsense-api.txt' (id dc0-edge-api, SEC-021) is expected by the matrix but NOT declared in the manifest -- the credential is either missing or undeclared
[FAIL] S5 ASYMMETRY: vr1-dc0 declares id=dcN-maas-power-key file=id_ed25519 on headend (custody=off-manifest-known) with no counterpart in vr1-dc1 -- a per-DC credential must exist at BOTH DCs in the same shape
[FAIL] S5 ASYMMETRY: vr1-dc0 declares id=dcN-maas-power-key file=maas-virsh_ed25519 on headend (custody=off-manifest-known) with no counterpart in vr1-dc1 -- a per-DC credential must exist at BOTH DCs in the same shape
[FAIL] S5 ASYMMETRY: vr1-dc1 declares id=dcN-maas-power-key file=id_dcN_power on headend (custody=off-manifest-known) with no counterpart in vr1-dc0 -- a per-DC credential must exist at BOTH DCs in the same shape
[FAIL] S6 IDENTITY CONFLATION: id 'maas-region-admin' serves 2 principal types (human via gui; service via api, cli-profile) -- ruling 5 requires one identity to serve one principal type
[FAIL] E4 UNCHECKABLE: 2 row(s) have no declared location for their (role, site) and can never be verified -- add a location row or correct the matrix: capi-mgmt-kubeconfig 'config' (cloud/-); rbd-mirror-peer-token 'rbd-mirror-bootstrap-token' (unit/-)
FAIL: creds-matrix tier 1 -- 101 row(s), 19 check group(s) clean, 6 finding(s)
================ P7: Octavia amphora PKI ================
=== octavia-pki verify: vr1-dc0 ===
ok host: 'voffice1' is the declared headend, so its filesystem is the right one to measure
(expect CA label 'VR1 DC0'; provider VIP v4=10.12.4.57 v6=2602:f3e2:f02:11::57)
ok A1 workspace present: ~/octavia-pki/vr1-dc0
ok A2 all 10 expected artifacts present
ok A3 private issuing-ca/passphrase.txt is 0600
ok A3 private issuing-ca/issuing-ca.key.enc is 0600
ok A3 private controller-ca/passphrase.txt is 0600
ok A3 private controller-ca/controller-ca.key.enc is 0600
ok A3 private controller/controller.key is 0600
ok A3 private controller/controller.bundle.pem is 0600
ok A3 cert issuing-ca/issuing-ca.cert.pem is 600 -- not group/world writable
ok A3 cert controller-ca/controller-ca.cert.pem is 600 -- not group/world writable
ok A3 cert controller-ca/controller-ca.cert.srl is 600 -- not group/world writable
ok A3 cert controller/controller.cert.pem is 600 -- not group/world writable
ok A4 issuing CA subject names this DC: contains 'VR1 DC0 Omega Cloud Octavia Issuing CA'
ok A5 controller CA subject names this DC: contains 'VR1 DC0 Omega Cloud Octavia Controller CA'
ok A6 issuing CA self-signature verifies
ok A7 controller CA self-signature verifies
ok A8 controller cert verifies against the CONTROLLER CA
ok A8 controller cert correctly does NOT verify against the issuing CA
ok A9 SAN carries 2 DNS names
ok A9 SAN carries this DC's provider v4 VIP (10.12.4.57)
ok A9 SAN carries this DC's provider v6 VIP (2602:f3e2:f02:11::57)
ok A12 DNS SANs are INERT -- os-public-hostname is set in no deploy artifact (B5 IP-only), so nothing resolves them; this assertion ARMS ITSELF when D-106 sets it
ok A12 DNS SANs are all in this DC's expected zone 'omega.dc0.vr1.cloud.neumatrix.local'
ok A13 controller cert CN is 'octavia-controller.omega.dc0.vr1.cloud.neumatrix.local'
ok A14 controller.key and controller.cert.pem carry the SAME public key (they are a pair)
ok A14 bundle carries exactly one CERTIFICATE block and one PRIVATE KEY block
ok A14 the bundle's CERTIFICATE block is byte-identical to controller.cert.pem
ok A14 the bundle's PRIVATE KEY block is byte-identical to controller.key
ok A15 controller cert carries keyUsage (critical: digitalSignature, keyEncipherment) and EKU (clientAuth, serverAuth)
ok A16 controller cert is valid and not expiring within 30 days
ok A10 overlay is 0600
ok A10 overlay declares 5 lb-mgmt-* keys
ok A10 overlay is ASCII clean
ok A10 overlay is gitignored (F4)
ok A17 the overlay's controller cert and CA values decode byte-identically to this workspace's bundle and controller CA
ok A11 all 3 compared artifacts differ from vr1-dc1 -- per-DC independence holds
octavia-pki verify (vr1-dc0): PASS -- 37 assertion(s), 0 failed
[ok] P7 octavia PKI verified for vr1-dc0, and its DNS SANs are in this DC's own zone
================ P6: stage-2 reminders (NOT run here) ================
- after 'juju add-model': bash scripts/juju-spaces-check.sh
- with sudo: bash scripts/osd-blank-check.sh
- phase-01 Step 1.2: juju deploy --dry-run (plan: 50 apps / 97 relations)
PREFLIGHT: FAIL (DC=vr1-dc0) -- do NOT deploy
