diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 5cdbdd7..e4049af 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2013,6 +2013,48 @@ at `life: dead`** -- that moved it BACK to `dying` and re-armed the undertaker loop. It did not help and it changed state. Both are now written into Path M as explicit prohibitions. + **>>> THE dc0 JUJU CONTROLLER WAS REBUILT FROM MAAS 2026-08-03 AND THE ENVIRONMENT IS READY + FOR THE DEPLOY. <<<** Operator ruling: *"There is no reason to troubleshoot the controller + issues or gracefully try to repair at this point. It will be easier and cleaner just the + remove the machine and rebuild fresh from MAAS. We have the configuration the controller is + supposed to have including naming, numbering, networking, and software so a full rebuild + will be clean."* Executed as `runbooks/dc-dc-teardown-rollback.md` **Path C**, every gate + captured. + **C.2b(1)** `juju unregister vr1-dc0-controller` rc=0 (client-side only). + **C.2b(2)** MAAS **release**, not delete -- release returns the VM to `Ready` while PRESERVING + the interface carve and the `juju-controller-vr1-dc0` tag that C.6/C.7 target, and MAAS + re-images the OS at bootstrap, which IS the fresh rebuild; a delete would have destroyed the + carve and forced re-enlistment. Pre-release gate: tag matched EXACTLY ONE machine + (`subtle-grouse`/`arfr7p`), nine role nodes `Ready/owner=None`. + **C.3** census **10 machines, count UNCHANGED -- no cascade**; `arfr7p` `Ready/owner=None`; + **the tag SURVIVED**, which is the property bootstrap depends on. + **C.5** credential proven to WORK, not merely exist: key extracted from the client store + WITHOUT being printed (3 parts), `AUTH: OK` against `http://10.12.8.6:5240/MAAS`, rack + identity `hot-kid`/`c3aqh8` confirming the **DC-LOCAL** region rather than Office1, + superuser scope sees 10 machines, temp profile logged out. **The cloud definition and + credential SURVIVED the unregister** -- they belong to the client, not the controller. + **C.6** tag gate PASS (exactly one, `Ready`). + **C.7 BOOTSTRAP COMPLETE, EXIT 0**, 04:09:12Z -> 04:15:28Z (~6 min). Took `arfr7p`, the + tagged VM. Both ruled constraint flags plus `--bootstrap-base ubuntu@22.04`. juju 3.6.27. + **C.8 -- AND THE WARNING IS NOW EMPIRICALLY CONFIRMED: A FRESH CONTROLLER HAS NO + MODEL-DEFAULTS.** Measured BEFORE: all five keys empty. Set and read back on CONTENT: + `apt-http-proxy=http://10.12.8.4:3142`, `snap-http-proxy`/`snap-https-proxy=http://10.12.8.4:3129`; + `apt-mirror` deliberately LEFT UNSET per D-135 amendment (b); `snap-store-proxy` empty. + **The snap keys had never been set on the OLD controller either** -- RULING 2 (07-31) was + built and never wired, so this rebuild closes a gap rather than merely restoring one. + **C.9** `juju add-model vr1-dc0` with `vr1-dc0-cred`; **defaults INHERITED** (all three + proxies present in the model, `apt-mirror` empty); `juju-spaces-check.sh vr1-dc0` **PASS, + 0 fatal, exit 0**, all six ruled spaces. + **C.10** all five staged deploy inputs re-verified against HEAD -- the machines overlay was + re-staged (`dd914358` -> `e8ca794c`) because the origin block was deleted at convergence; + PKI overlay unchanged `5fc117f1` mode `0600`. + **STEP 4.2 DRY-RUN GATE: ALL PASS.** Nine constraint lines, ALL `tags=openstack-vr1-dc0`, + split 3 control / 2 compute / 4 storage per D-121 Option C; nine machines; no overlay path + error (the three `ERROR|cannot` greps are the documented benign R11 `name`/`variables` + unmarshal pair and juju RPC teardown noise). **Four designate applications present, which is + CORRECT under D-106** -- their absence would be the defect. + **THE DEPLOY (Step 4.3) IS NOT RUN** -- it is the gated mutation and awaits the operator. + **>>> CAUSATION SETTLED 2026-08-03, AND THE HEADLINE IS PROCEDURAL, NOT A FLAG: I RAN AN AD-HOC COMMAND INSTEAD OF THE TESTED D-061 TEARDOWN SCRIPTS THIS REPO BUILT FOR EXACTLY THIS OPERATION. <<<** diff --git a/docs/audit/stage5-controller-rebuild-dc0-20260803.txt b/docs/audit/stage5-controller-rebuild-dc0-20260803.txt new file mode 100644 index 0000000..41f497b --- /dev/null +++ b/docs/audit/stage5-controller-rebuild-dc0-20260803.txt @@ -0,0 +1,397 @@ +04:30:46 INFO juju.cmd supercommand.go:56 running juju [3.6.27 4f49d69aab83fe68390126013713a228d57d0edf gc go1.26.5] +04:30:46 DEBUG juju.cmd supercommand.go:57 args: []string{"/snap/juju/35621/bin/juju", "deploy", "./bundle.yaml", "--overlay", "overlays/vr1-dc0-vips.yaml", "--overlay", "overlays/vr1-dc0-machines.yaml", "--overlay", "overlays/vr1-dc0-octavia-pki.yaml", "-m", "vr1-dc0", "--dry-run", "--debug"} +04:30:46 INFO juju.juju api.go:90 connecting to API addresses: [10.12.8.5:17070 10.12.4.5:17070 [fd50:840e:74e2:220::5]:17070] +04:30:46 DEBUG juju.api apiclient.go:1318 api dial attempt starting: url=wss://[fd50:840e:74e2:220::5]:17070/model/9bc2650e-5b17-4747-84e0-ec3f57237a74/api ip=[fd50:840e:74e2:220::5]:17070 attempt=1 proxy=direct +04:30:46 DEBUG juju.api apiclient.go:1341 api dial attempt failed: url=wss://[fd50:840e:74e2:220::5]:17070/model/9bc2650e-5b17-4747-84e0-ec3f57237a74/api address=wss://[fd50:840e:74e2:220::5]:17070 ip=[fd50:840e:74e2:220::5]:17070 attempt=1 proxy=direct elapsed=0s err=dial tcp [fd50:840e:74e2:220::5]:17070: connect: network is unreachable +04:30:46 DEBUG juju.api apiclient.go:1318 api dial attempt starting: url=wss://10.12.8.5:17070/model/9bc2650e-5b17-4747-84e0-ec3f57237a74/api ip=10.12.8.5:17070 attempt=1 proxy=direct +04:30:46 DEBUG juju.api apiclient.go:1365 successfully dialed "wss://10.12.8.5:17070/model/9bc2650e-5b17-4747-84e0-ec3f57237a74/api" +04:30:46 DEBUG juju.api apiclient.go:1329 api dial attempt succeeded: url=wss://10.12.8.5:17070/model/9bc2650e-5b17-4747-84e0-ec3f57237a74/api ip=10.12.8.5:17070 attempt=1 proxy=direct elapsed=9ms +04:30:46 INFO juju.api apiclient.go:668 connection established to "wss://10.12.8.5:17070/model/9bc2650e-5b17-4747-84e0-ec3f57237a74/api" +04:30:46 INFO juju.juju api.go:90 connecting to API addresses: [10.12.8.5:17070 10.12.4.5:17070 [fd50:840e:74e2:220::5]:17070] +04:30:46 DEBUG juju.api apiclient.go:1318 api dial attempt starting: url=wss://[fd50:840e:74e2:220::5]:17070/api ip=[fd50:840e:74e2:220::5]:17070 attempt=1 proxy=direct +04:30:46 DEBUG juju.api apiclient.go:1341 api dial attempt failed: url=wss://[fd50:840e:74e2:220::5]:17070/api address=wss://[fd50:840e:74e2:220::5]:17070 ip=[fd50:840e:74e2:220::5]:17070 attempt=1 proxy=direct elapsed=0s err=dial tcp [fd50:840e:74e2:220::5]:17070: connect: network is unreachable +04:30:46 DEBUG juju.api apiclient.go:1318 api dial attempt starting: url=wss://10.12.8.5:17070/api ip=10.12.8.5:17070 attempt=1 proxy=direct +04:30:46 DEBUG juju.api apiclient.go:1365 successfully dialed "wss://10.12.8.5:17070/api" +04:30:46 DEBUG juju.api apiclient.go:1329 api dial attempt succeeded: url=wss://10.12.8.5:17070/api ip=10.12.8.5:17070 attempt=1 proxy=direct elapsed=8ms +04:30:46 INFO juju.api apiclient.go:668 connection established to "wss://10.12.8.5:17070/api" +04:30:46 WARN cmd bundle.go:190 These fields + unmarshal document 0: yaml: unmarshal errors: + line 62: field name not found in type charm.bundleData + line 96: field variables not found in type charm.bundleData +will be ignored during deployment +04:30:47 DEBUG juju.cmd.juju.application.deployer bundlehandler.go:262 model: &bundlechanges.Model{ + Applications: { + }, + Machines: { + }, + Relations: nil, + ConstraintsEqual: func(string, string) bool {...}, + Sequence: {"space":7, "subnet":13}, + sequence: {}, + MachineMap: {}, + logger: nil, +} +04:30:47 INFO cmd bundlehandler.go:373 Located charm "barbican" in charm-hub, channel 2024.1/stable +04:30:48 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:30:48 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +04:30:48 INFO cmd bundlehandler.go:373 Located charm "barbican-vault" in charm-hub, channel 2024.1/stable +04:30:49 INFO cmd bundlehandler.go:373 Located charm "ceph-mon" in charm-hub, channel squid/stable +04:30:49 INFO cmd bundlehandler.go:373 Located charm "ceph-osd" in charm-hub, channel squid/stable +04:30:49 INFO cmd bundlehandler.go:373 Located charm "ceph-radosgw" in charm-hub, channel squid/stable +04:30:49 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:30:50 INFO cmd bundlehandler.go:373 Located charm "ceph-rbd-mirror" in charm-hub, channel squid/stable +04:30:50 INFO cmd bundlehandler.go:373 Located charm "cinder" in charm-hub, channel 2024.1/stable +04:30:50 INFO cmd bundlehandler.go:373 Located charm "cinder-backup" in charm-hub, channel 2024.1/stable +04:30:51 INFO cmd bundlehandler.go:373 Located charm "cinder-ceph" in charm-hub, channel 2024.1/stable +04:30:51 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:30:51 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +04:30:52 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +04:30:52 INFO cmd bundlehandler.go:373 Located charm "designate" in charm-hub, channel 2024.1/stable +04:30:52 INFO cmd bundlehandler.go:373 Located charm "designate-bind" in charm-hub, channel 2024.1/stable +04:30:53 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:30:53 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +04:30:54 INFO cmd bundlehandler.go:373 Located charm "glance" in charm-hub, channel 2024.1/stable +04:30:54 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:30:55 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +04:30:55 INFO cmd bundlehandler.go:373 Located charm "glance-simplestreams-sync" in charm-hub, channel 2024.1/stable +04:30:55 INFO cmd bundlehandler.go:373 Located charm "keystone" in charm-hub, channel 2024.1/stable +04:30:56 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:30:56 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +04:30:56 INFO cmd bundlehandler.go:373 Located charm "magnum" in charm-hub, channel 2024.1/stable +04:30:57 INFO cmd bundlehandler.go:373 Located charm "magnum-dashboard" in charm-hub, channel 2024.1/stable +04:30:57 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:30:57 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +04:30:58 INFO cmd bundlehandler.go:373 Located charm "memcached" in charm-hub, channel latest/stable +04:30:58 INFO cmd bundlehandler.go:373 Located charm "mysql-innodb-cluster" in charm-hub, channel 8.0/stable +04:30:58 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +04:30:59 INFO cmd bundlehandler.go:373 Located charm "neutron-api" in charm-hub, channel 2024.1/stable +04:30:59 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:30:59 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +04:31:00 INFO cmd bundlehandler.go:373 Located charm "neutron-api-plugin-ovn" in charm-hub, channel 2024.1/stable +04:31:00 INFO cmd bundlehandler.go:373 Located charm "nova-cloud-controller" in charm-hub, channel 2024.1/stable +04:31:01 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:31:02 INFO cmd bundlehandler.go:373 Located charm "nova-compute" in charm-hub, channel 2024.1/stable +04:31:02 INFO cmd bundlehandler.go:373 Located charm "octavia" in charm-hub, channel 2024.1/stable +04:31:02 INFO cmd bundlehandler.go:373 Located charm "octavia-dashboard" in charm-hub, channel 2024.1/stable +04:31:03 INFO cmd bundlehandler.go:373 Located charm "octavia-diskimage-retrofit" in charm-hub, channel 2024.1/stable +04:31:03 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:31:04 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +04:31:04 INFO cmd bundlehandler.go:373 Located charm "openstack-dashboard" in charm-hub, channel 2024.1/stable +04:31:05 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:31:05 INFO cmd bundlehandler.go:373 Located charm "ovn-central" in charm-hub, channel 24.03/stable +04:31:05 INFO cmd bundlehandler.go:373 Located charm "ovn-chassis" in charm-hub, channel 24.03/stable +04:31:05 INFO cmd bundlehandler.go:373 Located charm "ovn-chassis" in charm-hub, channel 24.03/stable +04:31:06 INFO cmd bundlehandler.go:373 Located charm "placement" in charm-hub, channel 2024.1/stable +04:31:06 INFO cmd bundlehandler.go:373 Located charm "hacluster" in charm-hub, channel 2.4/stable +04:31:07 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +04:31:07 INFO cmd bundlehandler.go:373 Located charm "rabbitmq-server" in charm-hub, channel 3.9/stable +04:31:07 INFO cmd bundlehandler.go:373 Located charm "vault" in charm-hub, channel 1.8/stable +04:31:08 INFO cmd bundlehandler.go:373 Located charm "mysql-router" in charm-hub, channel 8.0/stable +Changes to deploy bundle: +- upload charm barbican from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application barbican from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- upload charm hacluster from charm-hub for base ubuntu@22.04/stable from channel 2.4/stable with architecture=amd64 +- deploy application barbican-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- upload charm mysql-router from charm-hub for base ubuntu@22.04/stable from channel 8.0/stable with architecture=amd64 +- deploy application barbican-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- upload charm barbican-vault from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application barbican-vault from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- upload charm ceph-mon from charm-hub for base ubuntu@22.04/stable from channel squid/stable with architecture=amd64 +- deploy application ceph-mon from charm-hub on ubuntu@22.04/stable with squid/stable +- upload charm ceph-osd from charm-hub for base ubuntu@22.04/stable from channel squid/stable with architecture=amd64 +- deploy application ceph-osd from charm-hub on ubuntu@22.04/stable with squid/stable +- upload charm ceph-radosgw from charm-hub for base ubuntu@22.04/stable from channel squid/stable with architecture=amd64 +- deploy application ceph-radosgw from charm-hub on ubuntu@22.04/stable with squid/stable +- deploy application ceph-radosgw-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- upload charm ceph-rbd-mirror from charm-hub for base ubuntu@22.04/stable from channel squid/stable with architecture=amd64 +- deploy application ceph-rbd-mirror from charm-hub on ubuntu@22.04/stable with squid/stable +- upload charm cinder from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application cinder from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- upload charm cinder-backup from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application cinder-backup from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- upload charm cinder-ceph from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application cinder-ceph from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- deploy application cinder-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- deploy application cinder-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- deploy application dashboard-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- upload charm designate from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application designate from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- upload charm designate-bind from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application designate-bind from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- deploy application designate-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- deploy application designate-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- upload charm glance from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application glance from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- deploy application glance-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- deploy application glance-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- upload charm glance-simplestreams-sync from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application glance-simplestreams-sync from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- upload charm keystone from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application keystone from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- deploy application keystone-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- deploy application keystone-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- upload charm magnum from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application magnum from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- upload charm magnum-dashboard from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application magnum-dashboard from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- deploy application magnum-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- deploy application magnum-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- upload charm memcached from charm-hub for base ubuntu@22.04/stable from channel latest/stable with architecture=amd64 +- deploy application memcached from charm-hub on ubuntu@22.04/stable with latest/stable +- upload charm mysql-innodb-cluster from charm-hub for base ubuntu@22.04/stable from channel 8.0/stable with architecture=amd64 +- deploy application mysql-innodb-cluster from charm-hub on ubuntu@22.04/stable with 8.0/stable +- deploy application ncc-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- upload charm neutron-api from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application neutron-api from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- deploy application neutron-api-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- deploy application neutron-api-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- upload charm neutron-api-plugin-ovn from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application neutron-api-plugin-ovn from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- upload charm nova-cloud-controller from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application nova-cloud-controller from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- deploy application nova-cloud-controller-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- upload charm nova-compute from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application nova-compute from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- upload charm octavia from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application octavia from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- upload charm octavia-dashboard from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application octavia-dashboard from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- upload charm octavia-diskimage-retrofit from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application octavia-diskimage-retrofit from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- deploy application octavia-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- deploy application octavia-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- upload charm openstack-dashboard from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application openstack-dashboard from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- deploy application openstack-dashboard-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- upload charm ovn-central from charm-hub for base ubuntu@22.04/stable from channel 24.03/stable with architecture=amd64 +- deploy application ovn-central from charm-hub on ubuntu@22.04/stable with 24.03/stable +- upload charm ovn-chassis from charm-hub for base ubuntu@22.04/stable from channel 24.03/stable with architecture=amd64 +- deploy application ovn-chassis from charm-hub on ubuntu@22.04/stable with 24.03/stable +- deploy application ovn-chassis-octavia from charm-hub on ubuntu@22.04/stable with 24.03/stable using ovn-chassis +- upload charm placement from charm-hub for base ubuntu@22.04/stable from channel 2024.1/stable with architecture=amd64 +- deploy application placement from charm-hub on ubuntu@22.04/stable with 2024.1/stable +- deploy application placement-hacluster from charm-hub on ubuntu@22.04/stable with 2.4/stable using hacluster +- deploy application placement-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- upload charm rabbitmq-server from charm-hub for base ubuntu@22.04/stable from channel 3.9/stable with architecture=amd64 +- deploy application rabbitmq-server from charm-hub on ubuntu@22.04/stable with 3.9/stable +- upload charm vault from charm-hub for base ubuntu@22.04/stable from channel 1.8/stable with architecture=amd64 +- deploy application vault from charm-hub on ubuntu@22.04/stable with 1.8/stable +- deploy application vault-mysql-router from charm-hub on ubuntu@22.04/stable with 8.0/stable using mysql-router +- add new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "arch=amd64 tags=openstack-vr1-dc0,control" +- add new machine 1 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "arch=amd64 tags=openstack-vr1-dc0,control" +- add new machine 2 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "arch=amd64 tags=openstack-vr1-dc0,control" +- add new machine 3 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "arch=amd64 tags=openstack-vr1-dc0,compute" +- add new machine 4 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "arch=amd64 tags=openstack-vr1-dc0,compute" +- add new machine 5 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "arch=amd64 tags=openstack-vr1-dc0,storage" +- add new machine 6 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "arch=amd64 tags=openstack-vr1-dc0,storage" +- add new machine 7 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "arch=amd64 tags=openstack-vr1-dc0,storage" +- add new machine 8 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "arch=amd64 tags=openstack-vr1-dc0,storage" +- add relation nova-cloud-controller:memcache - memcached:cache +- add relation vault-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation vault:shared-db - vault-mysql-router:shared-db +- add relation mysql-innodb-cluster:certificates - vault:certificates +- add relation keystone-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation keystone-mysql-router:shared-db - keystone:shared-db +- add relation keystone:certificates - vault:certificates +- add relation keystone:ha - keystone-hacluster:ha +- add relation glance-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation glance-mysql-router:shared-db - glance:shared-db +- add relation glance:identity-service - keystone:identity-service +- add relation glance:certificates - vault:certificates +- add relation glance:ha - glance-hacluster:ha +- add relation glance-simplestreams-sync:identity-service - keystone:identity-service +- add relation glance-simplestreams-sync:certificates - vault:certificates +- add relation ncc-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation ncc-mysql-router:shared-db - nova-cloud-controller:shared-db +- add relation nova-cloud-controller:identity-service - keystone:identity-service +- add relation nova-cloud-controller:amqp - rabbitmq-server:amqp +- add relation nova-cloud-controller:image-service - glance:image-service +- add relation nova-cloud-controller:neutron-api - neutron-api:neutron-api +- add relation nova-cloud-controller:cloud-compute - nova-compute:cloud-compute +- add relation nova-cloud-controller:cinder-volume-service - cinder:cinder-volume-service +- add relation nova-cloud-controller:certificates - vault:certificates +- add relation nova-cloud-controller:ha - nova-cloud-controller-hacluster:ha +- add relation nova-compute:amqp - rabbitmq-server:amqp +- add relation nova-compute:image-service - glance:image-service +- add relation placement-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation placement-mysql-router:shared-db - placement:shared-db +- add relation placement:identity-service - keystone:identity-service +- add relation placement:placement - nova-cloud-controller:placement +- add relation placement:certificates - vault:certificates +- add relation placement:ha - placement-hacluster:ha +- add relation neutron-api-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation neutron-api-mysql-router:shared-db - neutron-api:shared-db +- add relation neutron-api:identity-service - keystone:identity-service +- add relation neutron-api:amqp - rabbitmq-server:amqp +- add relation neutron-api:certificates - vault:certificates +- add relation neutron-api-plugin-ovn:neutron-plugin - neutron-api:neutron-plugin-api-subordinate +- add relation neutron-api-plugin-ovn:ovsdb-cms - ovn-central:ovsdb-cms +- add relation neutron-api-plugin-ovn:certificates - vault:certificates +- add relation ovn-central:certificates - vault:certificates +- add relation ovn-chassis:ovsdb - ovn-central:ovsdb +- add relation ovn-chassis:nova-compute - nova-compute:neutron-plugin +- add relation ovn-chassis:certificates - vault:certificates +- add relation neutron-api:ha - neutron-api-hacluster:ha +- add relation cinder-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation cinder-mysql-router:shared-db - cinder:shared-db +- add relation cinder:identity-service - keystone:identity-service +- add relation cinder:amqp - rabbitmq-server:amqp +- add relation cinder:image-service - glance:image-service +- add relation cinder:certificates - vault:certificates +- add relation cinder-ceph:storage-backend - cinder:storage-backend +- add relation cinder-ceph:ceph - ceph-mon:client +- add relation cinder-ceph:ceph-access - nova-compute:ceph-access +- add relation cinder:ha - cinder-hacluster:ha +- add relation cinder-backup:backup-backend - cinder:backup-backend +- add relation cinder-backup:ceph - ceph-mon:client +- add relation ceph-mon:osd - ceph-osd:mon +- add relation ceph-mon:client - nova-compute:ceph +- add relation ceph-mon:client - glance:ceph +- add relation ceph-radosgw:mon - ceph-mon:radosgw +- add relation ceph-radosgw:identity-service - keystone:identity-service +- add relation ceph-radosgw:certificates - vault:certificates +- add relation ceph-radosgw:ha - ceph-radosgw-hacluster:ha +- add relation ceph-rbd-mirror:ceph-local - ceph-mon:rbd-mirror +- add relation dashboard-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation dashboard-mysql-router:shared-db - openstack-dashboard:shared-db +- add relation openstack-dashboard:identity-service - keystone:identity-service +- add relation openstack-dashboard:certificates - vault:certificates +- add relation openstack-dashboard:ha - openstack-dashboard-hacluster:ha +- add relation octavia-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation octavia-mysql-router:shared-db - octavia:shared-db +- add relation octavia:identity-service - keystone:identity-service +- add relation octavia:amqp - rabbitmq-server:amqp +- add relation octavia:neutron-api - neutron-api:neutron-load-balancer +- add relation octavia:certificates - vault:certificates +- add relation octavia-dashboard:dashboard - openstack-dashboard:dashboard-plugin +- add relation ovn-chassis-octavia:ovsdb - ovn-central:ovsdb +- add relation ovn-chassis-octavia:ovsdb-subordinate - octavia:ovsdb-subordinate +- add relation ovn-chassis-octavia:certificates - vault:certificates +- add relation octavia-diskimage-retrofit:juju-info - glance-simplestreams-sync:juju-info +- add relation octavia-diskimage-retrofit:identity-credentials - keystone:identity-credentials +- add relation octavia:ha - octavia-hacluster:ha +- add relation barbican-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation barbican-mysql-router:shared-db - barbican:shared-db +- add relation barbican:identity-service - keystone:identity-service +- add relation barbican:amqp - rabbitmq-server:amqp +- add relation barbican:certificates - vault:certificates +- add relation barbican:secrets - barbican-vault:secrets +- add relation barbican-vault:certificates - vault:certificates +- add relation barbican-vault:secrets-storage - vault:secrets +- add relation barbican:ha - barbican-hacluster:ha +- add relation magnum-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation magnum:shared-db - magnum-mysql-router:shared-db +- add relation magnum:identity-service - keystone:identity-service +- add relation magnum:amqp - rabbitmq-server:amqp +- add relation magnum:certificates - vault:certificates +- add relation magnum-dashboard:dashboard - openstack-dashboard:dashboard-plugin +- add relation magnum:ha - magnum-hacluster:ha +- add relation designate-mysql-router:db-router - mysql-innodb-cluster:db-router +- add relation designate-mysql-router:shared-db - designate:shared-db +- add relation designate:identity-service - keystone:identity-service +- add relation designate:amqp - rabbitmq-server:amqp +- add relation designate:certificates - vault:certificates +- add relation designate:coordinator-memcached - memcached:cache +- add relation designate:dns-backend - designate-bind:dns-backend +- add relation designate:ha - designate-hacluster:ha +- add unit ceph-osd/0 to new machine 5 +- add unit ceph-osd/1 to new machine 6 +- add unit ceph-osd/2 to new machine 7 +- add unit ceph-osd/3 to new machine 8 +- add unit nova-compute/0 to new machine 3 +- add unit nova-compute/1 to new machine 4 +- add lxd container 0/lxd/0 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,provider-public arch=amd64" +- add lxd container 0/lxd/1 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,replication,storage arch=amd64" +- add lxd container 1/lxd/0 on new machine 1 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,replication,storage arch=amd64" +- add lxd container 2/lxd/0 on new machine 2 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,replication,storage arch=amd64" +- add lxd container 0/lxd/2 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,provider-public,storage arch=amd64" +- add lxd container 0/lxd/3 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,replication arch=amd64" +- add lxd container 0/lxd/4 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,provider-public,storage arch=amd64" +- add lxd container 0/lxd/5 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,provider-public arch=amd64" +- add lxd container 2/lxd/1 on new machine 2 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal arch=amd64" +- add lxd container 0/lxd/6 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,provider-public,storage arch=amd64" +- add lxd container 1/lxd/1 on new machine 1 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal arch=amd64" +- add lxd container 0/lxd/7 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,provider-public arch=amd64" +- add lxd container 0/lxd/8 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,provider-public arch=amd64" +- add lxd container 0/lxd/9 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal arch=amd64" +- add lxd container 0/lxd/10 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal arch=amd64" +- add lxd container 1/lxd/2 on new machine 1 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal arch=amd64" +- add lxd container 2/lxd/2 on new machine 2 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal arch=amd64" +- add lxd container 0/lxd/11 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,provider-public arch=amd64" +- add lxd container 0/lxd/12 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,provider-public arch=amd64" +- add lxd container 0/lxd/13 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=data-tenant,metal-admin,metal-internal,provider-public arch=amd64" +- add lxd container 0/lxd/14 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,provider-public arch=amd64" +- add lxd container 0/lxd/15 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal arch=amd64" +- add lxd container 1/lxd/3 on new machine 1 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal arch=amd64" +- add lxd container 2/lxd/3 on new machine 2 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal arch=amd64" +- add lxd container 0/lxd/16 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal,provider-public arch=amd64" +- add lxd container 0/lxd/17 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal arch=amd64" +- add lxd container 0/lxd/18 on new machine 0 +04:31:28 INFO cmd bundlehandler.go:1035 with base "ubuntu@22.04/stable", with constraints "spaces=metal-admin,metal-internal arch=amd64" +- add unit barbican/0 to 0/lxd/0 +- add unit ceph-mon/0 to 0/lxd/1 +- add unit ceph-mon/1 to 1/lxd/0 +- add unit ceph-mon/2 to 2/lxd/0 +- add unit ceph-radosgw/0 to 0/lxd/2 +- add unit ceph-rbd-mirror/0 to 0/lxd/3 +- add unit cinder/0 to 0/lxd/4 +- add unit designate/0 to 0/lxd/5 +- add unit designate-bind/0 to 2/lxd/1 +- add unit glance/0 to 0/lxd/6 +- add unit glance-simplestreams-sync/0 to 1/lxd/1 +- add unit keystone/0 to 0/lxd/7 +- add unit magnum/0 to 0/lxd/8 +- add unit memcached/0 to 0/lxd/9 +- add unit mysql-innodb-cluster/0 to 0/lxd/10 +- add unit mysql-innodb-cluster/1 to 1/lxd/2 +- add unit mysql-innodb-cluster/2 to 2/lxd/2 +- add unit neutron-api/0 to 0/lxd/11 +- add unit nova-cloud-controller/0 to 0/lxd/12 +- add unit octavia/0 to 0/lxd/13 +- add unit openstack-dashboard/0 to 0/lxd/14 +- add unit ovn-central/0 to 0/lxd/15 +- add unit ovn-central/1 to 1/lxd/3 +- add unit ovn-central/2 to 2/lxd/3 +- add unit placement/0 to 0/lxd/16 +- add unit rabbitmq-server/0 to 0/lxd/17 +- add unit vault/0 to 0/lxd/18 +04:31:28 DEBUG juju.api monitor.go:35 RPC connection died +04:31:28 DEBUG juju.rpc server.go:356 error closing codec: write tcp 10.12.8.2:43254->10.12.8.5:17070: write: broken pipe +04:31:28 DEBUG juju.api monitor.go:35 RPC connection died +04:31:28 DEBUG juju.rpc server.go:356 error closing codec: write tcp 10.12.8.2:43258->10.12.8.5:17070: write: broken pipe +04:31:28 INFO cmd supercommand.go:578 command finished