# Phase 02 -- Vault Bring-up (PKI root; secret-handling)

Initialize, unseal, and authorize Vault -- the cloud's PKI/CA root. This is the
SECRET-HANDLING phase: every step is DISCRETE and individually gated (never
batched), secrets go through hidden prompts (never on argv / in a var / in
scrollback / in a juju action log), and the init key material is saved OFF-HOST
immediately.

Decisions: vault-on-mysql backend (etcd/easyrsa dropped -- C1). Troubleshooting:
appendix-A -- DOCFIX-006 (init one-shot capture), DOCFIX-011 (authorize-charm token),
DOCFIX-014 (generate-root-ca required), L4 (unseal via hidden prompt), R3 (HA Enabled
false is correct here).

!!! IRREVERSIBLE ONE-SHOT -- `vault operator init` runs EXACTLY ONCE per vault. It
    prints the 5 unseal-key shares and the root token ONE TIME ONLY. A stdout-only
    `>` redirect drops the key block to a file that loses stderr and you can be left
    with NOTHING (DOCFIX-006 / the B15 incident). Run the init command VERBATIM as
    written in 2.1 (`2>&1 | tee`), confirm the gate, and SAVE the captured file
    off-host before doing anything else. Lost shares = unrecoverable vault.

---

## Prerequisites (must be true entering phase-02)
- phase-01 done: bundle deployed; mysql-innodb-cluster ACTIVE (vault's backend --
  it bootstrapped before vault init).
- vault/0 sits BLOCKED needing initialization (a fresh, uninitialized vault).

## Constants and env-literals
- vault loopback: `http://127.0.0.1:8200` (on the unit; NOT a VIP -- this is B14's
  on-unit loopback model, deliberately not the jumphost-CLI + unit-IP path).
- key-shares=5, key-threshold=3.

## Run-location legend
- `# RUN: on vault/0` -- inside an interactive `juju ssh -m openstack vault/0` session
  (the init/unseal/token-mint steps need a tty for hidden prompts -- do NOT pipe them).
- `# RUN: jumphost` -- `juju run` client calls (authorize / generate-root-ca / status).

---

## Step 2.1 -- Vault init  [IRREVERSIBLE ONE-SHOT -- run verbatim]  DISCRETE
`# RUN: on vault/0`  Open the session, set the loopback addr, pre-check fresh, then
init with the `2>&1 | tee` capture (NOT `>`). Save `~/vault-init/init.txt` off-host
the moment the gate passes.
```bash
# RUN: jumphost -- open the interactive session ONLY (paste this line alone; DOCFIX-029)
juju ssh -m openstack vault/0
```
WAIT for the remote prompt (`ubuntu@juju-...`) before pasting the next block -- a combined
paste buffers the in-session lines and feeds them to the session on connect.
```bash
# --- inside the vault/0 session: ---
export VAULT_ADDR=http://127.0.0.1:8200 ; umask 077 ; mkdir -p ~/vault-init
vault status 2>&1 | grep -E 'Initialized|Sealed|Storage Type|HA Enabled' || true   # pre-check: Initialized false (fresh)
vault operator init -key-shares=5 -key-threshold=3 2>&1 | tee ~/vault-init/init.txt # DOCFIX-006: 2>&1|tee, NEVER '>'
grep -c '^Unseal Key' ~/vault-init/init.txt                                         # GATE: MUST print 5
grep -q '^Initial Root Token:' ~/vault-init/init.txt && echo TOKEN_OK || echo MISSING
```
GATE: `5` unseal keys AND `TOKEN_OK`. If the count is not 5 or the token is MISSING,
STOP -- do not proceed (the empty-file case is the DOCFIX-006 catch). Now SAVE the 5
shares + root token off-host (operator secret store) before continuing. Do NOT batch
this with unseal.

## Step 2.2 -- Vault unseal (3 of 5)  DISCRETE  (re-runnable)
`# RUN: on vault/0`  Use Vault's OWN hidden prompt -- the key is never on the command
line, in a var, or in scrollback (appendix-A: L4). Do NOT use `vault operator unseal $K`
(that puts the key in `ps`/argv).
```bash
# --- inside the vault/0 session: ---
export VAULT_ADDR=http://127.0.0.1:8200
vault operator unseal      # prompts hidden; paste share 1   -> Unseal Progress 1/3
vault operator unseal      # prompts hidden; paste share 2   -> 2/3
vault operator unseal      # prompts hidden; paste share 3   -> 3/3
vault status 2>&1 | grep -E 'Sealed|Initialized|Storage Type|HA Enabled'
```
GATE: progress 1/3 -> 2/3 -> 3/3, then `Sealed false`. Expected final: Initialized
true / Sealed false / Storage Type mysql / **HA Enabled false** (CORRECT for
single-unit vault-on-mysql -- appendix-A: R3; any "HA true / etcd" reference is stale).

NOTE (unseal policy, v1): MANUAL unseal is the v1 standard -- after any vault unit
reboot, re-run this 3-of-5 step at the hidden prompt. Auto-unseal (e.g. a transit/KMS
seal so the unit returns unsealed after a reboot) is an available option, adopted
case-by-case; it is NOT configured in v1. D-011.6 (phase-08) re-confirms manual unseal.

## Step 2.3 -- Authorize-charm + generate-root-ca  DISCRETE
First confirm the action schema (DOCFIX-011), then authorize with a SHORT-LIVED CHILD
token (not the root token -- `juju run` persists action params in the operation log,
so a minutes-lived token self-limits), then generate the root CA (DOCFIX-014 -- without
it vault stays blocked "Missing CA cert").
```bash
# RUN: jumphost -- schema (read-only): authorize-charm requires `token` (direct-token path)
juju actions vault --schema --format yaml -m openstack | sed -n '/authorize-charm:/,/^[a-z]/p'
```
```bash
# RUN: jumphost -- open the interactive session ONLY (paste this line alone; DOCFIX-029)
juju ssh -m openstack vault/0
```
WAIT for the remote prompt (`ubuntu@juju-...`). This in-session block contains a hidden
`read -s` -- a combined paste would let read swallow the next buffered line as the secret.
NO trailing `exit`: exit MANUALLY after copying the child token (a paste-ahead `exit` could
self-terminate the session and mask the swallow).
```bash
# --- inside the session: mint a short-lived child token (root entered hidden, never on argv/history) ---
export VAULT_ADDR=http://127.0.0.1:8200
read -s -p "root token: " VAULT_TOKEN; echo ; export VAULT_TOKEN
vault token create -ttl=10m -field=token        # prints ONLY the child token -- copy it
unset VAULT_TOKEN
# (exit manually after you have copied the child token)
```
```bash
# RUN: jumphost -- authorize + root CA + status (each juju run blocks to completion)
# ENHANCEMENT-2: enter the child token via hidden read (keeps it out of jumphost shell
# history). The token still transits the Juju operation log (inherent to the action;
# mitigated by the 10m TTL) -- this narrows exposure, it does not eliminate it.
read -s -p "child token: " TOK; echo
juju run vault/leader authorize-charm token="$TOK" -m openstack
unset TOK
juju run vault/leader generate-root-ca -m openstack
juju status vault -m openstack
```
GATE: authorize-charm completes; generate-root-ca returns the root CA PEM ("Vault Root
Certificate Authority (charm-pki-local)"); vault/0 -> active/idle "Unit is ready". The
"Missing CA cert" block clears straight to active (validates DOCFIX-014).
(`mlock: disabled` is expected/benign for snap/container vault without IPC_LOCK.)

---

## EXIT GATE (phase-02 complete)
- Vault Initialized true / Sealed false; 5 shares + root token saved OFF-HOST.
- vault/0 active/idle; root CA generated (the cloud's PKI anchor).
- The narrow cert cascade to the Vault consumers (ovn-central x3, ovn-chassis x3,
  ovn-chassis-octavia, neutron-api-plugin-ovn, barbican-vault) now proceeds -- it is
  watched and accepted in phase-03.
- POST-INIT SWEEP (FINDING-2 / DOCFIX-028 cross-check) -- after the cert cascade settles:
  * magnum/0 -> active "Unit is ready"; magnum-api is now served by apache2 on *:9501 (all
    interfaces; haproxy backends reachable; [api] port moved to the wsgi backend). The
    phase-01 pre-vault 9501 BLOCK was the expected loopback-bound posture and self-resolves
    here at the TLS cutover (confirmed 2026-06-12). If it is STILL loopback-bound after certs
    settle, escalate to charm diagnosis BEFORE phase-03 (then the phase-01 line is a defect).
  * keystone/0 PO state UNCHANGED ("PO (broken): Unit is ready") -- still default policy
    (FINDING-1: use-policyd-override=true with no zip). Not a regression; no mutation.

## As-built reference (2026-06-03 run -- audit trail)
- init: 5 shares / threshold 3, "Vault initialized with 5 key shares and a key
  threshold of 3"; captured via `2>&1 | tee ~/vault-init/init.txt`.
- unseal: 1/3 -> 2/3 -> 3/3 -> Sealed false; Storage Type mysql; HA Enabled false;
  Version 1.8.8, vault-cluster-872a43d1.
- authorize: op 1/task 2 OK (short-lived child token); generate-root-ca op 3/task 4
  returned the root CA (valid 2026-06-03 -> 2036-05-31); vault/0 + vault-mysql-router active.

## Next
phase-03 -- core verify (cert-cascade settle, admin-openrc, Horizon).