Newer
Older
openstack-caracal-ipv4 / clientdocs / tenant-skill / references / kubernetes.md

Kubernetes clusters on Omega Cloud

The one non-negotiable: the -cluster account, with its PASSWORD

Cluster create and delete run ONLY as {{TENANT_SHORT_NAME}}-cluster, signed in with its password. The platform's cluster machinery cannot operate through an application credential -- a cluster create attempted with the -svc credential fails every time, no matter how it is phrased. This is a platform constraint, not a configuration you (or the operator) can change. Do not "simplify" cluster automation to the application credential; it will never work.

Sign in (prompt the human for the password -- never read it from a file into your context, never store it in CI):

export OS_AUTH_TYPE=password
export OS_AUTH_URL={{AUTH_URL}}
export OS_IDENTITY_API_VERSION=3
export OS_CACERT=<path to the delivered CA bundle>
export OS_USERNAME={{TENANT_SHORT_NAME}}-cluster
export OS_USER_DOMAIN_NAME={{TENANT_SHORT_NAME}}
export OS_PROJECT_NAME={{TENANT_SHORT_NAME}}-prod
export OS_PROJECT_DOMAIN_NAME={{TENANT_SHORT_NAME}}
read -rs -p "password for {{TENANT_SHORT_NAME}}-cluster: " OS_PASSWORD && export OS_PASSWORD
openstack token issue        # smoke test

The keypair-ownership trap

The SSH keypair {{TENANT_SHORT_NAME}}-key is OWNED by the -cluster account, and the platform validates keypair ownership in the cluster creator's context. Consequences:

  • Never delete {{TENANT_SHORT_NAME}}-key or recreate it under another account. A key with the same name owned by -svc or a team user will make every cluster create fail.
  • "Keypair not found" at cluster create almost always means you are signed in as the wrong account -- check OS_USERNAME before anything else.

Cluster lifecycle

Your cluster template was delivered at handover:

openstack coe cluster template show {{TENANT_SHORT_NAME}}-k8s

Create (node counts are limited by your quota; builds take TENS OF MINUTES -- poll slowly or just wait):

openstack coe cluster create <cluster-name> \
  --cluster-template {{TENANT_SHORT_NAME}}-k8s \
  --keypair {{TENANT_SHORT_NAME}}-key \
  --master-count 1 --node-count <N>

openstack coe cluster show <cluster-name>   # poll >=10s apart
# done when status = CREATE_COMPLETE

Fetch the kubeconfig (treat it as a credential -- never print its contents; store it like any other secret):

openstack coe cluster config <cluster-name> --dir <secure-dir>
export KUBECONFIG=<secure-dir>/config
kubectl get nodes

Delete (confirm with the human first, by name):

openstack coe cluster delete <cluster-name>

Deploying workloads: Services of type LoadBalancer

The cluster gets its own load balancer for the Kubernetes API, and it can publish Service objects of type: LoadBalancer through the same platform mechanism -- no extra setup needed:

kubectl expose deployment <app> --type=LoadBalancer --port=80

The Service's external address is allocated from the platform. Each such Service consumes a load balancer (and typically a floating IP) from YOUR quota -- prefer a single ingress controller of type LoadBalancer fronting many Services over one LoadBalancer per app.

  • Create long-lived clusters manually (or on a controlled schedule) as -cluster, OUTSIDE CI.
  • Pipelines then deploy INTO the cluster using its kubeconfig, stored in the CI secret store like any other credential.
  • Do NOT put the -cluster password into CI so pipelines can create clusters. If your workflow genuinely needs cluster-per-run, raise it with {{ACCOUNT_CONTACT}} first -- there are capacity and quota implications.

Day-2 cloud work around the cluster (networks, floating IPs, volumes, extra load balancers) still runs as -svc per references/day2-operations.md; only cluster create/delete needs the -cluster password login.