Building an IAM Platform on Keycloak 25 in Kubernetes

Keycloak is the default answer when you need OIDC and SAML, a user federation layer, and an admin surface that other teams can operate — without writing an authorization server yourself. It joined the CNCF as an incubating project in 2023, and the 25.0 release (June 2024) is the cleanest it has ever been to run on Kubernetes: the WildFly distribution is gone, health and metrics moved off the public port, and hostname configuration was rebuilt.

This post is the deployment blueprint I wish existed — what each moving part is for, and the exact configuration that makes a Keycloak cluster behave correctly behind an ingress. Everything here targets Keycloak 25.0.x and the matching Keycloak Operator.

The distribution: build vs. start

Since Keycloak 17 the server runs on Quarkus, and this reshapes deployment more than anything else. Configuration is split into two phases:

kc.sh build — augmentation. Bakes in the choices that can’t change at runtime without a rebuild: the database vendor, enabled features, the metrics/health toggles, cache type, transaction handling. Produces an optimized, closed-world server image.
kc.sh start — boot. Applies runtime configuration: hostnames, credentials, connection URLs, replica count.

In containers you do the build step once, at image build time, and then start with --optimized so the pod skips re-augmentation on every restart. Never rely on the default image’s implicit build — you pay a multi-second penalty on every cold start and lose reproducibility.

# Containerfile — bake an optimized image
FROM quay.io/keycloak/keycloak:25.0.1 AS builder

ENV KC_DB=postgres
ENV KC_HEALTH_ENABLED=true
ENV KC_METRICS_ENABLED=true
ENV KC_CACHE=ispn
ENV KC_FEATURES=token-exchange,admin-fine-grained-authz

RUN /opt/keycloak/bin/kc.sh build

FROM quay.io/keycloak/keycloak:25.0.1
COPY --from=builder /opt/keycloak/ /opt/keycloak/
ENTRYPOINT ["/opt/keycloak/bin/kc.sh", "start", "--optimized"]

Every option has three equivalent spellings — CLI flag --db, env var KC_DB, and a key in conf/keycloak.conf. In Kubernetes, env vars win: they map cleanly onto ConfigMap/Secret and onto the Operator’s additionalOptions. The full option reference is the All configuration guide.

Never ship start-dev. It enables the in-memory H2 database, a local cache with no clustering, disables hostname strictness, and turns on the caching of nothing. It exists for a laptop, not a cluster.

Deployment path: the Operator

You can deploy the raw StatefulSet by hand, but the Keycloak Operator earns its keep: it owns the StatefulSet, the Services (including the headless service used for clustering), the management-port probes, and realm imports, and it reconciles them from two CRDs — Keycloak and KeycloakRealmImport.

Install the CRDs and the Operator, pinned to the version that matches your server:

VERSION=25.0.1
BASE=https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/$VERSION/kubernetes

kubectl apply -f $BASE/keycloaks.k8s.keycloak.org-v1.yml
kubectl apply -f $BASE/keycloakrealmimports.k8s.keycloak.org-v1.yml
kubectl apply -f $BASE/kubernetes.yml

The manifests live in keycloak/keycloak-k8s-resources. A minimal but production-shaped Keycloak resource:

apiVersion: k8s.keycloak.org/v2alpha1
kind: Keycloak
metadata:
  name: iam
  namespace: iam
spec:
  instances: 3                       # HA — see clustering below
  image: registry.example.com/iam/keycloak:25.0.1   # your optimized image
  startOptimized: true               # matches the baked image
  db:
    vendor: postgres
    host: keycloak-db.iam.svc
    port: 5432
    database: keycloak
    usernameSecret:
      name: keycloak-db
      key: username
    passwordSecret:
      name: keycloak-db
      key: password
    poolMinSize: 5
    poolMaxSize: 20
  hostname:
    hostname: https://auth.example.com
  http:
    tlsSecret: keycloak-tls          # omit to terminate TLS at the ingress
  proxy:
    headers: xforwarded              # trust X-Forwarded-* from the ingress
  resources:
    requests: { cpu: "1", memory: 1300Mi }
    limits:   { memory: 2Gi }
  additionalOptions:
    - name: log-console-output
      value: json

spec.db maps onto the KC_DB_* options; anything the CR doesn’t model explicitly goes under additionalOptions (rendered to KC_* env). Because vendor: postgres was already baked at build time, startOptimized: true is safe — a mismatch here is the single most common “optimized image augmentation” failure.

State: Postgres is the source of truth

Keycloak’s durable state — realms, clients, users, credentials, offline sessions, events — lives in a relational database. Run Postgres externally (a managed service, or an operator like CloudNativePG); do not co-locate it in the Keycloak pod. The database is your backup, migration, and disaster-recovery unit. On upgrade, Keycloak runs Liquibase migrations against it automatically at boot — take a snapshot first, and never start two different Keycloak versions against one schema concurrently.

Two sizing notes that bite people:

The JDBC pool (db-pool-max-size, default 100 per instance) multiplies by replica count against Postgres’s max_connections. Three replicas at 100 is 300 connections — size Postgres accordingly or lower the pool as above.
Keycloak 25 makes Argon2 the default password hashing algorithm on JDK 17+. Argon2 is memory-hard; under a login storm it competes with the JVM heap for RAM. Budget memory for it, and keep hashIterations at the shipped default unless you have measured headroom.

Clustering: Infinispan over DNS_PING

Keycloak keeps an embedded Infinispan cache for data that is either hot or intentionally not persisted every request: realm/client metadata (local, replicated per node) and authentication sessions, login failures, and action tokens (distributed, owner count 2). For instances > 1 these caches must form a single cluster, or a request that lands on a different pod mid-login fails.

Clustering is JGroups discovery. In Kubernetes the right stack is kubernetes, which uses DNS_PING against a headless service to find peers — the Operator creates that service (<name>-discovery) and wires it for you. If you deploy by hand, set it explicitly:

KC_CACHE=ispn
KC_CACHE_STACK=kubernetes
# JGroups resolves SRV/A records from the headless service to find peers
JAVA_OPTS_APPEND=-Djgroups.dns.query=iam-discovery.iam.svc.cluster.local

Verify the cluster actually formed — split-brain is silent until a user hits it. Look for the JGroups view in the logs:

ISPN000094: Received new cluster view for channel ISPN:
  [iam-0|2] (3) [iam-0, iam-1, iam-2]

Three members, one view. If you see three separate single-member views, DNS_PING isn’t resolving — check the headless service, the publishNotReadyAddresses: true setting on it, and that pods share a KUBE_PING/DNS namespace.

For multi-datacenter or active/active across regions, embedded Infinispan is the wrong tool — cross-site replication moved to an external Infinispan cluster, documented under Keycloak’s multi-site / high availability guide. Persistent user sessions (sessions written through to the database rather than held only in-memory) landed in 25 as a preview feature, --features=persistent-user-sessions; it’s worth watching but not yet the default.

The management port: health and metrics

The most consequential operational change in 25.0: /health and /metrics moved to a separate management port, 9000, and it’s on by default. They are no longer reachable on the public 8443/8080 listener. This is exactly what you want in Kubernetes — probes and scrape targets never sit on the same port as user traffic, so you can firewall 9000 to the cluster internal network.

Wire probes to 9000, not 8080:

readinessProbe:
  httpGet: { path: /health/ready, port: 9000 }
livenessProbe:
  httpGet: { path: /health/live,  port: 9000 }
startupProbe:
  httpGet: { path: /health/started, port: 9000 }
  failureThreshold: 30
  periodSeconds: 5

The Operator sets these automatically. Metrics are Micrometer/Prometheus format at :9000/metrics — point a ServiceMonitor at the management port, and remember KC_HEALTH_ENABLED/KC_METRICS_ENABLED are build-time options, so they must be baked into the image (they are, in the Containerfile above).

Edge: hostname v2 and proxy correctness

Two settings cause the majority of “it works in the pod but issues broken tokens” incidents. Both are about Keycloak knowing its own public identity.

Hostname v2, new in 25 and now the default, replaced a pile of granular flags with a single URL. The value Keycloak puts in the OIDC discovery document, the issuer claim, and every redirect comes from here:

KC_HOSTNAME=https://auth.example.com
# separate admin console origin, optional:
KC_HOSTNAME_ADMIN=https://auth-admin.internal.example.com

If this is wrong, .well-known/openid-configuration advertises the wrong endpoints and clients fail token validation against the issuer.

Proxy headers. Keycloak almost always sits behind an ingress that terminates TLS. It must trust the forwarded headers to reconstruct the original scheme and host — otherwise it thinks it’s serving plain HTTP and emits http:// URLs. In 25 the old --proxy=edge shorthand is deprecated in favor of an explicit --proxy-headers:

KC_PROXY_HEADERS=xforwarded        # or 'forwarded' (RFC 7239)

Only enable this when a trusted proxy is actually in front of Keycloak. X-Forwarded-* headers are client-spoofable; trusting them on a directly exposed server lets a caller lie about the origin host. Terminate TLS and strip/set these headers at one controlled ingress hop.

Bootstrapping realms as code

Click-ops in the admin console doesn’t survive a cluster rebuild. Manage realms declaratively with KeycloakRealmImport, which the Operator applies on top of a running instance:

apiVersion: k8s.keycloak.org/v2alpha1
kind: KeycloakRealmImport
metadata:
  name: platform-realm
  namespace: iam
spec:
  keycloakCRName: iam
  realm:
    realm: platform
    enabled: true
    # full realm representation (same schema as an admin-console export)
    registrationAllowed: false
    loginWithEmailAllowed: true

Import is additive and idempotent for realm creation; it does not continuously reconcile every nested resource, so treat it as bootstrap. For ongoing client/role/user drift management, drive the Admin REST API from a GitOps job, or use a config-management tool such as keycloak-config-cli. Keep secrets (client secrets, SMTP credentials) out of the CR and inject them by reference.

A deployment checklist

Concretely, a sound Keycloak 25 platform on Kubernetes has:

An optimized custom image — kc.sh build at image time, start --optimized at boot, DB vendor and health/metrics baked in.
External Postgres as the single source of truth, with a snapshot taken before every version upgrade.
instances >= 3 with Infinispan clustered over DNS_PING, and the JGroups view verified in the logs.
Probes and scraping on port 9000, firewalled off from public traffic.
KC_HOSTNAME set to the exact public URL and KC_PROXY_HEADERS set only because a trusted ingress terminates TLS.
Realms as code via KeycloakRealmImport for bootstrap plus a REST/GitOps loop for drift.

None of this is exotic — it’s the difference between a Keycloak that survives a node drain and one that logs users out (or hands them broken tokens) the first time a pod moves.

Upstream: keycloak/keycloak · Server & Operator guides · keycloak-k8s-resources · 25.0 release notes

The distribution: build vs. start#

Deployment path: the Operator#

State: Postgres is the source of truth#

Clustering: Infinispan over DNS_PING#

The management port: health and metrics#

Edge: hostname v2 and proxy correctness#

Bootstrapping realms as code#

A deployment checklist#