ClickHouse Operator configuration guide - ClickHouse Documentation

This guide covers how to configure ClickHouse and Keeper clusters using the operator.

ClickHouseCluster configuration

Basic configuration

apiVersion: clickhouse.com/v1alpha1
kind: ClickHouseCluster
metadata:
  name: my-cluster
spec:
  replicas: 3           # Number of replicas per shard
  shards: 2             # Number of shards
  keeperClusterRef:
    name: my-keeper     # Reference to KeeperCluster
  dataVolumeClaimSpec:
    resources:
      requests:
        storage: 10Gi

Replicas and shards

Replicas: Number of ClickHouse instances per shard (for high availability)
Shards: Number of horizontal partitions (for scaling)

spec:
  replicas: 3  # Default: 3
  shards: 2    # Default: 1

A cluster with replicas: 3 and shards: 2 will create 6 ClickHouse pods total.

Keeper integration

Every ClickHouse cluster needs a Keeper ensemble for coordination. Set exactly one of keeperClusterRef or externalKeeper; a cluster that sets both or neither is rejected. Point keeperClusterRef at a KeeperCluster this operator manages:

spec:
  keeperClusterRef:
    name: my-keeper
    # namespace: keeper-system  # Optional, defaults to the ClickHouseCluster namespace

When keeperClusterRef.namespace is set, the operator must watch both namespaces. If WATCH_NAMESPACE is configured, include the ClickHouse and Keeper namespaces in that list.

Externally managed Keeper

Use externalKeeper to run ClickHouse against a Keeper ensemble the operator does not manage — one owned by another operator, running in a different Kubernetes cluster, on virtual machines or bare metal, or shared between several systems:

spec:
  externalKeeper:
    nodes:
      - host: keeper-1.example.internal
        port: 9181
      - host: keeper-2.example.internal
        port: 9181
      - host: keeper-3.example.internal
        port: 9181

port defaults to 9181. host is a DNS name or an IP address; enclose IPv6 addresses in square brackets. Set tls: Enabled to connect over TLS, and give port the value the ensemble listens on for secure connections:

spec:
  externalKeeper:
    tls: Enabled
    nodes:
      - host: keeper-1.example.internal
        port: 9281

ClickHouse verifies the Keeper certificates against the system trust store, plus any caBundle you configure. See ClickHouse-Keeper communication over TLS. When several ClickHouse clusters share one ensemble, give each its own root znode through settings.extraConfig, so their replication metadata, SQL-defined users and named collections don’t collide. Create the znode before ClickHouse starts, and don’t change it afterwards:

spec:
  settings:
    extraConfig:
      zookeeper:
        root: /clickhouse/production

Don’t put zookeeper.root in settings.extraReloadableConfig: a reload would switch the root of a running cluster and leave its replicated tables read-only. If the root znode has ACLs, they must let the cluster’s keeper-identity create child znodes. The operator does not change these servers, and it does not wait for them before it reconciles ClickHouse. Keeping the ensemble available is up to the user. Authentication works the same way as for an operator-managed Keeper: the operator reads the keeper-identity key from the cluster Secret and passes it to ClickHouse. See External Secret for the Secret layout.

KeeperCluster configuration

apiVersion: clickhouse.com/v1alpha1
kind: KeeperCluster
metadata:
  name: my-keeper
spec:
  replicas: 3  # Must be odd: 1, 3, 5, 7, 9, 11, 13, or 15
  dataVolumeClaimSpec:
    resources:
      requests:
        storage: 5Gi

Leadership handover on termination

The Keeper container has a default preStop hook that asks the replica to hand off Raft leadership before it stops, and waits for the handover to complete before letting the container terminate. This runs on every graceful stop of the container, not just operator-initiated rollouts: a descheduler eviction, a node drain, or an autoscaler action all trigger the same hook, since it is a kubelet-level preStop handler rather than reconcile-loop logic. Without this, restarting the current leader leaves the ensemble without a leader until a new election completes, which can take several seconds under load and shows up to ClickHouse as Keeper connection loss and elevated query latency. The hook asks the replica to yield leadership (Keeper’s ydld four-letter-word command, sent over the http_control port) and then polls the replica’s status until it is no longer the leader, rather than firing the request and immediately letting the container stop — a fire-and-forget request can still lose the race with SIGTERM under write load, so waiting is what keeps the leaderless window bounded. The hook needs no configuration and degrades gracefully: on Keeper versions older than 26.3, which don’t yet serve the leadership commands used here, the request is a no-op and the container stops as it would without the hook. It runs a shell script, so it requires bash in the image; if you run a distroless or scratch Keeper image, override or remove it via containerTemplate.lifecycle — see API Reference.

Storage configuration

Configure persistent storage with dataVolumeClaimSpec, a standard Kubernetes PersistentVolumeClaimSpec. The operator turns it into a per-replica PersistentVolumeClaim mounted at the data path /var/lib/clickhouse:

spec:
  dataVolumeClaimSpec:
    storageClassName: fast-ssd  # Optional: consider your storage class based on the installed CSI
    resources:
      requests:
        storage: 100Gi

The operator can modify an existing PVC only if the underlying StorageClass supports volume expansion.

Attaching extra disks in a multi-disk (JBOD) layout, running without a persistent volume, expanding capacity, custom storage policies, at-rest encryption, and the rules for what cannot change after creation are covered in the dedicated Storage and volumes guide.

Cluster domain

spec.clusterDomain sets the Kubernetes DNS suffix the operator uses when it builds the fully-qualified pod host names it writes into the ClickHouse server configuration. It defaults to cluster.local and exists on both ClickHouseCluster and KeeperCluster.

spec:
  clusterDomain: cluster.local   # default; override only for a custom domain

For ClickHouse replicas the operator creates a governing headless Service <cluster-name>-clickhouse-headless that serves client traffic. And per-replica services <cluster-name>-clickhouse-internal-<shard>-<index> that publish unready Pods for internal traffic and management requests from the operator. This can be used for replica recovery if it cannot become ready itself. Keeper nodes use pod names through their headless Service: <pod>.<headless-service>.<namespace>.svc.<clusterDomain>.

Only override this when your cluster’s kubelet runs with a --cluster-domain other than cluster.local. If the value does not match the real cluster domain, ClickHouse cannot resolve the Keeper and replica host names — coordination and Distributed queries fail with DNS resolution errors. Set the same value on the ClickHouseCluster and the KeeperCluster it references.

Pod configuration

Automatic topology spread and affinity

Distribute pods across availability zones:

spec:
  podTemplate:
    topologyZoneKey: topology.kubernetes.io/zone
    nodeHostnameKey: kubernetes.io/hostname

Ensure your Kubernetes cluster has enough nodes in different zones to satisfy the spread constraints.

With a node autoscaler (Karpenter, Cluster Autoscaler) the spread constraint alone cannot provision a zone that has no nodes yet — the scheduler settles for the zones it can see. Set topologyMinDomains to the number of availability zones to force spreading:

spec:
  podTemplate:
    topologyZoneKey: topology.kubernetes.io/zone
    topologyMinDomains: 3   # number of availability zones

topologyMinDomains requires topologyZoneKey. Pods stay Pending while fewer zones than requested are available, so keep it within the zones your cluster can provide.

Manual configuration

Arbitrary pod affinity/anti-affinity rules and topology spread constraints can be specified.

spec:
  podTemplate:
    affinity:
      <your-affinity-rules-here>
    topologySpreadConstraints:
      <your-topology-spread-constraints-here>

See API Reference for all supported Pod template options.

Pod disruption budgets

The operator creates a PodDisruptionBudget (PDB) for each cluster so that voluntary disruptions — node drains, rolling upgrades, autoscaler evictions — cannot take down enough pods to lose quorum or break availability. For ClickHouse clusters with more than one shard, one PDB is created per shard so a disruption in one shard cannot count against another.

Defaults

The operator picks safe defaults based on the cluster size so that a fresh apply already protects against accidental quorum loss.

Resource	Topology	Default PDB
`ClickHouseCluster`	`replicas: 1` (single-replica shard)	`maxUnavailable: 1` — disruption is allowed for a single-node cluster so that node drains are not blocked
`ClickHouseCluster`	`replicas: 2+` (multi-replica shard)	`minAvailable: 1` — at least one replica per shard must stay up
`KeeperCluster`	`replicas: 1`	`maxUnavailable: 1` — disruption is allowed for a single-node cluster so that node drains are not blocked
`KeeperCluster`	`replicas: 3+`	`maxUnavailable: replicas/2` — preserves the RAFT quorum for a `2F+1` cluster (3 replicas tolerate 1 down, 5 replicas tolerate 2 down)

For a 3-shard ClickHouseCluster with replicas: 3, the operator creates three PDBs, one per shard, each with minAvailable: 1.

Overriding the defaults

Use spec.podDisruptionBudget to override either minAvailable or maxUnavailable (exactly one):

spec:
  replicas: 3
  shards: 2
  podDisruptionBudget:
    minAvailable: 2   # keep at least 2 of 3 replicas in every shard up during a disruption

Or the maxUnavailable form, with a percentage:

spec:
  replicas: 5
  podDisruptionBudget:
    maxUnavailable: 40%

Setting both minAvailable and maxUnavailable is rejected by the validating webhook. Pick one — Kubernetes itself does not allow both either.

You can also pass the unhealthyPodEvictionPolicy field through to the generated PDB — useful when you need to allow eviction of pods that are still in NotReady:

spec:
  podDisruptionBudget:
    minAvailable: 2
    unhealthyPodEvictionPolicy: AlwaysAllow

Policies

spec.podDisruptionBudget.policy lets you choose how aggressively the operator manages PDBs:

Policy	Behavior
`Enabled` (default)	The operator creates and updates the PDB on every reconcile. This is the safe production default.
`Disabled`	The operator does not create PDBs and deletes any existing ones with matching labels. Useful for development clusters where every voluntary disruption should be allowed.
`Ignored`	The operator neither creates nor deletes PDBs. Existing PDBs are left alone. Use this when another system (e.g. policy admission, GitOps tool) owns PDB management for you.

Example — disable PDB management completely on a development cluster:

spec:
  podDisruptionBudget:
    policy: Disabled

Example — keep your hand-crafted PDB next to the cluster and stop the operator from touching it:

spec:
  podDisruptionBudget:
    policy: Ignored

Cluster-wide opt-out

PDB management can also be disabled cluster-wide via the operator’s ENABLE_PDB environment variable. With ENABLE_PDB=false, the operator skips the PDB reconcile step for every ClickHouseCluster and KeeperCluster regardless of their spec.podDisruptionBudget.policy, and does not watch PodDisruptionBudget resources at all. The operator’s ServiceAccount therefore does not need RBAC permissions on poddisruptionbudgets.policy/v1, which is useful when running the operator under a restricted ServiceAccount that intentionally omits those permissions.

# in the operator Deployment spec
env:
- name: ENABLE_PDB
  value: "false"

This is intended for environments that ship their own disruption policies (e.g. through Gatekeeper / Kyverno) and want the operator out of the loop entirely.

Container configuration

Custom image

Use a specific ClickHouse image:

spec:
  containerTemplate:
    image:
      repository: clickhouse/clickhouse-server
      tag: "25.12"
    imagePullPolicy: IfNotPresent

Container resources

Configure CPU and memory for ClickHouse containers:

# default values
spec:
  containerTemplate:
    resources:
      requests:
        cpu: "250m"
        memory: "512Mi"
      limits:
        cpu: "1"
        memory: "512Mi"

Environment variables

Add custom environment variables:

spec:
  containerTemplate:
    env:
    - name: CUSTOM_ENV_VAR
      value: "1"

Volume mounts

Add additional volume mounts:

spec:
  containerTemplate:
    volumeMounts:
    - name: custom-config
      mountPath: /etc/clickhouse-server/config.d/custom.xml
      subPath: custom.xml

It is allowed to specify multiple volume mounts to the same mountPath. Operator will create projected volume with all specified mounts.

Health probes

The operator configures liveness and readiness probes on every ClickHouse and Keeper container. No startup probe is set by default.

Resource	Probe	Default check
`ClickHouseCluster`	liveness	HTTP `GET /ping` on the interserver port `9009`
`ClickHouseCluster`	readiness	HTTP `GET /ping` on the management HTTP port `9002`
`KeeperCluster`	liveness	TCP connect to the client port `2181` (`2281` when both `settings.tls.enabled` and `settings.tls.required` are `true`)
`KeeperCluster`	readiness	HTTP `GET /ready` on the `http_control` port `9123`

ClickHouse and Keeper use the same default timing settings:

Probe	`initialDelaySeconds`	`periodSeconds`	`timeoutSeconds`	`successThreshold`	`failureThreshold`
liveness	60	5	10	1	10
readiness	5	1	10	5	10

ClickHouse opens the interserver listener before it loads tables and the client listeners after, so the liveness probe tolerates slow table loading, while a passing readiness probe means the replica serves queries. Override any probe through spec.containerTemplate on either resource. For example, customize the ClickHouse liveness probe:

spec:
  containerTemplate:
    livenessProbe:
      httpGet:
        path: /ping
        port: 9009
      initialDelaySeconds: 120
      periodSeconds: 10
      timeoutSeconds: 10
      failureThreshold: 10

A probe you set replaces the operator default entirely — it isn’t merged field by field. Specify the handler (httpGet, tcpSocket, or exec) and every timing field you care about; omitted fields fall back to the Kubernetes defaults (for example timeoutSeconds: 1, failureThreshold: 3), not to the operator values above.

Startup probe

Use startupProbe when a container needs more time to start than the liveness probe allows — with the defaults, about 60 seconds of initial delay plus 10 failed checks 5 seconds apart. Kubernetes holds back the liveness and readiness probes until the startup probe succeeds, so a slow start doesn’t trigger a restart loop. For example, give a Keeper replica that replays a large snapshot up to 10 minutes to start accepting connections:

apiVersion: clickhouse.com/v1alpha1
kind: KeeperCluster
metadata:
  name: sample
spec:
  replicas: 3
  containerTemplate:
    startupProbe:
      tcpSocket:
        port: 2181
      periodSeconds: 10
      failureThreshold: 60

After the startup probe succeeds, the regular liveness and readiness probes take over.

See API Reference for all supported Container template options.

TLS/SSL configuration

Configure secure endpoints

Pass a reference to a Kubernetes Secret containing TLS certificates to enable secure endpoints

spec:
  settings:
    tls:
      enabled: true
      required: true # Insecure ports are disabled if set
      serverCertSecret:
        name: <certificate-secret-name>

SSL certificate secret format

It is expected that the Secret contains the server keypair:

tls.crt - PEM encoded server certificate
tls.key - PEM encoded private key

This format is compatible with cert-manager generated certificates.

ClickHouse-Keeper communication over TLS

If the KeeperCluster referenced by keeperClusterRef has TLS enabled, ClickHouseCluster would use secure connection to Keeper nodes automatically. With externalKeeper, set tls: Enabled instead, and set each node’s port to the ensemble’s secure client port (for example 9281). ClickHouseCluster verifies Keeper node certificates against the system trust store, plus any caBundle you configure. To trust a private CA (for example, a self-signed or internal CA), provide a custom CA bundle reference:

spec:
    settings:
        tls:
          caBundle:
            name: <ca-certificate-secret-name>
            key: <ca-certificate-key>

External Secret

By default the operator creates and owns a Secret containing the cluster’s internal credentials (interserver password, management password, keeper identity, cluster secret, named-collections key). The Secret is named after the cluster and lives in the cluster’s namespace. If you want to manage these credentials yourself — for example, sourcing them from HashiCorp Vault, AWS Secrets Manager, or External Secrets Operator — point the operator at a pre-existing Secret using spec.externalSecret:

apiVersion: clickhouse.com/v1alpha1
kind: ClickHouseCluster
metadata:
  name: sample
spec:
  replicas: 2
  keeperClusterRef:
    name: sample
  dataVolumeClaimSpec:
    resources:
      requests:
        storage: 10Gi
  externalSecret:
    name: my-clickhouse-credentials
    policy: Observe

The referenced Secret must reside in the same namespace as the ClickHouseCluster. The operator never deletes a Secret it did not create.

Required keys

The Secret must contain the following keys:

Key	Format	When required
`interserver-password`	plaintext password	Always
`management-password`	plaintext password	Always
`keeper-identity`	`clickhouse:<password>`	Always
`cluster-secret`	plaintext password	Always
`named-collections-key`	hex-encoded 16-byte AES key (32 hex chars)	ClickHouse `>= 25.12` only
`disk-encryption-key`	hex-encoded 16-byte AES key (32 hex chars)	Only when `settings.encryption` is set

A complete Secret looks like this:

apiVersion: v1
kind: Secret
metadata:
  name: my-clickhouse-credentials
  namespace: sample
type: Opaque
stringData:
  interserver-password: "a-strong-random-password"
  management-password: "another-strong-password"
  keeper-identity: "clickhouse:keeper-auth-password"
  cluster-secret: "cluster-internal-secret"
  named-collections-key: "0123456789abcdef0123456789abcdef"   # 32 hex chars = 16 bytes
  disk-encryption-key: "00112233445566778899aabbccddeeff"     # only when settings.encryption is set

Policy: Observe vs Manage

spec.externalSecret.policy controls how the operator handles missing required keys:

Policy	Behavior on missing keys
`Observe` (default)	Reconciliation is blocked until every required key is present. The operator reports each missing key — and the format hint for it — via the `ExternalSecretValid` condition (with reason `ExternalSecretInvalid`) and a `Warning` event.
`Manage`	The operator generates any missing required keys and writes them back to the same Secret. Useful for bootstrapping: create an empty Secret, let the operator fill it, then optionally tighten access. The operator still never deletes the Secret.

Even with policy: Manage the Secret must already exist in the namespace — the operator never creates the Secret itself, it only writes generated keys into an existing one. If the referenced Secret is missing, reconciliation is blocked with the ExternalSecretNotFound reason regardless of policy.

Pick Observe when an external system (Vault, ESO, sealed-secrets, GitOps) is the source of truth and you want the operator to fail loudly on misconfiguration. Pick Manage when you want self-sufficient bootstrapping but still want to retain ownership of the Secret object itself (for example, to back it up).

Status condition and troubleshooting

The operator exposes a ExternalSecretValid condition on ClickHouseCluster.status.conditions. Inspect it when reconciliation looks stuck:

# Plain kubectl — works out of the box
kubectl describe clickhousecluster sample | sed -n '/Conditions:/,$p'

# Same data as YAML
kubectl get clickhousecluster sample -o yaml | sed -n '/conditions:/,/^[^ ]/p'

# Pretty-printed JSON (requires jq)
kubectl get clickhousecluster sample -o jsonpath='{.status.conditions}' | jq

Possible reasons:

`reason`	Meaning	Fix
`ExternalSecretNotFound`	The referenced Secret does not exist in the namespace.	Create the Secret, or fix `spec.externalSecret.name`.
`ExternalSecretInvalid`	The Secret exists but lacks required keys (only with `Observe`). The message lists each missing key together with its expected format.	Add the missing keys, or switch to `policy: Manage`.
`ExternalSecretValid`	All required keys are present and the operator is using the Secret.	—

The operator requeues reconciliation while the Secret is invalid, so once you add the missing keys the next reconcile picks them up automatically — no need to bounce pods.

The set of required keys depends on the running ClickHouse version. named-collections-key is only validated once the operator’s version probe has detected ClickHouse 25.12 or newer. On older versions the key may be absent from the Secret. disk-encryption-key is required only when spec.settings.encryption is set.

Additional ports

The operator exposes a fixed set of ports on every ClickHouse Pod and its public headless Service: 8123 HTTP, 9000 native, 9009 interserver, 9001/9002 management, 9363 Prometheus metrics, and the TLS variants 8443/9440 when TLS is enabled. The interserver and management ports are additionally exposed through the per-replica internal Services so replicas and the operator can communicate before a replica becomes ready. To make ClickHouse listen on additional protocols — MySQL, PostgreSQL, gRPC, or any custom port — declare them in spec.additionalPorts:

spec:
  additionalPorts:
    - name: mysql
      port: 9004
    - name: postgres
      port: 9005
    - name: grpc
      port: 9100

The operator adds those ports to the Pod’s containerPorts and the public headless Service. The complete example lives at examples/custom_protocols.yaml.

additionalPorts only opens the ports on the Kubernetes side. It does not configure the ClickHouse server to listen on them. You also have to enable the matching protocol in spec.settings.extraConfig.protocols. Without that, the port is open on the Service but nothing inside the pod is answering.

End-to-end example: MySQL wire protocol

To expose ClickHouse over the MySQL wire protocol on port 9004:

apiVersion: clickhouse.com/v1alpha1
kind: ClickHouseCluster
metadata:
  name: sample
spec:
  replicas: 1
  keeperClusterRef:
    name: sample
  dataVolumeClaimSpec:
    resources:
      requests:
        storage: 2Gi

  # 1) Open the port on the Pod and the public headless Service.
  additionalPorts:
    - name: mysql
      port: 9004

  # 2) Tell ClickHouse server to actually listen on it.
  settings:
    extraConfig:
      protocols:
        mysql:
          type: mysql
          port: 9004
          description: "MySQL wire protocol"

After applying, verify from inside the cluster:

kubectl exec sample-clickhouse-0-0-0 -- \
  clickhouse-client --port 9004 --query "SELECT 1"

Field constraints

Field	Rule
`name`	Must match the DNS_LABEL pattern `^[a-z]([-a-z0-9]*[a-z0-9])?$`, max 63 characters. Uniqueness is enforced by the CRD as a list-map key.
`port`	Integer in `[1, 65535]`. The webhook rejects duplicate port numbers within the list.

Reserved ports and names

The validating webhook rejects additionalPorts entries that would collide with ports the operator binds itself. All TLS-related ports are reserved unconditionally so that flipping spec.settings.tls.enabled later cannot break a previously valid cluster.

Port	Reserved for
`8123`	HTTP
`8443`	HTTPS
`9000`	native TCP
`9440`	native TLS
`9009`	interserver
`9001`	management
`9363`	Prometheus metrics

The following names are also rejected — they are the operator’s internal protocol-type identifiers (not the human-readable aliases):

Name
`http`
`http-secure`
`tcp`
`tcp-secure`
`interserver`
`management`
`prometheus`

A rejected request produces an error such as:

spec.additionalPorts[0].port: 8123 is reserved for the operator-managed HTTP port
spec.additionalPorts[0].name: "http" is reserved by the operator

Version probe and upgrade channel

The operator does two independent things with cluster versions:

Version reporting — for ClickHouseCluster, a Kubernetes Job runs the container image once to detect the running ClickHouse version; for KeeperCluster, the operator reads the server-reported version from running replicas. The detected version is recorded in .status.version and used by other reconciliation steps (e.g. the External Secret named-collections key is only required from ClickHouse 25.12).
Upgrade channel — a periodic check against the public ClickHouse release feed (https://clickhouse.com/data/version_date.tsv). The operator reports whether a newer version is available via the VersionUpgraded status condition. It never upgrades the cluster on its own — the user is in control of the image tag.

Choosing a release channel

spec.upgradeChannel selects which set of upstream releases the operator compares against. Same field exists on both ClickHouseCluster and KeeperCluster.

spec:
  upgradeChannel: lts   # or "stable", or "25.8", or omitted

Allowed values (validated by the CRD with the pattern ^(lts|stable|\d+\.\d+)?$):

Value	Behavior
empty (default)	The operator proposes only minor updates within the currently-running major.minor line. A cluster on `25.8.3.1` will be told about `25.8.4.x` but not `25.9.x`.
`stable`	Tracks the upstream `stable` channel — the latest release that ClickHouse Inc. flags as stable on the main release line. Receives major upgrades sooner than the `lts` channel.
`lts`	Tracks the upstream `lts` channel — long-term support releases. Receives major upgrades less frequently, with longer support windows.
`25.8` (or any `<major>.<minor>`)	Pins the channel to a specific major.minor line. Major upgrades beyond it are not proposed even if a newer version exists upstream.

For production, pinning the channel to an explicit <major>.<minor> (e.g. 25.8) is generally preferred. It locks the cluster to the intended major release line and lets the operator surface a WrongReleaseChannel warning if any replica somehow drifts onto a different major — which matters especially when the image is referenced by a digest (@sha256:...) rather than by a human-readable tag. The empty default is fine for development clusters where major-version jumps are not a concern.

Status conditions

Two conditions surface the result of the probe and the upgrade check:

Condition	Reason	Meaning
`VersionInSync`	`VersionMatch`	All replicas report the same version
`VersionInSync`	`VersionMismatch`	Replicas are running different versions. This reason is suppressed during a planned rolling upgrade. It typically surfaces when a mutable image tag has been pinned (for example `latest` or a bare major like `26.3`) and the underlying registry has shifted between pulls, so different replicas ended up on different patches of the same tag.
`VersionInSync`	`VersionPending`	Version probe Job has not finished yet, or no Keeper replica version has been observed yet
`VersionInSync`	`VersionProbeFailed`	ClickHouse probe Job failed; the operator cannot determine the running version
`VersionUpgraded`	`UpToDate`	The cluster is on the latest version available in the selected channel
`VersionUpgraded`	`MinorUpdateAvailable`	A newer patch is available in the same `major.minor` line
`VersionUpgraded`	`MajorUpdateAvailable`	A newer `major.minor` is available within the chosen channel
`VersionUpgraded`	`VersionOutdated`	The running version is out of date and will no longer receive fixes from the selected channel — typically because the major line has been dropped from `lts` or `stable` upstream
`VersionUpgraded`	`WrongReleaseChannel`	The running image does not belong to the selected `upgradeChannel`. Example: a cluster running `26.5` with `upgradeChannel: lts`, since `26.5` is not part of the upstream `lts` line.
`VersionUpgraded`	`UpgradeCheckFailed`	The operator could not reach the upstream release feed

Inspect them with:

kubectl get clickhousecluster sample -o yaml | sed -n '/conditions:/,/^[^ ]/p'

Overriding the version probe Job

This applies to ClickHouseCluster only. KeeperCluster no longer runs a version-probe Job — its version is read directly from the running Keeper replicas — so spec.versionProbeTemplate is deprecated and has no effect there. The probe is implemented as a regular Kubernetes Job. If your cluster has admission policies that require specific Tolerations, node selectors, security contexts, or you want to limit how long completed probe Jobs linger, override the template via spec.versionProbeTemplate:

spec:
  versionProbeTemplate:
    spec:
      ttlSecondsAfterFinished: 600   # delete completed probe Jobs 10 minutes after completion
      template:
        spec:
          nodeSelector:
            kubernetes.io/arch: amd64
          tolerations:
            - key: dedicated
              operator: Equal
              value: clickhouse
              effect: NoSchedule
          containers:
            - name: version-probe
              resources:
                requests:
                  cpu: 50m
                  memory: 64Mi

The container name version-probe is the operator’s default — the entry under containers: matches it by name, so the operator deep-merges the user-provided fields on top of the defaults.

Operator-wide controls

Two flags on the operator manager control the upgrade-check loop globally:

Flag	Default	Effect
`--version-update-interval`	`24h`	How often the operator re-fetches the upstream version list. Must be greater than zero when version update checks are enabled
`--disable-version-update-checks`	`false`	Disables the upgrade checker entirely. The `VersionUpgraded` condition is not set, and no outbound HTTP traffic to `clickhouse.com` is generated

Set --disable-version-update-checks=true in air-gapped environments or when egress to clickhouse.com is not allowed.

ClickHouse settings

Default user password

spec.settings.defaultUserPassword sets the password for the built-in default user. Provide the value from a key in a Secret (recommended) or a ConfigMap that you create, rather than inline in the CR:

spec:
  settings:
    defaultUserPassword:
      passwordType: password   # default; see "Password types" below
      secret:                  # exactly one of secret or configMap
        name: clickhouse-password   # name of the Secret/ConfigMap
        key: password               # the key inside it, not the password value

Provide exactly one of secret or configMap, each with both name (the object) and key (the entry that holds the password).

Password types

passwordType tells ClickHouse how to interpret the value. It defaults to password (plaintext); the alternatives are hashed forms such as password_sha256_hex and password_double_sha1_hex. Prefer a hashed type so the plaintext is never stored. See the ClickHouse user settings for the full list.

Full example with a Secret

Create the Secret, then reference its key:

kubectl create secret generic clickhouse-password \
  --from-literal=password='your-secure-password'

apiVersion: clickhouse.com/v1alpha1
kind: ClickHouseCluster
metadata:
  name: my-cluster
spec:
  settings:
    defaultUserPassword:
      passwordType: password
      secret:
        name: clickhouse-password
        key: password

With passwordType: password, the in-pod clickhouse-client is configured with this password, which is handy for debugging.

For a hashed password, store the hash instead of the plaintext:

echo -n 'your-secure-password' | sha256sum   # use the hex digest as the value
kubectl create secret generic clickhouse-password \
  --from-literal=password='<sha256-hex-digest>'

spec:
  settings:
    defaultUserPassword:
      passwordType: password_sha256_hex
      secret:
        name: clickhouse-password
        key: password

Using a ConfigMap

A ConfigMap works the same way, but its contents are not protected like a Secret. Use it only for non-sensitive or already-hashed values, such as a password_sha256_hex digest:

spec:
  settings:
    defaultUserPassword:
      passwordType: password_sha256_hex
      configMap:
        name: clickhouse-config
        key: default_password

Do not put a plaintext password in a ConfigMap. Use a Secret for any plaintext (passwordType: password) value.

Custom users in configuration

Configure additional users in configuration files. Create a ConfigMap and Secret for user:

apiVersion: v1
kind: ConfigMap
metadata:
  name: user-config
data:
  reader.yaml: |
    users:
      reader:
        password:
          - '@from_env': READER_PASSWORD
        profile: default
        grants:
          query:
            - "GRANT SELECT ON *.*"
---
apiVersion: v1
kind: Secret
metadata:
  name: reader-password
data:
  password: "c2VjcmV0LXBhc3N3b3Jk"  # base64("secret-password")

Add custom configuration to ClickHouseCluster:

spec:
  podTemplate:
    volumes:
      - name: reader-user
        configMap:
          name: user-config
  containerTemplate:
    env:
      - name: READER_PASSWORD
        valueFrom:
          secretKeyRef:
            name: reader-password
            key: password
    volumeMounts:
      - mountPath: /etc/clickhouse-server/users.d/
        name: reader-user
        readOnly: true

Database sync

Enable automatic database synchronization for new replicas:

spec:
  settings:
    enableDatabaseSync: true  # Default: true

When enabled, the operator synchronizes Replicated and integration tables to new replicas. Every replica Pod carries the clickhouse.com/ReplicaInitialized readiness gate, so a new replica is published through the public headless Service only after the operator completes its initialization: the default database is converted to the Replicated engine and the schema is synchronized. Until then, the operator and the other replicas reach it through its internal Service. With enableDatabaseSync: false, the operator marks replicas initialized immediately, so readiness follows the container probes alone. The operator never drops a populated non-Replicated default database. Such a replica still starts serving client traffic after schema preparation, but the cluster reports SchemaInSync=False with reason DefaultDatabaseNotReplicated until you resolve that database yourself. Before a replica is removed on scale-down, the operator first unloads client traffic from it, waits until it stops being published, replicates its remaining data to the surviving replicas, and only then deletes it.

Server logging

Configure the ClickHouse server log through spec.settings.logger. Every field is optional with a safe default, so a cluster you never touch already logs at information to both the container console and a rotated file on disk.

spec:
  settings:
    logger:
      logToFile: true   # Default: true. Set false to log only to the console
      jsonLogs: false   # Default: false. Set true for structured JSON log lines
      level: information # Default: information
      size: 100M        # Default: 100M. Rotate a log file once it reaches this size
      count: 10         # Default: 10. Number of rotated files to keep

Field	Default	Description
`logToFile`	`true`	When `false`, the operator drops the file targets and the server logs only to the container console.
`jsonLogs`	`false`	When `true`, the operator adds `formatting.type: json` so each line is a JSON object.
`level`	`information`	Log verbosity. One of `test`, `trace`, `debug`, `information`, `notice`, `warning`, `error`, `critical`, `fatal`.
`size`	`100M`	Maximum size of a single log file before rotation.
`count`	`10`	Number of rotated log files the server retains.

The operator always keeps console logging on so that kubectl logs works, and layers file logging on top when logToFile is true. A cluster with the defaults renders this logger block:

logger:
  console: true
  level: information
  log: /var/log/clickhouse-server/clickhouse-server.log
  errorlog: /var/log/clickhouse-server/clickhouse-server.err.log
  size: 100M
  count: 10

The same spec.settings.logger block applies to a KeeperCluster; the operator writes its files under /var/log/clickhouse-keeper/ instead, and sets the Raft logger (raft_logs_level) to the same level.

Console logging stays on regardless of logToFile, so kubectl logs keeps working even when you disable file logging. Set jsonLogs: true when you ship logs to a structured log store that parses JSON.

System log tables

The operator enables the query_log, part_log, text_log, metric_log and asynchronous_metric_log system tables. Because they live on the same volume as your data, spec.settings.systemLogsTTLDays bounds them with a table TTL. The defaulting webhook sets it to 30 when you create a cluster without the field. Set 0 to disable the TTL explicitly.

spec:
  settings:
    systemLogsTTLDays: 14

The default applies only to new clusters. A cluster created before systemLogsTTLDays existed keeps unbounded system log tables until you set the field yourself.

The TTL takes effect on the next server restart. Override it per table, tune any other system log setting, or disable a table entirely with the @remove attribute — all through extraConfig:

spec:
  settings:
    extraConfig:
      text_log:
        ttl: event_date + INTERVAL 90 DAY DELETE
      metric_log:
        collect_interval_milliseconds: 60000
      part_log:
        "@remove": "1"

A disabled table is not dropped: the existing data stays on disk until you drop the table yourself.

ClickHouse recreates a system table whose TTL changed and keeps the old data in a renamed table such as system.text_log_0. Drop those tables once you no longer need the history.

Custom configuration

Embedded extra configuration

Instead of mounting custom configuration files, you can directly specify additional ClickHouse configuration options. Add custom ClickHouse configuration using extraConfig:

spec:
  settings:
    extraConfig:
      background_pool_size: 20

Useful links:

Reloadable extra configuration

Any change to extraConfig restarts the ClickHouse pods, because the operator cannot know which settings apply at runtime. For settings ClickHouse reloads on the fly — named_collections is the common case — use extraReloadableConfig instead. The operator applies its changes through a configuration reload, without restarting pods:

spec:
  settings:
    extraReloadableConfig:
      named_collections:
        customers:
          url: "http://my-service.my-ns.svc:5400/Customers"
          format: "JSONEachRow"

Keep each top-level section in exactly one of extraConfig and extraReloadableConfig; the operator warns when both set the same section.

By using extraReloadableConfig you assert its settings are reload-safe. A restart-only setting placed here takes effect only at the next restart, without any error.

Embedded extra users configuration

You can also specify additional ClickHouse users configuration using extraUsersConfig. This is useful for defining users, profiles, quotas, and grants directly in the cluster specification.

spec:
  settings:
    extraUsersConfig:
      users:
        analyst:
          password:
            - '@from_env': ANALYST_PASSWORD
          profile: "readonly"
          quota: "default"
      profiles:
        readonly:
          readonly: 1
          max_memory_usage: 10000000000
      quotas:
        default:
          interval:
            duration: 3600
            queries: 1000
            errors: 100

The extraUsersConfig is stored in k8s ConfigMap object. Avoid plain text secrets there.

See documentation for all supported ClickHouse users configuration options.

Configuration example

Complete configuration example:

apiVersion: clickhouse.com/v1alpha1
kind: KeeperCluster
metadata:
  name: sample
spec:
  replicas: 3
  dataVolumeClaimSpec:
    storageClassName: <storage-class-name>
    resources:
      requests:
        storage: 10Gi
  podTemplate:
    topologyZoneKey: topology.kubernetes.io/zone
    nodeHostnameKey: kubernetes.io/hostname
  containerTemplate:
    resources:
      requests:
        cpu: "2"
        memory: "4Gi"
      limits:
        cpu: "4"
        memory: "8Gi"
  settings:
    tls:
      enabled: true
      required: true
      serverCertSecret:
        name: <keeper-certificate-secret>
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: default-user-password
data:
  # secret-password
  password: "..." # sha256 hex of the password
---
apiVersion: clickhouse.com/v1alpha1
kind: ClickHouseCluster
metadata:
  name: sample
spec:
  replicas: 2
  dataVolumeClaimSpec:
    storageClassName: <storage-class-name>
    resources:
      requests:
        storage: 200Gi
  keeperClusterRef:
    name: sample
  podTemplate:
    topologyZoneKey: topology.kubernetes.io/zone
    nodeHostnameKey: kubernetes.io/hostname
  settings:
    tls:
      enabled: true
      required: true
      serverCertSecret:
        name: clickhouse-cert
    defaultUserPassword:
      passwordType: password_sha256_hex
      configMap:
        key: password
        name: default-password

Periodic resync

Besides reacting to Kubernetes events, the operator re-reconciles every cluster on a fixed interval. This keeps status conditions fresh for state that no watch can observe — ClickHouse replica health, Keeper quorum — and retries remediation such as stuck-pod cleanup. The interval is controlled by the operator’s RESYNC_PERIOD environment variable (default 30s). It applies to KeeperCluster resources, where 0 disables the periodic resync. ClickHouseCluster resources poll ClickHouse system.warnings every 30 seconds, which already re-triggers reconciliation, so RESYNC_PERIOD currently has no effect on them.

# in the operator Deployment spec
env:
- name: RESYNC_PERIOD
  value: "1m"

The operator reconciles clusters concurrently; the number of parallel reconciles per controller is set by MAX_CONCURRENT_RECONCILES (default 4).

Pausing reconciliation

Set the clickhouse.com/pause-reconciliation: "true" annotation on a ClickHouseCluster or KeeperCluster to stop the operator from reconciling it. The operator reports ReconcileSucceeded=False with reason ReconciliationPaused on the paused resource and otherwise leaves it untouched — useful for manual intervention when the operator would fight your changes. While paused, the operator reports all state conditions (including Ready and Healthy) as Unknown with reason ReconciliationPaused — it does not observe the cluster. Remove the annotation to resume; the pending spec changes are applied on the next reconciliation.

kubectl annotate clickhousecluster my-cluster clickhouse.com/pause-reconciliation=true
# ... manual surgery ...
kubectl annotate clickhousecluster my-cluster clickhouse.com/pause-reconciliation-

Last modified on October 8, 2026

​ClickHouseCluster configuration

​Basic configuration

​Replicas and shards

​Keeper integration

​Externally managed Keeper

​KeeperCluster configuration

​Leadership handover on termination

​Storage configuration

​Cluster domain

​Pod configuration

​Automatic topology spread and affinity

​Manual configuration

​See API Reference for all supported Pod template options.

​Pod disruption budgets

​Defaults

​Overriding the defaults

​Policies

​Cluster-wide opt-out

​Container configuration

​Custom image

​Container resources

​Environment variables

​Volume mounts

​Health probes

​Startup probe

​See API Reference for all supported Container template options.

​TLS/SSL configuration

​Configure secure endpoints

​SSL certificate secret format

​ClickHouse-Keeper communication over TLS

​External Secret

​Required keys

​Policy: Observe vs Manage

​Status condition and troubleshooting

​Additional ports

​End-to-end example: MySQL wire protocol

​Field constraints

​Reserved ports and names

​Version probe and upgrade channel

​Choosing a release channel

​Status conditions

​Overriding the version probe Job

​Operator-wide controls

​ClickHouse settings

​Default user password

​Password types

​Full example with a Secret

​Using a ConfigMap

​Custom users in configuration

​Database sync

​Server logging

​System log tables

​Custom configuration

​Embedded extra configuration

​Useful links:

​Reloadable extra configuration

​Embedded extra users configuration

​See documentation for all supported ClickHouse users configuration options.

​Configuration example

​Periodic resync

​Pausing reconciliation

ClickHouseCluster configuration

Basic configuration

Replicas and shards

Keeper integration

Externally managed Keeper

KeeperCluster configuration

Leadership handover on termination

Storage configuration

Cluster domain

Pod configuration

Automatic topology spread and affinity

Manual configuration

See API Reference for all supported Pod template options.

Pod disruption budgets

Defaults

Overriding the defaults

Policies

Cluster-wide opt-out

Container configuration

Custom image

Container resources

Environment variables

Volume mounts

Health probes

Startup probe

See API Reference for all supported Container template options.

TLS/SSL configuration

Configure secure endpoints

SSL certificate secret format

ClickHouse-Keeper communication over TLS

External Secret

Required keys

Policy: Observe vs Manage

Status condition and troubleshooting

Additional ports

End-to-end example: MySQL wire protocol

Field constraints

Reserved ports and names

Version probe and upgrade channel

Choosing a release channel

Status conditions

Overriding the version probe Job

Operator-wide controls

ClickHouse settings

Default user password

Password types

Full example with a Secret

Using a ConfigMap

Custom users in configuration

Database sync

Server logging

System log tables

Custom configuration

Embedded extra configuration

Useful links:

Reloadable extra configuration

Embedded extra users configuration

See documentation for all supported ClickHouse users configuration options.

Configuration example

Periodic resync

Pausing reconciliation