Kafka KRaft Setup with Strimzi

This skill documents the successful transition from ZooKeeper to KRaft mode and resource tuning for stability.

Core Setup

1. KRaft Migration

Use KafkaNodePool to manage brokers and controllers without ZooKeeper.

Example Kafka Resource:

yaml

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 4.0.0 # Use modern version for KRaft
    metadataVersion: 4.0-IV1
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      default.replication.factor: 1
      min.insync.replicas: 1
  entityOperator:
    topicOperator: {}
    userOperator: {}

Resource Optimization

On local machines (Minikube), default limits are often too low or too high for the available node capacity.

1. Kafka Pod Resources

Kafka requires at least 1Gi RAM to initialize reliably.

yaml

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: dual-role
spec:
  replicas: 1
  roles:
    - controller
    - broker
  resources:
    limits:
      memory: 1Gi
      cpu: 1000m
    requests:
      memory: 512Mi
      cpu: 500m

2. Operator Stability

The Strimzi Cluster Operator can OOM if it has too many resources to manage with low memory. Ensure it has at least 512Mi-1Gi.

Storage

Always use jbod with persistent claims for local persistence.

yaml

storage:
  type: jbod
  volumes:
  - id: 0
    type: persistent-claim
    size: 5Gi
    deleteClaim: false

Troubleshooting

•Pod Forbidden (Quota/LimitRange): If pods fail to create, check for LimitRange or ResourceQuota in the namespace that might be capping memory below the 1Gi required by Kafka.
•Entity Operator CrashLoop: If the operator crashes, check if the node is under heavy CPU pressure; scale down other deployments during the initial Kafka startup.