AgentSkillsCN

kafka

在Apache Kafka中生产和消费消息——列出主题、检查消费者组延迟，并将数据流式传输至DuckDB。

SKILL.md
--- frontmatter
name: kafka
description: >
  Produce and consume messages from Apache Kafka — list topics, check
  consumer group lag, and stream data into DuckDB.
metadata:
  openclaw:
    requires:
      bins: [clawdata]
    primaryEnv: KAFKA_BOOTSTRAP_SERVERS
    tags: [streaming, kafka, messages, topics, events]

Kafka

Produce and consume messages from Apache Kafka clusters.

Commands

TaskCommand
List topicsclawdata kafka topics
Describe topicclawdata kafka describe <topic>
Consume messagesclawdata kafka consume <topic> --limit 10
Produce messageclawdata kafka produce <topic> --message '{"key":"value"}'
Consumer group lagclawdata kafka lag <group>
Consume to DuckDBclawdata kafka ingest <topic> --table <table>

Configuration

Env VarDescription
KAFKA_BOOTSTRAP_SERVERSComma-separated broker list (e.g. localhost:9092)
KAFKA_SASL_USERNAMESASL username (optional)
KAFKA_SASL_PASSWORDSASL password (optional)
KAFKA_SECURITY_PROTOCOLPLAINTEXT, SASL_SSL, etc.

When to use

  • •User wants to check what events are flowing → clawdata kafka consume
  • •User needs to load streaming data into DuckDB → clawdata kafka ingest
  • •User asks about topic lag → clawdata kafka lag

Example: Stream to DuckDB

bash
# Consume 1000 messages from 'events' topic into DuckDB table
clawdata kafka ingest events --table raw_events --limit 1000

# Then transform with dbt
clawdata dbt run --select raw_events+