AgentSkillsCN

clawdsense

通过 ClawdSense ESP32 适配器进行实时图像分析。监测媒体接收器，从设备中抓取照片，并借助 Groq Vision 实时分析图像。当 ClawdSense 通过 /photo 命令或按钮控制发送照片时，可启用此功能。它能够即时分析房间状况、检测人员占用情况，并提升环境感知能力。

SKILL.md
--- frontmatter
name: clawdsense
description: Real-time image analysis from ClawdSense ESP32 dongle. Monitors media receiver, captures photos from device, analyzes instantly with Groq Vision. Use when ClawdSense sends photos via /photo command or button controls. Provides instant room analysis, occupancy detection, and environmental awareness.

ClawdSense Skill

Real-time image capture and analysis from ClawdSense ESP32 dongle.

Quick Start

Start Services

bash
# Terminal 1: Media receiver (accepts photo uploads from ESP32)
node ~/clawd/clawdsense-skill/scripts/media-receiver.js

# Terminal 2: Analyzer (monitors inbound folder, analyzes with Groq)
node ~/clawd/clawdsense-skill/scripts/analyzer.js

# Terminal 3: Health monitor (keeps both services alive)
node ~/clawd/clawdsense-skill/scripts/health-monitor.js

Usage

  1. •Send /photo command to ClawdSense via Telegram
  2. •Device captures and POSTs to media receiver (port 5555)
  3. •Analyzer detects new photo and analyzes with Groq Vision
  4. •Results printed to console

Architecture

Three Components

Media Receiver (port 5555)

  • •Accepts multipart/form-data uploads from ESP32
  • •Stores photos in ~/.clawdbot/media/inbound/
  • •Endpoints:
    • •POST /inbound/photo - JPEG photos
    • •POST /inbound/audio - WAV audio
    • •POST /inbound/video - AVI video

Analyzer (real-time polling)

  • •Polls inbound folder every 500ms
  • •Detects new photos automatically
  • •Sends to Groq Vision API for analysis
  • •Uses pixtral-12b model for instant results

Health Monitor

  • •Checks both services every 30s
  • •Restarts if either dies
  • •Logs status to console

Performance

  • •Detection latency: ~500ms (polling interval)
  • •Analysis time: 1-3s (Groq API)
  • •Total end-to-end: ~2-5s from capture to results

Configuration

ESP32 Firmware Settings

Device must be configured with:

code
MEDIA_RECEIVER_URL = "http://localhost:5555"
or for public: "https://your-ngrok-url"

Groq API Key

Stored in environment:

bash
export GROQ_API_KEY="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"

Troubleshooting

"Media receiver is DOWN"

  • •Check if port 5555 is in use
  • •Restart: node ~/clawd/clawdsense-skill/scripts/media-receiver.js

"No new photos detected"

  • •Is device sending to media receiver? Check device logs
  • •Is media receiver running? Curl http://localhost:5555/health
  • •Check inbound folder permissions

"Groq API errors"

  • •Verify API key is set
  • •Check account quota/billing

References

  • •See references/groq-vision-api.md for Groq setup
  • •See references/esp32-setup.md for device configuration