Generating Assets with ComfyUI

This skill helps you generate images and videos using ComfyUI, the powerful node-based generative AI platform.

Prerequisites

ComfyUI must be running locally or accessible via network. Default: http://127.0.0.1:8188

Quick Setup:

bash

# Install comfy-cli
pip install comfy-cli

# Install ComfyUI
comfy install

# Launch ComfyUI server
comfy launch

Core Capabilities

•Text-to-Image: Generate images from text prompts (Flux, SDXL, SD1.5)
•Text-to-Video: Generate videos from text (HunyuanVideo, Wan, LTX-Video)
•Image-to-Video: Animate images into video sequences
•Image-to-Image: Transform/enhance existing images
•Upscaling: Enhance resolution with ESRGAN/SwinIR
•LoRA Integration: Apply style/concept fine-tunes

Workflow Execution Process

Step 1: Check ComfyUI Status

bash

curl -s http://127.0.0.1:8188/system_stats | jq .

Step 2: Choose Workflow Template

Select based on the user's needs:

•Image Generation: See ./workflow-templates.md#text-to-image
•Video Generation: See ./workflow-templates.md#text-to-video
•Image Enhancement: See ./workflow-templates.md#upscaling

Step 3: Submit Workflow via API

Use the provided Node.js scripts in ./scripts/ or submit directly:

javascript

// See ./scripts/comfy-api.mjs for full implementation
const response = await fetch('http://127.0.0.1:8188/prompt', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ prompt: workflowJson, client_id: clientId })
});

Step 4: Monitor Progress

Connect to WebSocket for real-time updates:

code

ws://127.0.0.1:8188/ws?clientId={client_id}

Step 5: Retrieve Results

bash

# Get execution history
curl http://127.0.0.1:8188/history/{prompt_id}

# Download generated image
curl "http://127.0.0.1:8188/view?filename={filename}&subfolder={subfolder}&type=output" -o output.png

Quick Commands

Task	Script
Generate Image	`node ~/.claude/skills/generating-with-comfyui/scripts/comfy-api.mjs generate-image "prompt"`
Generate Video	`node ~/.claude/skills/generating-with-comfyui/scripts/comfy-api.mjs generate-video "prompt"`
Check Status	`node ~/.claude/skills/generating-with-comfyui/scripts/comfy-api.mjs status`
List Models	`node ~/.claude/skills/generating-with-comfyui/scripts/comfy-api.mjs list-models`

API Endpoints Reference

Endpoint	Method	Purpose
`/prompt`	POST	Submit workflow to queue
`/prompt`	GET	Get current queue status
`/queue`	GET	View execution queue
`/queue`	POST	Clear/manage queue
`/history`	GET	Get execution history
`/history/{id}`	GET	Get specific prompt history
`/interrupt`	POST	Stop current execution
`/object_info`	GET	List all available nodes
`/models`	GET	List available models
`/view`	GET	Retrieve generated outputs
`/upload/image`	POST	Upload input images

Model Recommendations

For Images

•Flux.1 Dev/Schnell: Latest high-quality, fast generation
•SDXL 1.0: Excellent quality, wide compatibility
•SD 1.5: Fast, vast LoRA ecosystem

For Videos

•HunyuanVideo 1.5: 8.3B params, consumer GPU friendly, 480p-1080p
•Wan 2.1: High quality text-to-video
•LTX-Video: Fast video generation

For Enhancement

•4x-UltraSharp: Best general upscaler
•RealESRGAN x4: Realistic photo enhancement
•SwinIR: High-quality restoration

Directory Structure

ComfyUI expects models in specific locations:

code

ComfyUI/
├── models/
│   ├── checkpoints/     # Main model files (.safetensors)
│   ├── vae/             # VAE models
│   ├── loras/           # LoRA fine-tunes
│   ├── controlnet/      # ControlNet models
│   ├── clip/            # CLIP text encoders
│   ├── upscale_models/  # Upscaler models
│   └── diffusion_models/ # Flux/video models
├── input/               # Input images
├── output/              # Generated outputs
└── custom_nodes/        # Extensions

Installing Models

bash

# Download checkpoint
comfy model download <url> models/checkpoints

# Or manually with curl/wget
curl -L "https://huggingface.co/model/file.safetensors" -o models/checkpoints/model.safetensors

Custom Nodes Installation

bash

# Install via comfy-cli
comfy node install ComfyUI-VideoHelperSuite
comfy node install ComfyUI-Manager

# Or git clone to custom_nodes/
cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-HunyuanVideoWrapper

Troubleshooting

Server Not Running

bash

# Check if ComfyUI is running
curl -s http://127.0.0.1:8188/ > /dev/null && echo "Running" || echo "Not running"

# Start it
comfy launch

Out of Memory

•Reduce resolution (512x512 for SD, 1024x1024 for SDXL)
•Enable --lowvram or --cpu flags
•Use FP8 quantized models

Missing Model

bash

# List available models
curl http://127.0.0.1:8188/models | jq .

Best Practices

•Always check server status before submitting workflows
•Use unique client_id for WebSocket connections
•Monitor queue to avoid duplicate submissions
•Save successful workflows as JSON templates
•Use seed values for reproducible results

Reference Documentation

•Workflow templates: ./workflow-templates.md
•Video generation specifics: ./video-generation-guide.md
•Full API script: ./scripts/comfy-api.mjs