Caddy Server Management Skill

Version: 1.0.0 Author: Claude Code Purpose: Comprehensive Caddy web server management across multiple environments

Overview and Purpose

This skill provides enterprise-grade Caddy server management across local development and production environments. It handles:

•Multi-server operations: Manage Caddy on local (macOS), bigbox (production), and registry (production) servers
•Zero-downtime deployments: Graceful reloads with automatic validation
•Secret management: Integrated 1Password support for secure credential handling
•Custom builds: Build Caddy with required plugins (security, cloudflare, cgi, tlsredis)
•Systemd management: Proper service lifecycle management
•Log analysis: Parse and analyze JSON logs for troubleshooting
•Pattern library: Ready-to-use configurations for common use cases

Critical Mandates:

•Zero-Downtime: Always use graceful reloads, never kill processes
•Validation-First: Mandatory syntax validation before any deployment
•Secret Safety: Never display plaintext secrets, always use 1Password
•Backup-First: Always create timestamped backups before modifications

Prerequisites

Required

•
Server Access
- •Local: macOS with mdfind available
- •bigbox: SSH access configured (ssh bigbox)
- •registry: SSH access configured (ssh registry)

•

Caddy Installation

bash

# Check Caddy on servers
ssh bigbox "which caddy && caddy version"
ssh registry "which caddy && caddy version"

•

1Password CLI (for secret management)

bash

# Verify 1Password skill is available
which op && op --version
# See: ~/.claude/skills/1password-secrets.md

•

Systemd (on production servers)

bash

# Verify systemd available
ssh bigbox "systemctl --version"
ssh registry "systemctl --version"

Optional

•xcaddy: For building custom Caddy binaries
•jq: For JSON log parsing (usually available)
•Go toolchain: Required for xcaddy builds

Server Configuration

This skill operates across 3 servers with different characteristics:

Local (Development)

•Platform: macOS
•Search: mdfind (fast Spotlight search)
•Caddy: Not running (development configs only)
•Use Cases: Testing configs, pattern development

bigbox (Production - Migration Required)

•Platform: Linux
•Caddy Version: v2.6.4 (custom build)
•Binary: /home/alex/caddy_terraphim/caddy
•Active Config: /home/alex/caddy_terraphim/conf/Caddyfile_auth
•Status: Currently running in tmux (needs systemd migration)
•Plugins: caddy-security, caddy-tlsredis, cloudflare, cgi
•⚠️ Action Required: Migrate to systemd + 1Password

registry (Production - Best Practice)

•Platform: Linux
•Caddy Version: v2.10.2 (standard)
•Binary: /usr/bin/caddy
•Active Config: /etc/caddy/Caddyfile
•Status: Properly managed via systemd
•✓ Reference: Clean 41-line config with snippets

Server configuration details: ~/.docs/caddy-skill/servers.json

Workflow 1: Discover Caddyfiles

Purpose: Find all Caddyfiles across servers quickly (<5 seconds).

When to Use

•Auditing Caddyfile locations
•Before making configuration changes
•Identifying active vs backup configs
•Searching for specific configuration patterns

How It Works

•User specifies server(s): local, bigbox, registry, or all
•Skill loads server configuration from servers.json
•Skill executes fast search command (mdfind locally, find remotely)
•Skill filters out backup files (.backup., *.bak)
•Skill identifies active configuration (from systemd service)
•Skill displays sorted results (active first, then by modification time)

Usage Patterns

Pattern 1: Find on Single Server

code

User: "Find all Caddyfiles on bigbox"

Assistant Process:
1. Load bigbox config from servers.json
2. Execute via SSH: find /home/alex/caddy_terraphim /home/alex/infrastructure -name 'Caddyfile*' ! -name '*.backup.*' -type f
3. Identify active config from systemd service
4. Sort results (active first)
5. Display with metadata

Example Output:
Found 14 Caddyfiles on bigbox:

ACTIVE:
  ✓ /home/alex/caddy_terraphim/conf/Caddyfile_auth (324 lines, modified 2 days ago)
    Service: caddy-terraphim (currently tmux-based)
    Status: Running (PID 24566, 35+ days uptime)

INACTIVE:
  /home/alex/infrastructure/atomic-server-turso/Caddyfile.enhanced (112 lines)
  /home/alex/infrastructure/atomic-server-turso/Caddyfile.production (86 lines)
  /home/alex/caddy_terraphim/conf/Caddyfile_auth2 (63 lines)
  ... (10 more files)

💡 Tip: Use 'Validate' workflow to check syntax before deployment

Pattern 2: Find on All Servers

code

User: "Find all Caddyfiles across all servers"

Assistant Process:
1. Load all server configs
2. Execute searches in parallel (local + SSH to bigbox + SSH to registry)
3. Aggregate results by server
4. Mark active configs
5. Display grouped by server

Example Output:
Found 28 Caddyfiles across 3 servers:

LOCAL (14 files):
  /Users/alex/projects/zestic-ai/charm/charm-impact/infrastructure/Caddyfile.production (167 lines)
  /Users/alex/projects/zestic-ai/charm/charm-impact/infrastructure/Caddyfile_docker (31 lines)
  ... (12 more)

BIGBOX (14 files):
  ✓ ACTIVE: /home/alex/caddy_terraphim/conf/Caddyfile_auth (324 lines)
  /home/alex/infrastructure/atomic-server-turso/Caddyfile.enhanced (112 lines)
  ... (12 more)

REGISTRY (1 file):
  ✓ ACTIVE: /etc/caddy/Caddyfile (41 lines)

⚠️  bigbox: Migration to systemd + 1Password recommended
✓  registry: Properly configured (reference implementation)

Pattern 3: Search with Pattern

code

User: "Find Caddyfiles with 'auth' in the name on bigbox"

Assistant Process:
1. Load bigbox config
2. Add pattern filter: -name '*auth*'
3. Execute search
4. Display results

Example Output:
Found 3 Caddyfiles matching 'auth' on bigbox:
  ✓ /home/alex/caddy_terraphim/conf/Caddyfile_auth (ACTIVE, 324 lines)
  /home/alex/caddy_terraphim/conf/Caddyfile_auth2 (63 lines)
  /home/alex/caddy_terraphim/conf/Caddyfile_localauth (123 lines)

Implementation Details

Step 1: Load Server Configuration

bash

# Read servers.json
cat ~/.docs/caddy-skill/servers.json | jq -r '.servers.bigbox'

Step 2: Execute Search

For local (macOS):

bash

mdfind -name Caddyfile | grep -v ".backup\." | head -20

For remote servers (Linux):

bash

ssh bigbox "find /home/alex/caddy_terraphim /home/alex/infrastructure \
  -name 'Caddyfile*' ! -name '*.backup.*' ! -name '*~' -type f"

Step 3: Identify Active Configuration

bash

# Check systemd service for active config
ssh bigbox "systemctl show caddy-terraphim --property=ExecStart"
# Parse output to extract config path

Step 4: Get File Metadata

bash

# For each file, get line count and modification time
ssh bigbox "wc -l /path/to/Caddyfile; stat -c %y /path/to/Caddyfile"

Step 5: Sort and Display

•Active configs first
•Then by modification time (newest first)
•Show metadata: lines, modification date, status

Performance Optimization

•Local search: mdfind is instant (Spotlight index)
•Remote search: find with path limits (<5 seconds)
•Parallel execution: Search all servers concurrently
•Caching: Results cached for 60 seconds (optional)

Error Handling

SSH Connection Failed:

code

✗ Error: Cannot connect to bigbox
  Check: ssh bigbox "echo test"
  Solution: Verify SSH config and connectivity

No Caddyfiles Found:

code

ℹ No Caddyfiles found on local
  Searched paths: /Users/alex/projects/
  Tip: Caddyfiles may be in different locations

Permission Denied:

code

✗ Error: Permission denied accessing /etc/caddy/Caddyfile
  Solution: Add sudo to remote command or adjust permissions

Security Guarantees

✓ Read-only operations (no file modifications) ✓ SSH uses existing configuration (no password prompts) ✓ No secrets displayed (even if found in configs) ✓ Respects file permissions (won't force access)

Workflow 2: Validate Configuration

Purpose: Validate Caddyfile syntax before deployment to prevent downtime.

When to Use

•Before deploying configuration changes
•After editing Caddyfiles
•When troubleshooting syntax errors
•As part of CI/CD pipeline

How It Works

•User provides Caddyfile path and server
•Skill reads server configuration
•Skill executes caddy validate --config <path>
•Skill parses validation output for errors
•Skill checks for environment variables (coordinates with 1Password skill if needed)
•Skill reports results with clear error messages

Usage Patterns

Pattern 1: Validate Single File

code

User: "Validate /home/alex/caddy_terraphim/conf/Caddyfile_auth on bigbox"

Assistant Process:
1. Load bigbox server config
2. Check Caddy binary exists
3. Execute: ssh bigbox "/home/alex/caddy_terraphim/caddy validate --config /home/alex/caddy_terraphim/conf/Caddyfile_auth"
4. Parse output
5. Check for undefined environment variables
6. Report results

Example Output (Valid):
✓ Configuration is valid
  File: /home/alex/caddy_terraphim/conf/Caddyfile_auth
  Lines: 324
  Binary: /home/alex/caddy_terraphim/caddy (v2.6.4)
  Validation time: 0.8s

Environment variables detected:
  ⚠️  GITHUB_CLIENT_ID (defined in caddy_complete.env)
  ⚠️  GITHUB_CLIENT_SECRET (plaintext in caddy_complete.env)
  ⚠️  JWT_SHARED_KEY (plaintext in caddy_complete.env)

Recommendation: Migrate secrets to 1Password
  See: Workflow "Migrate Secrets" or 1Password skill

Pattern 2: Validate with Errors

code

Example Output (Invalid):
✗ Configuration has errors

Error 1 (Line 45):
  /home/alex/caddy_terraphim/conf/Caddyfile_auth:45
  parsing Caddyfile tokens: missing closing brace

  Context:
  43: reverse_proxy localhost:3000 {
  44:     header_up X-Real-IP {remote_host}
  45:     # Missing closing brace here
  46:
  47: handle /api {

Fix: Add closing brace after line 44

Error 2 (Line 102):
  Unrecognized directive: 'auth_portal'
  Did you mean: 'authentication portal' ?

Recommendation: Fix these 2 errors before deployment

Pattern 3: Validate Before Deployment

code

User: "Validate the active Caddyfile on bigbox before I deploy changes"

Assistant Process:
1. Identify active config (Workflow 1)
2. Validate current active config
3. If user modified a file, validate that too
4. Compare and report

Example Output:
✓ Current active config is valid
  File: /home/alex/caddy_terraphim/conf/Caddyfile_auth
  Status: Currently running, no errors in logs

✓ Modified config is valid
  File: /home/alex/caddy_terraphim/conf/Caddyfile_auth
  Changes: 15 lines modified (comparing to last backup)

Validation Summary:
  - Syntax: ✓ Valid
  - Directives: ✓ All recognized
  - Env vars: ⚠️  3 plaintext secrets detected
  - Conflicts: ✓ No port conflicts

Ready for deployment: Yes
Recommendation: Deploy with 'Workflow 4: Deploy Changes'

Implementation Details

Step 1: Check Caddy Binary

bash

# Verify Caddy is available
ssh bigbox "which /home/alex/caddy_terraphim/caddy && /home/alex/caddy_terraphim/caddy version"

Step 2: Execute Validation

bash

# Validate Caddyfile
ssh bigbox "/home/alex/caddy_terraphim/caddy validate --config /path/to/Caddyfile --adapter caddyfile"

Step 3: Parse Output

bash

# Validation successful if exit code 0
# Parse stderr for error messages
# Extract line numbers and error types

Step 4: Check Environment Variables

bash

# Scan Caddyfile for {env.VAR_NAME} or {$VAR_NAME} syntax
grep -oP '\{(env\.)?\$?[A-Z_]+\}' Caddyfile

# Check if variables defined in env file or 1Password template
cat caddy_complete.env | grep "VAR_NAME="

Step 5: Detect Secrets

bash

# Coordinate with 1Password skill to detect plaintext secrets
# See: ~/.claude/skills/1password-secrets.md Workflow 1

Validation Checks

Syntax Validation:

•Valid Caddyfile format
•Matching braces and brackets
•Recognized directives
•Valid matcher syntax
•Proper nesting

Semantic Validation:

•No duplicate site blocks
•No conflicting matchers
•Valid port bindings (not in use)
•Valid upstream addresses
•Proper TLS configuration

Environment Validation:

•All referenced variables defined
•No plaintext secrets (warn if found)
•Env file readable if specified

Security Validation:

•Secrets in 1Password (recommended)
•No exposed credentials
•Proper TLS configuration
•Security headers present (recommended)

Error Messages

Syntax Errors:

code

Line 45: Missing closing brace
  Expected '}' to close block started at line 43

  43: reverse_proxy localhost:3000 {
  44:     header_up X-Real-IP {remote_host}
  45:     # Missing '}'

Directive Errors:

code

Line 102: Unrecognized directive 'auth_portal'
  Available in module: http.handlers.authentication
  Suggestion: authentication portal myportal
  Documentation: https://caddyserver.com/docs/...

Environment Errors:

code

Line 15: Undefined environment variable 'API_KEY'
  Referenced as: {env.API_KEY}
  Solution 1: Define in caddy_complete.env
  Solution 2: Migrate to 1Password: op://Vault/Item/api_key

Safety Guarantees

✓ Read-only validation (no file changes) ✓ No service disruption (validation only) ✓ No secret exposure (secrets masked in output) ✓ Detailed error reporting (line numbers, context)

Workflow 3: Format Caddyfile

Purpose: Format Caddyfile for consistent style and readability.

When to Use

•After manual edits
•Before committing to version control
•When adopting team style standards
•After merging configuration changes

How It Works

•User provides Caddyfile path and server
•Skill creates timestamped backup
•Skill executes caddy fmt --overwrite <path>
•Skill validates formatted output
•Skill shows diff (before/after) if requested
•Skill provides rollback instructions

Usage Pattern

code

User: "Format /home/alex/caddy_terraphim/conf/Caddyfile_auth on bigbox"

Assistant Process:
1. Load bigbox config
2. Create backup: Caddyfile_auth.backup.20251229_143000
3. Execute: caddy fmt --overwrite Caddyfile_auth
4. Validate formatted file
5. Show summary

Example Output:
✓ Caddyfile formatted successfully
  File: /home/alex/caddy_terraphim/conf/Caddyfile_auth
  Backup: /home/alex/caddy_terraphim/conf/Caddyfile_auth.backup.20251229_143000
  Changes: Indentation normalized, spacing adjusted

Validation: ✓ Formatted file is valid

To rollback:
  ssh bigbox "cp /home/alex/caddy_terraphim/conf/Caddyfile_auth.backup.20251229_143000 \
              /home/alex/caddy_terraphim/conf/Caddyfile_auth"

Recommendation: Review changes before deployment

Implementation Details

Step 1: Create Backup

bash

# Timestamped backup
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
cp Caddyfile_auth Caddyfile_auth.backup.$TIMESTAMP

Step 2: Format

bash

# Execute format with overwrite
caddy fmt --overwrite /path/to/Caddyfile

Step 3: Validate

bash

# Ensure formatting didn't break syntax
caddy validate --config /path/to/Caddyfile

Step 4: Show Diff (Optional)

bash

# Compare before/after
diff -u Caddyfile_auth.backup.$TIMESTAMP Caddyfile_auth

Safety Guarantees

✓ Always creates backup before formatting ✓ Validates after formatting to ensure no breakage ✓ Provides rollback instructions immediately ✓ Never formats active config without confirmation

Workflow 4: Deploy Changes (Graceful Reload)

Purpose: Deploy Caddyfile changes with zero downtime using graceful reload.

When to Use

•After validating configuration changes
•Deploying new site configurations
•Updating TLS settings
•Modifying reverse proxy rules

How It Works

•Pre-deployment checks (validation, service status, secret detection)
•Create backup of current config
•Detect deployment method (systemd reload or caddy reload API)
•Execute graceful reload
•Post-deployment verification (service status, logs, endpoint tests)
•Automatic rollback on failure

Usage Pattern

code

User: "Deploy Caddyfile changes to bigbox"

Assistant Process:
1. Pre-deployment Checks:
   ✓ Validate configuration
   ✓ Check service is running
   ⚠️  Detect 3 plaintext secrets (recommend 1Password migration)
   ✓ Create backup

2. Deployment:
   Method: systemd reload (zero-downtime)
   Command: ssh bigbox "systemctl reload caddy-terraphim"
   Status: ✓ Reload initiated

3. Verification:
   ✓ Service status: active (running)
   ✓ No errors in logs (last 50 lines)
   ✓ Test endpoint: https://alpha.truthforge.terraphim.cloud/health → 200 OK
   ✓ Requests dropped during reload: 0

Deployment Summary:
  - Status: ✓ Success
  - Duration: 2.3 seconds
  - Downtime: 0 seconds (graceful reload)
  - Backup: Caddyfile_auth.backup.20251229_143500

⚠️  Recommendation: Migrate secrets to 1Password for enhanced security
    See: 1Password skill or Workflow "Migrate Secrets"

Pre-Deployment Checks

Check 1: Validate Configuration

bash

# Must pass validation
caddy validate --config /path/to/Caddyfile

Check 2: Service Running

bash

# Verify Caddy is running
systemctl is-active caddy-terraphim
# Or check process for tmux-based
ps aux | grep caddy | grep -v grep

Check 3: Detect Secrets

bash

# Scan for plaintext secrets (coordinate with 1Password skill)
# Warn user but don't block deployment

Check 4: Create Backup

bash

# Always backup current active config
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
cp /path/to/Caddyfile /path/to/Caddyfile.backup.$TIMESTAMP

Deployment Methods

Method 1: Systemd Reload (Recommended)

bash

# Graceful reload via systemd
systemctl reload caddy-terraphim

# Benefits:
# - Zero downtime (in-place reload)
# - No dropped connections
# - Automatic rollback on failure
# - Logs in journalctl

Method 2: Caddy Reload API

bash

# Direct reload via Caddy admin API
curl -X POST http://localhost:2019/load \
  -H "Content-Type: application/json" \
  -d @config.json

# Benefits:
# - Finer control
# - Can preview config before apply
# - Works without systemd

Method 3: Signal Reload

bash

# Send USR1 signal (for tmux-based or non-systemd)
kill -USR1 $(cat /var/run/caddy.pid)

# Note: Less reliable, use systemd if possible

Post-Deployment Verification

Step 1: Check Service Status

bash

systemctl status caddy-terraphim --no-pager
# Verify: active (running), no recent errors

Step 2: Check Recent Logs

bash

# Last 50 log lines for errors
journalctl -u caddy-terraphim -n 50 --no-pager | grep -i error

Step 3: Test Endpoints

bash

# Test configured domains
curl -f https://alpha.truthforge.terraphim.cloud/health
curl -f https://ci.terraphim.cloud/health

# Verify: 200 OK responses

Step 4: Check Dropped Requests

bash

# Parse logs for any dropped connections during reload
# Caddy's graceful reload should have 0 drops

Rollback on Failure

If deployment fails (validation error, service won't start, endpoints down):

bash

# Automatic rollback process:
1. Restore backup:
   cp Caddyfile.backup.20251229_143500 Caddyfile

2. Reload with backup config:
   systemctl reload caddy-terraphim

3. Verify rollback successful:
   systemctl status caddy-terraphim
   curl -f https://domain.com/health

4. Report failure details to user with logs

Safety Guarantees

✓ Zero-downtime reloads (graceful, no dropped requests) ✓ Always validates before deployment ✓ Always creates backup before changes ✓ Automatic rollback on failure ✓ Post-deployment verification confirms success

Workflow 5: Check Status

Purpose: Check Caddy service health and current configuration.

Usage Pattern

code

User: "Check Caddy status on bigbox"

Assistant Process:
1. Query systemd service status
2. Check process details (PID, uptime, memory)
3. Verify listening ports
4. Check recent log entries
5. Test configured endpoints

Example Output:
Caddy Status on bigbox:

Service: caddy-terraphim
  Status: ✓ active (running)
  PID: 24566
  Uptime: 35 days 14 hours
  Memory: 27.6 MB
  User: caddy

Configuration:
  Active: /home/alex/caddy_terraphim/conf/Caddyfile_auth (324 lines)
  Loaded: 3 hours ago
  Last reload: 2 hours ago (successful)

Listening Ports:
  ✓ :80 (HTTP)
  ✓ :443 (HTTPS)

Recent Activity (last 10 minutes):
  - 245 requests processed
  - 0 errors
  - Average response time: 45ms

Domains Configured:
  ✓ alpha.truthforge.terraphim.cloud (responding)
  ✓ ci.terraphim.cloud (responding)
  ✓ logs.terraphim.cloud (responding)

Health: ✓ All systems operational

⚠️  Note: Service running in tmux, migration to systemd recommended

Implementation

bash

# Service status
systemctl status caddy-terraphim

# Process details
ps aux | grep caddy | grep -v grep

# Listening ports
ss -tulpn | grep caddy

# Recent logs
journalctl -u caddy-terraphim --since "10 minutes ago"

# Test endpoints
curl -sf https://domain.com/health

Integration with 1Password Skill

This skill coordinates with the 1Password Secret Management Skill for secure credential handling.

Secret Detection

When scanning Caddyfiles, this skill detects:

•Environment variable references: {env.VAR_NAME}
•Plaintext secrets in env files
•Basic auth credentials
•API keys and tokens

Then coordinates with 1Password skill (Workflow 1: Detect Secrets) to:

•Identify secret types
•Calculate confidence scores
•Recommend migration to op:// references

Secret Migration

For bigbox migration, this skill will:

•Detect plaintext secrets in caddy_complete.env
•Use 1Password skill (Workflow 2: Generate Template) to create .env.template
•Guide user to create 1Password items
•Update systemd service to use op run --no-masking
•Verify migration successful

See: ~/.claude/skills/1password-secrets.md for detailed secret management workflows.

Quick Reference

Common Commands

bash

# Discovery
"Find all Caddyfiles on bigbox"
"Find Caddyfiles with 'auth' in name on all servers"

# Validation
"Validate /path/to/Caddyfile on bigbox"
"Validate the active Caddyfile before deployment"

# Format
"Format /path/to/Caddyfile on registry"

# Deploy
"Deploy Caddyfile changes to bigbox"

# Status
"Check Caddy status on registry"

Server Names

•local: Local development machine (macOS)
•bigbox: Production server (migration needed)
•registry: Production server (reference implementation)
•all: All servers

File Locations

•Local: /Users/alex/projects/*/infrastructure/Caddyfile*
•bigbox: /home/alex/caddy_terraphim/conf/Caddyfile_auth (active)
•registry: /etc/caddy/Caddyfile (active)

Related Skills

•1Password Secret Management: ~/.claude/skills/1password-secrets.md
•Systemd Management: For service lifecycle operations
•SSH Management: For remote server access

Version History:

•1.0.0 (2025-12-29): Initial release with core workflows (Discovery, Validation, Format, Deploy, Status)

Maintainer: Claude Code License: MIT Documentation: This file Server Configuration: ~/.docs/caddy-skill/servers.json Pattern Library: ~/.docs/caddy-skill/patterns/ (coming soon)

Next Workflows Coming Soon:

•Workflow 6: Build Custom Caddy (xcaddy with plugins)
•Workflow 7: Analyze Logs (JSON parsing, error detection)
•Workflow 8: Extract Snippets (DRY principle, pattern identification)
•Workflow 9: Migrate to Systemd (bigbox migration with 1Password)
•Workflow 10: Pattern Library (reverse proxy, auth, WebSocket, security headers)