Container Boundary Testing Skill

This skill provides guidance for writing and fixing container boundary tests - tests that validate how containers handle their input interfaces (HTTP endpoints, queue consumers, webhooks, file triggers).

CRITICAL: Test File Format Standardization

ALL boundary tests MUST use Shell Script (.test.sh) format.

This ensures:

•Direct curl commands against running containers
•No additional runtime dependencies (Node/Python not required on host)
•Simple, portable test execution
•Clear pass/fail output with exit codes
•Compatible with CI/CD pipelines

Test File Location

code

{container}/tests/container-inputs/
├── {input-slug}.input.md           # Input definition (analysis)
├── {input-slug}.test.sh            # Executable test (Shell ONLY)
└── fixtures/
    └── {input-slug}-*.json         # Test fixtures

Test File Naming Convention

Input Type	Pattern	Example
HTTP API	`http-api-{path-slug}.test.sh`	`http-api-users-create.test.sh`
HTTP Webhook	`http-webhook-{name}.test.sh`	`http-webhook-payments.test.sh`
Queue Consumer	`queue-{queue-name}.test.sh`	`queue-job-processing.test.sh`
Health Check	`health-{name}.test.sh`	`health-endpoints.test.sh`

Standard Test File Template

All boundary tests MUST follow this template:

bash

#!/bin/bash
# =============================================================================
# Boundary Test: {Input Name}
#
# Tests the {path/queue/event} input handling including:
# - Happy path scenarios
# - Validation error scenarios
# - Authentication scenarios (if applicable)
#
# @see {input-slug}.input.md for full behavior analysis
# =============================================================================

set -e

# =============================================================================
# Configuration
# =============================================================================

BASE_URL="${API_BASE_URL:-http://localhost:{port}}"
ENDPOINT="${BASE_URL}/{path}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
FIXTURES_DIR="${SCRIPT_DIR}/fixtures"

# Counters
PASS=0
FAIL=0
SKIP=0

# =============================================================================
# Helper Functions
# =============================================================================

log_pass() {
    echo -e "\033[32m[PASS]\033[0m $1"
    ((PASS++))
}

log_fail() {
    echo -e "\033[31m[FAIL]\033[0m $1"
    ((FAIL++))
}

log_skip() {
    echo -e "\033[33m[SKIP]\033[0m $1"
    ((SKIP++))
}

log_info() {
    echo -e "\033[34m[INFO]\033[0m $1"
}

# Generate mock JWT token for LOCAL_RUN mode
generate_mock_token() {
    local user_id="${1:-testuser$(date +%s)}"
    local header=$(echo -n '{"alg":"HS256","typ":"JWT"}' | base64 -w0 | tr '/+' '_-' | tr -d '=')
    local payload=$(echo -n "{\"iss\":\"local-dev\",\"sub\":\"usr_${user_id}\",\"aud\":\"mock-client-id\",\"exp\":$(($(date +%s) + 3600)),\"iat\":$(date +%s),\"name\":\"Test User\",\"email\":\"${user_id}@example.com\",\"tfp\":\"B2C_1_SignIn\"}" | base64 -w0 | tr '/+' '_-' | tr -d '=')
    echo "${header}.${payload}.mocksignature"
}

# Check if container is running
check_container_running() {
    local container_name="$1"
    if docker ps --filter "name=${container_name}" --format "{{.Status}}" | grep -q "Up"; then
        return 0
    else
        return 1
    fi
}

# Make HTTP request and capture response
make_request() {
    local method="$1"
    local url="$2"
    local data="$3"
    local auth_header="$4"

    local curl_args=(-s -w "\n%{http_code}" -X "$method")

    if [ -n "$data" ]; then
        curl_args+=(-H "Content-Type: application/json" -d "$data")
    fi

    if [ -n "$auth_header" ]; then
        curl_args+=(-H "Authorization: $auth_header")
    fi

    curl "${curl_args[@]}" "$url"
}

# Parse response body and status code
parse_response() {
    local response="$1"
    RESPONSE_BODY=$(echo "$response" | head -n -1)
    RESPONSE_CODE=$(echo "$response" | tail -n 1)
}

# =============================================================================
# Pre-flight Checks
# =============================================================================

echo "=========================================="
echo "Boundary Test: {Input Name}"
echo "Endpoint: $ENDPOINT"
echo "=========================================="
echo ""

# Check container is running
if ! check_container_running "local-{container}"; then
    echo "ERROR: {container} container is not running."
    echo "Start with: ./scripts/local-start-all.sh"
    exit 1
fi

# Generate auth token
TOKEN=$(generate_mock_token)

# =============================================================================
# Happy Path Tests
# =============================================================================

echo "--- Happy Path Tests ---"
echo ""

# HP-01: Valid input processing
log_info "HP-01: Testing valid input..."
RESPONSE=$(make_request "POST" "$ENDPOINT" \
    '{"field1": "value1", "field2": "value2"}' \
    "Bearer $TOKEN")
parse_response "$RESPONSE"

if [ "$RESPONSE_CODE" = "201" ] || [ "$RESPONSE_CODE" = "200" ]; then
    if echo "$RESPONSE_BODY" | grep -q '"id"'; then
        log_pass "HP-01: Valid input accepted (HTTP $RESPONSE_CODE)"
    else
        log_fail "HP-01: Response missing 'id' field"
    fi
else
    log_fail "HP-01: Expected 200/201, got HTTP $RESPONSE_CODE"
    echo "  Response: $RESPONSE_BODY"
fi

# =============================================================================
# Validation Error Tests
# =============================================================================

echo ""
echo "--- Validation Error Tests ---"
echo ""

# VAL-01: Missing required field
log_info "VAL-01: Testing missing required field..."
RESPONSE=$(make_request "POST" "$ENDPOINT" \
    '{"field2": "value2"}' \
    "Bearer $TOKEN")
parse_response "$RESPONSE"

if [ "$RESPONSE_CODE" = "400" ]; then
    log_pass "VAL-01: Missing field rejected (HTTP 400)"
else
    log_fail "VAL-01: Expected 400, got HTTP $RESPONSE_CODE"
    echo "  Response: $RESPONSE_BODY"
fi

# VAL-02: Invalid format
log_info "VAL-02: Testing invalid format..."
RESPONSE=$(make_request "POST" "$ENDPOINT" \
    '{"field1": "", "field2": "value2"}' \
    "Bearer $TOKEN")
parse_response "$RESPONSE"

if [ "$RESPONSE_CODE" = "400" ]; then
    log_pass "VAL-02: Invalid format rejected (HTTP 400)"
else
    log_fail "VAL-02: Expected 400, got HTTP $RESPONSE_CODE"
    echo "  Response: $RESPONSE_BODY"
fi

# =============================================================================
# Authentication Tests
# =============================================================================

echo ""
echo "--- Authentication Tests ---"
echo ""

# AUTH-01: Missing token
log_info "AUTH-01: Testing missing authentication..."
RESPONSE=$(make_request "POST" "$ENDPOINT" \
    '{"field1": "value1", "field2": "value2"}' \
    "")
parse_response "$RESPONSE"

if [ "$RESPONSE_CODE" = "401" ]; then
    log_pass "AUTH-01: Missing token rejected (HTTP 401)"
else
    log_fail "AUTH-01: Expected 401, got HTTP $RESPONSE_CODE"
    echo "  Response: $RESPONSE_BODY"
fi

# =============================================================================
# Summary
# =============================================================================

echo ""
echo "=========================================="
echo "Results: $PASS passed, $FAIL failed, $SKIP skipped"
echo "=========================================="

if [ "$FAIL" -gt 0 ]; then
    exit 1
fi
exit 0

Docker Usage Patterns

Checking Container Status

bash

# Check if containers are running
docker-compose -f docker-compose.local.yml ps

# Check specific container
docker ps --filter "name=local-{container}" --format "{{.Status}}"

Viewing Container Logs

bash

# View recent logs
docker-compose -f docker-compose.local.yml logs --tail 100 {container-name}

# Follow logs in real-time
docker-compose -f docker-compose.local.yml logs -f {container-name}

# Search for errors
docker-compose -f docker-compose.local.yml logs {container-name} 2>&1 | grep -i error

Rebuild and Restart Procedure

CRITICAL: After making code changes, you MUST rebuild and restart containers.

bash

# Step 1: Stop all containers
./scripts/local-stop-all.sh
# Windows: .\scripts\local-stop-all.ps1

# Step 2: Rebuild specific container (if changes were made to it)
docker-compose -f docker-compose.local.yml build {container-name}

# Step 3: Rebuild ALL containers (if changes affect multiple)
docker-compose -f docker-compose.local.yml build --no-cache

# Step 4: Start all containers
./scripts/local-start-all.sh
# Windows: .\scripts\local-start-all.ps1

# Step 5: Verify containers are healthy
docker-compose -f docker-compose.local.yml ps

Quick Rebuild Single Container

bash

# Rebuild and restart a single container
docker-compose -f docker-compose.local.yml build {container-name} && \
docker-compose -f docker-compose.local.yml up -d {container-name}

Running Tests

Execute Boundary Tests

Tests are run from the host machine against running Docker containers:

bash

# Run all boundary tests for a container
cd {container}/tests/container-inputs/
for test in *.test.sh; do bash "$test"; done

# Run specific test file
bash {container}/tests/container-inputs/{input-slug}.test.sh

# Run with verbose output (shows all curl responses)
bash -x {container}/tests/container-inputs/{input-slug}.test.sh

Environment Variables

Tests support these environment variables:

bash

# Set API base URL (optional, defaults to localhost)
export API_BASE_URL=http://localhost:3000

# Run test with custom URL
API_BASE_URL=http://localhost:8080 bash http-api-{input-slug}.test.sh

Test Fixture Guidelines

Fixture Location

code

{container}/tests/container-inputs/fixtures/
├── {input-slug}-valid-payload.json       # Valid request body
├── {input-slug}-invalid-payload.json     # Invalid for testing validation
├── {input-slug}-test-video.mp4           # Media fixtures (if needed)
└── {input-slug}-test-image.jpg           # Image fixtures (if needed)

Fixture Naming Convention

Purpose	Pattern	Example
Valid payload	`{slug}-valid-payload.json`	`users-create-valid-payload.json`
Invalid payload	`{slug}-invalid-{reason}.json`	`users-create-invalid-email.json`
Test media	`{slug}-test-{type}.{ext}`	`files-upload-test-image.jpg`

Using Fixtures in Tests

bash

# Load fixture from file
PAYLOAD=$(cat "${FIXTURES_DIR}/{slug}-valid-payload.json")

# Use in curl
RESPONSE=$(make_request "POST" "$ENDPOINT" "$PAYLOAD" "Bearer $TOKEN")

Creating Media Fixtures

Generate test video (5 seconds, 720p):

bash

ffmpeg -f lavfi -i testsrc=duration=5:size=1280x720:rate=30 \
  -f lavfi -i sine=frequency=1000:duration=5 \
  -c:v libx264 -c:a aac \
  "{container}/tests/container-inputs/fixtures/{slug}-test-video.mp4"

Generate test image:

bash

convert -size 1920x1080 xc:skyblue \
  -fill red -draw "circle 960,540 1010,540" \
  "{container}/tests/container-inputs/fixtures/{slug}-test-image.jpg"

Fix Workflow

When fixing failing boundary tests, follow this procedure:

Step 1: Identify the Failure

•Run the test and note which assertion failed
•Check if it's a test issue or an implementation issue
•Read the .input.md file for expected behavior

Step 2: Check Container Logs

bash

# Check for runtime errors
docker-compose -f docker-compose.local.yml logs --tail 100 {container-name} 2>&1 | grep -i error

Step 3: Make Code Fix

Fix the implementation in {container}/src/ to match expected behavior.

Step 4: Rebuild and Restart

CRITICAL: Changes are NOT applied until container is rebuilt.

bash

# Full rebuild and restart
./scripts/local-stop-all.sh && \
docker-compose -f docker-compose.local.yml build {container-name} && \
./scripts/local-start-all.sh

Step 5: Verify Fix

bash

# Wait for containers to be healthy
sleep 10

# Check container status
docker-compose -f docker-compose.local.yml ps

# Re-run the failing test
bash {container}/tests/container-inputs/{input-slug}.test.sh

Step 6: Document Fix

Update the .input.md file with test results:

markdown

## Test Results

**Test Date**: {timestamp}
**Status**: PASS

### Fixes Applied

| Issue | Resolution |
|-------|------------|
| {description} | {what was fixed} |

Common Issues and Fixes

Container Not Responding

Symptom: Connection refused or curl: (7) Failed to connect

Fix:

•Check container is running: docker-compose -f docker-compose.local.yml ps
•Check container logs for startup errors
•Restart containers: ./scripts/local-start-all.sh

Test Timeout

Symptom: curl hangs or times out

Fix:

•Add timeout to curl: curl --connect-timeout 5 --max-time 30 ...
•Check if container is healthy
•Check for slow operations in container logs

Validation Returns 500 Instead of 400

Symptom: Invalid input returns 500 instead of 400

Fix:

•Check error handling in route handler
•Ensure validation errors are caught and return 400
•Add try-catch around validation logic

Authentication Bypass Not Working

Symptom: 401 errors in LOCAL_RUN mode

Fix:

•Verify LOCAL_RUN=true in docker-compose.local.yml
•Check auth middleware uses LOCAL_RUN check
•Rebuild container to apply environment changes

JSON Parsing Errors in Bash

Symptom: Cannot extract fields from JSON response

Fix:

•Use jq for JSON parsing: echo "$RESPONSE_BODY" | jq -r '.id'
•Or use grep with regex: echo "$RESPONSE_BODY" | grep -o '"id":"[^"]*"'

Quality Checklist

Before Creating Test

• Read the .input.md file for behavior analysis
• Identify all test scenarios from behavior analysis
• Check fixtures exist or create them
• Test is executable: chmod +x {test}.test.sh

Before Submitting Fix

• Container was rebuilt after code changes
• Container shows "healthy" status
• Test passes consistently (run 3 times)
• No console errors in container logs
• .input.md updated with test results

Anti-Patterns to Avoid

Anti-Pattern	Problem	Solution
TypeScript/Python tests	Requires runtime setup	Use Shell `.test.sh` only
Running inside container	Wrong context	Run tests from host with curl
Skipping rebuild	Changes not applied	Always rebuild after code changes
Testing without health check	Container may be unhealthy	Verify container health first
Hardcoded ports	Breaks with config changes	Use environment variables
No exit code	CI can't detect failures	Exit 1 on any failure
Silent failures	Hard to debug	Log clear PASS/FAIL messages