Consuelo Test Runner

Test deployed changes against the original requirements. This skill runs after a successful Railway deploy to verify that the changes actually work.

When to Use This Skill

• •After a successful Railway deploy (triggered by pr-watcher or heartbeat)
• •When asked to test a specific feature on the live app
• •During heartbeat checks if there are recently-deployed PRs that haven't been tested

What You Need

• •PR number or Linear task ID — to know what was changed and what to test
• •Railway URL — the deployed Consuelo app (usually from Railway dashboard or env)

Testing Flow

Step 1: Gather Requirements

From Linear task (if available):

# Fetch the Linear task spec
python3 /Users/kokayi/Dev/claude-agent-workflow/.opencode/skills/github-dev/fetch_task.py CON-456
# Read the spec
cat /Users/kokayi/Dev/claude-agent-workflow/.opencode/skills/github-dev/tasks/CON-456.md

From PR description:

gh pr view <number> --json body,title,files --repo kokayicobb/consuelo_on_call_coaching

Step 2: Extract Verification Criteria

Look for these in the Linear task spec or PR body:

• •## Verification Criteria section with checkboxes
• •## Acceptance Criteria section
• •## What to Test section
• •Any description of expected behavior

If no explicit criteria exist, derive them from:

• •The PR title and description (what was the intent?)
• •The changed files (what areas were modified?)
• •The commit messages (what was implemented?)

Step 3: Run Tests

For each verification criterion, test it against the deployed app:

API Tests (curl/httpie):

# Test an API endpoint
curl -s https://your-railway-url.up.railway.app/api/endpoint | python3 -m json.tool

# Test with authentication
curl -s -H "Authorization: Bearer $TOKEN" https://your-railway-url.up.railway.app/api/protected

UI Tests (agent-browser if available): If the changes involve UI:

1. •Navigate to the relevant page
2. •Verify the UI elements are present
3. •Test user interactions (clicks, form submissions)
4. •Check error states

Health Check:

# Always start with a basic health check
curl -s -o /dev/null -w "%{http_code}" https://your-railway-url.up.railway.app/

Step 4: Report Results

Post test results as a PR comment:

## Test Results — PR #123

**Deployed URL:** https://...
**Tested at:** 2026-02-07 15:30 UTC

### Criteria Results

| Criterion | Result | Notes |
|-----------|--------|-------|
| User can log in | PASS | Tested with test credentials |
| Error on invalid login | PASS | Shows appropriate error message |
| Dashboard loads | FAIL | 500 error on /dashboard |

### Summary
2/3 criteria passed. Dashboard endpoint returning 500.

### Recommendation
Fix the dashboard endpoint before marking as complete.

Step 5: Update Linear Task

If all tests pass:

# Move task to Done (via linear_helper or direct API)
python3 /Users/kokayi/Dev/claude-agent-workflow/.opencode/skills/kiro/linear_helper.py label CON-456 --add tested

If tests fail:

• •Post the failure details to the PR
• •Send Slack notification
• •Leave the task in "In Review" state

Test Categories

Critical (Must Pass)

• •App starts without errors (health check returns 200)
• •Core user flow works (the main feature being tested)
• •No console errors in the response

Important (Should Pass)

• •Error handling works (invalid inputs, edge cases)
• •Response times are reasonable (<5s for API calls)
• •No regressions in existing functionality

Nice to Have

• •UI looks correct (screenshots if agent-browser available)
• •Responsive design works
• •Edge cases handled gracefully

Important Notes

• •Don't test destructive operations on production data
• •Use test/staging credentials if authentication is needed
• •Report partial results — even if some tests fail, report what passed
• •This skill complements the github-review skill's --test flag — the review skill tests during review, this skill tests after deploy
• •Railway URL should be discoverable from Railway CLI or environment config