AI Health Check
Before you ship an AI feature, it needs to pass 6 checks.
Most AI products fail because PMs skip the basics: no cost model, broken failure UX, terrible data quality. This skill stops you from launching garbage.
Entry Point
When this skill is invoked, start with:
code
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ AI HEALTH CHECK ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Before shipping an AI feature, it needs to pass 6 checks. What AI feature are you preparing to launch? ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Usage
code
/ai-health-check [feature-name]
Examples:
- •
/ai-health-check "AI product recommendations"- Audit specific feature - •
/ai-health-check "email composer AI"- Manual description - •
/ai-health-check --pre-launch- Full checklist for current sprint
What Happens
- •Invokes the ai-implementation-auditor agent
- •Asks hard questions about your AI feature
- •Grades each of 6 dimensions: Ready / Risk / Blocker
- •Tells you if you can ship
The 6 Dimensions
| Dimension | What It Checks |
|---|---|
| Model Selection | Did you try simple approaches first? |
| Data Quality | The thing you're probably ignoring |
| Cost Modeling | Can you afford this at scale? |
| Production Monitoring | How will you know if it breaks? |
| Failure UX | What happens when AI screws up? |
| System Optimization | Are you measuring the right things? |
Verdict Logic
| Condition | Verdict |
|---|---|
| Any Blocker | DON'T SHIP |
| 2+ Risks (no blockers) | NEEDS WORK |
| 0-1 Risks | READY |
Sample Output
code
AI Health Check: Email Composer
Overall Readiness: NEEDS WORK (4/6 dimensions ready)
---
Ready: Model Selection, Production Monitoring, System Optimization
Risk: Data Quality, Failure UX
Blocker: Cost Modeling
VERDICT: DON'T SHIP YET
You have 1 blocker:
- No cost model -> Run /ai-cost-check RIGHT NOW
You have 2 risks:
- Data quality strategy undefined
- Failure UX is broken ("Something went wrong" isn't helpful)
---
What To Do Now:
Option A: Fix everything (RECOMMENDED)
1. Run /ai-cost-check (10 min)
2. Define data quality strategy (2 hours)
3. Build better failure UX (3 hours)
4. Rerun /ai-health-check
Option B: Ship with known risks
1. Fix the blocker only
2. Ship knowing data quality and failure UX are weak
3. Plan to fix in week 1
Common Blockers
Cost Modeling missing:
"You're about to launch with zero idea if this bankrupts you at scale." Run
/ai-cost-checkfirst.
Failure UX broken:
"Something went wrong" tells users nothing. No confidence indicators = users don't know when to trust the AI.
No monitoring plan:
"Launching without monitoring = flying blind."
Philosophy (Chip Huyen)
- •"Most AI failures are UX problems, not technical ones."
- •"Data quality beats tool selection."
- •"Fine-tuning should be your last resort."
- •"The gap between a demo and a product is production engineering."
Related Commands
- •
/ai-cost-check- Detailed cost modeling (run if cost dimension is blocked) - •
/start-evals- Set up quality testing - •
/four-risks- Overall feature risk assessment
Best for: Pre-launch validation of AI features Key insight: "Fine-tuning is the last resort. Data quality beats tool selection. Most AI failures are UX problems."