Devil's Advocate
Critically examine a research design, methodology, or assumption with 5-7 specific challenges.
Steps
- •
Understand the target: Read
$ARGUMENTSand any relevant project files (CLAUDE.md, manuscript, estimation code). - •
Generate 5-7 challenges from these categories:
- •Identification: Could the causal claim be undermined by [specific threat]?
- •Specification: What if the functional form is wrong? Alternative specifications?
- •Data: Are there sample selection issues? Measurement error concerns?
- •Methodology: Could a different estimator give different results? Why this method?
- •Assumptions: Which assumptions are most vulnerable? What if they're violated?
- •Robustness: What robustness checks are missing?
- •External validity: Does this generalize beyond the sample?
- •
For each challenge, provide:
- •The question
- •Why it matters
- •Suggested resolution or robustness check
- •Severity (Critical / Major / Minor)
- •
Summary verdict: Strengths, critical changes needed, and suggested improvements.
Notes
- •Be constructive, not destructive.
- •Think like a skeptical referee at a top journal.
- •Focus on the most impactful challenges first.