Evidence Auditor (peer review)
Goal: for each claim, either (a) point to the supporting evidence in the manuscript, or (b) write a concrete gap with an actionable fix.
Inputs
- •
output/CLAIMS.md
Outputs
- •
output/MISSING_EVIDENCE.md
Output format (recommended)
For each claim:
- •
Claim: copy the claim text - •
Evidence present: what the paper provides (experiments/theory/citations) - •
Gap / concern: what is missing or weak - •
Minimal fix: the smallest additional evidence that would address the gap - •
Severity:major|minor(optional)
Workflow
- •Iterate claims in
output/CLAIMS.md. - •For empirical claims, check:
- •dataset/task definition is clear
- •baselines are appropriate
- •evaluation protocol is valid
- •ablations/sensitivity analyses exist where needed
- •For conceptual claims, check:
- •definitions are unambiguous
- •assumptions are stated
- •claims do not exceed what is argued
- •Write
output/MISSING_EVIDENCE.mdas a list of claim-by-claim entries.
Definition of Done
- • Every claim from
output/CLAIMS.mdhas an evidence note or a gap item. - • “Fix” items are actionable (what to add, not “more experiments”).
Troubleshooting
Issue: you cannot locate the evidence in the paper
Fix:
- •Mark the claim as “evidence not locatable” and ask for a clearer source pointer (or re-extract claims with better pointers).
Issue: the audit starts proposing new claims
Fix:
- •Stop; only critique what exists in
output/CLAIMS.mdand the manuscript.