Peer Review
Structured framework for reviewing academic manuscripts.
When to Use
- •Reviewing own paper before submission
- •Providing feedback to colleagues
- •Serving as conference/journal reviewer
- •Preparing author response to reviews
- •Self-assessment of draft quality
Review Framework
Overall Assessment First
Before detailed review, form initial impressions:
- •Contribution: What is the main contribution?
- •Significance: How important is this for the field?
- •Novelty: What is new compared to prior work?
- •Soundness: Are the claims supported by evidence?
- •Clarity: Is the paper well-written?
Section-by-Section Review
Title and Abstract
| Criterion | Question | Rating |
|---|---|---|
| Clarity | Does title clearly convey the contribution? | [ ] Pass [ ] Needs work |
| Accuracy | Does title accurately reflect content? | [ ] Pass [ ] Needs work |
| Self-contained | Can abstract be understood alone? | [ ] Pass [ ] Needs work |
| Complete | Does abstract cover problem, method, results? | [ ] Pass [ ] Needs work |
| Concise | Is abstract appropriately brief (150-250 words)? | [ ] Pass [ ] Needs work |
Introduction
| Criterion | Question | Rating |
|---|---|---|
| Motivation | Is the problem motivation clear and compelling? | [ ] Pass [ ] Needs work |
| Gap | Is the research gap clearly identified? | [ ] Pass [ ] Needs work |
| Contributions | Are contributions clearly stated? | [ ] Pass [ ] Needs work |
| Scope | Is the scope appropriate? | [ ] Pass [ ] Needs work |
| Organization | Is paper organization previewed? | [ ] Pass [ ] Needs work |
Related Work
| Criterion | Question | Rating |
|---|---|---|
| Coverage | Are relevant papers cited? | [ ] Pass [ ] Needs work |
| Organization | Is related work logically organized? | [ ] Pass [ ] Needs work |
| Positioning | Is this work positioned vs. prior work? | [ ] Pass [ ] Needs work |
| Fairness | Is prior work fairly characterized? | [ ] Pass [ ] Needs work |
| Recency | Are recent papers included? | [ ] Pass [ ] Needs work |
Method
| Criterion | Question | Rating |
|---|---|---|
| Clarity | Is the method clearly explained? | [ ] Pass [ ] Needs work |
| Reproducibility | Could someone reproduce this? | [ ] Pass [ ] Needs work |
| Justification | Are design choices justified? | [ ] Pass [ ] Needs work |
| Completeness | Are all components described? | [ ] Pass [ ] Needs work |
| Formalism | Is notation clear and consistent? | [ ] Pass [ ] Needs work |
Experiments
| Criterion | Question | Rating |
|---|---|---|
| Setup | Is experimental setup complete? | [ ] Pass [ ] Needs work |
| Baselines | Are baselines appropriate and fair? | [ ] Pass [ ] Needs work |
| Datasets | Are datasets appropriate? | [ ] Pass [ ] Needs work |
| Metrics | Are metrics appropriate and justified? | [ ] Pass [ ] Needs work |
| Significance | Is statistical significance reported? | [ ] Pass [ ] Needs work |
| Analysis | Is there analysis beyond raw numbers? | [ ] Pass [ ] Needs work |
| Ablations | Are ablation studies included? | [ ] Pass [ ] Needs work |
Discussion and Conclusion
| Criterion | Question | Rating |
|---|---|---|
| Limitations | Are limitations honestly discussed? | [ ] Pass [ ] Needs work |
| Implications | Are broader implications considered? | [ ] Pass [ ] Needs work |
| Future work | Is future work mentioned? | [ ] Pass [ ] Needs work |
| Summary | Does conclusion summarize key findings? | [ ] Pass [ ] Needs work |
Presentation
| Criterion | Question | Rating |
|---|---|---|
| Writing | Is the writing clear and grammatical? | [ ] Pass [ ] Needs work |
| Figures | Are figures clear and informative? | [ ] Pass [ ] Needs work |
| Tables | Are tables well-formatted? | [ ] Pass [ ] Needs work |
| Captions | Are captions self-contained? | [ ] Pass [ ] Needs work |
| Length | Is paper appropriate length? | [ ] Pass [ ] Needs work |
Review Template
Summary
[2-3 sentences summarizing the paper's contribution]
Strengths
- •[Strength 1]
- •[Strength 2]
- •[Strength 3]
Weaknesses
- •[Weakness 1]
- •[Weakness 2]
- •[Weakness 3]
Detailed Comments
Major Issues
[Issues that must be addressed for acceptance]
- •[Issue title]
- •Location: [Section/page]
- •Problem: [Description]
- •Suggestion: [How to address]
Minor Issues
[Issues that would improve the paper but aren't critical]
- •[Issue and suggestion]
- •[Issue and suggestion]
Questions for Authors
- •[Question 1]
- •[Question 2]
Recommendation
[ ] Accept [ ] Minor revisions [ ] Major revisions [ ] Reject
Confidence
[ ] High - I am an expert in this area [ ] Medium - I am familiar with this area [ ] Low - This is outside my expertise
Constructive Feedback Principles
Do
- •Be specific about problems
- •Suggest solutions when possible
- •Acknowledge strengths
- •Separate major from minor issues
- •Explain reasoning
- •Be professional and respectful
Don't
- •Be vague ("this is unclear")
- •Be dismissive
- •Focus only on negatives
- •Request unreasonable changes
- •Let personal preferences bias review
- •Be unprofessional
Phrasing Guide
| Instead of... | Try... |
|---|---|
| "This is wrong" | "I believe this may be incorrect because..." |
| "Poorly written" | "The writing could be clearer in section X" |
| "Missing citations" | "Consider citing [specific area/papers]" |
| "Not novel" | "The novelty over [prior work] isn't clear" |
| "Weak experiments" | "The experimental evaluation could be strengthened by..." |
Quality Checklist
Before finalizing review:
- • Summary accurately reflects paper
- • All major issues identified
- • Issues are specific and actionable
- • Suggestions for improvement provided
- • Strengths acknowledged
- • Tone is constructive and professional
- • Recommendation matches the review content
- • No confidential information disclosed
References
See references/ folder for:
- •
review_checklist.md: Detailed checklist by paper type