Academic Paper Review Skill
Overview
This skill provides comprehensive peer review of academic papers for top-tier computer science conferences and journals. It evaluates papers against standard academic criteria including originality, technical quality, experimental validation, clarity, and significance.
Supported Input Formats
- •PDF files: Direct analysis of compiled papers
- •LaTeX files: Analysis of source .tex files
Review Process
Step 1: Gather Review Context
Before starting the review, use AskUserQuestion to gather:
- •Output Language: Ask whether the review should be in Chinese or English
- •Target Venue: Ask for the conference or journal name (e.g., "NeurIPS 2026", "CVPR 2026", "ICML 2026", "TPAMI")
Example interaction:
AskUserQuestion with two questions: 1. "What language should the review be written in?" - Options: "English", "Chinese (中文)" 2. "What is the target conference or journal for this paper?" - Options: "NeurIPS", "CVPR", "ICML", "ICLR", "Other (please specify)"
Step 2: Extract Paper Content
For PDF files:
import pdfplumber
with pdfplumber.open(pdf_path) as pdf:
full_text = ""
for page in pdf.pages:
full_text += page.extract_text() + "\n\n"
For LaTeX files:
# Read main .tex file and any \input or \include files
import re
def read_latex_file(filepath):
with open(filepath, 'r', encoding='utf-8') as f:
content = f.read()
# Find \input{} and \include{} commands
includes = re.findall(r'\\(?:input|include)\{([^}]+)\}', content)
# Read included files recursively
for inc in includes:
if not inc.endswith('.tex'):
inc += '.tex'
inc_path = os.path.join(os.path.dirname(filepath), inc)
if os.path.exists(inc_path):
inc_content = read_latex_file(inc_path)
content = content.replace(f'\\input{{{inc}}}', inc_content)
content = content.replace(f'\\include{{{inc}}}', inc_content)
return content
Step 3: Analyze Paper Structure
Extract and identify key sections:
- •Abstract
- •Introduction
- •Related Work
- •Methodology/Approach
- •Experiments/Results
- •Discussion
- •Conclusion
- •References
Step 4: Conduct Comprehensive Review
Evaluate the paper across these dimensions:
1. Originality & Novelty (Score: X/10)
- •Is the problem new or is the approach novel?
- •How does it differ from existing work?
- •Are the contributions clearly stated?
- •Confidence Level: 1-5 (1=Not familiar, 5=Expert)
2. Technical Quality & Soundness (Score: X/10)
- •Are the methods technically sound?
- •Are claims supported by theory or evidence?
- •Are there any technical errors or gaps?
- •Confidence Level: 1-5
3. Experimental Validation (Score: X/10)
- •Are experiments comprehensive and well-designed?
- •Are baselines appropriate and comparisons fair?
- •Are results statistically significant?
- •Are ablation studies included?
- •Confidence Level: 1-5
4. Clarity & Presentation (Score: X/10)
- •Is the paper well-written and organized?
- •Are concepts explained clearly?
- •Is the notation consistent?
- •Are there grammar or language issues?
- •Confidence Level: 1-5
5. Significance & Impact (Score: X/10)
- •How important is this work to the field?
- •What is the potential impact?
- •Does it open new research directions?
- •Confidence Level: 1-5
6. Reproducibility (Score: X/10)
Check for:
- •Code availability (mentioned or provided)
- •Dataset descriptions and availability
- •Hyperparameter details
- •Experimental setup completeness
- •Computational requirements
- •Confidence Level: 1-5
7. Visual Elements Quality
Evaluate:
- •Figures: clarity, relevance, quality
- •Tables: completeness, formatting, readability
- •Diagrams: accuracy, helpfulness
- •Captions: informativeness
8. Ethics & Limitations
Assess:
- •Discussion of limitations
- •Potential negative societal impacts
- •Ethical considerations
- •Broader impact statement (if required by venue)
Step 5: Generate Overall Assessment
Calculate overall score as weighted average:
- •Originality: 20%
- •Technical Quality: 25%
- •Experimental Validation: 20%
- •Clarity: 15%
- •Significance: 15%
- •Reproducibility: 5%
Provide recommendation:
- •Strong Accept (9-10): Exceptional paper, clear accept
- •Accept (8-8.9): Good paper, should be accepted
- •Weak Accept (7-7.9): Above threshold, leaning accept
- •Borderline (6-6.9): On the fence, could go either way
- •Weak Reject (5-5.9): Below threshold, leaning reject
- •Reject (4-4.9): Clear reject, significant issues
- •Strong Reject (1-3.9): Fundamental flaws, strong reject
Step 6: Write Review Document
Create review-{paper-name}-{timestamp}.md with this structure:
# Paper Review: [Paper Title] **Venue:** [Conference/Journal Name] **Review Date:** [YYYY-MM-DD HH:MM] **Review Language:** [Chinese/English] **Reviewer:** Claude AI Assistant --- ## Overall Recommendation - **Decision:** [Strong Accept / Accept / Weak Accept / Borderline / Weak Reject / Reject / Strong Reject] - **Overall Score:** [X.X/10] - **Confidence:** [5-Expert in this area / 4-Knowledgeable / 3-Familiar / 2-Somewhat familiar / 1-Not familiar] --- ## Summary [2-3 paragraph summary covering: - What the paper is about - Main contributions - Key strengths and weaknesses - Overall assessment] --- ## Strengths - [Strength 1] - [Strength 2] - [Strength 3] - ... --- ## Weaknesses - [Weakness 1] - [Weakness 2] - [Weakness 3] - ... --- ## Detailed Evaluation ### 1. Originality & Novelty **Score:** X/10 | **Confidence:** X/5 [Detailed assessment of novelty, comparison with related work, clarity of contributions] ### 2. Technical Quality & Soundness **Score:** X/10 | **Confidence:** X/5 [Assessment of technical correctness, theoretical foundations, methodology soundness] ### 3. Experimental Validation **Score:** X/10 | **Confidence:** X/5 [Evaluation of experimental design, baselines, results, ablation studies, statistical significance] ### 4. Clarity & Presentation **Score:** X/10 | **Confidence:** X/5 [Assessment of writing quality, organization, notation, figures, readability] ### 5. Significance & Impact **Score:** X/10 | **Confidence:** X/5 [Evaluation of importance, potential impact, relevance to community] ### 6. Reproducibility **Score:** X/10 | **Confidence:** X/5 [Assessment of code availability, dataset details, hyperparameters, experimental setup] ### 7. Visual Elements Quality [Analysis of figures, tables, and diagrams - clarity, completeness, appropriateness] ### 8. Ethics & Limitations [Discussion of ethical considerations, limitations, broader impact, potential negative consequences] --- ## Questions for Authors 1. [Question 1] 2. [Question 2] 3. [Question 3] ... --- ## Suggestions for Rebuttal [Constructive guidance on how authors might address the main concerns raised in this review. Focus on the most critical issues that could change the recommendation if addressed.] --- ## Minor Issues - [Typo/formatting issue 1] - [Reference issue 2] - [Minor suggestion 3] ... --- ## Detailed Comments by Section ### Abstract [Comments on abstract] ### Introduction [Comments on introduction] ### Related Work [Comments on related work section] ### Methodology [Comments on methodology] ### Experiments [Comments on experiments] ### Conclusion [Comments on conclusion] --- **End of Review**
Step 7: Generate Improvement Plan
Create improvements-{paper-name}-{timestamp}.md with categorized TODO items:
# Paper Improvement Plan **Paper:** [Paper Title] **Generated:** [YYYY-MM-DD HH:MM] This document provides a structured improvement plan based on the review. Items are categorized by priority and type. --- ## 🔴 Critical Issues (Must Address) These issues significantly impact the paper's acceptance chances and must be addressed. - [ ] **[Issue 1 Title]** - **Location:** [Section/Page] - **Problem:** [Description of the issue] - **Suggested Fix:** [How to address it] - **Estimated Effort:** [High/Medium/Low] - [ ] **[Issue 2 Title]** - **Location:** [Section/Page] - **Problem:** [Description] - **Suggested Fix:** [Solution] - **Estimated Effort:** [High/Medium/Low] --- ## 🟡 Major Improvements (Strongly Recommended) These improvements would significantly strengthen the paper. - [ ] **[Improvement 1 Title]** - **Location:** [Section/Page] - **Current State:** [What's there now] - **Suggested Enhancement:** [What to add/change] - **Expected Impact:** [How this helps] - **Estimated Effort:** [High/Medium/Low] - [ ] **[Improvement 2 Title]** - **Location:** [Section/Page] - **Current State:** [Description] - **Suggested Enhancement:** [Enhancement] - **Expected Impact:** [Impact] - **Estimated Effort:** [High/Medium/Low] --- ## 🟢 Minor Suggestions (Nice to Have) These are polish items that would improve the paper but aren't critical. - [ ] **[Suggestion 1]** - [Brief description] - [ ] **[Suggestion 2]** - [Brief description] - [ ] **[Suggestion 3]** - [Brief description] --- ## 📝 Writing & Presentation - [ ] **Grammar and Language** - [List specific issues with line numbers if possible] - [ ] **Notation and Terminology** - [Inconsistencies or unclear notation] - [ ] **Figure and Table Improvements** - [Specific suggestions for visual elements] --- ## 🔬 Experimental Enhancements - [ ] **Additional Experiments** - [What experiments to add and why] - [ ] **Baseline Comparisons** - [Missing baselines to include] - [ ] **Ablation Studies** - [What ablations would be valuable] --- ## 📚 Related Work & Citations - [ ] **Missing References** - [Important papers to cite] - [ ] **Related Work Discussion** - [Gaps in related work coverage] --- ## 🎯 Priority Roadmap ### Phase 1: Critical Fixes (Before Resubmission) 1. [Critical issue 1] 2. [Critical issue 2] 3. [Critical issue 3] ### Phase 2: Major Improvements (Strengthen Paper) 1. [Major improvement 1] 2. [Major improvement 2] 3. [Major improvement 3] ### Phase 3: Polish (If Time Permits) 1. [Minor suggestion 1] 2. [Minor suggestion 2] 3. [Minor suggestion 3] --- ## 📊 Estimated Impact on Review Score If all critical issues and major improvements are addressed: - **Current Overall Score:** X.X/10 - **Potential Score After Improvements:** Y.Y/10 - **Expected Recommendation Change:** [Current] → [Potential] --- **End of Improvement Plan**
Review Tone Guidelines
Maintain a constructive and balanced tone throughout:
- •Be Specific: Provide concrete examples and line numbers when possible
- •Be Fair: Acknowledge both strengths and weaknesses
- •Be Constructive: Offer actionable suggestions, not just criticism
- •Be Professional: Maintain respectful and objective language
- •Be Encouraging: Recognize good work and potential
Example Usage
# User invokes the skill /paper-review path/to/paper.pdf # Or with LaTeX /paper-review path/to/paper.tex
Output Files
The skill generates two files in the current directory:
- •
review-{paper-name}-{timestamp}.md- Full review - •
improvements-{paper-name}-{timestamp}.md- Improvement plan
Notes
- •Always read the entire paper before starting the review
- •For LaTeX files, handle multi-file projects by following \input and \include commands
- •Adjust review depth based on paper length (conference papers vs journal papers)
- •Consider venue-specific requirements (e.g., NeurIPS emphasizes reproducibility, CVPR emphasizes experimental results)
- •Be aware of common pitfalls in CS papers: overclaiming, insufficient baselines, unclear notation, missing ablations
Common CS Conference Standards
- •NeurIPS: Strong emphasis on reproducibility, code submission, theoretical soundness
- •CVPR/ICCV/ECCV: Focus on experimental results, visual quality, benchmark performance
- •ICML: Balance of theory and experiments, statistical significance
- •ICLR: Open review process, reproducibility, code availability
- •ACL/EMNLP: Language quality, linguistic analysis, dataset details
- •SIGIR/WWW: Real-world applicability, scalability, user studies
Tips for Effective Reviews
- •Read Abstract and Conclusion First: Get the big picture
- •Skim Figures and Tables: Understand the experimental setup
- •Read Introduction: Understand motivation and contributions
- •Deep Dive into Methods: Check technical soundness
- •Analyze Experiments: Verify claims are supported
- •Check Related Work: Ensure proper positioning
- •Review Writing Quality: Note clarity issues
- •Consider Reproducibility: Check for missing details