Claim Extraction Skill

Extract all substantive claims from AI research content. A claim is any assertion that:

• •States something as true about AI capabilities, limitations, or progress
• •Predicts future developments
• •Hints at unreleased work
• •Expresses a positioned opinion on the field's direction
• •Critiques others' claims or work

Extraction Schema

For each claim, extract:

1. claimText

The claim in clear, standalone form. Paraphrase if needed for clarity.

2. claimType

• •fact: Assertion about current state ("GPT-4 can do X")
• •prediction: Forward-looking ("By 2026, we'll have...")
• •hint: Implies unreleased work ("We've been seeing interesting results with...")
• •opinion: Positioned take ("I think scaling is/isn't sufficient")
• •critique: Challenges others ("Marcus is wrong because...")
• •question: Genuine uncertainty expressed ("I'm not sure if...")

3. topic

Primary topic category:

• •scaling: Scaling laws, compute, training efficiency
• •reasoning: LLM reasoning, chain-of-thought, planning
• •agents: AI agents, tool use, autonomy
• •safety: AI safety, alignment, control
• •interpretability: Mechanistic interpretability
• •multimodal: Vision, audio, video models
• •rlhf: RLHF, preference learning, Constitutional AI
• •benchmarks: Evals, benchmarks, capability measurement
• •infrastructure: Training infra, chips, hardware
• •policy: AI policy, regulation, governance
• •general: General AI commentary

4. stance

• •bullish: Optimistic about AI progress/capabilities
• •bearish: Skeptical/pessimistic about AI progress
• •neutral: Balanced or factual without clear stance

5. bullishness

Float from 0.0 (maximally bearish) to 1.0 (maximally bullish)

6. confidence

How confident does the author seem? (0.0-1.0)

• •Hedging language: "might", "could", "I think", "possibly" → lower
• •Certainty language: "will", "definitely", "it's clear that" → higher

7. timeframe (for predictions)

• •near-term: < 1 year
• •medium-term: 1-3 years
• •long-term: 3-10 years
• •unspecified: No clear timeframe
• •null: Not a prediction

8. evidenceProvided

• •strong: Cites data, papers, or detailed reasoning
• •moderate: Some reasoning but not rigorous
• •weak: Assertion without support
• •appeal-to-authority: "Trust me, I work on this"

9. quoteworthiness

Is this claim notable enough to quote in a digest? (0.0-1.0)

Output Format

Return JSON:

{
  "claims": [
    {
      "claimText": "The claim in clear form",
      "claimType": "prediction",
      "topic": "reasoning",
      "stance": "bullish",
      "bullishness": 0.8,
      "confidence": 0.7,
      "timeframe": "medium-term",
      "evidenceProvided": "moderate",
      "quoteworthiness": 0.6,
      "relatedTo": ["o1", "chain-of-thought"],
      "originalQuote": "Brief relevant quote if notable"
    }
  ]
}

Guidelines

• •Extract MULTIPLE claims from a single piece of content if present
• •Don't over-extract - only substantive, meaningful claims
• •A tweet saying "Interesting paper" is NOT a claim
• •Look for IMPLICIT claims ("We've made a lot of progress" implies capability gains)
• •Pay attention to WHO is speaking - lab researchers hinting at their own work is high signal
• •Critics often make claims by contradiction ("X is wrong, therefore Y")

Author Context Matters

Consider the author's affiliation when assessing:

• •Lab researchers (Anthropic, OpenAI, DeepMind): May hint at unreleased work
• •Critics (Marcus, Chollet, Mitchell): Often make claims through critique
• •Independent (Simon Willison, Jim Fan): Provide practitioner perspectives