Evidence Self-loop (C3/C4 fix → rebind → redraft)

Purpose: make the evidence-first pipeline converge without writing filler prose.

This skill reads the intermediate evidence artifacts (briefs/bindings/packs) and produces an actionable TODO list that answers:

•Which subsections are under-supported?
•Is the problem mapping/coverage (C2) or evidence extraction (C3) or binding/planning (C4)?
•Which skill(s) should be rerun, in what order, to unblock high-quality writing?

Inputs

•outline/subsection_briefs.jsonl
•outline/evidence_bindings.jsonl (expects binding_gaps / binding_rationale if available)
•outline/evidence_drafts.jsonl (expects blocking_missing, comparisons, eval protocol, limitations)
•
Optional (improves routing):
- •outline/evidence_binding_report.md
- •outline/anchor_sheet.jsonl
- •papers/paper_notes.jsonl
- •papers/fulltext_index.jsonl
- •queries.md

Outputs

•output/EVIDENCE_SELFLOOP_TODO.md (report-class; always written)

Self-loop contract (what “fixing evidence” means)

•Prefer fixing upstream evidence, not writing around gaps.
•If an evidence pack has blocking_missing, treat it as a STOP signal: strengthen notes/fulltext/mapping, then regenerate packs.
•If bindings show binding_gaps, treat it as a ROUTING signal: either enrich the evidence bank for the mapped papers, expand mapping coverage, or adjust required_evidence_fields if unrealistic.

Recommended rerun chain (minimal):

•If C3 evidence is thin: pdf-text-extractor → paper-notes → evidence-binder → evidence-draft → anchor-sheet → writer-context-pack
•If C2 coverage is weak: section-mapper → outline-refiner → (then rerun C3/C4 evidence skills)

Workflow (analysis-only)

•Read queries.md (if present)

•Use it only as a soft config hint (evidence_mode / draft_profile); do not override the artifact contract.

•Read outline/subsection_briefs.jsonl

•For each sub_id, capture axes + required_evidence_fields (what evidence types this subsection expects).

•Read outline/evidence_bindings.jsonl

•For each sub_id, surface binding_rationale and binding_gaps (what the binder could/could not cover from the evidence bank).

•(Optional) Read outline/evidence_binding_report.md

•Use it as a human-readable summary; treat it as a view of outline/evidence_bindings.jsonl, not a separate truth source.

•Read outline/evidence_drafts.jsonl

•Surface blocking_missing (STOP signals), and check for missing comparisons / eval protocol / limitations that would force hollow writing.

•(Optional) Read outline/anchor_sheet.jsonl

•Check whether each subsection has at least a few citation-backed anchors (numbers / evaluation / limitations).

•(Optional) Read papers/paper_notes.jsonl and papers/fulltext_index.jsonl

•Use these to route fixes: if evidence is abstract-only and missing eval tokens, prefer enriching notes/fulltext before drafting prose.

What the report contains

•Summary counts: subsections with blocking_missing, with binding_gaps, and common failure reasons.
•Per-subsection TODO: the smallest upstream fix path (skills + artifacts) to make the subsection writeable.

Status semantics (unblock rules)

This skill is the prewrite router for evidence quality. Treat its Status: line as the unblock contract:

•PASS: no blocking_missing and no binding_gaps -> proceed to C5 writing (but still scan non-blocking writability smells: low comparisons/eval/anchors often predict hollow prose).
•OK: no blocking_missing, but some binding_gaps -> you may draft, but expect weaker specificity; prefer fixing gaps first.
•FAIL: missing inputs OR any blocking_missing -> do not write filler prose; fix upstream and rerun C3/C4.

Routing matrix (symptom -> root cause -> upstream fix)

Use this as a semantic routing table (not a script checklist). The goal is to fix the earliest broken intermediate artifact.

Symptom (where you see it)	Likely root cause	Inspect first	Smallest upstream fix chain
`evidence_drafts.blocking_missing: no usable citation keys`	mapped papers lack `bibkey` / bibkeys not in `citations/ref.bib`	`papers/paper_notes.jsonl` (bibkey fields), `citations/ref.bib`	C3 `paper-notes` (ensure bibkeys) -> C4 `citation-verifier` -> rerun `evidence-binder` -> rerun `evidence-draft`
`blocking_missing: title-only evidence`	retrieval/metadata lacks abstracts (or aggressive filtering)	`papers/papers_raw.jsonl` abstracts, `papers/paper_notes.jsonl` evidence_level	C1 `literature-engineer` (enrich metadata) OR C3 `pdf-text-extractor` (fulltext) -> rerun `paper-notes`
`blocking_missing: no evidence snippets extractable`	notes are too thin / evidence bank empty for mapped papers	`papers/evidence_bank.jsonl` (counts), `papers/paper_notes.jsonl`	C3 `paper-notes` (richer extraction; prefer fulltext when possible) -> rerun C4 packs
`blocking_missing: no concrete evaluation tokens`	notes/bank did not extract benchmarks/metrics/budgets	`papers/paper_notes.jsonl` (metrics/benchmarks fields), `outline/anchor_sheet.jsonl`	C3 `paper-notes` (extract eval anchors) -> rerun `anchor-sheet` + `evidence-draft`
`evidence pack comparisons` are sparse (signals: comparisons low)	clusters are not contrastable OR mapping coverage too weak	`outline/subsection_briefs.jsonl` (clusters), `outline/mapping.tsv`	C2 `section-mapper` (coverage) OR C3 `subsection-briefs` (better clusters) -> rerun `evidence-draft`
`bindings.binding_gaps` mentions benchmarks/metrics/protocol	binder cannot find evaluation-tagged evidence for this subsection	`outline/evidence_binding_report.md` (tag mix), `papers/evidence_bank.jsonl` tags	C3 `paper-notes` (tag/evidence extraction) OR C2 expand mapping for that subsection -> rerun `evidence-binder`
`binding_gaps` mentions security/threat model/attacks	mapped set lacks security-focused works or notes lack threat-model detail	`outline/mapping.tsv`, `papers/paper_notes.jsonl`	C2 expand mapping (+ C1 queries if needed) OR C3 enrich notes -> rerun binder/packs
`binding report` looks mechanically uniform across H3 (same mix, low tag variance)	binder selection too recipe-like OR evidence bank tags too coarse	`outline/evidence_binding_report.md` (tag mix), evidence bank tags	tighten `required_evidence_fields` + improve evidence bank tags, then rerun binder; avoid writing around non-specific bindings

Interface with the writer self-loop (avoid writing around evidence)

•If writer-selfloop is FAIL due to missing anchors/comparisons and the corresponding writer pack has pack_warnings, stop and run this evidence self-loop: the section is telling you the pack is not writeable.
•Prefer fixing evidence gaps once, upstream, rather than patching every H3 with generic filler.

What this skill does NOT do

•It does not edit papers/*, outline/*, or sections/*.
•It does not invent new facts/citations.
•It does not "relax" quality by changing thresholds; it routes you to the earliest artifact to fix.

Script

Quick Start

•python .codex/skills/evidence-selfloop/scripts/run.py --workspace workspaces/<ws>

All Options

•--workspace <dir>
•--unit-id <U###> (optional)
•--inputs <semicolon-separated> (optional override)
•--outputs <semicolon-separated> (optional override; default writes output/EVIDENCE_SELFLOOP_TODO.md)
•--checkpoint <C#> (optional)

Examples

•
Generate an evidence TODO list after C4 packs are generated:
- •python .codex/skills/evidence-selfloop/scripts/run.py --workspace workspaces/<ws>