Validate Documentation

Validate that ALL features claimed in README are documented in specs and tested.

Philosophy

The README is the product's promise to users. Every feature advertised must be:

1. •Specified in docs/specs/ (source of truth for behavior)
2. •Tested in frontend/tests/ (proof it works)

Workflow

Step 1: Extract ALL features from README

Read README.md and identify every feature claim. Look in:

• •How It Works section - core functionality
• •UI section - user-facing features
• •Agent Workflow section - automation features
• •Any diagrams, bullet points, or descriptions that promise functionality

For each feature, note:

• •What it claims to do
• •Whether it has a linked spec (some won't)

Step 2: Map features to specs

Check if each README feature has a corresponding spec:

Flag any README feature without a spec as ERROR.

Step 3: Verify specs have testable assertions

For each spec file:

• •Check it contains specific, testable statements
• •Assertions should mention exact UI text, behaviors, or states
• •Vague specs like "works well" are not testable

Step 4: Map spec assertions to tests

For each bullet point/assertion in a spec, search for a test that verifies it:

# Example: search for test covering "No sessions yet" message
grep -r "No sessions yet" frontend/tests/

Track coverage for each spec assertion.

Step 5: Output report

## Documentation Validation Report

### README Features → Specs

- [x] Main thread conversation → docs/specs/chat.md
- [x] Sessions → docs/specs/sessions.md
- [x] Layout → docs/specs/layout.md
- [ ] **Agent Workflow → NO SPEC** ← ERROR

### Spec Assertions → Tests

#### docs/specs/chat.md (7/9 = 78%)

- [x] "Input field with placeholder" → e2e/user-journey.spec.ts
- [ ] "Messages persist across reloads" → NO TEST

#### docs/specs/sessions.md (23/23 = 100%)

- [x] "No sessions yet" → sessions/01-session-list-empty.spec.ts
      ...

### Summary

| Category                   | Coverage    |
| -------------------------- | ----------- |
| README features with specs | 3/4 (75%)   |
| Spec assertions with tests | 38/40 (95%) |

### Errors

1. README "Agent Workflow" section has no spec
2. chat.md "Messages persist across reloads" has no test

Step 6: Exit status

• •ERROR if any README feature lacks a spec
• •ERROR if any spec assertion lacks test coverage
• •SUCCESS only if fully covered

Common Gaps to Watch For

1. •Advertised but unspecified - README promises feature, no spec exists
2. •Specified but untested - Spec describes behavior, no test verifies it
3. •Workflow features - Multi-step flows (like agent lifecycle) often lack e2e coverage

Tests to Flag for Removal

Not all tests are good tests. Flag these as problems:

1. •

   Tests without specs - If a test exists but no spec describes the behavior, either:

   • •The spec is missing (add it), or
   • •The test is testing implementation details (remove it)
2. •

   Too technical / low-level - Tests should verify user-facing behavior, not internals:

   • •❌ "store updates when API returns data"
   • •❌ "component re-renders on state change"
   • •✅ "user sees updated session status after refresh"
3. •

   Testing implementation, not behavior - Tests coupled to code structure:

   • •❌ "calls fetchSessions with correct params"
   • •❌ "dispatches ACTION_TYPE to store"
   • •✅ "sessions list shows new session after spawning"
4. •

   Duplicate coverage - Multiple tests verifying the same user behavior

5. •

   Tests for removed features - Spec was removed but test remains

The test suite should read like a user manual, not a code audit.