Fact Sheet Generation

After each fact extraction cycle, the system generates a scored "fact sheet" — a curated top-100 list of facts ranked by category weight and recency. Chat and highlights workers use this instead of raw facts.

Architecture Overview

Fact Extraction Worker (per user, after all conversations)
        │
        ▼
  generateAndSaveFactSheet(userId)
        │
        ├─ getUserFacts()                 ──► all facts from DB
        ├─ getRecentFactExtractions(1200) ──► recent extraction refs
        ├─ computeReferenceScores()       ──► time-decay scoring
        ├─ scoreFacts()                   ──► category weight × ref score
        ├─ assembleFacts()                ──► top 100 with category limits
        └─ saveFactSheet()               ──► fact_sheets table
                                          └─ deleteOldFactSheets() (7-day retention)

Usage:
  fact-extraction-worker (dedup)
        ├─ getLatestFactSheet(userId) → fact IDs for dedup
        ├─ getRecentFacts(userId, 30) → supplement with recent facts
        └─ Merge by ID → existingFactRefs for AI extraction

  chat-worker / highlights-worker (context injection)
        ├─ getLatestFactSheet(userId)
        ├─ Parse facts_json → FactSheetEntry[]
        └─ Pass to injectContextMessages({ facts })

Database Schema

fact_sheets table (src/lib/db/db-schema.ts)

Index: (user_id, created_at DESC)

Types (src/lib/db/db-types.ts)

interface FactSheet { id, user_id, facts_json, dedup_log, fact_count, created_at }
interface FactSheetEntry { id: string; category: string; fact: string; score: number }

The id field references the original fact ID from the facts table. This allows the fact extraction worker to use fact sheet entries directly for deduplication.

Key Files

Scoring Algorithm (src/lib/ai/fact-sheet.ts)

score = categoryWeight × referenceScore

Category Weights (line 16-21)

Time-Decay Points (getDecayPoints(), lines 36-45)

Each fact_extractions record contributes points based on age:

Reference score = sum of all extraction points per fact. A fact mentioned 3 times today = 3 × 10 = 30. One mentioned once two weeks ago = 1 × 1 = 1.

Category Slot Limits (lines 24-29)

Total max: 100 facts per sheet.

Assembly Process (assembleFacts(), lines 170-248)

1. •Group facts by category, sort each by score descending
2. •Take minimum guaranteed facts per category
3. •Fill up to max per category with next highest-scored
4. •Fill remaining slots (up to 100) with highest-scored across all categories
5. •Final sort: category order (core → technical → project → transient), then score descending

AI Dedup — DISABLED

The aiDedupFacts() function exists in code (lines 52-126) but is not called.

Reason: With ~60 facts, Grok reasoning spends ~8800 output tokens to drop just 1 fact, costing ~$0.009 per run. Runs every 30 min per user — adds up with minimal value.

Re-enable when: A cheaper model is available, or we switch to a batch/offline approach (e.g. daily).

Design: Sends all facts (with category labels) to AI for cross-category dedup. Can catch contradictions like "Prefers Brain" vs "Avoids Brain" across categories.

DB Operations (src/lib/db/db-fact-sheets.ts)

Context Injection

Chat worker and highlights worker load the latest fact sheet:

const factSheet = await getLatestFactSheet(userId);
if (factSheet) {
  const entries: FactSheetEntry[] = JSON.parse(factSheet.facts_json);
  facts = entries.map(e => ({ category: e.category, fact: e.fact }));
}

Facts are passed to injectContextMessages() and formatted by buildFactsContextPrompt() in src/lib/ai/system-prompts.ts.

API Routes

GET response: { facts: FactSheetEntry[], created_at, fact_count, dedup_log }

UI — /me Page Fact Sheet Tab

src/app/(main)/me/page.tsx — Third tab alongside Memory and Facts.

• •Lazy-loaded from /api/fact-sheet when tab clicked
• •Shows category badges (.me-fact-badge-{category}) and scores (.me-fact-score)
• •Meta info: fact count and generation timestamp (.me-fact-sheet-meta)
• •Empty state when no sheet exists (.me-empty-state)

E2E Tests (e2e/fact-sheet.spec.ts)

Helper functions: seedFacts(), seedFactExtractions(), getFactSheetFromDb(), seedFactSheet(), seedChatMessages() — all use direct DB access via better-sqlite3.

Common Modifications

Adjusting category weights

Edit CATEGORY_WEIGHTS in src/lib/ai/fact-sheet.ts:16-21.

Changing category slot limits

Edit CATEGORY_LIMITS in src/lib/ai/fact-sheet.ts:24-29.

Changing max facts per sheet

Edit MAX_FACTS in src/lib/ai/fact-sheet.ts:31.

Re-enabling AI dedup

In generateAndSaveFactSheet() (~line 263), replace the disabled block with:

const droppedIds = await aiDedupFacts(allFacts);

Changing retention period

Edit the '-7 days' in deleteOldFactSheets() in src/lib/db/db-fact-sheets.ts.