Context Post-Verify Skill
Verifies model response by:
- •Checking if retrieved context was actually used
- •Validating against known invariants
- •Flagging potential hallucinations
- •Emitting machine verdict (pass/fail/warn)
Phase
post_verify - Runs after model response
Priority
10 (runs first in post-verify phase)
Mutations
None (read-only verification)
Verification Checks
- •Context Usage: Checks if response mentions concepts from retrieved items
- •Invariant Validation: Validates response doesn't violate known invariants
- •Hallucination Detection: Flags responses that make unsupported claims
Verdict Values
- •
pass- Response is valid and used context appropriately - •
warn- Response may have issues but is acceptable - •
fail- Response violates invariants or shows clear issues
Implementation
Located at: app/services/chat-gateway/context_engineering/skills/post_verify.py
The skill:
- •Extracts key concepts from retrieved items
- •Checks if response mentions these concepts
- •Validates response against invariants
- •Flags potential hallucinations (claims not supported by context)