Validation & Governance Skill

Purpose

Define the go/no-go gates, data quality checks, staleness windows, IC thresholds, and governance requirements that every pipeline run must satisfy before producing output. This skill encodes Wake Robin's fail-closed philosophy: uncertain or stale data triggers exclusion, not graceful degradation.

Preconditions

• •Pipeline runs MUST have an explicit as_of_date parameter (never datetime.now()).
• •All validation uses Decimal arithmetic where scores are involved.
• •PIT cutoff: source_date <= as_of_date - 1 (standard) or source_date < as_of_date - 2 (strict mode).

Gate 1: Point-in-Time (PIT) Enforcement

Every record must pass PIT admissibility before entering any scoring module. There are no exceptions.

Gate 2: Data Staleness (Phase-Dependent)

Financial Data

Trial Data - Phase 3

Trial Data - Phase 2

Trial Data - Phase 1

Market Data

Short Interest Data (FINRA 2-week lag built in)

13F Holdings Data (45-day SEC filing lag)

SEC_13F_FILING_LAG_DAYS: 45 (built-in constant).

Gate 3: Data Quality Hard Gates

Gate 4: Circuit Breakers

Circuit breakers prevent silent data corruption from propagating through the pipeline.

Gate 5: Input Validation

Gate 6: Score Bounds Validation

All scores must fall within [0, 100]:

Any score outside [0, 100] is a pipeline error. Fail-closed.

Gate 7: Weight Sum Validation

Module 5 component weights must sum to 1.0 within tolerance:

Weights outside tolerance are a configuration error. Fail-closed.

Gate 8: Module Coverage Minimums

Gate 9: Severity System

Severity Levels

SEV3 is a hard gate. The ticker is removed from the rankable universe.

Gate 10: Pipeline Health Status

Run Status Classification:

• •OK: All thresholds met
• •DEGRADED: Optional components below threshold
• •FAIL: Critical catalyst pipeline broken (< 5% with events)

IC Quality Benchmarks

Information Coefficient Thresholds

IC Measurement Constants

Forward Return Horizons

Market Cap Buckets (IC Analysis)

Regime Data Staleness Haircuts

Production Hardening Limits

File Size Limits

Operation Timeouts

Logging Sanitization

Determinism Enforcement

Determinism Rules

1. •Same inputs MUST produce byte-identical outputs
2. •All JSON serialization uses sorted keys
3. •All list operations use deterministic sort keys
4. •Content hashes (SHA256) included in every output for verification
5. •No external API calls during scoring (stdlib only)
6. •All timestamps derived from as_of_date, never from wall clock

Governance Metadata Requirements

Every pipeline output MUST include:

{
  "_governance": {
    "run_id": "<deterministic-hash>",
    "score_version": "<version>",
    "schema_version": "<version>",
    "parameters_hash": "sha256:<hash>",
    "pit_cutoff": "<ISO-date>",
    "as_of_date": "<ISO-date>"
  }
}

Audit Stages

Audit Status Values

Standard Error Codes

Schema Version Support

Enhancement Engine Confidence Thresholds

Pre-Run Checklist

Before executing a pipeline run, verify:

1. •as_of_date is explicitly provided (never derived from wall clock)
2. •All input files exist and are within size limits
3. •PIT cutoff is computed and logged
4. •Schema versions match expected versions
5. •Weight sums are within tolerance
6. •No float arithmetic in scoring paths (only Decimal)
7. •No datetime.now() calls in any module
8. •No random module usage without explicit seed
9. •Audit log writer is initialized
10. •Run ID is deterministically generated

Post-Run Checklist

After a pipeline run completes, verify:

1. •All output scores are within [0, 100]
2. •Governance metadata is present in every output file
3. •Content hashes match recomputed hashes (determinism check)
4. •No SEV3 tickers appear in ranked output
5. •Coverage metrics are logged (per-module and per-signal)
6. •Circuit breaker did not trip silently
7. •Staleness penalties were applied where required
8. •Audit log contains entries for all stages (INIT through FINAL)

Source Files