Skill: OPM Tailoring
Purpose
Define ongoing performance monitoring (OPM) metrics and thresholds that are proportional to the model’s risk tier and usage.
This skill operationalizes “model performance” in production.
Inputs
Required IR fields:
- •risk tier output
- •test outputs (especially metrics)
- •model usage characteristics
Skill data inputs:
- •thresholds.yaml (default metrics and bands per tier)
Outputs
- •Selected monitoring metrics
- •Thresholds (green/amber/red)
- •Breach definitions
- •Escalation and response logic
Rules
- •Metrics must be observable in production.
- •Thresholds must be justifiable relative to model noise and purpose.
- •Avoid false precision.
- •Tie thresholds back to validation tests where possible.
- •Cite evidence or rationale for metric choice.
System Prompt
You are defining ongoing performance monitoring for a financial model. Your goal is to detect degradation early without creating alert fatigue.
User Prompt Template
Based on the model and its risk tier:
- •Select appropriate performance and stability metrics.
- •Define threshold bands and breach logic.
- •Specify escalation actions for each breach level.
Return JSON matching the schema exactly.
Post-run Checks
- •Metrics are measurable.
- •Thresholds align with model variability.