Test Strategy Skill
Overview
Enterprise-grade test strategy capabilities for risk-based test planning, coverage optimization, and quality gate management.
Input Schema
json
{
"type": "object",
"properties": {
"action": {
"type": "string",
"enum": ["create_strategy", "analyze_coverage", "assess_risk", "optimize", "generate_plan"],
"description": "Strategy action to perform"
},
"project_context": {
"type": "object",
"properties": {
"type": {"type": "string", "enum": ["web", "mobile", "api", "desktop", "embedded"]},
"size": {"type": "string", "enum": ["small", "medium", "large", "enterprise"]},
"methodology": {"type": "string", "enum": ["agile", "waterfall", "hybrid", "devops"]},
"team_size": {"type": "integer", "minimum": 1},
"timeline_weeks": {"type": "integer", "minimum": 1}
}
},
"requirements": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {"type": "string"},
"description": {"type": "string"},
"priority": {"type": "string", "enum": ["critical", "high", "medium", "low"]},
"complexity": {"type": "string", "enum": ["simple", "moderate", "complex"]}
}
}
},
"constraints": {
"type": "object",
"properties": {
"budget": {"type": "string"},
"resources": {"type": "integer"},
"deadline": {"type": "string", "format": "date"},
"automation_target": {"type": "number", "minimum": 0, "maximum": 100}
}
},
"quality_targets": {
"type": "object",
"properties": {
"code_coverage": {"type": "number", "minimum": 0, "maximum": 100},
"defect_escape_rate": {"type": "number", "minimum": 0, "maximum": 100},
"test_pass_rate": {"type": "number", "minimum": 0, "maximum": 100}
}
}
},
"required": ["action"]
}
Output Schema
json
{
"type": "object",
"properties": {
"status": {"type": "string", "enum": ["success", "partial", "failed"]},
"strategy": {
"type": "object",
"properties": {
"scope": {"type": "object"},
"approach": {"type": "string"},
"test_levels": {"type": "array"},
"test_types": {"type": "array"},
"automation_strategy": {"type": "object"},
"resource_allocation": {"type": "object"},
"timeline": {"type": "object"},
"quality_gates": {"type": "array"}
}
},
"risk_assessment": {
"type": "array",
"items": {
"type": "object",
"properties": {
"area": {"type": "string"},
"risk_level": {"type": "string"},
"mitigation": {"type": "string"},
"testing_focus": {"type": "string"}
}
}
},
"coverage_analysis": {
"type": "object",
"properties": {
"requirements_coverage": {"type": "number"},
"code_coverage_estimate": {"type": "number"},
"risk_coverage": {"type": "number"},
"gaps": {"type": "array"}
}
},
"recommendations": {"type": "array", "items": {"type": "string"}}
}
}
Parameter Validation
yaml
project_context.type:
required: false
default: web
validate:
- type: enum
values: [web, mobile, api, desktop, embedded]
project_context.size:
required: false
default: medium
validate:
- type: enum
values: [small, medium, large, enterprise]
quality_targets.code_coverage:
required: false
default: 80
validate:
- type: range
min: 0
max: 100
- type: realistic_check
warn_below: 50
warn_above: 95
constraints.automation_target:
required: false
default: 70
validate:
- type: range
min: 0
max: 100
Error Handling
yaml
retry_config:
strategy: exponential_backoff
max_retries: 3
base_delay_ms: 1000
max_delay_ms: 15000
retryable_errors:
- CALCULATION_TIMEOUT
- INCOMPLETE_ANALYSIS
error_categories:
input_errors:
- MISSING_REQUIREMENTS
- INVALID_CONSTRAINTS
- CONFLICTING_PRIORITIES
recovery: request_clarification
analysis_errors:
- COVERAGE_CALCULATION_FAILED
- RISK_ASSESSMENT_INCOMPLETE
- RESOURCE_ESTIMATION_FAILED
recovery: use_defaults_with_warning
optimization_errors:
- NO_FEASIBLE_SOLUTION
- CONSTRAINTS_TOO_STRICT
- TIMELINE_UNREALISTIC
recovery: suggest_constraint_relaxation
Strategy Templates
Agile Sprint Strategy
yaml
name: agile_sprint_strategy
methodology: agile
sprint_duration: 2_weeks
test_distribution:
unit_tests: 70%
integration_tests: 20%
e2e_tests: 10%
automation_approach:
sprint_1: Setup framework, core flows
sprint_2_onwards: Maintain 80% automation
regression: Fully automated
quality_gates:
- name: Sprint Ready
criteria:
- All stories have acceptance criteria
- Test cases written before development
- name: Development Complete
criteria:
- Unit tests pass (>80% coverage)
- No critical bugs open
- name: Sprint Done
criteria:
- All acceptance tests pass
- Regression suite green
- Performance baseline met
Risk-Based Strategy
yaml
name: risk_based_strategy
approach: focus_on_high_risk
risk_categories:
critical:
testing_effort: 40%
automation: 100%
test_depth: exhaustive
high:
testing_effort: 30%
automation: 90%
test_depth: comprehensive
medium:
testing_effort: 20%
automation: 70%
test_depth: standard
low:
testing_effort: 10%
automation: 50%
test_depth: basic
risk_factors:
- Business impact
- Technical complexity
- Change frequency
- Integration points
- Security sensitivity
Enterprise Strategy
yaml
name: enterprise_strategy
scale: large
testing_pyramid:
unit:
percentage: 60%
responsibility: developers
automation: 100%
integration:
percentage: 25%
responsibility: qa_and_developers
automation: 95%
e2e:
percentage: 10%
responsibility: qa
automation: 80%
exploratory:
percentage: 5%
responsibility: qa
automation: 0%
governance:
quality_gates: mandatory
sign_off: required
metrics_tracking: continuous
compliance: enforced
Coverage Analysis
Requirements Coverage
yaml
formula: (tested_requirements / total_requirements) * 100 categories: full_coverage: >= 95% adequate_coverage: 80-94% partial_coverage: 60-79% insufficient: < 60% tracking: - Requirements traceability matrix - Automated coverage reports - Gap analysis
Risk Coverage
yaml
formula: weighted_average( critical_area_coverage * 4 + high_area_coverage * 3 + medium_area_coverage * 2 + low_area_coverage * 1 ) / total_weight target: critical: 100% high: 95% medium: 80% low: 60%
Code Coverage
yaml
types: - line_coverage - branch_coverage - function_coverage - statement_coverage targets: unit_tests: >= 80% integration_tests: >= 60% overall: >= 75% exclusions: - Generated code - Third-party libraries - Test utilities
Quality Gates
Gate 1: Requirements Ready
yaml
name: requirements_ready stage: pre_development criteria: - [ ] All requirements documented - [ ] Acceptance criteria defined - [ ] Edge cases identified - [ ] Non-functional requirements specified exit_criteria: completion: 100% sign_off: product_owner
Gate 2: Development Complete
yaml
name: development_complete stage: post_development criteria: - [ ] Unit tests passing (>80% coverage) - [ ] Code review completed - [ ] No critical/high bugs open - [ ] Integration tests passing exit_criteria: unit_test_pass_rate: >= 95% code_coverage: >= 80% critical_bugs: 0
Gate 3: Release Ready
yaml
name: release_ready stage: pre_release criteria: - [ ] All test types executed - [ ] Performance benchmarks met - [ ] Security scan passed - [ ] Regression suite green - [ ] UAT sign-off obtained exit_criteria: test_pass_rate: >= 98% defect_escape_rate: < 1% performance: within_sla security: no_critical_vulnerabilities
Troubleshooting
Issue: Coverage Targets Not Met
yaml
symptoms: - Code coverage below threshold - Requirements gaps identified - Risk areas untested diagnosis: 1. Analyze coverage reports 2. Identify untested areas 3. Review test case mapping 4. Check automation gaps solutions: - Add missing test cases - Increase automation coverage - Focus on high-risk areas - Review and update requirements traceability
Issue: Timeline Unrealistic
yaml
symptoms: - Test execution delayed - Resources overallocated - Quality compromised diagnosis: 1. Review resource utilization 2. Analyze bottlenecks 3. Check dependencies 4. Evaluate scope solutions: - Prioritize by risk - Increase automation - Add resources - Negotiate scope reduction - Extend timeline
Issue: High Defect Escape Rate
yaml
symptoms: - Production bugs increasing - Customer complaints - Release rollbacks diagnosis: 1. Analyze escaped defects 2. Review test coverage 3. Check quality gates 4. Evaluate testing depth solutions: - Enhance test coverage for escape patterns - Add missing test types - Strengthen quality gates - Implement shift-left testing - Add exploratory testing
Best Practices
yaml
planning: - Start with risk assessment - Define clear quality gates - Set realistic targets - Plan for contingencies execution: - Automate early and often - Track metrics continuously - Communicate status regularly - Adapt to changes quickly optimization: - Review and refine regularly - Learn from defect escapes - Balance coverage and efficiency - Focus on value delivery
Metrics Dashboard Template
yaml
key_metrics:
- name: Test Execution Rate
formula: executed_tests / planned_tests * 100
target: >= 100%
- name: Test Pass Rate
formula: passed_tests / executed_tests * 100
target: >= 95%
- name: Defect Detection Rate
formula: defects_found / (defects_found + defects_escaped) * 100
target: >= 95%
- name: Automation Rate
formula: automated_tests / total_tests * 100
target: >= 70%
- name: Coverage Score
formula: weighted_average(requirements, code, risk)
target: >= 85%
Logging & Observability
yaml
log_events: - strategy_created - quality_gate_evaluated - coverage_calculated - risk_assessed metrics: - strategy_generation_time - coverage_trend - quality_gate_pass_rate - risk_score_distribution dashboards: - Strategy Overview - Coverage Analysis - Quality Gate Status - Risk Heatmap
Version History
| Version | Date | Changes |
|---|---|---|
| 2.1.0 | 2025-01 | Production-grade with quality gates |
| 2.0.0 | 2024-12 | SASMP v1.3.0 compliance |
| 1.0.0 | 2024-11 | Initial release |