Vulnerability Validation

Validate security findings by assessing whether they are actually exploitable in the context of this codebase. This skill filters false positives, confirms real vulnerabilities, and generates proof-of-concept exploits.

When to Use This Skill

• •After commit-security-scan - Validate findings before creating issues or blocking PRs
• •HIGH/CRITICAL findings - Prioritize validation of severe findings
• •Before patching - Confirm vulnerability is real before investing in fixes
• •Security review - Deep-dive validation of specific findings

Prerequisites

• •.factory/threat-model.md must exist (from threat-model-generation skill)
• •security-findings.json must exist (from commit-security-scan skill)

Inputs

Instructions

Follow these steps for each finding to validate:

Step 1: Load Context

1. •Read security-findings.json from commit-security-scan
2. •Read .factory/threat-model.md for system context
3. •Identify which findings to validate based on inputs

Step 2: Reachability Analysis

For each finding, determine if the vulnerable code is reachable:

1. •

   Trace entry points

   • •Can external users reach this code path?
   • •What HTTP endpoints, CLI commands, or event handlers lead here?
   • •Is authentication required to reach this code?
2. •

   Map the call chain

   • •Starting from the entry point, trace the path to the vulnerable code
   • •Document each function call in the chain
   • •Note any branching conditions that must be satisfied
3. •

   Classify reachability

   • •EXTERNAL - Reachable from unauthenticated external input
   • •AUTHENTICATED - Requires valid user session
   • •INTERNAL - Only reachable from internal services
   • •UNREACHABLE - Dead code or blocked by conditions

Step 3: Control Flow Analysis

Determine if an attacker can control the vulnerable input:

1. •

   Identify the source

   • •Where does the tainted data originate?
   • •HTTP parameter, file upload, database query, environment variable?
2. •

   Trace data flow

   • •Follow the data from source to sink (vulnerable function)
   • •Document each transformation or validation step
   • •Note any sanitization, encoding, or type conversion
3. •

   Assess attacker control

   • •Can the attacker fully control the input?
   • •Are there length limits, character restrictions, or format validation?
   • •Does the data pass through any sanitization?

Step 4: Mitigation Assessment

Check if existing security controls prevent exploitation:

1. •

   Input validation

   • •Is the input validated before reaching the vulnerable code?
   • •What validation rules are applied?
2. •

   Framework protections

   • •Does the framework provide automatic protection? (e.g., ORM parameterization, React XSS escaping)
   • •Is the protection enabled and properly configured?
3. •

   Security middleware

   • •Are there WAF rules, rate limiting, or other controls?
   • •Do CSP headers or other browser protections apply?
4. •

   Reference threat model

   • •Check the "Existing Mitigations" section for this threat type
   • •Verify mitigations are actually in place

Step 5: Exploitability Assessment

Determine how difficult it is to exploit:

Consider:

• •Attack complexity
• •Required privileges
• •User interaction needed
• •Scope of impact

Step 6: Generate Proof-of-Concept

For confirmed vulnerabilities, create a proof-of-concept:

1. •

   Craft exploit payload

   • •Create a minimal payload that demonstrates the vulnerability
   • •Use benign payloads (no actual damage)
2. •

   Document the request

   • •HTTP method, URL, headers, body
   • •Or CLI command, file input, etc.
3. •

   Describe expected vs actual behavior

   • •What should happen (secure behavior)
   • •What actually happens (vulnerable behavior)

Example PoC structure:

{
  "payload": "' OR '1'='1",
  "request": "GET /api/users?search=' OR '1'='1",
  "expected_behavior": "Returns users matching search term",
  "actual_behavior": "Returns all users due to SQL injection"
}

Step 7: Calculate CVSS Score

Assign a CVSS 3.1 score based on:

Example: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N = 9.1 (Critical)

Step 8: Classify Finding

Based on analysis, classify each finding:

Step 9: Generate Output

Create validated-findings.json:

{
  "validation_id": "val-<timestamp>",
  "validation_date": "<ISO timestamp>",
  "scan_id": "<from security-findings.json>",
  "threat_model_version": "<from threat-model.md>",
  "validated_findings": [
    {
      "id": "VULN-001",
      "status": "CONFIRMED",
      "original_severity": "HIGH",
      "validated_severity": "HIGH",
      "exploitability": "EASY",
      "reachability": "EXTERNAL",
      "existing_mitigations": [],
      "exploitation_path": [
        "User submits search query via GET /api/users?search=<payload>",
        "Express router passes query to searchUsers() handler",
        "Handler passes unsanitized input to SQL template literal",
        "PostgreSQL executes malicious SQL"
      ],
      "proof_of_concept": {
        "payload": "' OR '1'='1",
        "request": "GET /api/users?search=' OR '1'='1",
        "expected_behavior": "Returns users matching search term",
        "actual_behavior": "Returns all users due to SQL injection"
      },
      "cvss_vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N",
      "cvss_score": 9.1,
      "validation_notes": "Confirmed via code tracing. No input validation or parameterization."
    }
  ],
  "false_positives": [
    {
      "id": "VULN-003",
      "original_severity": "MEDIUM",
      "reason": "Input is validated by Joi schema in middleware before reaching this code. Schema enforces UUID format which prevents injection.",
      "evidence": "See src/middleware/validation.js:45 - Joi.string().uuid()"
    }
  ],
  "needs_manual_review": [
    {
      "id": "VULN-005",
      "original_severity": "HIGH",
      "reason": "Complex data flow through message queue. Unable to fully trace if sanitization occurs in consumer service."
    }
  ],
  "summary": {
    "total_analyzed": 10,
    "confirmed": 5,
    "likely": 2,
    "false_positives": 2,
    "needs_manual_review": 1,
    "by_severity": {
      "CRITICAL": 1,
      "HIGH": 3,
      "MEDIUM": 1,
      "LOW": 0
    }
  }
}

Success Criteria

The skill is complete when:

• • All specified findings have been analyzed
• • Each finding has a status (CONFIRMED, LIKELY, FALSE_POSITIVE, NEEDS_MANUAL_REVIEW)
• • Confirmed findings have exploitation paths documented
• • Confirmed findings have proof-of-concept exploits
• • False positives have clear reasoning
• • validated-findings.json is valid JSON
• • CVSS scores are calculated for confirmed findings

Verification

Run these checks before completing:

# Verify output exists and is valid JSON
cat validated-findings.json | jq . > /dev/null && echo "✓ Valid JSON"

# Check all findings have status
jq '.validated_findings | all(.status)' validated-findings.json

# Check confirmed findings have PoC
jq '.validated_findings | map(select(.status == "CONFIRMED")) | all(.proof_of_concept)' validated-findings.json

# Check false positives have reasoning
jq '.false_positives | all(.reason)' validated-findings.json

Example Invocations

Validate all findings:

Validate the security findings from the last scan.

Validate specific findings:

Validate findings VULN-001 and VULN-002 from security-findings.json.

Validate only HIGH/CRITICAL:

Validate all HIGH and CRITICAL severity findings from the security scan.

Validate with specific files:

Validate findings in security-findings.json using threat model at .factory/threat-model.md.

References

• •CVSS 3.1 Calculator: https://www.first.org/cvss/calculator/3.1
• •OWASP Testing Guide: https://owasp.org/www-project-web-security-testing-guide/
• •Examples: validation-examples.md (in this skill directory)
• •Upstream: commit-security-scan skill
• •Downstream: security-patch-generation skill