Mutation Testing Analysis

Purpose

Run mutation testing on a target module, analyze surviving mutants, and produce actionable recommendations to strengthen the test suite.

When to Use

• •After achieving high line/branch coverage but suspecting tests lack assertion depth
• •During code review to evaluate test quality for critical modules
• •Before releasing financial, security, or safety-critical code
• •When setting up mutation testing for the first time in a project

Steps

Step 1: Identify the Target

Determine:

• •Which source files to mutate (focus on critical business logic first)
• •Which test files exercise those source files
• •The language and appropriate mutation tool:
  • •TypeScript/JavaScript: Stryker Mutator
  • •Python: mutmut
  • •Go: go-mutesting or gremlins

Step 2: Configure the Mutation Tool

Stryker (TypeScript/JavaScript):

{
  "mutate": ["src/services/transfer.ts"],
  "testRunner": "jest",
  "reporters": ["html", "clear-text"],
  "coverageAnalysis": "perTest",
  "thresholds": { "high": 80, "low": 60, "break": 50 }
}

mutmut (Python):

[tool.mutmut]
paths_to_mutate = "src/services/"
tests_dir = "tests/"
runner = "python -m pytest -x"

Go:

go-mutesting ./internal/service/...

Step 3: Run Mutation Testing

Execute the tool and capture the report. Note:

• •Total mutants generated
• •Mutants killed (tests detected the change)
• •Mutants survived (tests missed the change)
• •Equivalent mutants (change doesn't affect behavior)
• •Timeouts (infinite loops from mutation)

Step 4: Classify Surviving Mutants

For each surviving mutant, classify it into one of these categories:

Step 5: Prioritize Fixes

Sort surviving mutants by impact:

1. •Critical path mutants (payment, auth, transfers) -- fix immediately
2. •Business logic mutants (order processing, validation) -- fix in current sprint
3. •Utility mutants (formatting, logging) -- fix when convenient
4. •Equivalent mutants -- document and exclude from future runs

Step 6: Write Strengthening Tests

For each fixable surviving mutant:

1. •Identify the exact mutation (e.g., > changed to >=)
2. •Write a test that would fail if that mutation were applied
3. •Verify the test passes against the original code
4. •Verify the test fails against the mutated code (if tool supports re-run)

Step 7: Re-Run and Verify

After adding tests:

1. •Re-run mutation testing on the same module
2. •Verify the mutation score improved
3. •Check that no new surviving mutants appeared
4. •Update thresholds if the module now exceeds the previous target

Step 8: Document and Report

Produce a summary:

• •Module: path to source files
• •Before score: X% (Y killed / Z total)
• •After score: X% (Y killed / Z total)
• •Surviving mutants: categorized list with actions taken
• •Excluded: equivalent mutants with justification
• •Recommendation: target score for this module going forward

Score Thresholds

Output Format

Produce:

1. •Mutation testing report summary (scores, surviving mutant count)
2. •Categorized list of surviving mutants with recommended actions
3. •New or strengthened test files
4. •Updated mutation tool configuration (exclusions for equivalent mutants)