Memory

Search first: /mnemonic:search {relevant_keywords} Capture after: /mnemonic:capture {namespace} "{title}"

Run /mnemonic:list --namespaces to see available namespaces from loaded ontologies.

Validation Strategies

Choose and implement effective validation strategies for hook-driven tests.

Mnemonic Integration

Before advising on validation, check mnemonic for insights:

# Search for validation-related memories
rg -i "validation|assertion|regex|contains|flaky" ~/.claude/mnemonic/ ./.claude/mnemonic/ --glob "*.memory.md" -l | head -5

# Check learnings for validation experiences
rg -l "." ~/.claude/mnemonic/*/learnings/ --glob "*.memory.md" 2>/dev/null | xargs grep -l -i "validation\|expect" 2>/dev/null | head -5

Apply recalled insights:

• •Validation approaches that worked well
• •Flaky test patterns to avoid
• •Regex patterns that proved reliable
• •User's preferred validation strategies

Validation Approaches

String Matching (contains)

Strengths:

• •Simple and readable
• •Tolerant of formatting changes
• •Fast execution

Weaknesses:

• •May match unintended content
• •No structure awareness
• •Case-sensitive by default

Best for:

• •Success/error messages
• •UI text validation
• •Presence checks

expect:
  - contains: "Operation successful"
  - contains: "Created"

Negative Matching (not_contains)

Strengths:

• •Clear failure detection
• •Complements positive checks
• •Security validation

Weaknesses:

• •Can't validate what IS present
• •May miss variant error messages

Best for:

• •Error absence verification
• •Security checks (no sensitive data)
• •Deprecation validation

expect:
  - not_contains: "error"
  - not_contains: "exception"
  - not_contains: "password"

Pattern Matching (regex)

Strengths:

• •Precise structure validation
• •Value extraction capability
• •Multiple value matching

Weaknesses:

• •Complex to write correctly
• •Harder to debug failures
• •Performance overhead

Best for:

• •ID/UUID validation
• •Numeric output
• •Structured data
• •Variable capture

expect:
  - regex: "ID:\\s+[a-f0-9]{12}"
  - regex: "Count:\\s+\\d+"
  - regex: "(success|complete|done)"

Choosing the Right Strategy

Decision Matrix

Combination Strategies

Use multiple expectations for robust validation:

expect:
  # Positive confirmation
  - contains: "Memory created"
  # Negative confirmation
  - not_contains: "error"
  - not_contains: "failed"
  # Structured validation
  - regex: "ID:\\s+([a-f0-9]+)"

Regex Patterns Reference

Common Patterns

ID Patterns

# UUID v4
regex: "[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}"

# Short ID (12 hex chars)
regex: "[a-f0-9]{12}"

# Numeric ID
regex: "ID:\\s+(\\d+)"

# Alphanumeric ID
regex: "ID:\\s+([a-zA-Z0-9]+)"

Output Patterns

# Success with count
regex: "Found\\s+(\\d+)\\s+results"

# Status messages
regex: "Status:\\s+(active|inactive|pending)"

# Version numbers
regex: "v(\\d+\\.\\d+\\.\\d+)"

# Timestamps
regex: "\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}"

Handling Edge Cases

Empty Results

Validate empty states explicitly:

- id: search_no_results
  action: "Search for 'nonexistent_term_xyz'"
  expect:
    - regex: "(no results|0 found|empty)"
    - not_contains: "error"

Variable Output

Handle outputs that may vary:

expect:
  # Accept range of counts
  - regex: "Found\\s+[1-9]\\d*\\s+results"
  # Accept multiple success messages
  - regex: "(created|updated|saved)"

Whitespace Sensitivity

Use \s+ for flexible whitespace:

# Rigid (may fail)
- contains: "ID: abc123"

# Flexible (handles formatting)
- regex: "ID:\\s+abc123"

Case Sensitivity

Regex can handle case variations:

# Case-insensitive with (?i)
- regex: "(?i)success"

# Explicit alternatives
- regex: "(Success|SUCCESS|success)"

Handling Flaky Tests

Causes of Flakiness

1. •Timing issues: Async operations not complete
2. •Order dependence: Tests affect each other
3. •External dependencies: Network, services
4. •Non-deterministic output: Timestamps, random IDs

Mitigation Strategies

1. Flexible expectations:

expect:
  # Don't match exact timestamp
  - regex: "Created at: \\d{4}-\\d{2}-\\d{2}"
  # Don't match exact ID
  - regex: "ID:\\s+[a-f0-9]+"

2. Retry logic (in runner):

cmd_validate_with_retry() {
  local max_retries=3
  for i in $(seq 1 $max_retries); do
    if cmd_validate "$1"; then
      return 0
    fi
    sleep 1
  done
  return 1
}

3. Tag flaky tests:

- id: external_api_test
  tags: [flaky, external]

4. Isolate stateful tests:

- id: cleanup_before_test
  action: "Reset test state"
  tags: [setup]

- id: actual_test
  depends_on: cleanup_before_test

Debugging Validation Failures

Reading Failure Messages

❌ FAIL: search_basic
Failures:
  - Missing: 'results found'
  - Pattern not found: 'Count:\s+\d+'

Interpretation:

• •Missing: 'X' → contains: X failed
• •Unexpected: 'X' → not_contains: X failed
• •Pattern not found: 'X' → regex: X failed

Troubleshooting Steps

1. •Get actual response: Check what Claude returned
2. •Compare expectations: Look for typos, case issues
3. •Test regex separately: Use online regex tester
4. •Simplify expectations: Start broad, narrow down
5. •Check for invisible chars: Whitespace, newlines

Common Fixes

Additional Resources

Reference Files

• •references/validation-approaches.md - Validation strategies and patterns