pytest-recording (VCR.py) Testing

Overview

pytest-recording wraps VCR.py to record HTTP interactions as YAML cassettes, enabling deterministic tests without live API calls.

Quick Reference

Running Tests

bash

# Run all tests (uses existing cassettes)
uv run pytest tests/

# Run a single test
uv run pytest tests/test_module.py::test_function

# Rewrite all cassettes with fresh responses
uv run pytest tests/ --vcr-record=rewrite

# Record only missing cassettes
uv run pytest tests/ --vcr-record=new_episodes

# Disable VCR (make live requests)
uv run pytest tests/ --disable-recording

Recording Modes

Mode	Flag	Behavior
`none`	`--vcr-record=none`	Only replay, fail if no cassette
`once`	(default)	Record if no cassette exists
`new_episodes`	`--vcr-record=new_episodes`	Record new requests, keep existing
`all`	`--vcr-record=all`	Always record, overwrite existing
`rewrite`	`--vcr-record=rewrite`	Delete and re-record all cassettes

Writing VCR Tests

Basic test with VCR:

python

import pytest

@pytest.mark.vcr()
def test_api_call():
    response = my_api_function()
    assert response.status_code == 200

Custom cassette name:

python

@pytest.mark.vcr("custom_cassette_name.yaml")
def test_with_custom_cassette():
    pass

Multiple cassettes:

python

@pytest.mark.vcr("cassette1.yaml", "cassette2.yaml")
def test_with_multiple_cassettes():
    pass

VCR Configuration in conftest.py

The vcr_config fixture controls VCR behavior:

python

@pytest.fixture(scope="module")
def vcr_config():
    return {
        # Filter sensitive headers from recordings
        "filter_headers": ["authorization", "api-key", "x-api-key"],

        # Filter query parameters
        "filter_query_parameters": ["key", "api_key", "token"],

        # Match requests by these criteria
        "match_on": ["method", "scheme", "host", "port", "path", "query"],

        # Ignore certain hosts (don't record)
        "ignore_hosts": ["localhost", "127.0.0.1"],

        # Record mode
        "record_mode": "once",
    }

Filtering Sensitive Data

For LLM providers, filter authentication:

python

@pytest.fixture(scope="module")
def vcr_config():
    return {
        "filter_headers": [
            "authorization",      # OpenAI, Anthropic
            "api-key",            # Azure OpenAI
            "x-api-key",          # Anthropic
            "x-goog-api-key",     # Google AI
        ],
        "filter_query_parameters": ["key"],
    }

Response Processing

Use pytest_recording_configure for advanced processing:

python

def pytest_recording_configure(config, vcr):
    vcr.serializer = "yaml"
    vcr.decode_compressed_response = True

    # Sanitize response headers
    def sanitize_response(response):
        response['headers']['Set-Cookie'] = 'REDACTED'
        return response

    vcr.before_record_response = sanitize_response

Cassette Location

Cassettes are stored in tests/cassettes/ by default, organized by test module:

code

tests/
├── cassettes/
│   └── test_module/
│       └── test_function.yaml
└── test_module.py

Debugging

Cassette Not Found

If tests fail with "Can't find cassette":

•Run with --vcr-record=once to create missing cassettes
•Check cassette path matches test location
•Verify cassette file exists and is valid YAML

Request Mismatch

If VCR can't match requests:

•Check match_on criteria in vcr_config
•Compare request details in cassette vs actual request
•Use --vcr-record=new_episodes to add missing interactions

Stale Cassettes

When API responses change:

•Delete specific cassette file and re-run test
•Or use --vcr-record=rewrite to refresh all cassettes

View Cassette Contents

bash

# View a cassette file
cat tests/cassettes/test_module/test_function.yaml

# Search for specific content in cassettes
grep -r "error" tests/cassettes/

Adding New LLM Providers

When adding a new provider:

•Identify authentication headers (check provider docs)
•Add headers to filter_headers in vcr_config
•Add any query param auth to filter_query_parameters
•Test with --vcr-record=once to create cassettes
•Verify cassettes don't contain secrets

Common provider authentication:

Provider	Headers to Filter
OpenAI	`authorization`
Anthropic	`x-api-key`, `authorization`
Azure OpenAI	`api-key`
Google AI	`x-goog-api-key`
Cohere	`authorization`

Best Practices

•Never commit secrets: Always filter auth headers/params
•Use descriptive test names: Cassette names derive from test names
•Keep cassettes small: Mock only what you need to test
•Review cassettes in PRs: Check for sensitive data leaks
•Regenerate periodically: API responses may change over time
•Use scope appropriately: scope="module" for shared fixtures