Python Testing
Overview
Test observable behavior and contracts, not internal implementation. Keep unit tests fast, deterministic, and patched at module boundaries.
These are preferred defaults for common cases. When a default conflicts with project constraints, suggest a better-fit alternative, call out tradeoffs, and note compensating controls.
When to Use
- •Writing or reviewing unit, integration, or reliability-sensitive tests.
- •Tests are flaky, slow, or coupled to implementation details.
- •Adding regression tests after a bugfix.
- •Testing async lifecycles, cancellation, or cleanup paths.
- •Unsure what test coverage a change needs.
When NOT to use:
- •Pure data-shape or schema validation (see
python-types-contracts). - •Production observability or monitoring concerns (see
python-runtime-operations). - •Concurrency design decisions outside of test harnesses (see
python-concurrency-performance).
Quick Reference
- •Test observable behavior, not internals.
- •Keep unit tests fast and deterministic.
- •Patch at module boundaries and import locations used by the unit under test.
- •Add regression tests for bugfixes.
- •Include timeout/retry/cancellation/cleanup coverage where relevant.
Change-Specific Diagnostics
- •Dependency updates: run
uv run pytest scripts/test_pypi_security_audit.py -v - •Async-heavy lifecycle changes: run
pyleakdiagnostics.
Common Mistakes
- •Mocking too deep — patching internals instead of module-boundary seams makes tests brittle and coupled to implementation.
- •Testing the mock — verifying mock call counts without asserting on observable output proves nothing about behavior.
- •Missing regression test — fixing a bug without a test that reproduces it first; the bug will recur.
- •Non-deterministic time/order — relying on wall-clock time or dict/set ordering instead of injecting clocks and sorting explicitly.
- •Skipping cleanup assertions — verifying the happy path but never asserting that resources are released on failure or cancellation.
Scope Note
- •Treat these recommendations as preferred defaults for common cases, not universal rules.
- •If a default conflicts with project constraints or worsens the outcome, suggest a better-fit alternative and explain why it is better for this case.
- •When deviating, call out tradeoffs and compensating controls (tests, observability, migration, rollback).
Invocation Notice
- •Inform the user when this skill is being invoked by name:
python-design-modularity.
References
- •
references/testing-strategy.md - •
references/pytest-practices.md - •
references/async-and-concurrency-testing.md - •
references/reliability-lifecycle-testing.md