Python Guide
Applies to: Python 3.11+, APIs, CLIs, Data Pipelines, Automation
Core Principles
- •Type Hints Everywhere: All function signatures, class attributes, and module-level variables must have type annotations
- •Explicit Over Implicit: No
*imports, no mutable default arguments, no implicit type coercions - •Virtual Environments Always: Never install into system Python; use
venv,uv, orpoetry - •Pytest Over unittest: Use pytest for all testing; fixtures and parametrize over setUp/tearDown
- •PEP 8 + Ruff: Enforce style mechanically; never rely on manual formatting
Guardrails
Python Version
- •Target Python 3.11+ (use
matchstatements,ExceptionGroup,tomllib) - •Set
requires-python = ">=3.11"inpyproject.toml - •Use
from __future__ import annotationsfor forward references in 3.11 - •Never use features removed in 3.12+ (
distutils,imp, legacytypingaliases)
Code Style
- •Run
ruff checkandruff formatbefore every commit - •Max line length: 88 characters (Black default)
- •Imports: stdlib, blank line, third-party, blank line, local (enforced by
isort/ruff) - •Naming:
snake_casefor functions/variables,PascalCasefor classes,UPPER_SNAKEfor constants - •No bare
except:— always catch specific exceptions - •No mutable default arguments (
def f(items=None):notdef f(items=[]):) - •Prefer f-strings over
.format()or%formatting - •Use
pathlib.Pathinstead ofos.pathfor all file operations
Type Hints
- •All public functions MUST have full type annotations (params + return)
- •Use
collections.abctypes:Sequence,Mapping,Iterable(notList,Dict) - •Use
X | Noneunion syntax (notOptional[X]) - •Use
TypeAliasfor complex types:UserMap: TypeAlias = dict[str, User] - •Use
Protocolfor structural subtyping (duck typing with safety) - •Use
@overloadfor functions returning different types based on input - •Run
mypy --strictin CI (notype: ignorewithout explanation)
python
from collections.abc import Sequence
def find_users(
ids: Sequence[str],
*,
active_only: bool = True,
) -> list[User]:
"""Fetch users by ID list, optionally filtering inactive."""
...
Error Handling
- •Never use bare
except:orexcept Exception:without re-raising - •Create domain-specific exception hierarchies rooted in a base class
- •Use
raise ... from errto preserve exception chains - •Log at the boundary, raise in the interior (don't log-and-raise)
- •Use
contextlib.suppress()instead of empty except blocks - •Always close resources with
withstatements orcontextlib.closing
Dependencies
- •Define all deps in
pyproject.toml(notsetup.pyor barerequirements.txt) - •Pin exact versions in lock files (
uv.lock,poetry.lock,pip-compileoutput) - •Keep
requirements.txtonly as a generated artifact, never hand-edited - •Separate
[project.optional-dependencies]for dev, test, docs - •Audit with
pip-auditorsafetybefore adding new packages - •Prefer stdlib solutions:
tomllib,pathlib,dataclasses,enum,logging
Project Structure
code
myproject/ ├── src/ │ └── myproject/ # Importable package (src layout) │ ├── __init__.py │ ├── py.typed # PEP 561 marker for type stubs │ ├── domain/ # Business logic, entities │ │ ├── __init__.py │ │ ├── models.py │ │ └── exceptions.py │ ├── service/ # Application services │ │ └── __init__.py │ ├── repository/ # Data access layer │ │ └── __init__.py │ └── api/ # HTTP/CLI interface │ └── __init__.py ├── tests/ │ ├── conftest.py # Shared fixtures │ ├── unit/ │ └── integration/ ├── pyproject.toml # Single source of truth for config ├── uv.lock # Or poetry.lock └── README.md
- •Use src layout (
src/myproject/) to prevent accidental local imports - •Keep
conftest.pyat test root for shared fixtures; nest for scope - •Include
py.typedmarker for downstream type checking - •No
__init__.pyintests/(pytest discovers without it) - •One module = one responsibility; split at ~200 lines
Error Handling Patterns
Exception Hierarchy
python
class AppError(Exception):
"""Base exception for the application."""
def __init__(self, message: str, *, code: str = "UNKNOWN") -> None:
super().__init__(message)
self.code = code
class NotFoundError(AppError):
"""Raised when a requested resource does not exist."""
def __init__(self, resource: str, identifier: str) -> None:
super().__init__(
f"{resource} with id '{identifier}' not found",
code="NOT_FOUND",
)
self.resource = resource
self.identifier = identifier
class ValidationError(AppError):
"""Raised when input data fails validation."""
def __init__(self, field: str, reason: str) -> None:
super().__init__(
f"Validation failed for '{field}': {reason}",
code="VALIDATION_ERROR",
)
Context Managers for Cleanup
python
from contextlib import contextmanager
from collections.abc import Generator
@contextmanager
def managed_connection(url: str) -> Generator[Connection, None, None]:
conn = Connection(url)
try:
conn.open()
yield conn
except ConnectionError as err:
raise AppError("Database unavailable") from err
finally:
conn.close()
Error Chaining
python
def get_user(user_id: str) -> User:
try:
row = db.fetch_one("SELECT * FROM users WHERE id = %s", (user_id,))
except DatabaseError as err:
raise AppError(f"Failed to fetch user {user_id}") from err
if row is None:
raise NotFoundError("User", user_id)
return User.from_row(row)
Testing
Standards
- •Test files:
test_*.py(same name as module:models.py->test_models.py) - •Test functions:
test_<unit>_<scenario>_<expected>(e.g.,test_get_user_not_found_raises) - •Use
conftest.pyfor fixtures shared across a directory - •Coverage target: >80% for business logic, >60% overall
- •Mark slow tests:
@pytest.mark.slowand exclude from default runs - •No
unittest.TestCase— use plain functions with pytest assertions - •Use
tmp_pathfixture for file operations (auto-cleanup)
Fixtures and Parametrize
python
import pytest
from myproject.domain.models import User
@pytest.fixture
def sample_user() -> User:
return User(id="u-123", name="Ada Lovelace", email="ada@example.com")
@pytest.mark.parametrize(
("email", "is_valid"),
[
("user@example.com", True),
("user@.com", False),
("", False),
("user@domain", False),
],
)
def test_validate_email(email: str, is_valid: bool) -> None:
assert validate_email(email) == is_valid
def test_get_user_returns_user(sample_user: User) -> None:
repo = InMemoryUserRepo(users=[sample_user])
result = repo.get("u-123")
assert result == sample_user
def test_get_user_not_found_raises() -> None:
repo = InMemoryUserRepo(users=[])
with pytest.raises(NotFoundError, match="User.*not found"):
repo.get("nonexistent")
Mocking External Dependencies
python
from unittest.mock import AsyncMock, patch
async def test_send_notification_retries_on_failure() -> None:
mock_client = AsyncMock()
mock_client.post.side_effect = [ConnectionError, None]
with patch("myproject.service.notify.http_client", mock_client):
await send_notification(user_id="u-123", message="hello")
assert mock_client.post.call_count == 2
Tooling
pyproject.toml Configuration
toml
[project]
name = "myproject"
requires-python = ">=3.11"
[project.optional-dependencies]
dev = ["ruff", "mypy", "pytest", "pytest-cov", "pytest-asyncio"]
[tool.ruff]
target-version = "py311"
line-length = 88
[tool.ruff.lint]
select = [
"E", # pycodestyle errors
"W", # pycodestyle warnings
"F", # pyflakes
"I", # isort
"N", # pep8-naming
"UP", # pyupgrade
"B", # flake8-bugbear
"S", # flake8-bandit (security)
"A", # flake8-builtins
"C4", # flake8-comprehensions
"SIM", # flake8-simplify
"RUF", # ruff-specific rules
]
[tool.mypy]
strict = true
warn_return_any = true
disallow_untyped_defs = true
[tool.pytest.ini_options]
testpaths = ["tests"]
markers = ["slow: marks tests as slow (deselect with '-m \"not slow\"')"]
asyncio_mode = "auto"
[tool.coverage.run]
source = ["src/myproject"]
branch = true
[tool.coverage.report]
fail_under = 60
show_missing = true
exclude_lines = ["if TYPE_CHECKING:", "pragma: no cover"]
Essential Commands
bash
ruff check . # Lint (replaces flake8, isort, pyupgrade) ruff format . # Format (replaces black) mypy . # Type check (strict mode) pytest # Run all tests pytest --cov=src -q # Coverage summary pytest -m "not slow" # Skip slow tests pip-audit # Check dependencies for vulnerabilities python -m build # Build sdist + wheel
Advanced Topics
For detailed patterns and examples, see:
- •references/patterns.md -- Async patterns, dataclass/Pydantic models, context managers, decorators, type hints (generics, Protocol, TypeVar)
- •references/pitfalls.md -- Common Python gotchas and do/don't examples
- •references/security.md -- Input sanitization, secrets management, SQL injection prevention