Token Cost Computation

Compute token usage totals and estimated costs from Codex CLI (OpenAI) and Claude Code (Anthropic) SSE JSONL log streams.

Quick Start

Compute Token Costs from SSE Logs

bash

# Basic usage with both log types
python scripts/compute_token_costs.py \
  --codex ~/centralized-logs/codex/sse_lines.jsonl \
  --claude ~/centralized-logs/claude/sse_lines.jsonl

# With a pricing snapshot for reproducibility
python scripts/compute_token_costs.py \
  --codex codex_sse_lines.jsonl \
  --claude claude_sse_lines.jsonl \
  --pricing-snapshot pricing_snapshots/pricing_snapshot_2026-01-24.json

Fetch Latest Pricing

bash

# Fetch and save a dated pricing snapshot
python scripts/pull_pricing.py --snapshot-dir pricing_snapshots/ --print-summary

Extract a Time Window from Logs

bash

# Extract a specific week from Codex logs
python scripts/extract_codex_week.py \
  --src ~/centralized-logs/codex/sse_lines.jsonl \
  --dst extracted_week.jsonl \
  --start "2026-01-15 14:47" \
  --end "2026-01-22 14:47"

What It Computes

Token Metrics

Provider	Metric	Description
Codex (OpenAI)	`input_tokens`	Total input tokens across all calls
Codex (OpenAI)	`cached_input_tokens`	Subset of input tokens served from cache
Codex (OpenAI)	`output_tokens`	Total output tokens generated
Codex (OpenAI)	`reasoning_output_tokens`	Tokens used for reasoning (o-series models)
Claude (Anthropic)	`input_tokens`	Total input tokens
Claude (Anthropic)	`cache_creation_input_tokens`	Tokens written to cache
Claude (Anthropic)	`cache_read_input_tokens`	Tokens read from cache
Claude (Anthropic)	`output_tokens`	Total output tokens

Cost Estimation

Costs are computed using:

•OpenAI: Standard/Flex/Priority tier pricing per model
•Anthropic: Base pricing + cache multipliers (1.25x write-5m, 2.0x write-1h, 0.1x read)

Key Concepts

Token Aggregation Semantics

Codex (OpenAI):

•Uses last_token_usage from turn.token_count events (not cumulative totals)
•Deduplicates consecutive identical events to avoid double-counting
•cached_input_tokens is a subset of input_tokens, not additive

Claude (Anthropic):

•Uses max of message_start and message_delta usage per message (they're snapshots, not deltas)
•Cache pricing is writes (1.25x or 2x) + reads (0.1x), NOT a flat 90% discount

Pricing Sources (in order of reliability)

•OpenAI: https://platform.openai.com/docs/pricing
•Anthropic: https://platform.claude.com/docs/en/about-claude/pricing
•OpenRouter: https://openrouter.ai/api/v1/models (machine-readable, requires API key)

Output Format

Console Output

code

=== OpenAI / Codex ===
Model: gpt-5.2 | tier=standard | pricing source: snapshot:...
API calls counted (deduped token_count): 150
input_tokens=8,947,835 (cached=8,656,512, noncached=291,323)
output_tokens=65,051 (reasoning_output_tokens=0)
estimated cost: $1.42

=== Anthropic / Claude ===
Model: claude-opus-4-5 | pricing source: snapshot:...
  messages=75 | input=2,500,000 | cache_write=500,000 | cache_read=2,000,000 | output=150,000
  estimated cost: $4.25

=== Totals ===
Grand total: $5.67

JSON Output

The script also outputs a machine-readable JSON structure with full breakdown.

Scripts Reference

Script	Purpose
`compute_token_costs.py`	Main computation - totals + costs
`pull_pricing.py`	Fetch latest pricing from providers
`extract_codex_week.py`	Extract time windows from SSE logs

Important Notes

•Do NOT hardcode "90% cached discount" - varies by provider/model/tier
•Do NOT sum message_start + message_delta usage - they're snapshots, not increments
•Do NOT use total_token_usage for per-round deltas without accounting for session resume
•Use --pricing-snapshot for reproducible cost audits

Context Window Consumption

Note: This skill computes token totals and costs, not context window utilization. The raw logs contain model_context_window data, but utilization percentage is not currently computed.

To compute context utilization, you would need:

python

utilization_pct = (total_tokens_at_turn / model_context_window) * 100

Related Locations

•Centralized Codex logs: ~/centralized-logs/codex/sse_lines.jsonl
•Centralized Claude logs: ~/centralized-logs/claude/sse_lines.jsonl
•Source repo: ~/swe/token-computation/