Dev Module

This skill is written for LLMs working inside this repo. It focuses on the fastest, most reliable inner loop:

• •rebuild Zig bindings when needed
• •run the repo’s orchestrated test runner (ato dev test --llm, not raw test output)
• •use the generated test reports (artifacts/test-report.json, artifacts/test-report.html, artifacts/test-report.llm.json)
• •discover and use ConfigFlags correctly (and inventory them repo-wide)

Quick Start

source .venv/bin/activate

ato dev compile
ato dev test --llm -k solver
ato dev test --llm --view HEAD --open
ato dev test --reuse --baseline HEAD~1
ato dev flags

Relevant Files

• •CLI commands: src/atopile/cli/dev.py
  • •ato dev compile (triggers Zig build via import faebryk.core.zig)
  • •ato dev test --llm (runs test/runner/main.py with args; supports baseline/CI report helpers)
• •Zig build-on-import glue: src/faebryk/core/zig/__init__.py (ZIG_NORECOMPILE, ZIG_RELEASEMODE)
• •Config flags utility: src/faebryk/libs/util.py (ConfigFlag, ConfigFlagInt, …)
• •Test runner + reports: test/runner/main.py (artifacts/test-report.json, artifacts/test-report.html, artifacts/test-report.llm.json)
• •CI artifacts definition: .github/workflows/pytest.yml (test-report.json, test-report.html)

Dependants (Call Sites)

• •CI/CD: The dev commands are the primary interface for GitHub Actions workflows.
• •Local Development: Developers use ato dev compile after modifying Zig code.

How to Work With / Develop / Test

Core Commands

• •ato dev compile: compile native extensions (graph/typegraph/sexp bindings).
• •ato dev test --llm: runs the orchestrated test runner (defaults to -p test -p src); supports:
  • •-k filter (-- -k ... also works via passthrough args)
  • •--baseline comparisons (commit hash or HEAD~N style)
  • •--view / --open to fetch and open the test-report.html artifact from GitHub Actions (requires gh CLI)
  • •--ci to apply the CI marker expression (not not_in_ci and not regression and not slow)
  • •--direct -k <testname> to run a single test via test/runtest.py (tight single-test loops)

Test Reports (JSON as source of truth)

Local test runs write:

• •artifacts/test-report.json (single source of truth; outcomes/durations/memory/baseline compare status + stdout/stderr/logs/tracebacks; see tests[].output_full)
• •artifacts/test-report.html (human dashboard; derived from JSON; controlled by FBRK_TEST_GENERATE_HTML=1)
• •artifacts/test-report.llm.json (LLM-friendly; derived from JSON; ANSI stripped logs)

CI uploads both artifacts (see .github/workflows/pytest.yml):

• •test-report.json
• •test-report.html

Notes for LLM debugging:

• •Prefer artifacts/test-report.json or artifacts/test-report.llm.json over raw output; they include structured failures, logs, baseline compare, and collection errors.
• •The HTML is best for quickly scanning long-running tests, worker crashes, and per-test output.

Remote/baseline behavior:

• •ato dev test --llm --baseline <commit> uses the CI test-report.json artifact as the baseline (requires gh CLI).
• •ato dev test --llm --view <commit> --open currently fetches/opens only the HTML artifact; for JSON, download the test-report.json artifact via gh run download.
• •ato dev test --reuse --baseline <commit> rebuilds JSON/HTML/LLM against a baseline without rerunning tests.
• •ato dev test --keep-open keeps the live report server running after tests finish.

Useful test-runner environment variables (see test/runner/main.py):

• •FBRK_TEST_REPORT_INTERVAL (seconds; report refresh cadence)
• •FBRK_TEST_LONG_THRESHOLD (seconds; “long test” threshold)
• •FBRK_TEST_WORKERS (0 = cpu count, negative scales workers)
• •FBRK_TEST_GENERATE_HTML (1/0)
• •FBRK_TEST_PERIODIC_HTML (1/0)
• •FBRK_TEST_OUTPUT_MAX_BYTES (truncate preview output used by HTML; tests[].output_full remains complete)
• •FBRK_TEST_OUTPUT_TRUNCATE_MODE (head or tail)
• •FBRK_TEST_BIND_HOST (orchestrator bind host; default 0.0.0.0)
• •FBRK_TEST_REPORT_HOST (host used in printed report URL; default bind host)
• •FBRK_TEST_PERF_THRESHOLD_PERCENT (default 0.30)
• •FBRK_TEST_PERF_MIN_TIME_DIFF_S (default 1.0)
• •FBRK_TEST_PERF_MIN_MEMORY_DIFF_MB (default 50.0)

LLM quick usage:

• •artifacts/test-report.llm.json is always generated (ANSI stripped, full tests + logs).
• •ato dev test --llm prints a concise summary + schema + jq hints (stdout only).
• •jq recipes are embedded in the report under llm.jq_recipes.
• •Auto-LLM: ato dev test enables the summary automatically when running under claude-code/codex-cli/cursor.
• •Force on/off via FBRK_TEST_LLM=1 or FBRK_TEST_LLM=0.

ConfigFlags (how to use + how to inventory)

ConfigFlag is the repo’s “toggle-by-env-var” mechanism. The environment variable name is the first argument to ConfigFlag(...).

Usage:

export SOME_FLAG=1

Inventory all ConfigFlags in-tree (preferred over trying to maintain a manual list):

ato dev flags

Prefer using ato dev flags when you want the full picture (types/defaults/descriptions + callsite counts) in one place.

High-leverage flags you’ll use often:

• •Zig build: ZIG_NORECOMPILE, ZIG_RELEASEMODE
• •Solver debug: SLOG, SVERBOSE_TABLE, SPRINT_START, SMAX_ITERATIONS, SSHOW_SS_IS
• •Logs: COLOR_LOGS, LOG_TIME, LOG_FILEINFO

Development Workflow

1. •Zig Changes: Edit files under src/faebryk/core/zig/src/ -> Run ato dev compile.
2. •Profiling: If something is slow, use ato dev profile <command> to generate a flamegraph or stats.

Testing

• •Main test entrypoint: ato dev test --llm.
• •If you change CLI behavior, add/adjust tests under test/ that exercise the command surface.

Best Practices

• •Use ConfigFlags: For experimental features or verbose debugging, use a ConfigFlag instead of commenting out code.
• •Compile often: Zig errors won’t be caught by Python tooling.