Benchmark Comparison Workflow

Compare benchmark results against a baseline to measure optimization impact.

Usage

Invoke with optional benchmark filter:

• •/bench-compare - Run all benchmarks
• •/bench-compare scan - Run only scan-related benchmarks
• •/bench-compare throughput - Run throughput benchmarks

Workflow

1. Establish Baseline

Before making changes, save current benchmark results:

cargo bench --bench <name> -- --save-baseline before

Or for all benchmarks:

cargo bench -- --save-baseline before

2. Apply Changes

Make the optimization changes to the codebase.

3. Compare Results

Run benchmarks against the saved baseline:

cargo bench --bench <name> -- --baseline before

4. Analyze Output

Look for these patterns in Criterion output:

• •Performance has improved - Optimization successful
• •Performance has regressed - Changes hurt performance
• •No change in performance - Within noise threshold

5. Report Summary

Provide a summary table:

Available Benchmarks

The project has these benchmark files in benches/:

• •scan.rs - Core scanning performance
• •scanner_throughput.rs - End-to-end throughput
• •vectorscan_overhead.rs - Vectorscan integration overhead
• •rule_isolation.rs - Per-rule cost isolation
• •hotspots.rs - Known hot path benchmarks (requires bench feature)
• •validator.rs - Validation step benchmarks (requires bench feature)
• •Data structure benchmarks: ring_buffer, fixed_set, fixed_vec, timing_wheel, etc.

Tips

• •Use --bench <name> to run specific benchmarks for faster iteration
• •The hotspots and validator benches require: cargo bench --features bench
• •For memory bandwidth tests, use memory_bandwidth.rs
• •Compare multiple runs to account for variance