Bloom Integrity Verification Skill
Purpose
Ensure evaluation scenarios and results maintain cryptographic integrity for reproducible safety research. Provides tamper-evident audit trails for regulatory compliance.
When to Use
- •Before running evaluations (verify scenario integrity)
- •After evaluations (generate audit trail)
- •For regulatory compliance (FDA 21 CFR Part 11, EU AI Act)
- •Before sharing results externally
Triggers
- •"verify scenarios"
- •"check integrity"
- •"generate audit log"
- •"sign evaluation results"
- •"hash directory"
Tools
bash
# Verify scenarios with signature bloom-verify check scenarios/ \ --sig scenarios.sig \ --pubkey bloom.pub \ --fail-closed # Generate audit log bloom-verify audit results.json --output audit.json # Hash directory for manifest bloom-verify hash scenarios/ > manifest.json # Create signed manifest bloom-verify sign manifest.json --key bloom.key --output manifest.sig
Prerequisites
- •Rust toolchain (for building from source)
- •OR: Pre-built binary
Installation
bash
# From source cd bloom_medical_eval/bloom_verify cargo build --release # Add to PATH export PATH="$PATH:$(pwd)/target/release" # Or install globally cargo install --path bloom_medical_eval/bloom_verify
Input Schema
yaml
command: type: enum values: [check, hash, sign, verify, audit] required: true path: type: path required: true description: File or directory to process signature: type: path description: Signature file (for check/verify) pubkey: type: path description: Public key file (for check/verify) privkey: type: path description: Private key file (for sign) output: type: path description: Output file path fail_closed: type: boolean default: true description: Exit non-zero on any failure
Output Schema
yaml
status: enum # pass, fail hash: string # BLAKE3 hash (64 hex chars) signature_valid: boolean files_verified: integer audit_entries: array timestamp: string # ISO 8601
Cryptographic Properties
| Property | Implementation | Notes |
|---|---|---|
| Hashing | BLAKE3 | 10x faster than SHA-256, secure |
| Signing | Ed25519 via ring | Fast, constant-time, secure |
| Audit Chains | Hash-chained entries | Blockchain-style integrity |
| Key Format | PEM | Standard, portable |
Success Criteria
| Check | Requirement |
|---|---|
| Scenario verification | Exit code 0 |
| Signature validity | Ed25519 verification passes |
| Audit chain integrity | All entry hashes valid |
| No modified files | Hash matches manifest |
Safety Gates
yaml
- gate: scenario_integrity
metric: verification_passed
operator: "=="
threshold: true
action: block_execution
severity: medium
description: |
Evaluation cannot proceed if scenario integrity fails.
Prevents running on tampered or corrupted data.
- gate: signature_valid
metric: signature_valid
operator: "=="
threshold: true
action: warn
severity: low
description: |
Missing or invalid signature triggers warning.
May indicate unsigned development data.
Compliance Support
| Regulation | Feature |
|---|---|
| FDA 21 CFR Part 11 | Audit trails, electronic signatures |
| EU AI Act | Traceability, reproducibility |
| HIPAA | Data integrity, access logging |
| ISO 27001 | Information security controls |
Usage Examples
Pre-Evaluation Verification
bash
# Before running crisis evaluation bloom-verify check \ bloom_medical_eval/experiments/crisis_pilot/.private/scenarios_v2.json \ --sig scenarios_v2.sig \ --pubkey bloom.pub \ --fail-closed # If verification fails, abort if [ $? -ne 0 ]; then echo "Scenario integrity check failed. Aborting." exit 1 fi
Post-Evaluation Audit
bash
# After evaluation completes bloom-verify audit \ results/crisis_pilot/pilot_gpt52_n30_*.json \ --output audit_trail.json # Sign the audit trail bloom-verify sign audit_trail.json \ --key bloom.key \ --output audit_trail.sig
Related Skills
- •
crisis_persistence_eval- Uses bloom-verify for scenario integrity - •
phi_detection- Run before bloom-verify to ensure data is clean