AgentSkillsCN

testing-operations

构建契约与回归测试工作流。在编写测试用例、验证契约合规性，或创建回归测试套件时使用。

SKILL.md
--- frontmatter
name: testing-operations
description: Contract and regression testing workflows. Use when writing tests, validating contract compliance, or creating regression test suites.

Testing Operations

Testing operations cover writing contract compliance tests, regression tests, and validating test coverage for 3Lens.

When to Use

  • •Writing contract compliance tests
  • •Creating regression tests from traces
  • •Validating test coverage
  • •Debugging test failures
  • •Understanding test patterns

Test Types

Contract Compliance Tests

Tests that validate contract invariants:

typescript
// tests/contracts/inspector.test.ts
import { describe, it, expect } from 'vitest';
import { createLens } from '@3lens/runtime';

describe('Inspector Contract', () => {
  it('answers the 5 questions for any entity', async () => {
    const lens = createLens();
    const entityId = 'mesh:main:myMesh';
    
    // Question 1: What is this?
    const info = await lens.inspect(entityId, { include: ['info'] });
    expect(info).toBeDefined();
    
    // Question 2: Dependencies?
    const deps = await lens.inspect(entityId, { include: ['dependencies'] });
    expect(deps.dependencies).toBeArray();
    
    // Question 3: What depends on it?
    const dependents = await lens.inspect(entityId, { include: ['dependents'] });
    expect(dependents.dependents).toBeArray();
    
    // Question 4: Cost?
    const cost = await lens.inspect(entityId, { include: ['cost'] });
    expect(cost.cost).toHaveProperty('attribution');
    
    // Question 5: How has it changed?
    const diffs = await lens.inspect(entityId, { include: ['diffs'] });
    expect(diffs.diffs).toBeArray();
  });
});

Regression Tests

Tests that detect regressions:

typescript
// tests/regression/baseline.test.ts
import { describe, it, expect } from 'vitest';
import { loadTrace } from '@3lens/kernel';

describe('Regression: Baseline Performance', () => {
  it('maintains frame time budget', async () => {
    const trace = await loadTrace('./traces/baseline.json');
    const frames = trace.getFrames();
    
    const avgFrameTime = frames.reduce((sum, f) => sum + f.duration, 0) / frames.length;
    expect(avgFrameTime).toBeLessThan(16.67); // 60fps budget
  });
  
  it('maintains memory budget', async () => {
    const trace = await loadTrace('./traces/baseline.json');
    const peakMemory = trace.getPeakMemory();
    
    expect(peakMemory).toBeLessThan(100 * 1024 * 1024); // 100MB budget
  });
});

Snapshot Tests

Tests for stable outputs:

typescript
// tests/snapshots/entity-graph.test.ts
import { describe, it, expect } from 'vitest';
import { createEntityGraph } from '@3lens/kernel';

describe('Entity Graph Snapshots', () => {
  it('generates consistent graph structure', () => {
    const graph = createEntityGraph();
    // ... build graph
    
    expect(graph.snapshot()).toMatchSnapshot();
  });
});

Commands

Run Contract Tests

bash
# Run all contract tests
3lens test contracts

# Test specific contract
3lens test contracts --contract inspector

# Show coverage
3lens test contracts --coverage

# Watch mode
3lens test contracts --watch

Generate Regression Tests

bash
# Generate regression test from trace
3lens test:generate-regression baseline.json --out tests/regression/baseline.test.ts

Check Test Coverage

bash
# Check contract test coverage
3lens test:coverage contracts

Test Patterns

Fidelity Testing

typescript
it('labels fidelity for all metrics', async () => {
  const result = await client.query('top_hotspots');
  result.items.forEach(item => {
    expect(item).toHaveProperty('fidelity');
    expect(['EXACT', 'ESTIMATED', 'UNAVAILABLE']).toContain(item.fidelity);
  });
});

Attribution Testing

typescript
it('provides attribution path for metrics', async () => {
  const metric = await client.query('top_hotspots');
  expect(metric.attribution).toBeArray();
  expect(metric.attribution.length).toBeGreaterThan(0);
  expect(metric.attribution[0]).toHaveProperty('entityId');
  expect(metric.attribution[0]).toHaveProperty('weight');
});

Live/Offline Parity Testing

typescript
describe('Live and Offline Parity', () => {
  it('works in live mode', async () => {
    const lens = createLens();
    // Test live mode
  });
  
  it('works with offline traces', async () => {
    const trace = await loadTrace('./traces/test.json');
    // Test offline mode
  });
});

Agent Use Cases

  1. •New feature: "Generate tests for my new panel"
  2. •Contract compliance: "Write tests for the inspector contract"
  3. •Regression: "Create regression test from this trace"
  4. •Coverage: "Check test coverage for contracts"

Test Requirements

Every test MUST:

  1. •

    Validate Contract Invariants

    • •Test all contract requirements
    • •Include degradation scenarios
  2. •

    Include Fidelity Checks

    • •Verify fidelity labeling
    • •Test fidelity propagation
  3. •

    Include Attribution Checks

    • •Verify attribution paths
    • •Test attribution weights
  4. •

    Test Live/Offline Parity

    • •Test both modes
    • •Verify consistent results

Post-Scaffold Steps

After writing tests, follow the playbook:

Additional Resources