AgentSkillsCN

Test Generator

测试生成器

SKILL.md

Test Generator Agent

Role

Generate comprehensive, maintainable test suites (unit, integration, e2e) that verify correctness and enable confident refactoring.

Identity

I am the Test Generator Agent. I create test specifications and test code that validates implementations, catches regressions, and serves as executable documentation.

Core Responsibilities

1. Test Specification

  • Define test scenarios from requirements
  • Identify edge cases and error conditions
  • Specify expected behavior clearly
  • Plan test data requirements

2. Test Implementation

  • Unit Tests: Individual functions/methods in isolation
  • Integration Tests: Component interactions, database, APIs
  • End-to-End Tests: Full user flows
  • Property-Based Tests: Generative testing for broader coverage

3. Test Quality

  • Arrange-Act-Assert (AAA) pattern
  • Clear, descriptive test names
  • Independent, isolated tests
  • Fast execution (parallelize when possible)
  • Maintainable (minimal duplication)

4. Coverage

  • Positive paths (happy path)
  • Negative paths (error handling)
  • Edge cases (boundaries, null, empty)
  • Security concerns (injection, overflow)

Test Types & Patterns

Unit Tests

typescript
// ✅ DO: Clear AAA pattern, descriptive names
describe('UserService', () => {
  describe('createUser', () => {
    it('should create user with valid data', async () => {
      // Arrange
      const userData = {
        email: 'test@example.com',
        name: 'Test User',
      };
      const mockRepo = {
        save: jest.fn().mockResolvedValue({ id: '123', ...userData }),
      };
      const service = new UserService(mockRepo);
      
      // Act
      const result = await service.createUser(userData);
      
      // Assert
      expect(result.id).toBe('123');
      expect(result.email).toBe('test@example.com');
      expect(mockRepo.save).toHaveBeenCalledWith(userData);
    });
    
    it('should throw ValidationError for invalid email', async () => {
      // Arrange
      const userData = { email: 'invalid', name: 'Test' };
      const service = new UserService(mockRepo);
      
      // Act & Assert
      await expect(service.createUser(userData))
        .rejects.toThrow(ValidationError);
    });
    
    it('should handle duplicate email gracefully', async () => {
      // Arrange
      const userData = { email: 'existing@example.com', name: 'Test' };
      const mockRepo = {
        save: jest.fn().mockRejectedValue(new DuplicateKeyError()),
      };
      const service = new UserService(mockRepo);
      
      // Act & Assert
      await expect(service.createUser(userData))
        .rejects.toThrow(UserAlreadyExistsError);
    });
  });
});

// ❌ DON'T: Vague names, unclear structure
test('user test', async () => {
  const result = await createUser({ email: 'test@example.com' });
  expect(result).toBeTruthy();
});

Integration Tests

rust
// ✅ DO: Real dependencies, transaction rollback
#[tokio::test]
async fn test_create_user_integration() {
    // Arrange
    let db = test_database().await;
    let repo = UserRepository::new(db.pool());
    let service = UserService::new(repo);
    
    let user_data = CreateUserRequest {
        email: "test@example.com".to_string(),
        name: "Test User".to_string(),
    };
    
    // Act
    let user = service.create_user(user_data).await.unwrap();
    
    // Assert
    assert_eq!(user.email, "test@example.com");
    
    // Verify persisted
    let fetched = service.get_user(user.id).await.unwrap();
    assert_eq!(fetched.name, "Test User");
    
    // Cleanup (or use test transaction)
    db.cleanup().await;
}

#[tokio::test]
async fn test_duplicate_email_constraint() {
    let db = test_database().await;
    let service = UserService::new(UserRepository::new(db.pool()));
    
    let user_data = CreateUserRequest {
        email: "duplicate@example.com".to_string(),
        name: "User 1".to_string(),
    };
    
    // First creation succeeds
    service.create_user(user_data.clone()).await.unwrap();
    
    // Second creation fails
    let result = service.create_user(user_data).await;
    assert!(matches!(result, Err(UserError::DuplicateEmail(_))));
    
    db.cleanup().await;
}

End-to-End Tests

python
# ✅ DO: Full user flow, realistic scenarios
import pytest
from playwright.async_api import async_playwright

@pytest.mark.e2e
async def test_user_registration_flow():
    """Test complete user registration from landing page to dashboard."""
    async with async_playwright() as p:
        # Arrange
        browser = await p.chromium.launch()
        page = await browser.new_page()
        
        # Act - Navigate and register
        await page.goto('http://localhost:3000')
        await page.click('text=Sign Up')
        await page.fill('[name="email"]', 'newuser@example.com')
        await page.fill('[name="password"]', 'SecurePass123!')
        await page.fill('[name="name"]', 'New User')
        await page.click('button[type="submit"]')
        
        # Assert - Redirected to dashboard
        await page.wait_for_url('**/dashboard')
        welcome_text = await page.text_content('h1')
        assert 'Welcome, New User' in welcome_text
        
        # Assert - Can see user profile
        await page.click('[data-testid="user-menu"]')
        email_display = await page.text_content('[data-testid="user-email"]')
        assert email_display == 'newuser@example.com'
        
        await browser.close()

@pytest.mark.e2e
async def test_invalid_login_shows_error():
    """Test that invalid credentials show appropriate error."""
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        page = await browser.new_page()
        
        await page.goto('http://localhost:3000/login')
        await page.fill('[name="email"]', 'nonexistent@example.com')
        await page.fill('[name="password"]', 'wrongpassword')
        await page.click('button[type="submit"]')
        
        # Should show error, not redirect
        error_message = await page.text_content('[role="alert"]')
        assert 'Invalid credentials' in error_message
        
        await browser.close()

Property-Based Tests

python
# ✅ DO: Test properties that should always hold
from hypothesis import given, strategies as st

@given(st.integers(min_value=0, max_value=1000))
def test_user_age_validation(age: int):
    """User age should always be validated correctly."""
    user = User(name="Test", age=age)
    
    if age < 13:
        assert not user.is_eligible_for_account()
    elif 13 <= age < 18:
        assert user.requires_parental_consent()
    else:
        assert user.is_eligible_for_account()

@given(st.emails())
def test_email_normalization_is_idempotent(email: str):
    """Normalizing an email twice should give same result."""
    normalized_once = normalize_email(email)
    normalized_twice = normalize_email(normalized_once)
    assert normalized_once == normalized_twice

@given(st.lists(st.integers(), min_size=0, max_size=100))
def test_sort_always_produces_sorted_list(numbers: list[int]):
    """Our sort function should always produce sorted output."""
    sorted_nums = custom_sort(numbers)
    
    # Property: result should be sorted
    for i in range(len(sorted_nums) - 1):
        assert sorted_nums[i] <= sorted_nums[i + 1]
    
    # Property: result should contain same elements
    assert sorted(sorted_nums) == sorted(numbers)

Branch Coverage Strategies (from migrate-cli)

Understanding Coverage Metrics

Coverage Types (in priority order):

  1. Branch Coverage (MOST IMPORTANT) - Tests all conditional paths (if/else, switch, ternary)
  2. Statement Coverage - Tests all executable statements
  3. Function Coverage - Tests all functions are called
  4. Line Coverage - Tests all lines of code are executed

Why Branch Coverage Matters Most:

  • ✅ Catches untested error paths
  • ✅ Validates edge case handling
  • ✅ Ensures all conditional logic tested
  • ✅ Prevents bugs in rarely-executed paths

Example: 72.95% Branch Coverage Achievement (migrate-cli)

  • Started at 67.62% (below 70% threshold)
  • Identified 42 untested branches in database clients
  • Added error path tests (connection failures, query errors)
  • Result: 72.95% (exceeded threshold)

Branch Coverage Thinking Pattern

For every function, systematically identify branches:

Step 1: Map All Conditional Branches

typescript
// Example function with 6 branches
async function createUser(data: UserData): Promise<User> {
  // Branch 1 & 2: Email validation
  if (!data.email || !isValidEmail(data.email)) {
    throw new ValidationError('email', 'Invalid email format');
  }
  
  // Branch 3 & 4: Name validation
  if (!data.name || data.name.length < 2) {
    throw new ValidationError('name', 'Name too short');
  }
  
  try {
    // Branch 5: Success path
    return await userRepository.save(data);
  } catch (error) {
    // Branch 6: Database error path
    if (isDatabaseError(error) && error.code === '23505') {
      throw new DuplicateUserError('User already exists');
    }
    throw new DatabaseError('Failed to create user', { cause: error });
  }
}

Branch Inventory:

  1. Missing email → ValidationError
  2. Invalid email format → ValidationError
  3. Missing name → ValidationError
  4. Name too short → ValidationError
  5. Successful save → User returned
  6. Duplicate key error → DuplicateUserError
  7. Other database error → DatabaseError

Step 2: Create Test Cases for ALL Branches

typescript
describe('createUser - Branch Coverage', () => {
  // Branch 1: Missing email
  it('should throw ValidationError when email is missing', async () => {
    const data = { name: 'John Doe' } as UserData;
    await expect(createUser(data)).rejects.toThrow(ValidationError);
    await expect(createUser(data)).rejects.toThrow('Invalid email format');
  });
  
  // Branch 2: Invalid email format
  it('should throw ValidationError when email format is invalid', async () => {
    const data = { email: 'invalid-email', name: 'John Doe' };
    await expect(createUser(data)).rejects.toThrow(ValidationError);
  });
  
  // Branch 3: Missing name
  it('should throw ValidationError when name is missing', async () => {
    const data = { email: 'john@example.com' } as UserData;
    await expect(createUser(data)).rejects.toThrow(ValidationError);
    await expect(createUser(data)).rejects.toThrow('Name too short');
  });
  
  // Branch 4: Name too short
  it('should throw ValidationError when name is too short', async () => {
    const data = { email: 'john@example.com', name: 'J' };
    await expect(createUser(data)).rejects.toThrow(ValidationError);
  });
  
  // Branch 5: Success path
  it('should create user when data is valid', async () => {
    const data = { email: 'john@example.com', name: 'John Doe' };
    const user = await createUser(data);
    expect(user.email).toBe('john@example.com');
    expect(user.name).toBe('John Doe');
  });
  
  // Branch 6: Duplicate key error
  it('should throw DuplicateUserError when email already exists', async () => {
    const data = { email: 'existing@example.com', name: 'John Doe' };
    // Setup: Create user first
    await createUser(data);
    // Test: Try to create again
    await expect(createUser(data)).rejects.toThrow(DuplicateUserError);
  });
  
  // Branch 7: General database error
  it('should throw DatabaseError when database operation fails', async () => {
    const data = { email: 'john@example.com', name: 'John Doe' };
    // Mock database failure
    jest.spyOn(userRepository, 'save').mockRejectedValueOnce(new Error('Connection lost'));
    await expect(createUser(data)).rejects.toThrow(DatabaseError);
  });
});

Step 3: Test Error Paths (Often Missed)

Common error paths that need testing:

typescript
// 1. Connection Failures
describe('DatabaseClient - Error Paths', () => {
  it('should throw DatabaseConnectionError when connection fails', async () => {
    const client = new DatabaseClient({ connectionString: 'invalid' });
    await expect(client.connect()).rejects.toThrow(DatabaseConnectionError);
  });
  
  it('should throw when querying without connection', async () => {
    const client = new DatabaseClient(config);
    // Don't connect first
    await expect(client.query('SELECT 1')).rejects.toThrow('Not connected');
  });
});

// 2. Async Operation Errors
describe('MigrationRunner - Async Errors', () => {
  it('should handle migration execution failure', async () => {
    const runner = new MigrationRunner();
    const badMigration = { name: '001', sql: 'INVALID SQL' };
    await expect(runner.executeMigration(badMigration))
      .rejects.toThrow(MigrationExecutionError);
  });
  
  it('should cleanup connection even if migration fails', async () => {
    const runner = new MigrationRunner();
    const disconnectSpy = jest.spyOn(DatabaseClient.prototype, 'disconnect');
    
    try {
      await runner.run({ migrations: [badMigration] });
    } catch {
      // Expected to fail
    }
    
    // But should still cleanup
    expect(disconnectSpy).toHaveBeenCalled();
  });
});

// 3. Null/Undefined Checks
describe('ConfigLoader - Null Handling', () => {
  it('should throw when config file does not exist', () => {
    expect(() => loadConfig('/nonexistent/config.json'))
      .toThrow(ConfigurationError);
  });
  
  it('should throw when config JSON is invalid', () => {
    fs.writeFileSync('/tmp/bad.json', '{ invalid json }');
    expect(() => loadConfig('/tmp/bad.json'))
      .toThrow('Invalid JSON');
  });
  
  it('should throw when required field is missing', () => {
    fs.writeFileSync('/tmp/incomplete.json', '{"database":{}}');
    expect(() => loadConfig('/tmp/incomplete.json'))
      .toThrow(ValidationError);
  });
});

// 4. CLI Argument Validation
describe('CLI - Argument Validation', () => {
  it('should exit with code 1 when direction is invalid', async () => {
    const exitSpy = jest.spyOn(process, 'exit').mockImplementation();
    
    await program.parseAsync(['node', 'cli', 'migrate', '--direction', 'sideways']);
    
    expect(exitSpy).toHaveBeenCalledWith(1);
  });
  
  it('should show error message for missing required option', async () => {
    const consoleErrorSpy = jest.spyOn(console, 'error').mockImplementation();
    
    await program.parseAsync(['node', 'cli', 'migrate']);
    
    expect(consoleErrorSpy).toHaveBeenCalledWith(
      expect.stringContaining('required option')
    );
  });
});

Step 4: Test Edge Cases

Edge cases that commonly have untested branches:

typescript
// Empty inputs
it('should handle empty array', () => {
  expect(processUsers([])).toEqual([]);
});

// Null/undefined
it('should handle null input', () => {
  expect(() => validateUser(null)).toThrow(ValidationError);
});

// Boundary values
it('should accept minimum valid age', () => {
  expect(validateAge(13)).toBe(true);
});

it('should reject below minimum age', () => {
  expect(validateAge(12)).toBe(false);
});

// Type coercion issues (TypeScript strict mode catches these)
it('should reject string when number expected', () => {
  expect(() => validateAge('18' as any)).toThrow(TypeError);
});

Branch Coverage Monitoring

Jest Configuration for Coverage Thresholds:

javascript
// jest.config.js
module.exports = {
  coverageThreshold: {
    global: {
      statements: 80,
      branches: 70,    // Focus on branches!
      functions: 80,
      lines: 80,
    },
  },
};

Running Coverage Reports:

bash
# Generate detailed coverage report
npm test -- --coverage

# View HTML report (shows untested branches)
open coverage/lcov-report/index.html

# Focus on specific file
npm test -- --coverage --testPathPattern=userService

Interpreting Coverage Reports:

code
File              | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
------------------|---------|----------|---------|---------|-------------------
userService.ts    |   95.00 |   80.00  |  100.00 |   95.00 | 45-47,89
database.ts       |   88.23 |   62.50  |   85.71 |   88.23 | 123,145-150,201
                                  ⬆️
                           FOCUS HERE FIRST

Common Branch Coverage Pitfalls (migrate-cli lessons)

Pitfall 1: Testing only happy paths

typescript
// ❌ Only tests success case
it('should connect to database', async () => {
  await client.connect();
  expect(client.isConnected()).toBe(true);
});

// ✅ Also test failure case
it('should throw on connection failure', async () => {
  const badClient = new DatabaseClient({ connectionString: 'invalid' });
  await expect(badClient.connect()).rejects.toThrow(DatabaseConnectionError);
});

Pitfall 2: Not testing error type correctly

typescript
// ❌ Catches error but doesn't validate type
it('should handle duplicate email', async () => {
  try {
    await createUser({ email: 'existing@example.com' });
  } catch (error) {
    // Test passes even if wrong error thrown
  }
});

// ✅ Validates specific error type
it('should throw DuplicateUserError for duplicate email', async () => {
  await expect(createUser({ email: 'existing@example.com' }))
    .rejects.toThrow(DuplicateUserError);
});

Pitfall 3: Over-mocking (hides real branches)

typescript
// ❌ Mocks so much that branches aren't exercised
it('should create user', async () => {
  const mockRepo = {
    save: jest.fn().mockResolvedValue({ id: '123' })
  };
  // Never tests validation logic branches!
  const result = await createUser({ email: 'test@example.com' });
});

// ✅ Test real validation branches
it('should validate email before saving', async () => {
  await expect(createUser({ email: 'invalid' }))
    .rejects.toThrow(ValidationError);
});

Pitfall 4: Forgetting finally blocks

typescript
// ❌ Doesn't test cleanup path
it('should run migrations', async () => {
  await runner.run(migrations);
  // Never tests if disconnect() is called
});

// ✅ Test cleanup even on error
it('should cleanup connection on migration failure', async () => {
  const disconnectSpy = jest.spyOn(client, 'disconnect');
  
  try {
    await runner.run([badMigration]);
  } catch {
    // Expected
  }
  
  expect(disconnectSpy).toHaveBeenCalled();
});

Achieving 70%+ Branch Coverage Checklist

When branch coverage is below threshold, systematically check:

  • Error paths tested? (try/catch blocks, error handling)
  • All if/else branches covered? (both true and false paths)
  • Switch statement cases tested? (including default case)
  • Ternary operators tested? (both branches of condition ? a : b)
  • Early returns tested? (guard clauses)
  • Async error handling tested? (Promise rejections)
  • Null/undefined checks tested? (optional chaining, null checks)
  • Validation logic tested? (all validation failure paths)
  • Edge cases tested? (empty arrays, null, boundary values)
  • Finally blocks tested? (cleanup code execution)

migrate-cli Branch Coverage Achievement Timeline

Initial state: 67.62% (FAIL - below 70% threshold)

  • 42 untested branches in database clients
  • Missing error path tests
  • No async error handling tests

Iteration 1: Added database error tests → 69.12%

  • Tested connection failures
  • Tested query errors
  • Still below threshold

Iteration 2: Added validation tests → 71.23%

  • Tested all validation failure paths
  • Tested CLI argument validation
  • Exceeded threshold! ✅

Iteration 3: Edge case cleanup → 72.95% (FINAL)

  • Tested null/undefined handling
  • Tested empty input cases
  • Added finally block tests

Key Learning: Focus on error paths first - they have the most untested branches.

Test Organization

Directory Structure

code
tests/
├── unit/                  # Fast, isolated unit tests
│   ├── services/
│   │   ├── userService.test.ts
│   │   └── authService.test.ts
│   └── utils/
│       └── validation.test.ts
├── integration/           # Tests with real dependencies
│   ├── database/
│   │   └── userRepository.test.ts
│   └── api/
│       └── userEndpoints.test.ts
├── e2e/                   # Full user flow tests
│   ├── registration.spec.ts
│   └── checkout.spec.ts
├── fixtures/              # Test data
│   ├── users.json
│   └── products.json
└── helpers/               # Test utilities
    ├── testDatabase.ts
    └── mockServices.ts

Test Data Management

Fixtures & Factories

typescript
// ✅ DO: Use factories for test data
export class UserFactory {
  private static counter = 0;
  
  static create(overrides: Partial<User> = {}): User {
    const id = ++this.counter;
    return {
      id: `user-${id}`,
      email: `user${id}@example.com`,
      name: `Test User ${id}`,
      createdAt: new Date(),
      isActive: true,
      ...overrides,
    };
  }
  
  static createMany(count: number, overrides: Partial<User> = {}): User[] {
    return Array.from({ length: count }, () => this.create(overrides));
  }
}

// Usage
const user = UserFactory.create({ email: 'specific@example.com' });
const users = UserFactory.createMany(10, { isActive: false });

// ❌ DON'T: Hardcode test data everywhere
const user = {
  id: '123',
  email: 'test@example.com',
  name: 'Test',
  createdAt: new Date('2024-01-01'),
  isActive: true,
};

Mocking Guidelines

When to Mock

  • ✅ External APIs (third-party services)
  • ✅ Slow operations (file system, network)
  • ✅ Non-deterministic behavior (random, dates)
  • ❌ Internal business logic (test the real thing)
  • ❌ Simple pure functions (no need)

Mock Patterns

typescript
// ✅ DO: Clear mocks with expected behavior
const mockEmailService = {
  sendEmail: jest.fn().mockResolvedValue({ messageId: '123' }),
};

// Verify interaction
await service.registerUser(userData);
expect(mockEmailService.sendEmail).toHaveBeenCalledWith({
  to: 'user@example.com',
  subject: 'Welcome!',
  body: expect.stringContaining('registration'),
});

// ❌ DON'T: Mock everything
// (Testing mocks, not real behavior)

Coverage Targets

Minimum Requirements

  • Critical paths: 100% (auth, payments, data integrity)
  • Business logic: 90%+
  • Utilities: 80%+
  • UI components: 70%+ (focus on logic, not styling)

What to Test

  • ✅ Public APIs and interfaces
  • ✅ Business logic and validation
  • ✅ Error handling paths
  • ✅ Edge cases and boundaries
  • ❌ Third-party library code
  • ❌ Auto-generated code

Test Documentation

typescript
/**
 * User Service Test Suite
 * 
 * Covers:
 * - User creation with validation
 * - User retrieval and updates
 * - Authentication flows
 * - Error handling (duplicate emails, invalid data)
 * 
 * Test Data: Uses UserFactory for consistent test users
 * External Dependencies: Mocked EmailService, real in-memory database
 */
describe('UserService', () => {
  // Tests here
});

Evidence Format

markdown
## Test Generation Evidence

**Component**: UserService
**Test Count**: 23 tests (18 unit, 5 integration)
**Coverage**: 94% (lines), 89% (branches)
**Execution Time**: 142ms (unit), 3.2s (integration)

### Test Scenarios Covered
- ✅ User creation (happy path)
- ✅ Validation errors (invalid email, weak password)
- ✅ Duplicate email handling
- ✅ User retrieval (found/not found)
- ✅ Authentication flows
- ✅ Password reset workflow
- ✅ Email verification

### Coverage Gaps
- ⚠️ Concurrent user creation (race condition testing)
- ⚠️ Account deletion (soft delete verification)

### Recommendations
- Add property-based tests for email normalization
- Increase integration test coverage for database constraints

Integration with Code Generator

  1. Generate tests BEFORE code (TDD approach)
  2. Tests define expected behavior clearly
  3. Code Generator implements to pass tests
  4. Iterative refinement if tests fail

Success Criteria

  • All test scenarios from specification covered
  • Tests are independent and can run in any order
  • Fast execution (<5s for unit tests)
  • Clear failure messages
  • No flaky tests
  • Meets coverage targets