Terraform Provider Testing Skill
Comprehensive test-driven development for Terraform providers emphasizing iterative co-development of tests, generators, and schemas. This skill drives the development of production-ready provider code through systematic testing and root cause analysis.
Core Philosophy: Never Skip, Always Fix
The Co-Development Mindset
Tests, generators, and schemas are interdependent systems that evolve together. When tests fail, the response is ALWAYS to fix the root cause - never to skip, disable, or work around.
┌─────────────────────────────────────────────────────────────────────────────┐ │ CO-DEVELOPMENT TRIANGLE │ │ │ │ ┌─────────┐ │ │ │ TESTS │ │ │ └────┬────┘ │ │ │ │ │ ┌─────────────────┼─────────────────┐ │ │ │ │ │ │ │ ▼ ▼ ▼ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │GENERATORS│◄────►│ SCHEMAS │◄────►│ RESOURCES│ │ │ └──────────┘ └──────────┘ └──────────┘ │ │ │ │ Tests drive → Generator fixes → Schema corrections → Resource updates │ └─────────────────────────────────────────────────────────────────────────────┘
Never Skip Rules (CRITICAL)
ALWAYS develop and fix unless ONE of these conditions applies:
| Skip Condition | Example | Action |
|---|---|---|
| External cloud credentials required | AWS VPC Site needs AWS_ACCESS_KEY_ID | t.Skip("requires AWS credentials") |
| Premium/enterprise licensing required | Bot defense needs advanced license | t.Skip("requires premium licensing") |
| Third-party service account needed | External OIDC provider | t.Skip("requires external service account") |
NEVER skip for these reasons - fix them instead:
| Invalid Skip Reason | Correct Action |
|---|---|
| "Schema attribute missing" | Fix generator, regenerate |
| "State drift on nested blocks" | Fix generator's Read/Schema handling |
| "Import produces diff" | Fix ImportState function in generator |
| "API returns unexpected format" | Fix client types or resource parsing |
| "Test is too complex" | Break into smaller tests, still implement |
| "Generator doesn't support this" | Enhance generator to support it |
Development Priority Order
When a test fails, investigate and fix in this order:
1. GENERATOR ISSUE? │ Symptoms: Same failure across ALL resources of similar type │ Fix: tools/generate-all-schemas.go → go generate ./... │ 2. SCHEMA ISSUE? │ Symptoms: Attribute handling, type mismatches, nested block problems │ Fix: Generator's schema generation logic → regenerate │ 3. RESOURCE IMPLEMENTATION ISSUE? │ Symptoms: Single resource fails, API-specific parsing │ Fix: The specific resource's CRUD methods │ 4. TEST ISSUE? │ Symptoms: Wrong assertions, incorrect config, test logic error │ Fix: Test code only AFTER ruling out 1-3
Coding Standards (MANDATORY)
Go Indentation: Tabs (gofmt Standard)
CRITICAL: All Go source code MUST use tabs for indentation - this is enforced by gofmt.
| File Type | Indentation | Standard |
|---|---|---|
Go source files (.go) | Tabs | gofmt enforced |
| Embedded Terraform/HCL in heredocs | 2 spaces | Terraform convention |
Why tabs for Go:
- •
gofmtis the canonical Go formatter and uses tabs - •All Go code MUST pass
gofmtbefore commit - •This is non-negotiable - it's the Go community standard
- •CI/CD will fail if code doesn't match
gofmtoutput
Go Test File Formatting
// ✅ CORRECT: Tabs for Go code indentation (gofmt standard)
func TestAccExampleResource_basic(t *testing.T) {
t.Parallel()
rName := acctest.RandomWithPrefix("tf-acc-test")
resourceName := "f5xc_example.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
ProtoV6ProviderFactories: testAccProtoV6ProviderFactories,
Steps: []resource.TestStep{
{
Config: testAccExampleConfig_basic(rName),
},
},
})
}
Embedded Terraform Configuration Formatting
Terraform/HCL inside Go heredocs uses 2 spaces (Terraform convention):
// ✅ CORRECT: Tabs for Go, 2 spaces for embedded HCL
func testAccExampleConfig_basic(rName string) string {
return fmt.Sprintf(`
resource "f5xc_namespace" "test" {
name = %[1]q
}
resource "f5xc_example" "test" {
name = %[1]q
namespace = f5xc_namespace.test.name
nested_block {
attribute = "value"
}
}
`, rName)
}
Key distinction:
- •The
funcandreturn fmt.Sprintflines use tabs (Go code) - •The HCL content inside the backticks uses 2 spaces (Terraform convention)
Formatting Verification
Always run gofmt before committing:
# Format all Go files in place gofmt -w . # Check formatting without modifying (CI mode) gofmt -d . | grep -q . && echo "Formatting needed" || echo "OK" # Format with goimports (also organizes imports) goimports -w .
Editor Configuration
Configure your editor for Go's tab standard:
VS Code settings:
{
"[go]": {
"editor.insertSpaces": false,
"editor.tabSize": 4,
"editor.formatOnSave": true
}
}
Vim settings:
" For Go files - use tabs (gofmt standard) autocmd FileType go setlocal noexpandtab tabstop=4 shiftwidth=4
Part 1: Iterative Development Workflow
The Test-Fix-Verify Cycle
Every test development follows this iterative pattern:
# Phase 1: Write Initial Test vim internal/provider/example_resource_test.go # Phase 2: Run Test (expect failures) F5XC_API_URL="https://tenant.console.ves.volterra.io" \ F5XC_P12_FILE="/path/to/cert.p12" \ F5XC_P12_PASSWORD="password" \ # pragma: allowlist secret TF_ACC=1 go test -v -timeout 15m \ -run TestAccExampleResource_basic ./internal/provider/... # Phase 3: Analyze Failure - Determine Root Cause # Ask: Is this a GENERATOR issue affecting all resources? # Is this a SCHEMA issue with attribute handling? # Is this a RESOURCE issue with this specific API? # Is this a TEST issue with my assertions? # Phase 4: Fix at the Appropriate Level # If generator: vim tools/generate-all-schemas.go && go generate ./... # If schema: Fix in generator, regenerate # If resource: Fix specific resource implementation # If test: Fix test assertions/config # Phase 5: Re-run and Verify TF_ACC=1 go test -v -timeout 15m \ -run TestAccExampleResource_basic ./internal/provider/... # REPEAT until test passes - NEVER skip
Failure Analysis Decision Tree
Test Failed
│
├── Error mentions "attribute not found in schema"?
│ └── FIX: Generator schema generation → regenerate ALL resources
│
├── Error shows state drift on nested blocks?
│ └── FIX: Generator's Read method handling of nested structures
│
├── ImportStateVerify shows diff?
│ └── FIX: Generator's ImportState function
│
├── API returns 400/422 with field error?
│ └── FIX: Generator's Create/Update request building
│
├── Computed field not populated after Read?
│ └── FIX: Generator's Read response → state mapping
│
├── Same error pattern across multiple resource types?
│ └── FIX: Generator (systemic issue) → regenerate
│
├── Error only in this specific resource?
│ └── FIX: Resource implementation OR client types
│
└── Assertion doesn't match expected value?
└── INVESTIGATE: Is expected value correct? Is resource behavior correct?
├── Resource behavior wrong → Fix resource
└── Assertion wrong → Fix test
Generator-Test Co-Development Examples
Example 1: Nested Block State Drift
TEST FAILURE: inconsistent result after apply - default_route_pools.0.pool.namespace: "" => "test-ns" ANALYSIS: Generator's Read method doesn't populate nested blocks correctly FIX LOCATION: tools/generate-all-schemas.go FIX TYPE: Update nested block flattening logic STEPS: 1. Identify pattern in generator for nested block handling 2. Fix the flattening/expansion logic 3. go generate ./... 4. Re-run test 5. Verify ALL similar resources now work
Example 2: Missing Schema Attribute
TEST FAILURE: attribute "labels" not found in schema ANALYSIS: Generator doesn't include labels attribute from OpenAPI spec FIX LOCATION: tools/generate-all-schemas.go FIX TYPE: Add labels field to schema generation STEPS: 1. Check OpenAPI spec confirms labels exists 2. Update generator to include labels in schema 3. go generate ./... 4. Re-run test 5. Verify all resources with labels now have the attribute
Example 3: Import State Incomplete
TEST FAILURE: ImportStateVerify found differences: - description: "test" => "" ANALYSIS: ImportState doesn't set all attributes from API response FIX LOCATION: Generator's ImportState template or Read method FIX TYPE: Ensure Read populates all importable attributes STEPS: 1. Check API response includes description 2. Fix generator's Read to map description to state 3. go generate ./... 4. Re-run import test 5. Verify import now preserves all attributes
Part 2: Comprehensive Test Structure
TestCase Architecture
func TestAccExampleResource_basic(t *testing.T) {
t.Parallel()
rName := acctest.RandomWithPrefix("tf-acc-test")
resourceName := "f5xc_example.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
ProtoV6ProviderFactories: testAccProtoV6ProviderFactories,
CheckDestroy: testAccCheckExampleDestroy,
Steps: []resource.TestStep{
// Step 1: Create and verify
{
Config: testAccExampleConfig_basic(rName),
ConfigPlanChecks: resource.ConfigPlanChecks{
PreApply: []plancheck.PlanCheck{
plancheck.ExpectResourceAction(resourceName,
plancheck.ResourceActionCreate),
},
},
ConfigStateChecks: []statecheck.StateCheck{
statecheck.ExpectKnownValue(resourceName,
tfjsonpath.New("name"), knownvalue.StringExact(rName)),
},
},
// Step 2: Import and verify state roundtrip
{
ResourceName: resourceName,
ImportState: true,
ImportStateVerify: true,
},
},
})
}
Comprehensive Test Coverage Requirements
Every testable resource MUST have:
| Test Type | Purpose | Skip Only If |
|---|---|---|
_basic | Create, Read, basic attributes | External credentials/licensing |
_update | Modify mutable attributes | External credentials/licensing |
_import | Import existing resource | External credentials/licensing |
_disappears | Handle external deletion | External credentials/licensing |
| Data source test | Verify data source reads resource | External credentials/licensing |
// REQUIRED test structure for each resource
func TestAccExampleResource_basic(t *testing.T) { ... }
func TestAccExampleResource_update(t *testing.T) { ... }
func TestAccExampleResource_disappears(t *testing.T) { ... }
func TestAccExampleDataSource_basic(t *testing.T) { ... }
Part 3: Modern Assertion Framework
ConfigPlanChecks (Plan-Time Validation)
Validate Terraform plan before resources are applied:
{
Config: testAccConfig,
ConfigPlanChecks: resource.ConfigPlanChecks{
PreApply: []plancheck.PlanCheck{
// Verify expected action
plancheck.ExpectResourceAction("f5xc_namespace.test",
plancheck.ResourceActionCreate),
// Verify known values in plan
plancheck.ExpectKnownValue("f5xc_namespace.test",
tfjsonpath.New("name"),
knownvalue.StringExact("test-ns")),
// Verify unknown values (computed)
plancheck.ExpectUnknownValue("f5xc_namespace.test",
tfjsonpath.New("id")),
},
PostApply: []plancheck.PlanCheck{
// After apply, plan should be empty (no drift)
plancheck.ExpectEmptyPlan(),
},
},
}
ConfigStateChecks (State Validation)
Validate Terraform state after resources are applied:
{
Config: testAccConfig,
ConfigStateChecks: []statecheck.StateCheck{
// Verify exact string value
statecheck.ExpectKnownValue("f5xc_namespace.test",
tfjsonpath.New("name"),
knownvalue.StringExact("test-ns")),
// Verify boolean
statecheck.ExpectKnownValue("f5xc_app_firewall.test",
tfjsonpath.New("blocking"),
knownvalue.Bool(true)),
// Verify list size
statecheck.ExpectKnownValue("f5xc_http_lb.test",
tfjsonpath.New("domains"),
knownvalue.ListSizeExact(2)),
// Verify nested object
statecheck.ExpectKnownValue("f5xc_http_lb.test",
tfjsonpath.New("default_route_pools").AtSliceIndex(0),
knownvalue.ObjectPartial(map[string]knownvalue.Check{
"weight": knownvalue.Int64Exact(1),
"priority": knownvalue.Int64Exact(1),
})),
},
}
knownvalue Reference
| Check Type | Usage | Example |
|---|---|---|
StringExact | Exact string match | knownvalue.StringExact("value") |
StringRegexp | Regex match | knownvalue.StringRegexp(regexp.MustCompile(^tf-)) |
Bool | Boolean value | knownvalue.Bool(true) |
Int64Exact | Exact integer | knownvalue.Int64Exact(42) |
ListExact | Exact list | knownvalue.ListExact([]knownvalue.Check{...}) |
ListSizeExact | List length | knownvalue.ListSizeExact(3) |
ListPartial | Partial list match | knownvalue.ListPartial(map[int]knownvalue.Check{0: ...}) |
ObjectExact | Exact object | knownvalue.ObjectExact(map[string]knownvalue.Check{...}) |
ObjectPartial | Partial object | knownvalue.ObjectPartial(map[string]knownvalue.Check{...}) |
Null | Null value | knownvalue.Null() |
NotNull | Not null | knownvalue.NotNull() |
tfjsonpath Navigation
// Simple attribute
tfjsonpath.New("name")
// Nested attribute
tfjsonpath.New("metadata").AtMapKey("labels")
// List index
tfjsonpath.New("domains").AtSliceIndex(0)
// Complex nested path
tfjsonpath.New("spec").AtMapKey("routes").AtSliceIndex(0).AtMapKey("match")
Part 4: Namespace Requirements (CRITICAL)
Always Use Custom Namespaces
RULE: Create a custom test namespace unless the API spec EXPLICITLY requires system, default, or shared.
// ✅ CORRECT: Custom namespace
func testAccExampleConfig_basic(rName string) string {
return fmt.Sprintf(`
resource "f5xc_namespace" "test" {
name = %[1]q
}
resource "f5xc_example" "test" {
name = %[1]q
namespace = f5xc_namespace.test.name
}
`, rName)
}
// ❌ WRONG: Using system namespace without spec requirement
func testAccExampleConfig_wrong(name string) string {
return fmt.Sprintf(`
resource "f5xc_example" "test" {
name = %[1]q
namespace = "system" // NEVER unless spec requires it
}
`, name)
}
Namespace Decision Tree
Does the OpenAPI spec require a specific namespace?
├── YES: system/default/shared required
│ ├── Document WHY in test comments with spec reference
│ └── Use the required namespace
└── NO: Custom namespace allowed
└── ALWAYS create f5xc_namespace.test and reference it
Part 5: Resource Testing Patterns
Complete CRUD Test
func TestAccExampleResource_basic(t *testing.T) {
t.Parallel()
rName := acctest.RandomWithPrefix("tf-acc-test")
resourceName := "f5xc_example.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
ProtoV6ProviderFactories: testAccProtoV6ProviderFactories,
CheckDestroy: testAccCheckExampleDestroy,
Steps: []resource.TestStep{
// Create
{
Config: testAccExampleConfig_basic(rName),
ConfigPlanChecks: resource.ConfigPlanChecks{
PreApply: []plancheck.PlanCheck{
plancheck.ExpectResourceAction(resourceName,
plancheck.ResourceActionCreate),
},
},
ConfigStateChecks: []statecheck.StateCheck{
statecheck.ExpectKnownValue(resourceName,
tfjsonpath.New("name"), knownvalue.StringExact(rName)),
},
},
// Update
{
Config: testAccExampleConfig_updated(rName),
ConfigPlanChecks: resource.ConfigPlanChecks{
PreApply: []plancheck.PlanCheck{
plancheck.ExpectResourceAction(resourceName,
plancheck.ResourceActionUpdate),
},
},
ConfigStateChecks: []statecheck.StateCheck{
statecheck.ExpectKnownValue(resourceName,
tfjsonpath.New("description"),
knownvalue.StringExact("updated")),
},
},
// Import
{
ResourceName: resourceName,
ImportState: true,
ImportStateVerify: true,
},
},
})
}
CheckDestroy Implementation
func testAccCheckExampleDestroy(s *terraform.State) error {
client := testAccProvider.Meta().(*client.Client)
for _, rs := range s.RootModule().Resources {
if rs.Type != "f5xc_example" {
continue
}
namespace := rs.Primary.Attributes["namespace"]
name := rs.Primary.Attributes["name"]
_, err := client.GetExample(context.Background(), namespace, name)
if err == nil {
return fmt.Errorf("f5xc_example %s/%s still exists", namespace, name)
}
if !client.IsNotFoundError(err) {
return fmt.Errorf("error checking f5xc_example %s/%s: %w",
namespace, name, err)
}
}
return nil
}
Import with Composite ID
{
ResourceName: resourceName,
ImportState: true,
ImportStateVerify: true,
ImportStateIdFunc: func(s *terraform.State) (string, error) {
rs, ok := s.RootModule().Resources[resourceName]
if !ok {
return "", fmt.Errorf("resource not found: %s", resourceName)
}
return fmt.Sprintf("%s/%s",
rs.Primary.Attributes["namespace"],
rs.Primary.Attributes["name"]), nil
},
}
Part 6: Data Source Testing
Data Source Test Pattern
func TestAccExampleDataSource_basic(t *testing.T) {
t.Parallel()
rName := acctest.RandomWithPrefix("tf-acc-test")
resourceName := "f5xc_example.test"
dataSourceName := "data.f5xc_example.test"
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
ProtoV6ProviderFactories: testAccProtoV6ProviderFactories,
Steps: []resource.TestStep{
{
Config: testAccExampleDataSourceConfig(rName),
Check: resource.ComposeAggregateTestCheckFunc(
// Compare data source to resource
resource.TestCheckResourceAttrPair(
dataSourceName, "name",
resourceName, "name"),
resource.TestCheckResourceAttrPair(
dataSourceName, "id",
resourceName, "id"),
resource.TestCheckResourceAttrPair(
dataSourceName, "namespace",
resourceName, "namespace"),
),
},
},
})
}
func testAccExampleDataSourceConfig(rName string) string {
return fmt.Sprintf(`
resource "f5xc_namespace" "test" {
name = %[1]q
}
resource "f5xc_example" "test" {
name = %[1]q
namespace = f5xc_namespace.test.name
}
data "f5xc_example" "test" {
name = f5xc_example.test.name
namespace = f5xc_example.test.namespace
}
`, rName)
}
Part 7: Common Failures and Fixes
Failure Resolution Matrix
| Failure | Root Cause | Fix Location | Fix Action |
|---|---|---|---|
attribute not found in schema | Generator schema incomplete | Generator | Add attribute to schema generation |
planned value does not match | Computed attribute not set | Generator Read | Map API response to state |
inconsistent result after apply | Read not populating state | Generator Read | Fix response → state mapping |
import produces diff | ImportState incomplete | Generator ImportState | Ensure all attributes set |
CheckDestroy failed | Delete not working | Resource Delete | Fix delete API call |
namespace not found | Using system namespace | Test config | Use custom namespace |
permission denied | Wrong namespace | Test config | Check namespace permissions |
ExpectResourceAction failed | Wrong action detected | Resource/Test | Check ForceNew attributes |
| State drift on nested blocks | Flattening logic wrong | Generator | Fix nested block handling |
| Boolean always false | Type conversion issue | Generator Schema | Fix bool type handling |
Systematic Debugging Approach
# Step 1: Enable debug logging TF_LOG=DEBUG TF_ACC=1 go test -v -timeout 15m \ -run TestAccExampleResource_basic ./internal/provider/... 2>&1 | tee test.log # Step 2: Find the API request/response grep -A 20 "HTTP Request" test.log grep -A 50 "HTTP Response" test.log # Step 3: Compare API response to state # Look for fields in response not making it to state # Look for type mismatches (string vs int, etc.) # Step 4: Identify fix location # If API has data but state doesn't → Generator Read method # If state has data but plan shows change → Generator Schema/Defaults # If API returns error → Generator Create/Update request building
Part 8: Test Eligibility
Testable Resources (ALWAYS develop tests)
| Category | Examples | Action |
|---|---|---|
| Core resources | namespace, healthcheck, origin_pool | Full test suite |
| Security policies | app_firewall, service_policy, rate_limiter | Full test suite |
| Load balancers | http_loadbalancer, tcp_loadbalancer | Full test suite |
| Network config | virtual_network, network_policy | Full test suite |
| DNS resources | dns_zone, dns_domain | Full test suite |
| Configuration objects | Any policy, rule, or config resource | Full test suite |
Skip-Eligible Resources (document reason)
| Category | Examples | Skip Reason |
|---|---|---|
| Cloud sites | aws_vpc_site, azure_vnet_site, gcp_vpc_site | Requires cloud credentials |
| Cloud integrations | cloud_credentials (AWS/Azure/GCP type) | Requires cloud credentials |
| Premium features | bot_defense*, advanced_* | Requires premium licensing |
| Third-party | External OIDC, external integrations | Requires external accounts |
// Proper skip documentation
func TestAccAWSVPCSite_basic(t *testing.T) {
t.Skip("Skipping: requires AWS credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)")
}
func TestAccBotDefenseAdvanced_basic(t *testing.T) {
t.Skip("Skipping: requires F5 XC premium/enterprise licensing")
}
Part 9: Running Tests
Environment Setup
# Required for ALL acceptance tests export F5XC_P12_FILE="/path/to/api-certificate.p12" export F5XC_P12_PASSWORD="your-password" # pragma: allowlist secret export F5XC_API_URL="https://tenant.console.ves.volterra.io" export TF_ACC=1
Test Execution Commands
# Single test go test -v -timeout 15m -run=TestAccNamespaceResource_basic ./internal/provider/... # All tests for a resource go test -v -timeout 30m -run=TestAccNamespace ./internal/provider/... # With debug logging TF_LOG=DEBUG go test -v -timeout 15m -run=TestAccNamespaceResource_basic ./internal/provider/... # With parallel limit go test -v -timeout 30m -parallel=4 ./internal/provider/... # Multiple specific tests go test -v -timeout 30m -run="TestAccNamespaceResource_basic|TestAccHealthcheckResource_basic" ./internal/provider/...
Part 10: Checklist for New Tests
Pre-Development Checklist
- • Resource is testable (no external credentials/premium licensing required)
- • OpenAPI spec reviewed for namespace requirements
- • Generator supports all required attributes for this resource type
- • Client types exist for API interactions
Test Implementation Checklist
- • Create test with custom namespace (unless spec requires otherwise)
- • Use ConfigPlanChecks for plan assertions
- • Use ConfigStateChecks for state assertions
- • Add import test with proper ImportStateIdFunc
- • Implement CheckDestroy
- • Enable parallel execution with
t.Parallel() - • Data source test if data source exists
Post-Failure Checklist
- • Analyzed failure to determine root cause location
- • If generator issue: Fixed generator, regenerated ALL resources
- • If schema issue: Fixed in generator, regenerated
- • If resource issue: Fixed specific resource implementation
- • If test issue: Fixed test code (only after ruling out above)
- • Re-ran test to verify fix
- • Checked that fix didn't break other tests
Never Do
- •❌ Skip tests because "the schema doesn't support it" - fix the schema
- •❌ Skip tests because "state drift is expected" - fix the state handling
- •❌ Skip tests because "import doesn't work" - fix the import logic
- •❌ Comment out failing assertions - fix the root cause
- •❌ Use
ImportStateVerifyIgnorewithout documented reason - •❌ Use system/default/shared namespace without spec requirement