Babuza WAL Skill

Name: babuza-wal
Rating: 76
Author: fanaujie

Package: github.com/fanaujie/babuza/pkg/wal

Efficient Write-Ahead Log implementations for durable Raft storage.

You are an expert at babuza storage. Help users by:

•Writing code: Configure different WAL backends (Native, Badger, Pebble, ETCD).
•Answering questions: Explain memory efficiency and storage options.

Documentation

Refer to the local files for detailed API:

•./references/wal-api.md - WAL interfaces, backend types, and architecture.

Key Patterns

1. Configuring WAL via Builder

Select the WAL backend using BabuzaComponentBuilder.

import "github.com/fanaujie/babuza/pkg/builder"

func buildComponent() *builder.BabuzaComponent {
    return builder.NewBabuzaComponentBuilder(&builder.BabuzaComponentConfig{
        ClusterId:      1,
        StorageRootDir: "/var/lib/babuza",  // WAL stored in {StorageRootDir}/wal

        // Select WAL Type
        WalType: builder.BabuzaWal,  // Recommended

        // Other required fields
        TransportType: builder.HttpTransport,
        SessionType:   builder.NoOpSession,
        SnapshotType:  builder.DurableSnapshot,
    }).Build()
}

2. Full Example with Raft

import (
    "github.com/fanaujie/babuza/pkg/builder"
    "github.com/fanaujie/babuza/raft"
)

func startNode(stateMachine ibabuza.BaseStateMachine) (*raft.Raft, error) {
    component := builder.NewBabuzaComponentBuilder(&builder.BabuzaComponentConfig{
        ClusterId:      1,
        StorageRootDir: "/tmp/node1",
        TransportType:  builder.HttpTransport,
        WalType:        builder.BabuzaWal,      // Memory-efficient native WAL
        SessionType:    builder.NoOpSession,
        SnapshotType:   builder.DurableSnapshot,
    }).Build()

    cfg := raft.DefaultBabuzaConfig(1, 1, "127.0.0.1:9001")

    peers := raft.NewPeersConfiguration()
    peers.AddPeer(1, "127.0.0.1:9001", false)

    bootstrap, err := raft.NewBootstrapRaftCluster(
        cfg, *peers, stateMachine,
        component.Cluster, component.RaftNode, component.SessionManager,
        component.SnapshotManager, component.WalManager, component.Transport,
        component.Logger, component.MetricsController,
    )
    if err != nil {
        return nil, err
    }

    return raft.NewRaft(cfg, bootstrap, nil)
}

WAL Backends

Backend	Builder Constant	Description
Babuza (Native)	`builder.BabuzaWal`	Recommended. Index-based, 94%+ memory savings vs etcd.
ETCD	`builder.ETCDWal`	Standard etcd WAL. Higher memory usage.
Badger Disk	`builder.BadgerWalDisk`	LSM tree (Badger). Good for high write throughput.
Badger Memory	`builder.BadgerWalMemory`	In-memory Badger. Testing only.
Pebble Disk	`builder.PebbleWalDisk`	LSM tree (CockroachDB). High performance.
Pebble Memory	`builder.PebbleWalMemory`	In-memory Pebble. Testing only.

When to Use Each WAL

Scenario	Recommended WAL
General purpose, memory-constrained	`BabuzaWal`
Compatibility with etcd tooling	`ETCDWal`
Very high write throughput	`BadgerWalDisk` or `PebbleWalDisk`
Testing/Development	`*WalMemory` variants

Memory Efficiency

Babuza's native WAL uses an Index + Cache architecture:

•Index: Keeps only metadata (term, index, offset) in memory.
•Cache: Recent entries in ring buffer.
•Read: Older entries read from disk on demand.

Memory Comparison

Entries	Data Size	etcd Memory	Babuza Memory	Savings
100K	1 KB	102 MB	5.35 MB	94.8%
100K	10 KB	981 MB	5.35 MB	99.5%

Storage Directory Structure

When using StorageRootDir: "/tmp/node1":

•WAL: /tmp/node1/wal/
•Snapshots: /tmp/node1/snap/

When Answering Questions

•Recommendation: Default to BabuzaWal (native) unless the user has specific needs for LSM trees (Badger/Pebble) or etcd compatibility.
•Safety: In-Memory WALs lose data on restart - use only for testing.
•Architecture: Babuza's index-based approach is key to memory efficiency. Unlike etcd which keeps all log entries in RAM, Babuza only keeps metadata.
•Performance: For most workloads, BabuzaWal is sufficient. LSM-based WALs (Badger/Pebble) may help with very high write throughput but add complexity.