GrepAI Chunking Configuration

This skill covers how GrepAI splits code files into chunks for embedding, and how to optimize chunking for your codebase.

When to Use This Skill

•Optimizing search accuracy
•Adjusting for code style (verbose vs. concise)
•Troubleshooting search results
•Understanding how indexing works

What is Chunking?

Chunking is the process of splitting source files into smaller segments for embedding:

code

┌─────────────────────────────────────┐
│         Large Source File           │
│         (1000+ tokens)              │
└─────────────────────────────────────┘
                  ↓
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Chunk 1 │ │ Chunk 2 │ │ Chunk 3 │
│ ~512    │ │ ~512    │ │ ~512    │
│ tokens  │ │ tokens  │ │ tokens  │
└─────────┘ └─────────┘ └─────────┘
                  ↓
          Each chunk gets
          its own embedding

Why Chunking Matters

Embedding models have optimal input sizes:

•Too large chunks: Less precise search results
•Too small chunks: Lost context, fragmented results
•Just right: Good balance of precision and context

Configuration

Basic Settings

yaml

# .grepai/config.yaml
chunking:
  size: 512      # Tokens per chunk
  overlap: 50    # Overlap between chunks

Understanding Parameters

Chunk Size

The target number of tokens per chunk.

Size	Effect
256	More precise, less context
512	Balanced (default)
1024	More context, less precise

Overlap

Tokens shared between adjacent chunks. Preserves context at boundaries.

Overlap	Effect
0	No overlap, may lose context at boundaries
50	Standard overlap (default)
100	More context, larger index

Visualization

With size=512 and overlap=50:

code

File: auth.go (1000 tokens)

Chunk 1: tokens 1-512
         ┌────────────────────────────────────┐
         │ func Login(user, pass)...          │
         └────────────────────────────────────┘
                                    ↘
                              50 token overlap
                                    ↙
Chunk 2: tokens 463-974
         ┌────────────────────────────────────┐
         │ ...validate credentials...         │
         └────────────────────────────────────┘
                                    ↘
                              50 token overlap
                                    ↙
Chunk 3: tokens 925-1000
         ┌──────────────┐
         │ ...return    │
         └──────────────┘

Recommended Settings by Language

Verbose Languages (Java, C#)

yaml

chunking:
  size: 768    # Larger to capture full methods
  overlap: 75

Concise Languages (Go, Python)

yaml

chunking:
  size: 512    # Standard size
  overlap: 50

Very Concise (Rust, Zig)

yaml

chunking:
  size: 384    # Smaller for precise results
  overlap: 40

Recommended Settings by Codebase

Small Functions (Microservices)

yaml

chunking:
  size: 384    # Capture individual functions
  overlap: 40

Large Classes (Monolith)

yaml

chunking:
  size: 768    # Capture more context
  overlap: 100

Mixed Codebase

yaml

chunking:
  size: 512    # Balanced default
  overlap: 50

How Tokens are Counted

GrepAI uses approximate token counting:

•~4 characters = 1 token (for English text)
•Code varies based on identifiers and syntax

Example:

func calculateTotal(items []Item) float64 {
    total := 0.0
    for _, item := range items {
        total += item.Price * float64(item.Quantity)
    }
    return total
}

≈ 45 tokens

Impact on Index Size

Larger overlap = more chunks = larger index:

Size	Overlap	Chunks per 10K tokens	Index Impact
512	0	~20	Smallest
512	50	~22	Standard
512	100	~24	+10%
256	50	~44	+100%

Impact on Search Quality

Too Small Chunks (size: 128)

code

Query: "authentication middleware"

Result: "...c.AbortWithStatus(401)..."
        (Fragment, missing context)

Just Right (size: 512)

code

Query: "authentication middleware"

Result: "func AuthMiddleware() gin.HandlerFunc {
            return func(c *gin.Context) {
                token := c.GetHeader("Authorization")
                if token == "" {
                    c.AbortWithStatus(401)
                    return
                }
                // validate token...
            }
        }"
        (Complete function with context)

Too Large Chunks (size: 2048)

code

Query: "authentication middleware"

Result: "// Multiple unrelated functions...
        func AuthMiddleware()... (your match)
        func LoggingMiddleware()...
        func CORSMiddleware()..."
        (Too much noise)

Experimentation

Testing Different Settings

•Try smaller chunks for more precise results:

yaml

chunking:
  size: 384
  overlap: 40

•Re-index:

bash

rm .grepai/index.gob
grepai watch

•Test with searches:

bash

grepai search "your query"

•Adjust and repeat until satisfied.

Comparing Results

Before changing settings, save a search result:

bash

grepai search "authentication" > before.txt

After changing settings and re-indexing:

bash

grepai search "authentication" > after.txt
diff before.txt after.txt

Chunk Boundaries

GrepAI tries to split at logical boundaries:

•Empty lines (function/class boundaries)
•Closing braces
•Statement ends

This means actual chunk sizes may vary slightly from the target.

Best Practices

•Start with defaults: 512/50 works well for most codebases
•Adjust based on code style: Verbose = larger, concise = smaller
•Test with real queries: See what your searches return
•Re-index after changes: Must regenerate embeddings
•Consider overlap: Don't set to 0 unless index size is critical

Common Issues

❌ Problem: Search results are too fragmented ✅ Solution: Increase chunk size:

yaml

chunking:
  size: 768

❌ Problem: Search results have too much irrelevant context ✅ Solution: Decrease chunk size:

yaml

chunking:
  size: 384

❌ Problem: Results miss related code at function boundaries ✅ Solution: Increase overlap:

yaml

chunking:
  overlap: 100

❌ Problem: Index is too large ✅ Solutions:

•Decrease overlap
•Increase chunk size
•Add more ignore patterns

Output Format

Chunking status:

code

✅ Chunking Configuration

   Size: 512 tokens
   Overlap: 50 tokens

   Index Statistics:
   - Total files: 245
   - Total chunks: 1,234
   - Avg chunks/file: 5.0
   - Avg chunk size: 478 tokens

   Recommendations:
   - Current settings are balanced
   - Consider size: 384 for more precise results
   - Consider size: 768 for more context