RAG Research Skill

Use this skill when users ask about RAG (Retrieval-Augmented Generation), semantic search, document indexing, embeddings, vector databases, or chunking strategies. This skill provides best practices for working with the rag-research plugin and optimizing document retrieval.

When to Use

Trigger this skill when users:

• •Ask about RAG, embeddings, or semantic search concepts
• •Want to optimize their document indexing strategy
• •Need help with chunking or retrieval quality
• •Ask "how does rag-research work?" or "how to improve search results?"
• •Troubleshoot poor search results or missing information

Core Concepts

Document Indexing Pipeline

1. •Load Document: Extract text from PDF, Markdown, or Text files
2. •Chunk Text: Split into overlapping segments (default: 512 chars, 50 overlap)
3. •Generate Embeddings: Convert chunks to vectors using FastEmbed (BAAI/bge-small-en-v1.5)
4. •Store in Qdrant: Persist vectors with metadata for retrieval

Embedding Models

The plugin uses FastEmbed with ONNX Runtime for efficient CPU inference:

Chunking Strategies

Chunk size affects retrieval quality:

• •Smaller chunks (256-512): More precise, may lose context
• •Larger chunks (1024+): More context, may dilute relevance
• •Overlap (10-20%): Prevents information loss at boundaries

Recommend: Start with defaults (512/50), adjust based on results.

Search Quality Tips

1. •Use specific queries: "Mistral OCR API configuration" > "OCR"
2. •Check coverage: Run /rag-research:list to verify documents are indexed
3. •Increase limit: Use --limit 20 for comprehensive research
4. •Review scores: Scores > 0.7 are highly relevant, < 0.5 may be tangential

Troubleshooting

Poor Search Results

1. •Check if document is indexed: /rag-research:list --filter "keyword"
2. •Re-index with different chunking: Adjust CHUNK_SIZE in .env
3. •Use more specific queries: Add domain-specific terms
4. •Verify embeddings: Check model compatibility

PDF Extraction Issues

1. •Enable Mistral OCR: Set MISTRAL_API_KEY in .env for scanned PDFs
2. •Fallback to pypdf: Use --no-ocr flag for text-based PDFs
3. •Check file permissions: Ensure PDF is readable

Database Issues

1. •Reset database: rm -rf ~/.rag-research and re-index
2. •Check disk space: Qdrant needs space for vectors
3. •Verify installation: uv run rag-research stats

Configuration Reference

See references/configuration.md for detailed settings documentation.

Examples

See references/examples.md for common usage patterns.