Web Search Plus

Multi-provider web search with Intelligent Auto-Routing: Serper (Google), Tavily (Research), Exa (Neural).

NEW in v2.1.0: Intelligent multi-signal analysis with confidence scoring!

⚠️ Important: Don't Modify Core Clawdbot Config

Tavily, Serper, and Exa are NOT core Clawdbot providers.

❌ DON'T add this to ~/.clawdbot/clawdbot.json:

"tools": {
  "web": {
    "search": {
      "provider": "tavily"  // WRONG - will cause errors!
    }
  }
}

✅ DO use this skill's scripts with environment variables:

export TAVILY_API_KEY="your-key"
python3 scripts/search.py -q "your query"

Core Clawdbot only supports brave or perplexity as built-in providers. This skill adds Serper, Tavily, and Exa as additional options via its own scripts.

🧠 Intelligent Auto-Routing

No need to choose a provider — just search! The skill uses multi-signal analysis to understand your query intent:

# These queries are intelligently routed with confidence scoring:
python3 scripts/search.py -q "how much does iPhone 16 cost"     # → Serper (68% MEDIUM)
python3 scripts/search.py -q "how does quantum entanglement work"  # → Tavily (86% HIGH)
python3 scripts/search.py -q "startups similar to Notion"       # → Exa (76% HIGH)
python3 scripts/search.py -q "MacBook Pro M3 specs review"      # → Serper (70% HIGH)
python3 scripts/search.py -q "explain pros and cons of React"   # → Tavily (85% HIGH)
python3 scripts/search.py -q "companies like stripe.com"        # → Exa (100% HIGH)

How It Works

The routing engine analyzes multiple signals:

🛒 Shopping Intent → Serper

📚 Research Intent → Tavily

🔍 Discovery Intent → Exa

Confidence Scoring

Every routing decision includes a confidence level:

Debug Routing Decisions

See the full analysis:

python3 scripts/search.py --explain-routing -q "how much does iPhone 16 Pro cost"

Output:

{
  "query": "how much does iPhone 16 Pro cost",
  "routing_decision": {
    "provider": "serper",
    "confidence": 0.68,
    "confidence_level": "medium",
    "reason": "moderate_confidence_match"
  },
  "scores": {"serper": 7.0, "tavily": 0.0, "exa": 0.0},
  "top_signals": [
    {"matched": "how much", "weight": 4.0},
    {"matched": "brand + product detected", "weight": 3.0}
  ],
  "query_analysis": {
    "word_count": 7,
    "is_complex": false,
    "has_url": null,
    "recency_focused": false
  }
}

🔍 When to Use This Skill vs Built-in Brave Search

Use Built-in Brave Search when:

• •✅ General web searches (news, info, questions)
• •✅ Privacy is important
• •✅ Quick lookups without specific requirements

Use web-search-plus when:

→ Serper (Google results):

• •🛍️ Product specs, prices, shopping - "Compare iPhone 16 vs Samsung S24"
• •📍 Local businesses, places - "Best pizza in Vienna"
• •🎯 "Google it" - Explicitly wants Google results
• •📰 Shopping/images/news - --type shopping/images/news
• •🏆 Knowledge Graph - Structured info (prices, ratings, etc.)

→ Tavily (AI-optimized research):

• •📚 Research questions - "How does quantum computing work?"
• •🔬 Deep dives - Complex multi-part questions
• •📄 Full page content - Not just snippets (--raw-content)
• •🎓 Academic research - Synthesized answers
• •🔒 Domain filtering - --include-domains for trusted sources

→ Exa (Neural semantic search):

• •🔗 Similar pages - "Sites like OpenAI.com" (--similar-url)
• •🏢 Company discovery - "AI companies like Anthropic"
• •📝 Research papers - --category "research paper"
• •💻 GitHub projects - --category github
• •📅 Date-specific - --start-date / --end-date

Provider Comparison

Usage Examples

Auto-Routed (Recommended)

python3 scripts/search.py -q "iPhone 16 Pro Max price"          # → Serper
python3 scripts/search.py -q "how does HTTPS encryption work"   # → Tavily
python3 scripts/search.py -q "startups similar to Notion"       # → Exa

Explicit Provider

python3 scripts/search.py -p serper -q "weather Vienna" --type weather
python3 scripts/search.py -p tavily -q "quantum computing" --depth advanced
python3 scripts/search.py -p exa --similar-url "https://stripe.com" --category company

Configuration

config.json

{
  "auto_routing": {
    "enabled": true,
    "fallback_provider": "serper",
    "confidence_threshold": 0.3,
    "disabled_providers": []
  },
  "serper": {"country": "us", "language": "en"},
  "tavily": {"depth": "advanced"},
  "exa": {"type": "neural"}
}

Output Format

{
  "provider": "serper",
  "query": "iPhone 16 price",
  "results": [{"title": "...", "url": "...", "snippet": "...", "score": 0.95}],
  "answer": "Synthesized answer...",
  "routing": {
    "auto_routed": true,
    "provider": "serper",
    "confidence": 0.78,
    "confidence_level": "high",
    "reason": "high_confidence_match",
    "top_signals": [{"matched": "price", "weight": 3.0}]
  }
}

Environment Setup

# In your .env file (use 'export' prefix!):
export SERPER_API_KEY="your-key"   # https://serper.dev
export TAVILY_API_KEY="your-key"   # https://tavily.com  
export EXA_API_KEY="your-key"      # https://exa.ai

# Then load with: source .env

FAQ

General

Q: How does auto-routing decide which provider to use?

Multi-signal analysis scores each provider based on: price patterns, explanation phrases, similarity keywords, URLs, product+brand combos, and query complexity. Highest score wins. Use --explain-routing to see the decision breakdown.

Q: What if it picks the wrong provider?

Override with -p serper/tavily/exa. Check --explain-routing to understand why it chose differently.

Q: What does "low confidence" mean?

Query is ambiguous (e.g., "Tesla" could be cars, stock, or company). Falls back to Serper. Results may vary.

Q: Can I disable a provider?

Yes! In config.json: "disabled_providers": ["exa"]

API Keys

Q: Which API keys do I need?

At minimum ONE key. You can use just Serper, just Tavily, or all three. Missing keys = that provider is skipped.

Q: Where do I get API keys?

• •Serper: https://serper.dev (100 free searches/month)
• •Tavily: https://tavily.com (1000 free searches/month)
• •Exa: https://exa.ai (limited free tier)

Q: How do I set API keys?

Create .env in your workspace:

bash

export SERPER_API_KEY="your-key"
export TAVILY_API_KEY="your-key"
export EXA_API_KEY="your-key"

Then source .env or add to your shell profile.

Routing Details

Q: How do I know which provider handled my search?

Check routing.provider in JSON output, or [🔍 Searched with: Provider] in chat responses.

Q: Why does it sometimes choose Serper for research questions?

If the query has brand/product signals (e.g., "how does Tesla FSD work"), shopping intent may outweigh research intent. Override with -p tavily.

Q: What's the confidence threshold?

Default: 0.3 (30%). Below this = low confidence, uses fallback. Adjustable in config.json.

Troubleshooting

Q: "No API key found" error?

Make sure keys are exported (not just set): export SERPER_API_KEY="..." and sourced.

Q: Getting empty results?

1. •Check API key is valid
2. •Try a different provider with -p
3. •Some queries have no results (very niche topics)

Q: Rate limited?

Each provider has limits. Spread queries across providers or wait. Serper: 100/month free, Tavily: 1000/month free.

For Clawdbot Users

Q: How do I use this in chat?

Just ask! Clawdbot auto-detects search intent. Or explicitly: "search with web-search-plus for..."

Q: Does it replace built-in Brave Search?

No, it's complementary. Use Brave for quick lookups, web-search-plus for research/shopping/discovery.

Q: Can I see which provider was used?

Yes! SOUL.md can include attribution: [🔍 Searched with: Serper/Tavily/Exa]