Web Search and Content Fetching with OpenClaw

Web Tools Overview

OpenClaw provides two primary tools for accessing the internet: web_search for discovering information and web_fetch for reading specific pages.

web_search

Performs keyword searches and returns structured results.

Usage

• "Search for the latest React 19 features"
• "Find the best Italian restaurants in Brooklyn"
• "What are the current trending topics in AI?"

How It Works

1. Generates an optimized search query from your request
2. Queries a search engine
3. Returns titles, snippets, and URLs
4. Optionally reads the top results for deeper answers

web_fetch

Reads and processes the content of a specific URL.

Usage

• "Read this article: https://example.com/post"
• "What does the documentation at this URL say?"
• "Fetch the pricing page of competitor.com"

Content Processing

web_fetch converts HTML to readable text, handling:

• Article extraction (removing ads, navigation)
• Table formatting
• Code block preservation
• Image description (alt text)

Combining Both Tools

For research tasks, OpenClaw combines search and fetch:

"Research the pros and cons of server-side rendering"

1. web_search finds relevant articles
2. web_fetch reads the top 3-5 results
3. OpenClaw synthesizes a comprehensive answer

Configuration

{
  "tools": {
    "web_search": {
      "enabled": true,
      "provider": "default"
    },
    "web_fetch": {
      "enabled": true,
      "timeout": 30000,
      "max_content_length": 50000
    }
  }
}

Limitations

• Some websites block automated access
• Paywalled content may not be readable
• Very dynamic (JavaScript-heavy) pages may need the browser tool instead
• Rate limits apply to prevent abuse

Privacy Note

Web search queries are sent to the search provider. The AI model processes the results locally (or on the model provider's servers). No conversation context is shared with the search engine.