io.scrapfly.mcp/mcp

平台与服务

by scrapfly

借助 AI agents 大规模抓取任意网站、提取结构化数据并采集网页内容,适合自动化数据获取与分析场景。

什么是 io.scrapfly.mcp/mcp

借助 AI agents 大规模抓取任意网站、提取结构化数据并采集网页内容,适合自动化数据获取与分析场景。

README

Scrapfly MCP Server

<p align="center"> <a href="https://scrapfly.io"> <img src="https://avatars.githubusercontent.com/u/54183743?s=400&u=5279c1aaea18805aa5cc4fec1053ac2a2cfaac5d&v=4" alt="Scrapfly" width="200"/> </a> </p> <p align="center"> <strong>Give your AI real-time access to any website</strong> </p> <p align="center"> <a href="https://scrapfly.io/mcp-cloud">🌐 Landing Page</a> • <a href="https://scrapfly.io/docs/mcp/getting-started">📖 Documentation</a> • <a href="https://scrapfly.io/mcp-cloud/n8n">🎮 Live Demo</a> • <a href="https://scrapfly.io/register">🔑 Get API Key</a> </p>

What is Scrapfly MCP?

The Scrapfly MCP Server connects your AI assistants to live web data through the Model Context Protocol. Transform your AI from being limited by training data to having real-time access to any website.

✨ What Your AI Can Do

CapabilityDescription
🌐 Scrape Live DataPull current prices, listings, news, or any webpage content in real-time
🛡️ Bypass Anti-Bot SystemsAutomatically handle CAPTCHAs, proxies, JavaScript rendering, and rate limits
Extract Structured DataParse complex websites into clean JSON using AI-powered extraction
📸 Capture ScreenshotsTake visual snapshots of pages or specific elements for analysis

🏆 Why Scrapfly?

Built on battle-tested infrastructure used by thousands of developers:

📖 Learn more: Why Scrapfly MCP?


🚀 Quick Install

Click one of the buttons below to install the MCP server in your preferred IDE:

Install in VS Code Install in VS Code Insiders Install in Visual Studio Install in Cursor


📦 Manual Installation

Standard Configuration

Works with most MCP-compatible tools:

json
{
  "servers": {
    "scrapfly-cloud-mcp": {
      "type": "http",
      "url": "https://mcp.scrapfly.io/mcp"
    }
  }
}

Cloud Configuration (NPX)

For tools that require a local process:

json
{
  "mcpServers": {
    "scrapfly": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://mcp.scrapfly.io/mcp"
      ]
    }
  }
}

🔧 IDE-Specific Setup

<details> <summary><strong>VS Code</strong></summary>

One-Click Install

Install in VS Code

Manual Install

Follow the VS Code MCP guide or use the CLI:

bash
code --add-mcp '{"name":"scrapfly-cloud-mcp","type":"http","url":"https://mcp.scrapfly.io/mcp"}'

After installation, Scrapfly tools will be available in GitHub Copilot Chat.

📖 Full guide: VS Code Integration

</details> <details> <summary><strong>VS Code Insiders</strong></summary>

One-Click Install

Install in VS Code Insiders

Manual Install

bash
code-insiders --add-mcp '{"name":"scrapfly-cloud-mcp","type":"http","url":"https://mcp.scrapfly.io/mcp"}'

📖 Full guide: VS Code Integration

</details> <details> <summary><strong>Visual Studio</strong></summary>

One-Click Install

Install in Visual Studio

Manual Install

  1. Open Visual Studio
  2. Navigate to GitHub Copilot Chat window
  3. Click the tools icon (🛠️) in the chat toolbar
  4. Click + Add Server to open the configuration dialog
  5. Configure:
    • Server ID: scrapfly-cloud-mcp
    • Type: http/sse
    • URL: https://mcp.scrapfly.io/mcp
  6. Click Save

📖 Full guide: Visual Studio MCP documentation

</details> <details> <summary><strong>Cursor</strong></summary>

One-Click Install

Install in Cursor

Manual Install

  1. Go to Cursor SettingsMCPAdd new MCP Server
  2. Use the standard configuration above
  3. Click Edit to verify or add arguments

📖 Full guide: Cursor Integration

</details> <details> <summary><strong>Claude Code</strong></summary>

Use the Claude Code CLI:

bash
claude mcp add scrapfly-cloud-mcp --url https://mcp.scrapfly.io/mcp

📖 Full guide: Claude Code Integration

</details> <details> <summary><strong>Claude Desktop</strong></summary>

Add to your Claude Desktop configuration file:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

json
{
  "mcpServers": {
    "scrapfly": {
      "command": "npx",
      "args": ["mcp-remote", "https://mcp.scrapfly.io/mcp"]
    }
  }
}

📖 Full guide: Claude Desktop Integration

</details> <details> <summary><strong>Cline</strong></summary>

Add to your Cline MCP settings:

json
{
  "scrapfly-cloud-mcp": {
    "type": "http",
    "url": "https://mcp.scrapfly.io/mcp"
  }
}

📖 Full guide: Cline Integration

</details> <details> <summary><strong>Windsurf</strong></summary>

Follow the Windsurf MCP documentation using the standard configuration.

📖 Full guide: Windsurf Integration

</details> <details> <summary><strong>Zed</strong></summary>

Add to your Zed settings:

json
{
  "context_servers": {
    "scrapfly-cloud-mcp": {
      "type": "http",
      "url": "https://mcp.scrapfly.io/mcp"
    }
  }
}

📖 Full guide: Zed Integration

</details> <details> <summary><strong>OpenAI Codex</strong></summary>

Create or edit ~/.codex/config.toml:

toml
[mcp_servers.scrapfly-cloud-mcp]
url = "https://mcp.scrapfly.io/mcp"

📖 More info: Codex MCP documentation

</details> <details> <summary><strong>Gemini CLI</strong></summary>

Follow the Gemini CLI MCP guide using the standard configuration.

</details> <details> <summary><strong>OpenCode</strong></summary>

Add to ~/.config/opencode/opencode.json:

json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "scrapfly-cloud-mcp": {
      "type": "http",
      "url": "https://mcp.scrapfly.io/mcp",
      "enabled": true
    }
  }
}

📖 More info: OpenCode MCP documentation

</details>

🛠️ Available Tools

The Scrapfly MCP Server provides 5 powerful tools covering 99% of web scraping use cases:

ToolDescriptionUse Case
scraping_instruction_enhancedGet best practices & POW tokenAlways call first!
web_get_pageQuick page fetch with smart defaultsSimple scraping tasks
web_scrapeFull control with browser automationComplex scraping, login flows
screenshotCapture page screenshotsVisual analysis, monitoring
info_accountCheck usage & quotaAccount management

📖 Full reference: Tools & API Specification

Example: Scrape a Page

code
User: "What are the top posts on Hacker News right now?"

AI: Uses web_get_page to fetch https://news.ycombinator.com and returns current top stories

Example: Extract Structured Data

code
User: "Get all product prices from this Amazon page"

AI: Uses web_scrape with extraction_model="product_listing" to return structured JSON

📖 More examples: Real-World Examples


🔐 Authentication

Scrapfly MCP supports multiple authentication methods:

MethodBest ForDocumentation
OAuth2Production, multi-user appsOAuth2 Setup
API KeyPersonal use, developmentAPI Key Setup
Header AuthCustom integrationsHeader Auth

🔑 Get your API key: Scrapfly Dashboard


📊 Configuration Reference

SettingValue
Server Namescrapfly-cloud-mcp
TypeRemote HTTP Server
URLhttps://mcp.scrapfly.io/mcp
ProtocolMCP over HTTP/SSE

🖥️ Self-Hosted / Local Deployment

You can run the Scrapfly MCP server locally or self-host it.

CLI Arguments

FlagDescription
-http <address>Start HTTP server at the specified address (e.g., :8080). Takes precedence over PORT env var.
-apikey <key>Use this API key instead of the SCRAPFLY_API_KEY environment variable.

Environment Variables

VariableDescription
PORTHTTP port to listen on. Used if -http flag is not set.
SCRAPFLY_API_KEYDefault Scrapfly API key. Can also be passed via query parameter ?apiKey=xxx at runtime.

Examples

bash
# Start HTTP server on port 8080
./scrapfly-mcp -http :8080

# Start HTTP server using PORT env var
PORT=8080 ./scrapfly-mcp

# Start with API key
./scrapfly-mcp -http :8080 -apikey scp-live-xxxx

# Start in stdio mode (for local MCP clients)
./scrapfly-mcp

Docker

bash
# Build
docker build -t scrapfly-mcp .

# Run (Smithery compatible - uses PORT env var)
docker run -p 8080:8080 scrapfly-mcp

# Run with custom port
docker run -e PORT=9000 -p 9000:9000 scrapfly-mcp

🤝 Framework Integrations

Scrapfly MCP also works with AI frameworks and automation tools:

FrameworkDocumentation
LangChainLangChain Integration
LlamaIndexLlamaIndex Integration
CrewAICrewAI Integration
OpenAIOpenAI Integration
n8nn8n Integration
MakeMake Integration
ZapierZapier Integration

📖 All integrations: Integration Index


📚 Resources


💬 Need Help?


<p align="center"> <a href="https://scrapfly.io"> <img src="https://avatars.githubusercontent.com/u/54183743?s=400&u=5279c1aaea18805aa5cc4fec1053ac2a2cfaac5d&v=4" alt="Scrapfly" width="50"/> </a> <br/> <strong>Made with ❤️ by <a href="https://scrapfly.io">Scrapfly</a></strong> <br/> <sub>The Web Scraping API for Developers</sub> </p>

常见问题

io.scrapfly.mcp/mcp 是什么?

借助 AI agents 大规模抓取任意网站、提取结构化数据并采集网页内容,适合自动化数据获取与分析场景。

相关 Skills

MCP构建

by anthropics

Universal
热门

聚焦高质量 MCP Server 开发,覆盖协议研究、工具设计、错误处理与传输选型,适合用 FastMCP 或 MCP SDK 对接外部 API、封装服务能力。

想让 LLM 稳定调用外部 API,就用 MCP构建:从 Python 到 Node 都有成熟指引,帮你更快做出高质量 MCP 服务器。

平台与服务
未扫描114.1k

Slack动图

by anthropics

Universal
热门

面向Slack的动图制作Skill,内置emoji/消息GIF的尺寸、帧率和色彩约束、校验与优化流程,适合把创意或上传图片快速做成可直接发送的Slack动画。

帮你快速做出适配 Slack 的动图,内置约束规则和校验工具,少踩上传与播放坑,做表情包和演示都更省心。

平台与服务
未扫描114.1k

MCP服务构建器

by alirezarezvani

Universal
热门

从 OpenAPI 一键生成 Python/TypeScript MCP server 脚手架,并校验 tool schema、命名规范与版本兼容性,适合把现有 REST API 快速发布成可生产演进的 MCP 服务。

帮你快速搭建 MCP 服务与后端 API,脚手架完善、扩展顺手,尤其适合想高效验证服务能力的开发者。

平台与服务
未扫描10.2k

相关 MCP Server

Slack 消息

编辑精选

by Anthropic

热门

Slack 是让 AI 助手直接读写你的 Slack 频道和消息的 MCP 服务器。

这个服务器解决了团队协作中需要 AI 实时获取 Slack 信息的痛点,特别适合开发团队让 Claude 帮忙汇总频道讨论或发送通知。不过,它目前只是参考实现,文档有限,不建议在生产环境直接使用——更适合开发者学习 MCP 如何集成第三方服务。

平台与服务
83.4k

by netdata

热门

io.github.netdata/mcp-server 是让 AI 助手实时监控服务器指标和日志的 MCP 服务器。

这个工具解决了运维人员需要手动检查系统状态的痛点,最适合 DevOps 团队让 Claude 自动分析性能数据。不过,它依赖 NetData 的现有部署,如果你没用过这个监控平台,得先花时间配置。

平台与服务
78.4k

by d4vinci

热门

Scrapling MCP Server 是专为现代网页设计的智能爬虫工具,支持绕过 Cloudflare 等反爬机制。

这个工具解决了爬取动态网页和反爬网站时的头疼问题,特别适合需要批量采集电商价格或新闻数据的开发者。不过,它依赖外部浏览器引擎,资源消耗较大,不适合轻量级任务。

平台与服务
35.4k

评论