什么是 io.scrapfly.mcp/mcp?
借助 AI agents 大规模抓取任意网站、提取结构化数据并采集网页内容,适合自动化数据获取与分析场景。
README
Scrapfly MCP Server
<p align="center"> <a href="https://scrapfly.io"> <img src="https://avatars.githubusercontent.com/u/54183743?s=400&u=5279c1aaea18805aa5cc4fec1053ac2a2cfaac5d&v=4" alt="Scrapfly" width="200"/> </a> </p> <p align="center"> <strong>Give your AI real-time access to any website</strong> </p> <p align="center"> <a href="https://scrapfly.io/mcp-cloud">🌐 Landing Page</a> • <a href="https://scrapfly.io/docs/mcp/getting-started">📖 Documentation</a> • <a href="https://scrapfly.io/mcp-cloud/n8n">🎮 Live Demo</a> • <a href="https://scrapfly.io/register">🔑 Get API Key</a> </p>What is Scrapfly MCP?
The Scrapfly MCP Server connects your AI assistants to live web data through the Model Context Protocol. Transform your AI from being limited by training data to having real-time access to any website.
✨ What Your AI Can Do
| Capability | Description |
|---|---|
| 🌐 Scrape Live Data | Pull current prices, listings, news, or any webpage content in real-time |
| 🛡️ Bypass Anti-Bot Systems | Automatically handle CAPTCHAs, proxies, JavaScript rendering, and rate limits |
| ⚡ Extract Structured Data | Parse complex websites into clean JSON using AI-powered extraction |
| 📸 Capture Screenshots | Take visual snapshots of pages or specific elements for analysis |
🏆 Why Scrapfly?
Built on battle-tested infrastructure used by thousands of developers:
- 99.9% Uptime — Enterprise-grade reliability
- 100+ Countries — Global proxy network with datacenter & residential IPs
- Anti-Bot Bypass — Advanced ASP technology defeats modern protections
- OAuth2 Security — Enterprise authentication for production deployments
📖 Learn more: Why Scrapfly MCP?
🚀 Quick Install
Click one of the buttons below to install the MCP server in your preferred IDE:
📦 Manual Installation
Standard Configuration
Works with most MCP-compatible tools:
{
"servers": {
"scrapfly-cloud-mcp": {
"type": "http",
"url": "https://mcp.scrapfly.io/mcp"
}
}
}
Cloud Configuration (NPX)
For tools that require a local process:
{
"mcpServers": {
"scrapfly": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.scrapfly.io/mcp"
]
}
}
}
🔧 IDE-Specific Setup
<details> <summary><strong>VS Code</strong></summary>One-Click Install
Manual Install
Follow the VS Code MCP guide or use the CLI:
code --add-mcp '{"name":"scrapfly-cloud-mcp","type":"http","url":"https://mcp.scrapfly.io/mcp"}'
After installation, Scrapfly tools will be available in GitHub Copilot Chat.
</details> <details> <summary><strong>VS Code Insiders</strong></summary>📖 Full guide: VS Code Integration
One-Click Install
Manual Install
code-insiders --add-mcp '{"name":"scrapfly-cloud-mcp","type":"http","url":"https://mcp.scrapfly.io/mcp"}'
</details> <details> <summary><strong>Visual Studio</strong></summary>📖 Full guide: VS Code Integration
One-Click Install
Manual Install
- Open Visual Studio
- Navigate to GitHub Copilot Chat window
- Click the tools icon (🛠️) in the chat toolbar
- Click + Add Server to open the configuration dialog
- Configure:
- Server ID:
scrapfly-cloud-mcp - Type:
http/sse - URL:
https://mcp.scrapfly.io/mcp
- Server ID:
- Click Save
</details> <details> <summary><strong>Cursor</strong></summary>📖 Full guide: Visual Studio MCP documentation
One-Click Install
Manual Install
- Go to
Cursor Settings→MCP→Add new MCP Server - Use the standard configuration above
- Click Edit to verify or add arguments
</details> <details> <summary><strong>Claude Code</strong></summary>📖 Full guide: Cursor Integration
Use the Claude Code CLI:
claude mcp add scrapfly-cloud-mcp --url https://mcp.scrapfly.io/mcp
</details> <details> <summary><strong>Claude Desktop</strong></summary>📖 Full guide: Claude Code Integration
Add to your Claude Desktop configuration file:
macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"scrapfly": {
"command": "npx",
"args": ["mcp-remote", "https://mcp.scrapfly.io/mcp"]
}
}
}
</details> <details> <summary><strong>Cline</strong></summary>📖 Full guide: Claude Desktop Integration
Add to your Cline MCP settings:
{
"scrapfly-cloud-mcp": {
"type": "http",
"url": "https://mcp.scrapfly.io/mcp"
}
}
</details> <details> <summary><strong>Windsurf</strong></summary>📖 Full guide: Cline Integration
Follow the Windsurf MCP documentation using the standard configuration.
</details> <details> <summary><strong>Zed</strong></summary>📖 Full guide: Windsurf Integration
Add to your Zed settings:
{
"context_servers": {
"scrapfly-cloud-mcp": {
"type": "http",
"url": "https://mcp.scrapfly.io/mcp"
}
}
}
</details> <details> <summary><strong>OpenAI Codex</strong></summary>📖 Full guide: Zed Integration
Create or edit ~/.codex/config.toml:
[mcp_servers.scrapfly-cloud-mcp]
url = "https://mcp.scrapfly.io/mcp"
</details> <details> <summary><strong>Gemini CLI</strong></summary>📖 More info: Codex MCP documentation
Follow the Gemini CLI MCP guide using the standard configuration.
</details> <details> <summary><strong>OpenCode</strong></summary>Add to ~/.config/opencode/opencode.json:
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"scrapfly-cloud-mcp": {
"type": "http",
"url": "https://mcp.scrapfly.io/mcp",
"enabled": true
}
}
}
</details>📖 More info: OpenCode MCP documentation
🛠️ Available Tools
The Scrapfly MCP Server provides 5 powerful tools covering 99% of web scraping use cases:
| Tool | Description | Use Case |
|---|---|---|
scraping_instruction_enhanced | Get best practices & POW token | Always call first! |
web_get_page | Quick page fetch with smart defaults | Simple scraping tasks |
web_scrape | Full control with browser automation | Complex scraping, login flows |
screenshot | Capture page screenshots | Visual analysis, monitoring |
info_account | Check usage & quota | Account management |
📖 Full reference: Tools & API Specification
Example: Scrape a Page
User: "What are the top posts on Hacker News right now?"
AI: Uses web_get_page to fetch https://news.ycombinator.com and returns current top stories
Example: Extract Structured Data
User: "Get all product prices from this Amazon page"
AI: Uses web_scrape with extraction_model="product_listing" to return structured JSON
📖 More examples: Real-World Examples
🔐 Authentication
Scrapfly MCP supports multiple authentication methods:
| Method | Best For | Documentation |
|---|---|---|
| OAuth2 | Production, multi-user apps | OAuth2 Setup |
| API Key | Personal use, development | API Key Setup |
| Header Auth | Custom integrations | Header Auth |
🔑 Get your API key: Scrapfly Dashboard
📊 Configuration Reference
| Setting | Value |
|---|---|
| Server Name | scrapfly-cloud-mcp |
| Type | Remote HTTP Server |
| URL | https://mcp.scrapfly.io/mcp |
| Protocol | MCP over HTTP/SSE |
🖥️ Self-Hosted / Local Deployment
You can run the Scrapfly MCP server locally or self-host it.
CLI Arguments
| Flag | Description |
|---|---|
-http <address> | Start HTTP server at the specified address (e.g., :8080). Takes precedence over PORT env var. |
-apikey <key> | Use this API key instead of the SCRAPFLY_API_KEY environment variable. |
Environment Variables
| Variable | Description |
|---|---|
PORT | HTTP port to listen on. Used if -http flag is not set. |
SCRAPFLY_API_KEY | Default Scrapfly API key. Can also be passed via query parameter ?apiKey=xxx at runtime. |
Examples
# Start HTTP server on port 8080
./scrapfly-mcp -http :8080
# Start HTTP server using PORT env var
PORT=8080 ./scrapfly-mcp
# Start with API key
./scrapfly-mcp -http :8080 -apikey scp-live-xxxx
# Start in stdio mode (for local MCP clients)
./scrapfly-mcp
Docker
# Build
docker build -t scrapfly-mcp .
# Run (Smithery compatible - uses PORT env var)
docker run -p 8080:8080 scrapfly-mcp
# Run with custom port
docker run -e PORT=9000 -p 9000:9000 scrapfly-mcp
🤝 Framework Integrations
Scrapfly MCP also works with AI frameworks and automation tools:
| Framework | Documentation |
|---|---|
| LangChain | LangChain Integration |
| LlamaIndex | LlamaIndex Integration |
| CrewAI | CrewAI Integration |
| OpenAI | OpenAI Integration |
| n8n | n8n Integration |
| Make | Make Integration |
| Zapier | Zapier Integration |
📖 All integrations: Integration Index
📚 Resources
- 🌐 MCP Cloud Landing Page — Product overview & features
- 🎮 Live n8n Demo — Try it in your browser
- 📖 Full Documentation
- 🛠️ Tools Reference
- 💡 Examples & Use Cases
- ❓ FAQ
- 🔐 Authentication Guide
💬 Need Help?
<p align="center"> <a href="https://scrapfly.io"> <img src="https://avatars.githubusercontent.com/u/54183743?s=400&u=5279c1aaea18805aa5cc4fec1053ac2a2cfaac5d&v=4" alt="Scrapfly" width="50"/> </a> <br/> <strong>Made with ❤️ by <a href="https://scrapfly.io">Scrapfly</a></strong> <br/> <sub>The Web Scraping API for Developers</sub> </p>
常见问题
io.scrapfly.mcp/mcp 是什么?
借助 AI agents 大规模抓取任意网站、提取结构化数据并采集网页内容,适合自动化数据获取与分析场景。
相关 Skills
MCP构建
by anthropics
聚焦高质量 MCP Server 开发,覆盖协议研究、工具设计、错误处理与传输选型,适合用 FastMCP 或 MCP SDK 对接外部 API、封装服务能力。
✎ 想让 LLM 稳定调用外部 API,就用 MCP构建:从 Python 到 Node 都有成熟指引,帮你更快做出高质量 MCP 服务器。
Slack动图
by anthropics
面向Slack的动图制作Skill,内置emoji/消息GIF的尺寸、帧率和色彩约束、校验与优化流程,适合把创意或上传图片快速做成可直接发送的Slack动画。
✎ 帮你快速做出适配 Slack 的动图,内置约束规则和校验工具,少踩上传与播放坑,做表情包和演示都更省心。
MCP服务构建器
by alirezarezvani
从 OpenAPI 一键生成 Python/TypeScript MCP server 脚手架,并校验 tool schema、命名规范与版本兼容性,适合把现有 REST API 快速发布成可生产演进的 MCP 服务。
✎ 帮你快速搭建 MCP 服务与后端 API,脚手架完善、扩展顺手,尤其适合想高效验证服务能力的开发者。
相关 MCP Server
Slack 消息
编辑精选by Anthropic
Slack 是让 AI 助手直接读写你的 Slack 频道和消息的 MCP 服务器。
✎ 这个服务器解决了团队协作中需要 AI 实时获取 Slack 信息的痛点,特别适合开发团队让 Claude 帮忙汇总频道讨论或发送通知。不过,它目前只是参考实现,文档有限,不建议在生产环境直接使用——更适合开发者学习 MCP 如何集成第三方服务。
by netdata
io.github.netdata/mcp-server 是让 AI 助手实时监控服务器指标和日志的 MCP 服务器。
✎ 这个工具解决了运维人员需要手动检查系统状态的痛点,最适合 DevOps 团队让 Claude 自动分析性能数据。不过,它依赖 NetData 的现有部署,如果你没用过这个监控平台,得先花时间配置。
by d4vinci
Scrapling MCP Server 是专为现代网页设计的智能爬虫工具,支持绕过 Cloudflare 等反爬机制。
✎ 这个工具解决了爬取动态网页和反爬网站时的头疼问题,特别适合需要批量采集电商价格或新闻数据的开发者。不过,它依赖外部浏览器引擎,资源消耗较大,不适合轻量级任务。