Guide

Best MCP Servers in 2026 by Job: Scraping, Search, Browser Automation and Databases

Which MCP server should I use for web scraping, web search, browser automation or database access?

For scraping sites that block bots, use Bright Data; for clean Markdown or JSON, use Firecrawl. For cheap web search, start with Parallel Search; for Google results, SerpApi. For browser automation while you code or in CI, use Playwright MCP; for cloud scale, Browserbase or Browserless. For databases, Neon is the safest Postgres pick (read-only switch and free branches), and Supabase is the pick if you already use it. Point any database server at a dev branch, never production.

Coding agent settings change often

Get one email a week when a setting, flag or permission rule changes, checked against the vendor docs.

Quick picks

  • Web scraping, sites that block bots: Bright Data (anti-bot, CAPTCHA and proxy rotation stated in its MCP docs; 5,000 free requests a month).
  • Web scraping, clean Markdown or JSON: Firecrawl (1,000 free credits a month).
  • Web search, lowest cost: Parallel Search (hosted MCP usable without an API key; $1 per 1,000 fast API searches).
  • Web search, Google results: SerpApi (250 free searches a month).
  • Web search, independent index: Brave Search.
  • Browser automation, local and CI: Playwright MCP (open source, no account).
  • Browser automation, cloud scale: Browserbase or Browserless.
  • Database, Postgres with safe migrations: Neon (readonly switch, 10 free branches per project).
  • Database, already on Supabase: Supabase with read_only=true and project_ref.
  • Database, documents: MongoDB with --readOnly.

Full comparisons

Each comparison lists free tiers, prices, install commands for Claude Code and other agents, the main tools and the gotchas, checked against each vendor's own docs and pricing pages.

Before you add any MCP server

  • Prefer the vendor's official server, published under its own namespace in the official MCP registry.
  • Keep API keys out of committed files: use environment variables, headers or the vendor's OAuth endpoint.
  • Give database servers read-only access and a dev branch first.
  • Load only the tools you need; most servers have an option to limit them.

Get coding agent settings changes by email

When a setting, flag or permission rule in this guide changes, it goes in the weekly AgentAtlas update, checked against the vendor docs. One email a week.

Sources

Web scraping MCP servers compared (vendor sources listed)Web search MCP servers compared (vendor sources listed)Browser automation MCP servers compared (vendor sources listed)Database MCP servers compared (vendor sources listed)

More guides

Other guides

How to allow Cursor to run all commands: Run Everything, allowlist and sandboxHow to allow Claude Code to run all commands: permission modes, allow rules and skip permissionsClaude Code permissions: allow, ask and deny rules, rule syntax and allowed toolsCodex CLI: approval and sandbox modesGemini CLI: YOLO mode, auto_edit and allowing specific shell commandsCursor Privacy Mode: telemetry, training and data retentionClaude Code telemetry and data retention: what is sent and how to turn it offGemini CLI: turn off usage statistics and telemetryCLAUDE.md: where it goes, how it loads, and how it works with AGENTS.mdClaude Code hooks: all events, exit codes, PreToolUse and PermissionRequest JSON, examplesClaude Code subagents: create one, limit its tools, and call itAdd MCP servers to Cursor with mcp.json: Keep files out of Cursor with .cursorignore: Configure Codex with config.toml: Configure Gemini CLI with settings.json: Add MCP servers to Claude Code (claude mcp add, .mcp.json, scopes): Claude Code custom slash commands and skills (SKILL.md): GitHub Copilot custom instructions: Adding MCP servers to GitHub Copilot: Claude Code settings.json: file locations, precedence, and key settingsClaude Code plugins and marketplaces: install, create, and shareGemini CLI extensions and custom commands: Configuring OpenCode with opencode.json: Cline Rules: workspace, global, and conditional rulesKiro steering files: .kiro/steering, inclusion modes, and AGENTS.mdZed agent: instruction files, MCP servers, tool permissions, and ACP agentsCursor CLI: install, headless mode, permissions and CIAider: CONVENTIONS.md, .aider.conf.yml and API keysJunie guidelines, MCP and the Action Allowlist: Amp AGENTS.md, settings.json and MCP: Configure Goose extensions, hints, recipes and permissions: Customize OpenHands with skills, AGENTS.md, setup.sh and MCP: Configure Factory Droid CLI: AGENTS.md, settings.json, autonomy, custom droids and MCPConfigure Qwen Code: settings.json, providers, QWEN.md, MCP and approval modesCursor hooks: hooks.json, beforeShellExecution, afterFileEdit and exit codesCursor sandbox: sandbox.json, network allowlist and run modesCursor settings: settings.json, Cursor Settings and the ~/.cursor config filesCursor browser agent: @browser, browser automation and approval modesCursor subagents: .cursor/agents, frontmatter, built-in Explore and BrowserCursor telemetry and OpenTelemetry export (Privacy Mode, OTLP, Analytics API): Cursor rules setup: project, user and Team Rules, .mdc frontmatter, globs and AGENTS.mdClaude Code headless mode: claude -p, --output-format json, --bare and CI flagsClaude Code statusline: custom status bar in settings.json with context %, cost and git branchClaude Code /model: switch models, aliasesClaude Code /cost and /usage: check token spend, usage limits and reduce costsClaude Code VS Code extension: install, shortcuts, settings and /ideCodex AGENTS.md: global, project and override instruction files explainedCodex MCP servers: codex mcp add and [mcp_servers] in config.tomlCodex sandbox modes: read-only, workspace-write, danger-full-access and network accesscodex exec: run Codex non-interactively in scripts and CI with JSON outputCodex skills: create SKILL.md, skill folder locations and $skill invocationGEMINI.md in Gemini CLI: context file locations, imports and /memoryGemini CLI MCP servers: gemini mcp add and mcpServers in settings.jsonGemini CLI custom slash commands: TOML files, {{args}}, !{shell} and @{file}Gemini CLI headless mode: gemini -p, JSON output and exit codes for scripts and CIGemini CLI sandbox: enable Docker, Podman, macOS Seatbelt or gVisorcopilot-setup-steps.yml: set up the GitHub Copilot cloudGitHub Copilot CLI permissions: --allow-tool, --deny-tool, --allow-all-tools and trusted directoriesGitHub Copilot custom agents: .github/agents/*.agent.md profiles, tools and frontmatterGitHub Copilot code review settings: automatic PR reviews, rulesets, effort level and approvalsOpenCode MCP Server Setup: Local and Remote Servers in opencode.jsonOpenCode Permissions: Allow, Ask or Deny Bash, Edit and WebfetchOpenCode Custom Agents and Subagents: Markdown Files, Modes and @ MentionsCline MCP Setup: cline_mcp_settings.json for Local and Remote ServersCline Auto Approve and YOLO Mode: Run Commands and Edits Without PromptsCline Skills: SKILL.md Location, Format and Slash Command UseAmp MCP Setup: amp mcp add, amp.mcpServers and Workspace ApprovalAmp AGENTS.md: File Locations, @ Mentions and Glob-Scoped GuidanceAmp Skills: SKILL.md Folders, amp skill add and Search OrderKiro MCP Setup: .kiro/settings/mcp.json, autoApprove and OAuthKiro Hooks: .kiro/hooks JSON, PreToolUse, PostFileSave and StopKiro Permissions: permissions.yaml Allow, Ask and Deny RulesFactory Droid MCP Setup: droid mcp add, mcp.json Locations and /mcpFactory Droid Hooks: hooks.json, PreToolUse Matchers and Exit CodesFactory Droid Custom Droids (Subagents): .factory/droids Markdown FilesJunie MCP Setup: .junie/mcp/mcp.json for CLI and /mcp CommandJunie Brave Mode and Action Allowlist: Auto-Approve Commands in Junie CLIJunie Custom Slash Commands: .junie/commands Markdown Files and Argumentsgoose .goosehints and AGENTS.md: Global and Project Instructionsgoose Permission Modes: GOOSE_MODE auto, approve, smart_approve and chatgoose Recipes: recipe.yaml Format, Parameters and goose run --recipeZed Agent Tool Permissions: Auto-Approve Terminal Commands with tool_permissionsUse Claude, Codex and Gemini CLI in Zed: External Agents and agent_serversZed with Ollama, LM Studio and llama.cpp: Local Model Setup and Context LengthMCP 2026-07-28 Migration Guide: Stateless Protocol, No Sessions, server/discover, MRTR, and SDK v2 UpgradeSubagents and Custom Agents Compared: Claude Code vs Codex vs Cursor vs Gemini CLI vs GitHub CopilotAllow all commands in any AI coding agent: Cursor, Claude Code, Codex, Gemini CLI, Copilot CLI and 7 moreSettings file locations for every AI coding agent: Claude Code, Cursor, Codex, Gemini CLI, OpenCode, Qwen Code, Amp and Factory DroidRules and instruction files in every AI coding agent: .cursor/rules, CLAUDE.md, AGENTS.md, GEMINI.md, Copilot instructions and moreGemini CLI hooks: settings.json, BeforeTool, AfterTool, exit codes and /hooksGitHub Copilot hooks: .github/hooks/*.json for Copilot CLI and the cloud agentHooks in every AI coding agent: Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Kiro and Factory DroidAgent Skills (SKILL.md) in every AI coding agent: Claude Code, Codex, Cline, Amp and OpenHandsHow to deploy a remote MCP server (Streamable HTTP): Cloudflare Workers, Vercel, Railway and moreHeadless mode in every AI coding agent: claude -p, codex exec, gemini -p, Cursor agent -p and copilot -p for scripts and CIClaude Code sandbox: /sandbox, sandbox.enabled, allowedDomains, excludedCommands and strict modeSandbox modes in every AI coding agent: Claude Code, Cursor, Codex and Gemini CLI comparedCustom slash commands in every AI coding agent: file locations, arguments and examplesMigrate from Gemini CLI to Antigravity CLI (2026): Config, MCP, Skills, FlagsRun GitHub Copilot CLI with copilot -p in scripts and CI (2026): Agent Loops 2026: Ralph Loop, /loop, /goal and Headless Loops for AI Coding AgentsCursor skills and custom slash commands: .cursor/skills SKILL.md, .cursor/commands and /migrate-to-skillsCodex hooks: hooks.json, PreToolUse allow/deny, exit code 2 and config.toml [hooks]GitHub Copilot prompt files in VS Code: .github/prompts/*.prompt.md slash commandsOpenCode custom commands: .opencode/commands, $ARGUMENTS, shell output and subtasksQwen Code custom commands: .qwen/commands Markdown files, {{args}}, !{shell} and @{file}

Reviewed Oct 23, 2026. Settings change often; the linked vendor docs are the source of truth.