Let your AI coding agent (Claude Code or Codex CLI) drive your real Chrome — clicks, fills forms, grabs API keys, writes them to .env.
Install · Tools · Website · Security
--- **The problem**: Playwright, Browser Use, and Puppeteer launch a fresh empty browser every time — no cookies, no sessions, no 2FA. Your AI agent gets stuck at the first login screen. **Chromeflow**: drives your **actual Chrome**, where you're already logged into Stripe, AWS, Supabase, Canvas, GitHub. The agent automates what it can (clicks, fills, uploads, captures keys) and pauses for anything that needs you (passwords, 2FA, payment). One MCP server, 28 browser tools, works with both Claude Code and Codex CLI. **Battle-tested**: shaped by **400+ hours of real agentic browser work** before public release — every gnarly edge case (React Selects, isTrusted-gated forms, shadow-DOM clicks, page-CSP fetches, silent redirects, cookie leaks, 0×0 hidden elements) is in the error-handling recipes. Not a weekend hackathon project. ## Why Chromeflow? Existing browser automation tools (Playwright, Browser Use, Puppeteer) launch a **fresh, empty browser** — no cookies, no sessions, no extensions. Every time they start, you're logged out of everything and can't handle 2FA. Chromeflow works in **your actual Chrome browser**, where you're already logged into Stripe, AWS, Supabase, and everything else. Claude automates what it can (clicking buttons, filling forms, uploading files) and pauses for anything that needs you (passwords, 2FA, payment details). | | Chromeflow | Playwright / Browser Use | |---|---|---| | **Browser** | Your real Chrome (sessions intact) | Fresh instance, logged out of everything | | **Auth / 2FA** | Already handled — pauses when needed | Can't handle — blocks completely | | **Page understanding** | DOM queries (fast, cheap, reliable) | Screenshots + vision model (slow, expensive) | | **Human-in-the-loop** | Highlights, pauses on sensitive input | Fully autonomous, no interaction | | **Integration** | MCP server for Claude Code & Codex CLI | Standalone, not agent-aware | | **Credential capture** | Reads API keys → writes to `.env` | Not designed for this | ## How it works Chromeflow is two things that work together: - **MCP server** — gives your coding agent (Claude Code or Codex) a set of browser tools (`open_page`, `click_element`, `fill_form`, `set_file_input`, `read_element`, `write_to_env`, etc.) - **Chrome extension** — receives those commands and acts on the active tab (highlights, clicks, fills, uploads files, captures screenshots) Claude drives the flow. You only touch the browser for things that genuinely need you — login, passwords, payment details, personal choices. ## Setup ### Claude Code **1. Install the plugin** (one-time, machine-wide): ``` /plugin marketplace add https://gitlab.com/NeoDrew/chromeflow.git /plugin install chromeflow ``` Run these inside Claude Code. The plugin registers the MCP server, pre-approves Chromeflow tools, and ships the usage skill — no per-project setup needed. **2. Install the Chrome extension** (one time): [chromewebstore.google.com/detail/chromeflow/lkdchdgkbkodliefobkkhiegjdiidime](https://chromewebstore.google.com/detail/chromeflow/lkdchdgkbkodliefobkkhiegjdiidime) — click **Add to Chrome**. The extension persists across Chrome restarts. You only do this once. **3. Restart Claude Code.** That's it. Claude will automatically reach for Chromeflow whenever a task needs browser interaction, in any project. The plugin's SessionStart hook also cleans up any stale config left over from older `npx chromeflow setup`-based installs on first run. ### Codex CLI **1. Install the plugin** (one-time, machine-wide): ``` codex plugin marketplace add https://gitlab.com/NeoDrew/chromeflow.git ``` Then inside Codex: ``` /plugins install chromeflow ``` The plugin registers the MCP server and ships the same usage skill that Claude Code uses, host-adjusted for Codex. **2. Install the Chrome extension** (one time) — same Web Store link as above. The extension is host-agnostic and serves both Claude Code and Codex from the same install. **3. Restart Codex.** Codex will reach for Chromeflow on browser tasks the same way Claude Code does. ## Usage Just ask Claude normally: > "Set up Stripe for this project — create a product with monthly and annual pricing, capture the price IDs into .env" > "Go to Supabase and get my project's anon key and service role key" > "Help me configure SendGrid webhooks for this app" Claude will navigate, highlight steps, click what it can, pause for anything sensitive, and write values to your `.env` automatically. ## What Claude can do | Capability | Tools | |------------|-------| | Navigate pages, open new tabs | `open_page`, `list_tabs`, `switch_to_tab` | | Click buttons and links | `click_element` (with `nth` for duplicates) | | Fill single fields | `fill_input` (with `nth` for duplicates) | | Fill multiple fields in one call | `fill_form` | | Upload files (even hidden inputs) | `set_file_input` | | Read page content as text | `get_page_text` (with `selector` scoping) | | Inspect all form fields | `get_form_fields` | | Scroll to a known element | `scroll_to_element` | | Highlight elements for the user | `highlight_region`, `find_and_highlight` | | Wait for the user to click | `wait_for_click` | | Wait for async changes | `wait_for_selector` | | Run arbitrary JS | `execute_script` | | Read browser console output | `get_console_logs` | | Capture credentials to `.env` | `read_element`, `write_to_env` | | Screenshot (Claude-only by default; pass `copy_to_clipboard` / `save_to` to share) | `take_screenshot` | | Screenshot the terminal window | `capture_terminal` | | Save/restore form state across tabs | `save_page_state`, `restore_page_state` | ### File uploads `set_file_input` uses Chrome DevTools Protocol to bypass the browser's file-input script restriction — the same mechanism used by Playwright and Puppeteer. It works even when the `` is hidden behind a custom drag-and-drop zone. ``` set_file_input("Upload", "/Users/you/Downloads/task.zip") ``` ### Terminal screenshots `capture_terminal` screenshots the terminal window (Terminal, iTerm2, Warp, VS Code, Ghostty, etc.) and saves it as a PNG. Use this with `set_file_input` to upload terminal output to a web form. ### Dedicated Claude window Click the Chromeflow extension icon and use **"Use this window for Claude"** to lock Claude's browser operations to a specific Chrome window. This lets you freely use other Chrome windows without Claude interfering. ### Running multiple agent sessions in parallel Chromeflow supports up to 11 agent sessions (Claude Code, Codex, or a mix) running in parallel, each automating a different Chrome window without touching the others. **How it works:** - Each agent session spawns its own Chromeflow MCP server, which auto-discovers a free port in the range `7878-7888` (first session gets 7878, second gets 7879, etc.). - The Chrome extension maintains one WebSocket connection per port and tracks per-port window assignments. - Every browser tool call is routed to the Chrome window assigned to the port the request came in on. **Setup:** 1. Start your first agent session as normal — its Chromeflow will claim port 7878. 2. Start a second agent session in another terminal — its Chromeflow auto-falls-back to 7879. 3. Click the Chromeflow extension icon. The popup now shows **one row per instance** (Port 7878, Port 7879, ...) each with a green dot when live. 4. In Chrome **window A**, open the popup and click **"Use this window"** next to Port 7878. 5. Switch to **window B**, open the popup, and click **"Use this window"** next to Port 7879. That's it. Each session now drives its own Chrome window — you can run a research task in one window while the other session fills out a Stripe dashboard in another, with zero collision. Single-instance usage is unchanged and fully backwards compatible — the old per-window assignment is auto-migrated on first load. ## Adding to another project Nothing to do — the plugin install above is machine-wide. Open any project and Chromeflow is ready. ## Upgrading In Claude Code: ``` /plugin update chromeflow /reload-plugins ``` In Codex: ``` /plugins update chromeflow ``` Restart the host afterward to pick up the new MCP server binary the plugin ships. ## Migrating from the old `npx chromeflow setup` flow If you previously installed Chromeflow via `npx chromeflow setup`, do nothing — install the plugin (Setup step 1 above) and the plugin's first SessionStart cleans up: - The stale `mcpServers.chromeflow` entry in `~/.claude.json` - Any leftover `## Chromeflow` section in `~/.claude/CLAUDE.md` Per-project `CLAUDE.md` files and `.claude/settings.local.json` allowlists are left alone — they may have content you want to keep, so the plugin won't touch them. You can delete the `# Chromeflow — Claude Instructions` section from each project's `CLAUDE.md` by hand whenever it's convenient (the plugin's skill carries the same content now), or just leave it. ## Development ```bash git clone https://gitlab.com/NeoDrew/chromeflow cd chromeflow npm install packages/plugin/scripts/build-server.sh # bundle the MCP server into the plugin ``` The MCP server source lives at `packages/mcp-server/src/` and is bundled with esbuild into `packages/plugin/server/chromeflow.mjs` — a single-file ESM binary the plugin ships directly. Iterate locally by pointing the plugin at this checkout. In Claude Code: ``` /plugin marketplace add /absolute/path/to/this/repo /plugin install chromeflow ``` In Codex: ``` codex plugin marketplace add /absolute/path/to/this/repo /plugins install chromeflow ``` Rebuild the bundle after editing MCP server source, then `/reload-plugins` (Claude Code) or restart the host (Codex) to pick it up. The Chrome extension lives at `packages/extension/`: ```bash npm run dev:ext # watches extension ``` After rebuilding the extension, reload it from `chrome://extensions`. ## Requirements - Claude Code or Codex CLI - Chrome (or any Chromium browser) - Node.js 22+ ================================================================================ SECTION 2 — Agent usage skill (every tool, every pattern, every escape hatch) ================================================================================ # Chromeflow — Claude Instructions ## What chromeflow is Chromeflow is a browser guidance tool. When a task requires the user to interact with a website (create accounts, set up billing, retrieve API keys, configure third-party services), use chromeflow to guide them through it visually instead of giving text instructions. ## When to use chromeflow (be proactive) Use chromeflow automatically whenever a task requires: - Creating or configuring a third-party account (Stripe, SendGrid, Supabase, Vercel, etc.) - Retrieving API keys, secrets, or credentials to place in `.env` - Setting up pricing tiers, webhooks, or service configuration in a web UI - Any browser-based step that is blocking code work Do NOT ask "should I open the browser?" — just do it. The user expects seamless handoff. **Never end a response with a "you still need to" list of browser tasks.** If code changes are done and browser steps remain (e.g. creating a Stripe product, adding an env var), continue immediately with chromeflow — don't hand them back to the user. ## HARD RULES — never break these 1. **Never use Bash as a fallback for browser tasks.** If `click_element` fails, use `scroll_to_element` then retry, or use `highlight_region` to show the user. Never use `osascript`, `applescript`, or any shell command to control the browser. 2. **Never use `take_screenshot` to read page content.** After `click_element`, after navigation — always call `get_page_text` (or `find_text` if you only need to check for a specific phrase). `get_page_text` returns up to 10,000 characters; if truncated it tells you the next `startIndex` to paginate. Screenshots are only for locating an element's pixel position when DOM queries have already failed. Never take more than 1–2 screenshots in a row. 3. **Use `wait_for(selector=…)` to wait for async page changes** (build completion, modals, toasts). Never poll with repeated `take_screenshot` calls. ## Guided flow pattern ``` 1. open_page(url) — navigate to the right page (add new_tab=true to keep current tab open; add background=true to keep the current tab focused if its form auto-saves on blur) 2. For each step: a. Claude acts directly: click_element("Save") — press buttons/links Claude can press click_element("Save", until_selector=".success-toast") — when synthetic clicks may silently no-op on a React-heavy site, require an observable post-click condition (or until_url_contains / until_text_contains) fill_form([{label, value}, ...], exact=true) — fill multiple fields in one call; pass exact=true on dense forms to refuse fuzzy text-walk matches fill_input(textHint="Product name", value="Pro") — fill a single field by label hint (works on React, CodeMirror, and contenteditable). Always check the response — it names the matched element so you can spot wrong-field matches fill_input(textHint="Rate", value="5", exact=true) — exact-match mode for short generic labels that may collide with neighbouring fields fill_input(selector="input[name=email]", value="x@y") — selector-mode (replaces the old react_set_input). Bypasses fuzzy matching and uses the React-aware native value-setter so React's onChange picks up the change. Supports `frame` for same-origin iframe inputs. type_text("hello world") — type via trusted keyboard events (use when fill_input fails isTrusted checks) type_text("description", frame="iframe.se-rte") — type into a same-origin iframe's contenteditable (eBay description editor pattern) set_file_input("Upload", "/abs/path/to/file.zip") — upload a file; returns success only after the upload is observably committed (no manual sleep needed between rapid uploads) clear_overlays() — call this immediately after fill_input/fill_form succeeds scroll_to_element("label text") — jump directly to a known field by CSS selector or visible text; use execute_script("window.scrollBy(0, 400)") only for blind incremental scrolls b. Check results with text, not vision: get_page_text() — read errors/status after actions wait_for(selector=".success") — wait for a CSS selector to appear (replaces wait_for_selector) wait_for(text="Saved") — wait for a text substring to appear (replaces wait_for_text) wait_for(change_in=".toast") — wait for an existing element's subtree to mutate, then read its text (replaces wait_for_change) execute_script("return await fetch('/api/x').then(r => r.json())") — top-level await is supported, no window.__variable + sleep dance needed c. When an element can't be found or clicked: scroll_to_element("label text") and retry — always try this first get_elements() — get EXACT DOM coords when needed highlight_region(selector,msg) — highlight by CSS selector (preferred; scrolls element into view automatically) highlight_region(x,y,w,h,msg) — highlight by coords only if no selector available (coords go stale on scroll) [absolute last resort] take_screenshot() — only if you genuinely can't identify the element from DOM d. Pause for the user when needed: find_and_highlight(text, msg) — show the user what to do wait_for_click() — wait for user interaction [after fill_input] clear_overlays() — always clear after filling 3. clear_overlays() — clean up when done ``` **Default to automation.** Only pause for human input when the step genuinely requires personal data or a human decision. ## What to do automatically vs pause for the user **Claude acts directly** (`click_element` / `fill_input`): - Any button: Save, Continue, Create, Add, Confirm, Next, Submit, Update - Product names, descriptions, feature lists - Prices and amounts specified in the task - URLs, redirect URIs, webhook endpoints - Selecting billing period, currency, or other known options - Dismissing cookie banners, cookie dialogs, "not now" prompts **Pause for the user** (`find_and_highlight` + `wait_for_click`): - Email address / username / login - Password or passphrase - Payment method / billing / card details - Phone number / 2FA / OTP codes - Any legal consent the user must personally accept - Choices that depend on user preference Claude wasn't told ## Capturing credentials After a secret key or API key is revealed: 1. `read_element(hint)` — capture the value 2. `write_to_env(KEY_NAME, value, envPath)` — write to `.env` 3. Tell the user what was written Use the absolute path for `envPath` — it's the Claude Code working directory + `/.env`. ## Privileged context — `fetch_url`, `download_file`, `read_attachment` `execute_script` runs in **page context**: it can touch the DOM but is subject to the page's `Content-Security-Policy` (specifically `connect-src`, which routinely blocks `fetch()` against authenticated APIs on sites like Canvas, banking dashboards, internal portals). For authenticated network access, use the **privileged-context** tools — they run in the extension's background service worker with full host_permissions, automatically include the user's Chrome cookies, and bypass page CSP entirely: - `fetch_url(url, method?, headers?, body?, binary?)` — generic HTTP request, returns `{status, headers, body_text or body_base64, truncated, total_bytes}`. Use for AJAX endpoints, JSON APIs, anything where you need a clean response object. Pass `binary: true` for non-text bodies (PDFs, images, zips). - `download_file(url, filename?)` — Chrome's authenticated download flow, returns the absolute path the file landed at. Use when you need the bytes saved to disk for another tool to read (or to hand the path to the user). - `read_attachment(url, format?, max_chars?)` — privileged fetch + format-aware text extraction in one call. Supports docx (via in-extension ZIP extraction, no local CLI), txt, md, csv, json, xml, html. PDF returns a structured error pointing you at `download_file` + your local `pdftotext`/`textutil` (PDF native support is planned for v0.9.4). **Mental model**: page context for DOM manipulation, privileged context for network access. If `execute_script("...await fetch(...)...")` returns "Failed to fetch" or a `Content-Security-Policy` error, switch to `fetch_url` (or `read_attachment` if you just want the text). ## Discoverability — `list_frames` Before reaching into an iframe with `find_text({frame: "selector"})` or other frame-targeted tools, call `list_frames()` to see what's actually on the page. Each result includes: - `selector` — drop this directly into another tool's `frame` parameter - `origin` — the iframe's origin (parsed from `src`) - `accessible` — `true` for same-origin frames (`find_text` etc. work), `false` for cross-origin (use `read_attachment(src)` or `take_screenshot` instead) This is how you avoid "frame not accessible" errors after the fact. Top-level frames only — nested cross-origin trees aren't enumerated. To capture and share a screenshot (e.g. for uploading to a form or pasting into a chat), use `take_screenshot(copy_to_clipboard=true, save_to="downloads")` — saves a PNG to ~/Downloads and copies it to the clipboard. The defaults (`copy_to_clipboard=false, save_to="none"`) return the image to Claude only. ## Working with complex forms - Before filling a large or unfamiliar form, call `get_form_fields()` to get a full inventory of every field (type, label, current value, vertical position, and section heading). Use `get_elements()` when you need pixel coordinates of visible elements; use `get_form_fields()` when you need to understand the full structure of a form including fields below the fold. If you only need one or two specific fields, use `find_input("hint")` instead — targeted lookup is much cheaper than the full inventory and returns labels you can pipe straight into `fill_input`. - `get_form_fields()` includes `[type=file]` fields even when they are visually hidden behind custom drag-and-drop zones. Use `set_file_input(hint, filePath)` to upload a file — provide the label/hint text and the absolute path to the file on disk. - For forms with multiple fields, use `fill_form([{label, value}, ...])` to fill them all in a single call. It returns a per-field success/failure report so you can immediately see which fields weren't found. Use `fill_input` only for a single field. - `fill_input` and `fill_form` work on React-controlled inputs, contenteditable (Stripe, Notion), and **CodeMirror 6 editors** — auto-detected. After filling, the value is read back and a warning is shown if React did not accept it. - **Monaco editors** (VS Code-style in-browser code editors) appear in `get_form_fields()` as type "monaco". They cannot be filled via `fill_input` — use `execute_script` with the Monaco API instead: ```js // Read content from the first Monaco model monaco.editor.getModels()[0].getValue() // Write content to the first Monaco model monaco.editor.getModels()[0].setValue('new content here') ``` - `set_file_input` accepts CSS selectors as the hint (e.g. `#import-problem-file`, `.upload-input`) in addition to label text. Use selectors when file inputs are hidden behind custom UIs and have no visible label. - **Replacing an already-uploaded file**: after `set_file_input` succeeds, the input becomes invisible and a "Remove" span/button typically appears near the upload area. To replace the file: `click_element("Remove", nth=N)` (the right `nth` if there are multiple), then call `set_file_input(hint, newPath)` again — the same hidden input is recycled and accepts the new file. Verify with `get_form_fields()` between the two steps so you're sure the input has reappeared. - **Forcing auto-save on idempotent text edits** (e.g. keep-alive loop on an auto-saving long-form review dashboard): some auto-save logic diffs against the last-saved value and skips no-op writes. To force a real save on each tick without changing visible content, toggle a trailing space — add when absent, remove when present. `fill_input` value comparison handles both directions transparently. **Caveat:** long-running heartbeats that toggle whitespace on a real form field have been observed to drift other fields' React state out of sync (the re-render reset a separate radio's checked state to the form-level store value). For heartbeat loops, prefer writing to `localStorage` via `execute_script` instead — the auto-save handler usually fires on any input event, but `localStorage` writes don't perturb React state at all. - After any radio/checkbox click that reveals new fields, call `get_form_fields()` again — the inventory will include the new fields and warn if more hidden ones still exist. - If a form has collapsible sections, expand them all before calling `get_form_fields()` so the field list is complete. Use the `[under: "section name"]` context in each field's entry to identify fields by section rather than by index — indices shift when sections expand. - Prefer `scroll_to_element("label text or #selector")` over `scroll_page` whenever you know which field or section you need — it scrolls precisely and confirms the matched element. - For multi-session tasks (long forms that may exceed context), call `save_page_state()` as a checkpoint. A future session can call `restore_page_state()` to reload all field values. ## Discovery — find without dumping the whole page Three lightweight tools save tokens vs `get_page_text` / `get_form_fields` when you don't need the full content: - `find_text("Saved successfully")` — grep the DOM. Returns surrounding context, a CSS selector, and a `clickable` flag for each match. Use this instead of `get_page_text` when you're checking whether a specific phrase is present, or to locate a button by its visible text. If `clickable=true`, pipe the matched text straight into `click_element`. - `find_input("Email")` — fuzzy form-field lookup, top-N. Returns labels you can pipe straight into `fill_input(label, value)` — both tools share the same match ranks (`aria-eq` → `placeholder-eq` → `label-text-eq` → `name-eq` → `id-eq` → `*-includes` → `fuzzy-text-walk`). Cheaper than `get_form_fields` when you just need a couple of specific fields. Pass `type_filter="email"` to restrict to a specific input type. - `wait_for_text("Saved")` — wait for text to appear without knowing the selector ahead of time. Complements `wait_for_selector` for the case where you only know the post-action message. All three pierce open shadow roots and accept `frame="iframe.selector"` for same-origin iframes. Pass `regex=true` on `find_text` / `wait_for_text` for case-insensitive regex matching. Pass `exact=true` on `find_input` to refuse fuzzy text-walk matches. ``` find_text("Build complete", scope_selector=".log-output") — only check the build log section find_input("Card number", type_filter="text") — find Stripe's card-number field wait_for_text("Deploy successful", timeout_ms=30000) — wait up to 30s after clicking Deploy ``` Reach for these BEFORE `get_page_text` / `get_form_fields` when the goal is "is X here?" or "where is X?". Reserve `get_page_text` for reading actual content, and `get_form_fields` for understanding a whole form's structure. ## Working with multiple tabs - Before opening a new tab, call `list_tabs()` to check if the target URL is already open — use `switch_to_tab` to return to it instead of opening a duplicate. - `open_page(url, new_tab=true)` opens a URL without losing the current tab. Use sparingly — prefer switching to an existing tab over opening a new one. - `switch_to_tab("1")` switches by tab number; `switch_to_tab("form")` matches by URL or title substring. - Before navigating away from a partially-filled form, call `save_page_state()` so the form can be restored if the tab reloads or the page loses its state on return. - **In long-lived self-rescheduling loops**, the active tab can silently drift mid-session (the user navigates manually while AFK, or another tab steals focus). At the start of every loop iteration, call `list_tabs` and verify the active tab's URL matches your expected target — if not, `switch_to_tab(