Grok Bot vs. Claude in Chrome vs. ChatGPT Work vs. Gemini: A 10-Task Website Test

xAI introduced Grok Bot in August 2026 as an early-beta, always-on teammate with a computer of its own. The official launch says Bots can sign into tools, work across applications and websites, continue after the user steps away, and handle products that lack a clean API or MCP surface. xAI subsequently expanded plan availability. Those facts justify testing Grok Bot as a persistent computer-using product, not merely sending a webpage to a Grok language model and calling the result a browser benchmark.^{1, 2}

The launch pages do not define a universal website conformance test, publish a site-owner crawler identity, or promise that every workflow will complete. A benchmark must record the Grok Bot application version, account plan, region, starting computer state, installed integrations, saved routine state, and whether the Bot has previously seen the site. Persistence and learned routines can be product benefits, but they are confounders in a cold-start comparison. Use a fresh Bot or report prior exposure explicitly.^{1, 2}

Anthropic made Claude in Chrome generally available on paid plans and describes it as able to view the current page, read and type text, click, navigate, fill forms, and use existing logins. Anthropic also says a safety classifier checks autonomous actions against the user's request, with prompt-injection defenses remaining part of the product design. Separately, Claude Cowork now has a built-in browser inside the desktop application. Those are two browser surfaces with different starting state: an existing Chrome session may already contain cookies, extensions, and open tabs, while a built-in browser has its own environment.^{3, 4}

Therefore the comparison row cannot simply say Claude. It must say Claude in Chrome or Claude Cowork built-in browser, along with approval mode, account plan, selected model when configurable, and authentication state. If a site works only because the local Chrome profile is already signed in, that is legitimate evidence for that surface but not proof that a fresh remote browser works. If the safety layer refuses or pauses on a risky instruction, record a safety outcome rather than treating it as a navigation defect.^{3, 4}

The current OpenAI help surface says the former ChatGPT Agent mode is no longer available and directs users to ChatGPT Work and its cloud browser. Accordingly, this article's search-oriented title retains ChatGPT, but the actual benchmark target is ChatGPT Work cloud browser. That cloud browser runs on a separate remote computer, can read pages, click, enter information, and handle supported public and signed-in tasks. It pauses for user input, sign-in, or confirmation, and the secure sign-in flow sends credentials to the remote browser rather than exposing them to the model.^{5, 6}

OpenAI also documents Web Bot Auth for cloud-browser traffic, allowing site operators to verify signed requests at supporting edge providers or with published keys. In the ChatGPT desktop app's built-in browser, site tools use the proposed WebMCP standard so a participating website can expose structured operations that ChatGPT may use instead of only clicking and typing. These are distinct surfaces. A cloud-browser run and a desktop built-in-browser site-tool run must occupy separate benchmark rows, and an older ChatGPT Agent result should be labeled historical rather than silently mixed into current data.^{7, 8}

Google's current Gemini in Chrome help documentation describes auto browse for multi-step tasks such as comparing products, adding items to a cart, finding travel accommodations, making reservations, and scheduling appointments. The product can ask the user to review a plan, confirm sensitive steps, take over, or grant sign-in permission. Depending on the flow it may use local Chrome state or a remote browser, and Google documents safeguards and user responsibility because prompt injection and unintended actions remain possible.⁹

Availability, plan, account type, region, device, and rollout state are part of the evidence. A test that one evaluator can run in a local signed-in browser may be unavailable to another evaluator or may resume in a remote session with a different state. Record which path was actually used, whether Password Manager assisted sign-in, and every take-over or confirmation event. Do not infer that a Gemini API model or a Google search result reproduces Gemini in Chrome auto browse.⁹

As reviewed on August 30, 2026, AgentReady's public diagnostic checks discovery policy, semantic HTML, accessible controls, content structure, browser stability, public safety signals, legal surfaces, and conditional API, MCP, repository, and commerce evidence. Its enhanced browser can attempt bounded navigation, fill a safe public form without submitting it, find documentation or contact information, and inspect a declared sandbox. It blocks non-GET and non-HEAD requests. AgentReady can produce a Fix Pack and verify stable findings after deployment.¹⁰

AgentReady does not currently sign into Grok Bot, Claude, ChatGPT Work, or Gemini accounts and run this cross-provider matrix. Its public runner uses a fresh anonymous browser context and does not receive credentials for the scanned customer's product; it does not currently make purchases or create accounts. It aborts observed non-GET and non-HEAD requests, although that cannot guarantee safety on a server incorrectly implemented to mutate on GET. A strong AgentReady score is useful prerequisite evidence, not a completed result for this benchmark. The distinction is explained in agent readiness score versus journey success. Any future comparison must preserve the public scan and provider journey run as separate evidence ledgers.¹⁰

Begin with a task that every available surface is permitted and reasonably expected to attempt. A public discovery contract could say: starting from the homepage, find the least expensive plan that includes feature X, report the price and two constraints, and cite the canonical source URL and as-of date. A public interaction contract could say: locate the demo form, identify every required field, populate synthetic values if permitted, and stop before submission. These tasks test retrieval, navigation, semantics, state recognition, and restraint without requiring money or customer data.^{11, 12}

For post-login work, create a disposable test tenant and follow how to test AI agents after login safely. The shared goal might be to locate a synthetic invoice and report its due date without editing, downloading, messaging, or paying. If one provider's current policy prohibits a step that another supports, set a provider-specific safe stop and mark the excluded step not applicable. Never prompt a system to bypass CAPTCHA, evade the WAF, defeat an approval, or violate its own safety controls merely to keep the rows symmetrical.^{5, 9}

Report each run as completed, partial, blocked by site, refused by provider policy, unsafe, unavailable, or unobservable. Add first-attempt completion, intervention count, retries, step count, citation correctness, assertion coverage, approval correctness, and time as secondary fields. A refusal at a prohibited purchase is not equivalent to a login page hidden from accessibility APIs. A rollout restriction is not a model failure. A WAF block should link to the access evidence described in robots.txt and WAF testing for AI agents.^{5, 9}

Avoid a universal model leaderboard. Results belong to a product surface, version, task, account state, and date. If aggregation is useful, publish cohorts with disclosed denominators, minimum sample sizes, missing-data treatment, ties, and methodology changes. AgentReady's proposed leaderboards provide an opt-in publication boundary; the Agentic Customer Journey Index demonstrates dated task contracts, safe stops, source records, coverage, and correction routes without claiming an unconsented numeric winner. This blueprint proposes future cross-provider rows and does not populate them.

A fair website evaluation should preserve a visual or semantic browser path because not every agent supports site tools and not every website exposes them. Stable headings, accessible names, native controls, explicit state, useful errors, and durable receipts remain the baseline. Google's agent-ready toolkit frames deterministic browser audits as a way to improve these conditions. Those audits help explain failures but do not replace the provider journey result.^{13, 11, 12}

Where current surfaces support WebMCP or site tools, add a separate structured-tool run. OpenAI says site tools can work with the current page and signed-in session and can expose read or change operations, subject to browser permissions and confirmation requirements. The WebMCP website-tools guide explains the implementation layer. Do not mix a tool-assisted success with a click-only result; report whether the agent discovered the right tool, supplied valid inputs, respected authorization, handled the output, and verified the visible page state.⁸

The available Cross-Agent Replay creates one versioned GoalContract and four explicit not-run evidence rows for Grok, Claude, ChatGPT Work, and Gemini. The shared contract freezes the scope, boundaries, safe stop, assertions, and authoritative-readback requirement before testing begins. This release does not edit or import provider result rows, contact a provider, open a browser, consume credits, collect passwords, or publish a ranking. A full Cross-Harness Journey Runner remains proposed and would attach separately authorized, privacy-safe provider receipts with intervention and first-failure fields.

A proposed Harness Change Log would monitor official provider pages and mark an old comparison stale when a product transitions, as ChatGPT Agent has transitioned to ChatGPT Work, or when a new browser or structured-tool surface appears. That prevents an SEO page from quietly comparing discontinued and current products as peers. Use the available replay plan and AgentReady public scan for prerequisites, follow the provider-specific safety documentation, and publish only evidence actually collected under consent. Also review Is your site usable by Grok Bot? and Claude in Chrome website readiness for the product-specific interpretation behind the shared matrix.^{1, 3, 5, 9}