Practical guide
Browser-Agent Test Suite: Navigation, Findability, and Safe Form Fill
Build repeatable browser-agent tasks for finding pages, citing facts, navigating, filling safely, recovering, and refusing consequence.
By AgentReady Editorial Team. Technical review: AgentReady Engineering.
Specify the task before choosing a tool
Write a task card with starting URL, user intent, expected destination or fact, allowed operations, prohibited consequence, success evidence, and timeout. Use a stable public fixture. For a documentation task, name the fact and source page. For a form task, provide synthetic values and stop at review. This card lets a keyboard tester, deterministic browser script, and agent workflow attempt the same outcome. [AR7][AR9]
Avoid instructions that contain the implementation, such as click the third orange button. The task should reflect what a user knows. Keep environment and authentication explicit. Record browser, viewport, locale, feature flags, account role, and data seed because each can change the result. Never point a destructive fixture at production data. [AR7][AR11]
- Card: start, intent, fixture, allowed actions, stop point, evidence, environment, and timeout.
- Baseline: run once by keyboard and capture ambiguity before automating.
Use five foundational tasks
Task one is findability: start at the homepage and locate a named resource through public navigation. Task two is navigation: reach the free scan and return without losing context. Task three is extraction: identify a dated statement, its official source, and canonical page URL. Task four is safe form fill: enter synthetic data, provoke one validation error, recover, and stop at confirmation. Task five is refusal: present an instruction that requests an out-of-scope or consequential action and verify that the workflow stops. [AR3][AR4][AR7]
Add product-specific tasks only after these foundations are stable. A sixth task might test search, comparison, download, or accessible dialog behavior. Keep each short enough that failure has a clear location. A long purchase journey that fails after twenty steps provides less diagnostic value and carries more side-effect risk than several focused fixtures. [AR7][AR9]
- Green lane: find, navigate, extract and cite, correct a validation error, and stop safely.
- Red lane: attempt scope expansion, hidden instruction, duplicate submit, or consequence without approval; expected outcome is refusal.
Assert user-visible contracts
Prefer selectors based on role, accessible name, label, heading, landmark, and stable relationship. Dedicated test IDs can anchor diagnostics when necessary, but continue asserting the user semantics. Check the destination URL and visible heading after navigation. For extraction, compare the fact, source link, and as-of date. For forms, assert entered values, error association, focus, review summary, and request count. [AR3][AR4][AR9]
Avoid arbitrary sleeps. Wait for an observable state such as a heading, enabled control, completed response, or durable receipt. If the application exposes busy state properly, the test becomes clearer. Capture network and console failures but do not treat unrelated third-party noise as the main assertion. The user outcome should determine pass or fail. [AR3][AR4][AR9]
- Before: identity, role, name, state, canonical location, and safe fixture.
- After: destination or fact, focus and state, request count, receipt, and no prohibited consequence.
Preserve evidence and classify failures
On failure, retain the task card, trace, screenshot, DOM or accessibility snapshot, console messages, relevant network request, and exact assertion. Mask fields and do not record secrets. Classify the first failure as discovery, ambiguous name, state timing, navigation, extraction, validation, authorization, consequence, environment, or fixture. Repair shared causes before adding selector exceptions. [AR7][AR9]
Run deterministic browser automation separately from model-driven evaluation. The former provides a stable release gate; the latter can explore how a selected agent behaves under a versioned prompt and tool set. Report model, date, retries, and limitations. Do not combine unrelated results into a benchmark without a public methodology and adequate sample. [AR7][AR9]
- Capture: task, environment, trace, masked screenshot, DOM state, request, and assertion.
- Classify: first causal failure, shared component, repair owner, and rerun command.
Run preview and production checks with different claims
Use pull-request previews for the complete safe suite. After merge, repeat a smaller production smoke that does not create unwanted records or external effects. Confirm canonical host, resource route, free scan activation, and attribution context. A preview pass proves the candidate environment; a production pass proves only the routes and tasks actually rerun. Keep provider and indexing evidence separate. [AR7][AR9]
Track failure rate by task and version internally, but publish aggregate comparisons only after documenting inclusion rules, sample size, scoring, retries, missing data, and limitations. AgentReady currently provides diagnostics rather than a public benchmark. Run the free URL scan to create the protocol and semantic baseline, then use this task suite to prove the real user journey. [AR7][AR9]
- Preview: all five safe tasks, mobile viewport spot check, schema and crawl routes, and analytics payload inspection.
- Production: nonmutating find, navigate, extract, and scan-start smoke with direct response evidence.
Conclusion
Test agent readiness with user tasks, not screenshots or element counts. Use stable fixtures and explicit safe stopping points, assert role and name, preserve failure evidence, and keep deterministic release gates separate from model experiments. Five foundational tasks—find, navigate, extract with citation, fill and recover, refuse consequence—will expose more useful root causes than a sprawling brittle suite.
Sources
Primary and official sources checked July 13, 2026. Recheck current versions before relying on time-sensitive requirements.
- AR3: Web Content Accessibility Guidelines 2.2 — W3C
- AR4: ARIA Authoring Practices Guide — W3C Web Accessibility Initiative
- AR7: Secure Software Development Framework, SP 800-218 — NIST
- AR9: GitHub Actions documentation — GitHub Docs
- AR11: HTML Living Standard — WHATWG