Website AI-Agent Readiness Checklist

Begin by writing the three to five public jobs that matter most. A documentation site might require finding an installation guide, identifying the current version, and locating an API operation. A commerce site might require finding return terms and adding an item to a cart, while stopping before payment. A service site might require finding scope, price context, and a contact route. These jobs become your audit fixtures. They prevent the team from declaring success because a crawler found many elements even though the central user journey remained ambiguous. Keep each job observable: name the starting URL, the requested fact or destination, the safe stopping point, and the evidence that counts as success. [AR7][AR11]

Separate discover, understand, navigate, and act. Discovery asks whether the page or capability can be found. Understanding asks whether names, relationships, and current state are exposed in machine-readable structure. Navigation asks whether a user or browser agent can reach the next step without brittle coordinate guesses. Action asks whether the interface protects consequence with confirmation, authorization, and a useful receipt. A site can pass discovery and still fail action safety. Score these stages separately in your notes so a new sitemap does not hide a dangerous submit flow or a polished form does not hide an orphaned policy page. [AR3][AR4][AR8]

Fetch the homepage, the intended canonical host, robots.txt, and sitemap.xml without relying on visual rendering. Record the final URL after redirects, HTTP status, Content-Type header, and the first meaningful bytes of the body. A route named sitemap.xml that returns a branded HTML shell is not a sitemap. A canonical tag pointing to a host that redirects elsewhere creates conflicting signals. Confirm that internal links, structured data, the sitemap, and redirects all agree on one public host and one URL form. Repeat the check from an incognito browser so account middleware does not mask the public response. [AR1][AR2]

Treat robots.txt as crawl guidance, not access control. The file is public, does not authenticate a requester, and should never contain secrets. Check the groups that apply to general crawlers and any named agents you intentionally address. Confirm that critical public content is not blocked by a broad Disallow rule and that the file declares the canonical sitemap URL. Then inspect sitemap entries: every listed page should return a useful indexable response, avoid redirect chains, use the canonical host, and carry a last modification date only when the page materially changed. Do not add search results, account pages, error routes, or duplicate URL variants simply to increase inventory. [AR1][AR2]

Classify each discovery artifact by authority. robots.txt and XML sitemaps have established syntax and well-documented crawler behavior. An OpenAPI document can formally describe HTTP operations, parameters, authentication, and response shapes when it matches the deployed API. By contrast, llms.txt and ai-agent-style manifests are emerging conventions whose support varies. They may provide useful orientation, but their presence does not prove that an agent can complete a task or that the listed capabilities are current. Label conventions honestly, version them, and maintain them from the same source of truth as the product whenever possible. [AR1][AR2][AR6]

Check content, not filenames. For a manifest, verify the media type, parseability, URLs, ownership, authentication description, and declared actions. For OpenAPI, validate the document and compare at least one harmless operation with the live service. For any human-readable agent guide, verify that commands and boundaries still match the repository or site. A stale instruction file can be worse than no file because it confidently sends automation down the wrong path. Add an owner and review trigger so changes to routes, authentication, or consequence levels prompt a corresponding documentation review. [AR6][AR7][AR12]

Turn off styles or inspect the accessibility tree and ask whether the document still explains itself. The page should expose a useful title, one primary heading, sensible heading levels, landmarks, lists, tables with headers where appropriate, and links whose text describes their destination. Native HTML carries behavior and semantics that generic containers do not. Use a real button for an operation and a real link for navigation. If the visual design requires a custom widget, follow a documented interaction pattern and verify keyboard behavior, focus movement, accessible name, role, value, and state rather than adding ARIA attributes at random. [AR3][AR4][AR11]

Accessible names are a reliability contract. A button whose only label is an unlabeled icon forces a person, test, or agent to infer intent from pixels. A field whose placeholder disappears during typing loses its instruction. Check every interactive control for a stable programmatic name and every repeated control for enough context to distinguish it. Ensure expanded, selected, pressed, invalid, busy, and disabled states are exposed when relevant. Then run the page with keyboard only: tab order should follow the task, focus should remain visible, dialogs should manage focus, and no action should require a pointer gesture alone. [AR3][AR4]

Put essential facts in server-rendered or prerendered HTML when they are intended for public discovery. A blank shell that depends on a long client-side chain can delay or prevent extraction and makes failures harder to diagnose. Use clear prose, descriptive headings, visible publication and modification dates, and source links near claims. Distinguish the date an article was reviewed from the date a regulation or specification changed. When facts are time-sensitive, state an explicit as-of date and tell readers what should be rechecked. This creates a usable evidence boundary instead of pretending evergreen guidance never ages. [AR5][AR11]

Structured data should match what a reader can see. Add vocabulary that accurately describes the page and its relationships, not markup intended to manufacture eligibility. Validate JSON-LD syntax, canonical URLs, dates, publisher, headline, and breadcrumbs against visible content. Avoid copying a schema example without replacing placeholder values. For documentation or APIs, link to the current specification and version. For articles, preserve a source list and change notes when a material correction occurs. The goal is not to maximize schema types; it is to reduce ambiguity while keeping one truthful representation of the page. [AR5]

A reliable form explains what information is requested, why it is needed, how it should be formatted, and what happens next. Associate every field with a label, use suitable input types and autocomplete tokens, and keep instructions available after the user begins typing. Mark required fields programmatically as well as visually. On validation, identify the field, explain the problem in plain language, preserve valid input, and move or announce focus deliberately. Generic messages such as invalid input force both people and agents to guess which constraint failed. [AR3][AR4][AR11]

Define consequence before submit. A search or preview is usually reversible; sending a message, publishing content, changing permissions, deleting data, or making a payment is not. Consequential flows should preview the exact effect, require an intentional confirmation, prevent duplicate submission, and return a durable receipt or status. Test with harmless fixtures and stop before real-world impact unless the environment is specifically designed for end-to-end testing. If an agent can fill the form but cannot identify the safe stopping point, the interface is not ready for autonomous operation. [AR7][AR8]

Assume that page content, uploaded documents, external links, and retrieved text can contain instructions intended to manipulate an agent. Keep untrusted content separate from system policy and user authorization. Restrict tools to the least privilege needed for the current task, validate targets and arguments server-side, and require fresh approval at high-consequence boundaries. Do not place secrets in public HTML, client bundles, logs, prompts, analytics payloads, or discovery files. Rotate credentials exposed through a repository or deployment rather than merely deleting the visible string. [AR7][AR8][AR10]

Design for audit and recovery. Record the authenticated actor, requested operation, validated target, approval decision, result, and time without logging sensitive content unnecessarily. Make retries idempotent where possible and provide undo or compensating actions for reversible changes. Rate-limit expensive or abusable operations. Clearly distinguish suggestions from actions in the interface. A high agent-readiness score must never mean that safeguards were removed to make clicking easier; safe refusal and escalation are successful outcomes when the requested operation exceeds policy or available evidence. [AR7][AR8]

Use the same fixtures for human, automated browser, and agent-assisted runs. Start with navigation and extraction: find a named page from the homepage, locate a dated fact, identify its source, and return the canonical URL. Add a safe form task that fills fields with synthetic data and stops at the review step. Add a recovery task such as correcting an invalid field or closing a dialog. Every run should capture the instruction, environment, trace or screenshot, result, and failure point. A single successful demo is not reliability; repeat the task after a content or dependency change. [AR4][AR7][AR9]

Measure outcomes that guide fixes. Useful measures include task completion, first ambiguous control, retries, wrong destination, unsupported claim, time to safe stopping point, and whether consequence was correctly avoided. Do not average unrelated tasks into a benchmark until the methodology, sample, version, and limitations are public. For your own release gate, a simple pass/fail table is enough. Fix shared causes before individual selectors: unclear navigation, missing names, unstable state, and weak confirmation often explain several failures at once. [AR7][AR9]

Rank findings using consequence, frequency, reach, and repair confidence. A destructive action without confirmation outranks a missing optional manifest. An unlabeled primary CTA used on every page outranks an obscure heading skip. A false canonical that affects an entire content type outranks one broken related link. Record the evidence for each ranking so urgency is explainable. Then group findings by root cause: a shared layout, design-system control, route generator, or form component can often repair many pages with one tested change. [AR7][AR9]

Convert the audit into small verification-ready work. Each ticket should include the failed user task, affected URLs or components, expected behavior, prohibited regression, and command or browser steps that prove completion. Add a failing test before the code change when the behavior is deterministic. After the fix, rerun the local test, the relevant task, and a production-like build. Keep cosmetic scoring changes separate from safety or crawl changes so reviewers can see what actually improves the user outcome. [AR7][AR9][AR12]

Add the stable checks to continuous integration: content inventory, internal links, canonical construction, JSON-LD parsing, sitemap membership, robots declaration, and lint or accessibility tests for shared controls. Run a focused browser smoke test against a preview before merge. After production deployment, fetch the canonical host, content index, representative article, sitemap, and robots file again because platform routing can differ from local behavior. Monitor the real activation path with first-party events, preserving source and campaign context from article view to scan start and completion. [AR1][AR2][AR9]

Schedule a material review instead of changing dates cosmetically. Recheck official sources, product claims, screenshots, route behavior, CTA destination, and test fixtures. Update the modified date only when the page meaningfully changes, and keep a concise correction note for material errors. If you later publish aggregate readiness data, disclose collection dates, inclusion rules, sample size, versioned scoring rules, missing-data treatment, privacy protections, and limitations before making comparisons. Until then, describe individual scan evidence only; do not imply a market benchmark that has not been measured. [AR2][AR7]