AI crawlers and robots.txt on Shopify: what to block, what to allow

The main AI bots and what they do

The roles below come from each company's own documentation, checked in October 2026. Bot names change; check the linked pages before you write rules.

Two practical consequences:

• Blocking Google-Extended does not remove you from Google Search, AI Overviews or AI Mode, which are part of Search. Blocking Googlebot would, and that is rarely what a store wants.
• Blocking Bingbot removes you from Bing and, with it, from the index Copilot uses. For a store hoping to appear in Copilot shopping, that is usually counterproductive.

The trade-off for a store

For a publisher, blocking AI bots can protect content. For a store, the calculation is different: you generally want shoppers using ChatGPT, Perplexity, Copilot or Gemini to find your products with correct details. A reasonable split:

• Search and user bots (OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot, Claude-User, Bingbot): blocking them makes it harder for assistants to cite or link your pages.
• Training bots (GPTBot, ClaudeBot, Google-Extended): blocking them is a policy choice about your content being used to train models. It does not stop the search bots of the same company.

Also note that being in an AI shopping surface often depends on feeds and integrations, not crawling alone: Shopify Catalog and Agentic Storefronts, Google Merchant Center, or the OpenAI merchant program. robots.txt is one piece, not the switch.

How robots.txt works on Shopify

Shopify generates a default robots.txt for every store. To change it, you add a robots.txt.liquid template to your theme:

1. Before anything, open https://yourstore.com/robots.txt and save a copy of the current output. Shopify's documentation asks you to compare it with your template's output before publishing.
2. Go to Online Store > Themes, click ... > Edit code.
3. Right-click the Templates folder, choose New file, and name it robots.txt.liquid. Shopify pre-fills it with Liquid that reproduces the default rules from robots.default_groups.
4. Keep that loop intact. Add your own groups after it, as plain lines. For example, to block OpenAI's training bot while leaving its search bot alone:
   • User-agent: GPTBot
   • Disallow: /
5. Save, then reload /robots.txt and check that the default rules are still there and your new group appears once.

Cautions from Shopify's documentation: adding the template replaces the generated file, so a mistake applies to your whole store; and you should use the provided Liquid objects rather than pasting a full static file, so Shopify's default rules stay current. If you use Shopify Markets with separate domains, request.host lets you vary rules per domain.

Things robots.txt cannot do

• It is a request, not a lock. Well-behaved bots follow it; others may not. Some user-initiated fetchers say they may not apply it.
• Changes are not instant. OpenAI and Perplexity both note that changes can take about 24 hours to be picked up.
• It does not fix wrong data. A bot that is allowed in can still read a stale price from a duplicate schema block or an outdated feed.

Reading your current robots.txt

Before changing anything, look at what you already have. Open /robots.txt and scan for User-agent: lines. Shopify's default file contains groups for all bots (User-agent: *) and a few specific crawlers, with Disallow rules for paths such as the cart, checkout, account and internal search. That is normal and protects you from duplicate or private pages being crawled.

If you see a group naming an AI bot that you did not add, it may come from an earlier developer, an app, or a template copied from another store. Ask whoever maintains your theme before removing it. If your store is on a headless setup, robots.txt is served by your own code, not by this template.

A rule only applies to the user agent named in its group. Disallow: / under User-agent: GPTBot does nothing to OAI-SearchBot, and a rule under * applies to bots that have no group of their own.

Quick checklist

• Saved a copy of the current /robots.txt before editing.
• Googlebot and Bingbot are not blocked.
• Each AI bot you block is blocked on purpose, with its role understood.
• Default Shopify rules (checkout, cart, admin paths) are still present after your edit.
• Rechecked after a theme switch, since the template lives in the theme.

What AI Sales Assurance does and does not check

To be clear: AI Sales Assurance does not check your robots.txt or tell you which AI bots are blocked. Use the steps above, or a crawler-access tool, for that.

What it does check is the next question: once an agent can read your store, is the data it gets correct? It compares what ChatGPT, Gemini / AI Mode, Copilot and Perplexity-style agents receive with your live product pages, covering price, currency, stock, structured data, variants, links, policies and the cart path up to (never through) payment. Run a free check.

Frequently asked questions

Related guides

Sources

• OpenAI: Overview of OpenAI crawlers
• Perplexity: Perplexity crawlers
• Anthropic: Does Anthropic crawl data from the web, and how can site owners block the crawler?
• Marie Haynes: Should you use Google-Extended in robots.txt? (quotes Google documentation)
• Shopify docs: Customize robots.txt
• Temso: Bingbot