Product Gale bot

This page describes the automated requests Product Gale sends to online stores. If you found productgale.com/bot in your logs, this is who we are and how to control what we do on your site.

Who operates it

Product Gale (productgale.com). Product Gale measures what AI shopping assistants can read on a store. A scan is started by one of our users, usually for their own store, or by us, to show a store's owner what AI shoppers see.

User agents

Both carry the robots.txt token ProductGale.

What a scan requests

• The store's product feed or sitemap.
• robots.txt, whose rules we follow.
• Product pages, up to the number the scan's plan allows. A scan never crawls a whole store.
• A small set of test requests to those pages that measure how the store responds to AI assistants. For ChatGPT, ChatGPT search, Claude and Perplexity our user agent carries that assistant's token, for example ProductGale-Scan/1.0 (as ChatGPT-User/1.0; +https://productgale.com/bot), so a rule that matches the token still applies. One more request comes from a headless browser. None of these is sent by those assistants, and every one is signed as Product Gale, the same way as every other request on this page.

An index run reads the product feed, sitemap and product pages of a store one of our users added, up to the number of pages their plan allows, under the same robots.txt rules.

How to verify a request is from us

Every request we send to a store is signed with Web Bot Auth (HTTP Message Signatures, RFC 9421). Each request carries Signature-Agent, Signature-Input and Signature headers. Our public keys are published at:

https://api.productgale.com/.well-known/http-message-signatures-directory

Every request from our production service is signed. Treat a request that names Product Gale in its user agent but carries no valid signature from one of those keys as not from us.

Where requests come from

The addresses our production service sends from are published, machine readable, at productgale.com/bot/ips.json. The signature above is the stronger check: an address can be shared, a valid signature cannot be forged.

How to opt out

Add this to your robots.txt and we will stop requesting your site:

User-agent: ProductGale
Disallow: /

To slow us down instead, we honour Crawl-delay in the group that matches ProductGale, or in the * group if there is none:

User-agent: ProductGale
Crawl-delay: 10

Contact

Questions about a request you saw, or a problem it caused: email contact@productgale.com. Including the time, the URL requested and the user agent helps us find it.