Krill

The standalone storefront analysis service operated by Waddle Labs Inc.

Krill analyzes public Shopify storefronts to generate store-intelligence reports for authorized demo and operations workflows.

Why Krill visits a storefront

Production intake requires a Shopify storefront. Krill visits a public storefront only as part of an authorized analysis workflow, to inspect shopper-facing pages and selected public resources and to produce rendered evidence for a report.

How to identify Krill

Krill identifies eligible HTTPS merchant-origin requests with Web Bot Auth HTTP message signatures. Signed requests carry Signature, Signature-Input, and Signature-Agent headers.

Signature-Agent / public key directory:

https://krill-gcp.gentooai.com/.well-known/http-message-signatures-directory

The directory response uses the media type application/http-message-signatures-directory+json and publishes the public verification material. The key identifier observed in the live response on 2026-08-05 was m7zyQj0IVCTCqb26jW9zGcJEyIo9LW8glkA72JkR9Vo. Key identifiers can rotate; the live directory is authoritative.

Declared User-Agent:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 Krill/1.0 (+https://krill-gcp.gentooai.com/.well-known/http-message-signatures-directory)

The signature and the public key directory, not the browser-compatible User-Agent tokens alone, are the authoritative way to verify a Krill request.

Robots.txt and Crawl-delay

Krill fetches an origin's /robots.txt once per analysis and host before deeper storefront requests. It applies the published rules for the Krill User-Agent and honors Crawl-delay; when a Krill-specific delay is not present, it falls back to the * policy.

• A 404 or 410 robots response is treated as no published robots policy.
• A 401 or 403 response stops storefront access because the policy cannot be verified.
• Other failed or non-successful responses permit only the submitted root URL; Krill does not continue into deeper paths until the policy is available.
• An explicit Disallow rule prevents access to the matching URL.

Request policy

The following are the reviewed production settings captured on 2026-08-05. They bound storefront traffic; they are not an unlimited crawl allowance.

• Per storefront host, at most one request is in flight at a time, with at least 2 seconds between request starts. A published Crawl-delay can make that interval longer.
• Across merchant hosts on the egress process, at most one storefront operation is in flight, with at least 1 second between starts. The first merchant document waits 2 seconds after robots discovery.
• Each analysis enforces a soft limit of 150 storefront operations and a hard limit of 240. It reserves 48 critical-resource starts: 16 for stylesheets and 32 for images. The current collection and product-page limits are 2 pages each.
• For retryable upstream responses (429, 403, 500, 502, 503, or 504), Krill makes at most one bounded retry and honors Retry-After for up to 6 seconds. A confirmed 429 stops further storefront I/O and records a shared cooldown instead of continuing the analysis.
• The configured cooldown schedule for repeated confirmed throttles is 24 hours, then 3 days, then 7 days maximum.

What Krill does not claim

• Krill is not an open-ended or general web crawler. Analyses are bounded by the request budget and page limits above.
• Web Bot Auth identification does not by itself mean that Shopify has verified Krill or approved a higher request tier.
• The public directory contains verification material only. It does not publish private keys, service tokens, or other internal credentials.

Contact and opt-out requests

For questions about a Krill request or to ask that storefront access be paused, contact support@waddlelab.com.