Scraper Information

MenuInformatics crawler disclosure

This page explains what our crawler does, how it identifies itself, how site operators can contact us, and the guardrails we follow while collecting public restaurant data.

Start coverage plan Review web scraping Contact support See the product

Purpose

Restaurant web discovery and evidence capture

MenuInformatics crawls public restaurant websites and ordering surfaces to find current menus, locations, prices, hours, specials, and related evidence used in our market-intelligence products.

Identity

Transparent crawler identification

Direct website-crawl traffic identifies as MenuInformaticsBot and points back to this disclosure page so site operators can understand what the bot is doing and how to contact us.

Respect

Robots-aware and rate-conscious

We honor robots.txt directives and crawl-delay on direct site crawling, avoid aggressive request bursts, back off on errors and throttling signals, and keep crawl activity focused on legitimate restaurant discovery and refresh needs.

Crawler Details

How the crawler behaves

We crawl for restaurant discovery, evidence refresh, and data quality review. We do not use the crawler to access private user accounts, authenticated sessions, or payment flows.
We focus on public restaurant web pages, menu endpoints, ordering surfaces, and related evidence that helps us keep restaurant records current.
We avoid harmful traffic patterns by applying queueing, throttling, host-aware scheduling, crawl-delay enforcement, and backoff when a site signals stress or blocks access.
We keep direct website crawler identity stable and reviewable so site operators, CDNs, and security teams can attribute our traffic correctly.
We review removal or concern requests through support intake and can suppress or re-scope crawl behavior where appropriate.

Verification

Operator-facing details

Primary crawler identity: `MenuInformaticsBot/1.0 (+https://menuinformatics.com/scraper)`
Public disclosure and contact path: this page plus the support intake at `/support`
Web Bot Auth key directory: `https://menuinformatics.com/.well-known/http-message-signatures-directory`
Expected request behavior: low-rate fetches, conditional refreshes where supported, `robots.txt` and `crawl-delay` enforcement on direct site crawling, and no attempts to bypass authentication or private surfaces
Expected verification headers on direct site-crawl requests when enabled: `Signature-Agent`, `Signature-Input`, and `Signature`
Opt-out / review requests: submit a support request with the site or path you want reviewed
Provider integrations: some third-party ordering surfaces may require provider-compatible request headers, but direct website crawl traffic keeps a stable MenuInformatics crawler identity
Verification target: transparent bot identity, public crawler documentation, and Web Bot Auth support are maintained for operator review and verified-bot enrollment

Crawler FAQ

Common operator questions

What does MenuInformaticsBot crawl?

MenuInformaticsBot crawls public restaurant websites, menu pages, ordering surfaces, location pages, hours, specials, and related evidence used to keep restaurant market-intelligence records current.

How does MenuInformaticsBot identify itself?

Direct website-crawl traffic uses the stable MenuInformaticsBot identity and links back to the public crawler disclosure page so operators, CDNs, and security teams can attribute requests correctly.

Does MenuInformatics honor robots.txt and crawl-delay?

Direct site crawling honors robots.txt directives and crawl-delay, applies host-aware throttling, avoids aggressive request bursts, and backs off when a site signals stress or blocks access.

How can site operators request a crawl review or suppression?

Operators can request a coverage review or support review with the relevant domain, URL pattern, headers, and timestamps so MenuInformatics can investigate and tune crawl behavior.