The ComupScanner crawler

ComupScanner is the store crawler of Comup (comup.net). Comup customers use it to measure their own store and the stores they added as competitors: public signals such as shipping and campaign information, structured data on product pages and page speed.

It identifies itself on every request with:

ComupScanner/1.0 (+https://comup.net/tarayici)

Only stores that a Comup customer added as a competitor. It does not roam the web, it is not a search engine and it does not follow links to other sites.

• It reads your robots.txt before every scan and follows its rules.
• It waits at least 2 seconds between requests.
• It opens only a few pages of a store per week: the home page, one category and a few product pages.
• It does not store the raw HTML of your pages; it keeps only the measured signals (for example the free shipping threshold or installment options).
• It never logs in, fills in forms, adds products to a cart or places orders.

Add these two lines to the robots.txt file at the root of your site. ComupScanner sees them on its next scan and will not open your site:

User-agent: ComupScanner
Disallow: /

If you have a problem or a question about the crawler, write to us: [email protected]