CompaBot

CompaBot is the crawler of Compa, a comparator of group trips. Compa does not sell trips and takes no paid placements: every trip links back to the operator's own site for booking.

Who runs it

• Operator: Mattia Morelli, an individual (Compa is not yet incorporated).
• Contact, removal requests and objections: bot@compatravel.com.

Identify CompaBot

• User-agent: CompaBot/0.1 (+https://compatravel.com/bot). The version number may change; match on CompaBot.
• robots.txt product token: CompaBot (case-insensitive).
• Fixed IP address: it will be published here before regular catalog crawling starts. Until then, requests come from a residential connection in the Netherlands, and every request is signed (see below).

Verify a request

CompaBot signs every request with Web Bot Auth (HTTP Message Signatures, RFC 9421), as profiled by draft-ietf-webbotauth-httpsig-protocol-00. To check that a request really comes from us:

1. The request carries Signature-Agent: "https://compatravel.com" (a plain string, the form Cloudflare verifies today), plus Signature-Input and Signature headers with tag="web-bot-auth".
2. The signature covers @authority and signature-agent, uses Ed25519, is valid for 60 seconds (created to expires) and carries a nonce.
3. Fetch our public keys from the key directory: https://compatravel.com/.well-known/http-message-signatures-directory.
4. Verify the signature with the key whose keyid matches. Active keys: PIxsFmaZ_aVqrWzbQnB16qlT0qJY9ap9DoRUen95qzQ.

When we rotate keys, the directory lists both the old and the new key until the crawler has switched; then the old one is removed.

A request that claims to be CompaBot but has no valid signature is not from us. Please report it to bot@compatravel.com.

What we collect and how we use it

• Only facts about trips: dates, prices, availability, duration, places and what is included.
• We never store or republish descriptions, itinerary text, photos, logos, reviews or personal data.
• Page text is processed only briefly, including with AI tools, to extract those facts so trips from different operators can be compared and matched to travellers. It is deleted within 30 days and is never used to train AI models.
• The facts appear on compatravel.com, and every trip links back to the operator's page.

How we crawl

• Before listing an operator, we run a one-off check of a few pages: robots.txt, sitemaps, legal pages and at most three trip pages.
• Catalog: one full refresh a week and a daily check of departures within the next 60 days only.
• All crawling, the weekly refresh included, runs at night (01:00–06:00 Italian time). A refresh that doesn't finish continues the next night.
• One connection per site, with at least 10 seconds between pages (or the site's Crawl-delay, if higher).
• Pages may be rendered in a headless browser (Chromium). It loads only the page's own scripts, styles and data requests, and never images, media, fonts or resources from other hosts. Legal pages are the one exception for other hosts, because their terms are sometimes embedded from another site. Resources that robots.txt disallows are never requested.
• Every request, page or resource, carries the CompaBot user-agent and signature.
• We never fetch pages in real time on behalf of a user.

What we respect

• robots.txt (RFC 9309), robots meta tags, X-Robots-Tag headers, TDMRep and ai.txt.
• Terms of use: if an operator's terms forbid automated access, scraping or reuse by comparators, we don't crawl its catalog without its written permission.
• We never log in, submit forms or solve CAPTCHAs.
• We stop at the first 403 or 429 response or bot challenge, and after three server errors in a row. We resume only after a manual review.

Block CompaBot

To stop CompaBot crawling your whole site, add this to your robots.txt:

User-agent: CompaBot
Disallow: /

To block only part of it, for example /private/:

User-agent: CompaBot
Disallow: /private/

We read robots.txt before every run, so a change applies from the next one.

Ask for removal

• Write to bot@compatravel.com and name the operator. We act directly on requests sent from an address on the operator's own domain, or from one published on its site.
• From any other address, we stop crawling that operator at once and ask the operator to confirm at its published address. We remove its trips once it confirms.
• Within 24 hours of an accepted request we unpublish the operator's trips (their pages return 410 Gone), stop the crawler, delete the stored text and facts, and confirm in writing to the sender and to the operator's published address.

Spotted a wrong price or date for a trip on Compa? Write to bot@compatravel.com and we will fix or remove it.

Privacy

• Data controller: Mattia Morelli, an individual (Compa is not yet incorporated).
• Trip pages sometimes name trip leaders. That text goes through the brief processing described above, including by an AI provider bound by a data processing agreement, which never uses it for training. If the provider is outside the EU, the transfer relies on the GDPR's safeguards, such as an adequacy decision or standard contractual clauses. Our legal basis is our legitimate interest in comparing trips (GDPR art. 6(1)(f)). Names are dropped when the facts are extracted, and the text is deleted within 30 days.
• If you write to us, we keep your message, our reply and the related evidence, to handle your request and to defend against legal claims (GDPR art. 6(1)(f)). We keep them for 10 years, the longest limitation period for claims that could arise.
• You can ask for access to your data, ask us to correct it, or object, at bot@compatravel.com. You can also complain to the data protection authority of your country.
• This page sets no cookies and uses no tracking.