BaskoBot
Коротко
BaskoBot читает цены, которые магазины Узбекистана публикуют на своих сайтах. Он соблюдает robots.txt и Crawl-delay, не входит в аккаунты и не выдаёт себя за браузер. Чтобы остановить его, добавьте в robots.txt строки User-agent: BaskoBot и Disallow: /.
BaskoBot reads the prices that stores in Uzbekistan publish on their own websites, so the Basko app can compare a shopping list across stores. It identifies itself as:
BaskoBot/1.0 (+https://basko.app/bot)How it behaves
- It reads only the product data a store publishes for search engines (schema.org), on pages listed in the store's sitemaps.
- It obeys
robots.txt, including Google's wildcard rules, and never goes faster than yourCrawl-delay. Without one, it waits 2 to 3 seconds between requests to a store. - It sends conditional requests (
If-None-Match,If-Modified-Since), backs off on errors and pauses after repeated failures. - It never signs in, solves captchas, rotates IP addresses or pretends to be a browser.
- Product photos shown on basko.app are fetched once by Basko's website (as
BaskoBot/1.0) and kept for a week, so visitors don't load them from your servers.
To stop it
Add this to your robots.txt; BaskoBot reads it again within a day:
User-agent: BaskoBot
Disallow: /Store owners who'd rather send a product feed (YML or Google Merchant XML), or who want their store's prices, name or logo removed from Basko, can write to support@basko.app.