OwedToYouBot

OwedToYouBot is the web fetcher for Owed To You. It reads public pages from government and other official sources that list unclaimed money and ways to claim it.

It identifies itself as OwedToYouBot/1.0 (+https://owedtoyou.com.au/bot).

How it behaves

• It reads robots.txt first and follows it, including any Crawl-delay.
• If robots.txt can't be read because of a server error, it doesn't fetch from that site.
• It fetches a path that robots.txt disallows only when the site owner has given written permission.
• It waits at least 2 seconds between requests to the same site.
• It never tries to get around a block. If a site refuses a request or shows a challenge page, it stops and we review the source by hand.
• It doesn't run a browser or execute scripts.

To ask us to stop fetching from your site, add a Disallow rule for OwedToYouBot to your robots.txt.