infoharness

About the infoharness crawler

If you found this in your server logs, someone asked infoharness to check the documentation on your site for contradictions and broken links.

What it does

  • Reads up to 300 public pages under the address it was given, plus your sitemap.
  • Obeys robots.txt. It never logs in, submits forms or runs JavaScript.
  • Checks each link it finds with one lightweight request.
  • Keeps a polite pace: a handful of requests at a time, with timeouts and size limits.

How it identifies itself

User agent: InfoharnessBot/0.1 (+https://infoharness.com/bot)

How to block it

Add this to your robots.txt and it will skip your whole site:

User-agent: InfoharnessBot
Disallow: /