DatayzeBot

DatayzeBot is the website crawler for our website analysis tool suite. If you see DatayzeBot in your access logs it is because someone has requested your site be analyzed. Don't want your site crawled in whole or in part? DatayzeBot respects the robots exclusion standard.

Example: To prevent the spider from crawling any URLs on your site simply add the following to your robots.txt file:

User-agent: DatayzeBot
Disallow: /

There are currently 2 apps that do not follow the robots exclusion standard:

These apps analyze a single page, and do not crawl a site.


Datayze does not store, cache or index pages crawled. Setting the noindex meta field will have no effect on the datayze spider.