Robots.txt

A text file placed in the root directory of a website that tells search engine crawlers which pages or sections of the site should not be indexed or crawled.

Ready to Put Our Award-Winning Team to Work?

Skip to content