# robots.txt for http://www.esate.ru User-Agent: * Disallow: /an_t/ User-agent: Slurp Crawl-delay: 500