User-agent: * Disallow: /info.php Disallow: /cgi-bin/ Disallow: /404.html Crawl-delay: 10