# # Exclusions for web crawlers # # http://www.searchengineworld.com/robots/robots_tutorial.htm # User-agent: * Disallow: /cgi-bin/ Disallow: /faq/ Disallow: /internals/ Disallow: /jukebox/ Disallow: /mrep/ Disallow: /old/ Disallow: /picbase/