# Robots.txt file from http://www.robottext.co.uk # # Built from information taken from http://www.searchengineworld.com & http://info.webcrawler.com/mak/projects/robots/active/all.txt # # User-Agent: Mediapartners-Google Disallow: /*.swf Disallow: /*.flv Disallow: /images/ Disallow: /itin/ User-Agent: Googlebot Disallow: /*.swf Disallow: /*.flv Disallow: /images/ Disallow: /itin/ User-Agent: MSNbot Disallow: /*.swf Disallow: /*.flv Disallow: /images/ Disallow: /itin/ # Yahoo and Inktomi both use Slurp User-agent: Slurp Disallow: /*.swf Disallow: /*.flv Disallow: /images/ Disallow: /itin/ User-Agent: WISENutBot Disallow: /*.swf Disallow: /*.flv Disallow: /images/ Disallow: /itin/ User-Agent: ASKJeeves Disallow: /*.swf Disallow: /*.flv Disallow: /images/ Disallow: /itin/ User-Agent: ia_archiver Disallow: /*.swf Disallow: /*.flv Disallow: /images/ Disallow: /itin/