Cooking with SQL - List of Bots

List of Bots

Written by Datagod
Wednesday, 14 May 2008
Here is a master list of bots that have been crawling my websites over the past year. I examine the UserAgent and apply a few rules in order to classify the visitors as Bots. This list may contain some false positives, but it is interesting to read none the less. Enjoy! 8484 Boston Project v 1.0 Acoon-Robot 4.0.2.17 (http://www.acoon.de) AdsBot-Google ( http://www.google.com/adsbot.html) AdsBot-Google (+http://www.google.com/adsbot.html) Agent Anonymous/3G bot 1.0 (anything...anything you want.; http://anything.you.want; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) AnotherBot 1.0 Apexoo Spider 1.1 Attentio/Nutch-0.9-dev (Attentio\'s beta blog crawler; www.attentio.com; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Azureus 3.0.3.4;Mac OS X;Java 1.5.0_13 Baiduspider ( http://www.baidu.com/search/spider.htm) Baiduspider ( http://www.baidu.com/search/spider_jp.html) Baiduspider+(+http://www.baidu.com/search/spider.htm) Baiduspider+(+http://www.baidu.com/search/spider_jp.html) Balihoo/Nutch-0.9 (Crawler for Balihoo.com search engine - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com) Balihoo/Nutch-1.0-dev (Crawler for Balihoo.com search engine - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com) Bigsearch.ca/Nutch-1.0-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) blackspider Bloodhound/Nutch-0.9 (Testing Crawler for Research - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com) boitho.com-dc/0.82 ( http://www.boitho.com/dcbot.html ) boitho.com-dc/0.85 ( http://www.boitho.com/dcbot.html ) boitho.com-dc/0.86 ( http://www.boitho.com/dcbot.html ) bot/1.0 (bot; http://; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) BrightCrawler (http://www.brightcloud.com/brightcrawler.asp) Canookle/1.1 ( http://www.canookle.ca/spider.html) CazoodleBot/CazoodleBot-0.1 (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) CCBot/1.0 ( http://www.commoncrawl.org/bot.html) CentiverseBot CentiverseBot - investigator CentiverseBot/3.0 (http://www.centiverse-project.net) ContactBot/0.2 ConveraCrawler/0.9e ( http://www.authoritativeweb.com/crawl) ConveraCrawler/2.0 ( http://ews.converasearch.com/crawl.htm) cytqqrmxxpthobottvljckjgx0j0sqqtts dgwkexnqfsogisyhbotfi2vjum2kdmbbsqebrun DiGi-RSSBot disco/Nutch-1.0-dev (experimental crawler; www.discoveryengine.com; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) DNSRight.com WebBot Link Ckeck Tool. Report abuse to: This e-mail address is being protected from spam bots, you need JavaScript enabled to view it Dot TK robot -- spidering your website. Rename your website at www.dot.tk DotTK-Robot/1.0 (Rename The Internet - http://www.dot.tk) DuckDuckBot/1.0; ( http://duckduckgo.com/duckduckbot.html) Dzesqclj qmjpbot nsexc e-SocietyRobot(http://www.yama.info.waseda.ac.jp/~yamana/es/) EnaBot/1.2 (http://www.enaball.com/crawler.html) Eurobot/Nutch-1.0-dev (1.0) FAST Enterprise Crawler 6 used by fast (jorgent@fast) FAST Enterprise Crawler 6 used by Singapore Press Holdings ( This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) FAST MetaWeb Crawler (helpdesk at fastsearch dot com) favorstarbot/1.0 ( http://favorstar.com/bot.html) Feedfetcher-Google; ( http://www.google.com/feedfetcher.html) Feedfetcher-Google; ( http://www.google.com/feedfetcher.html; 1 subscribers; feed-id=12476056983529841647) Feedfetcher-Google; ( http://www.google.com/feedfetcher.html; 2 subscribers; feed-id=12476056983529841647) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 1 subscribers; feed-id=12476056983529841647) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 2 subscribers; feed-id=12476056983529841647) froGgle/0.0 BETA! ( http://www.jabea.net/bot.php) Gaisbot/3.0 ( This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ; http://www.widelab.org/robot.html) Gaisbot/3.0 ( This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ; http://gais.cs.ccu.edu.tw/robot.php) Gaisbot/3.0+( This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ;+http://gais.cs.ccu.edu.tw/robot.php) genieBot This e-mail address is being protected from spam bots, you need JavaScript enabled to view it GeonaBot/1.2; http://www.geona.com/ Gigabot/1.0 Gigabot/2.0 (http://www.gigablast.com/spider.html) Gigabot/2.0att Gigabot/3.0 (http://www.gigablast.com/spider.html) Google-Sitemaps/1.0 googlebot Googlebot-Image/1.0 Googlebot/2.0 (http://www.googlebot.com/bot.html) Googlebot/2.1 Googlebot/2.1 ( http://www.google.com/bot.html) Googlebot/2.1 ( http://www.googlebot.com/bot.html) Googlebot/2.1 ( http://www.googlebot.com/bot.html; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; TheWorld) Googlebot/2.1 (http://www.googlebot.com/bot.html) great-plains-web-spider/gpws (Flatland Industries Web Spider; http://www.flatlandindustries.com/flatlandbot.php; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) gsa-crawler (Enterprise; S4-EHN4YSQ2XLJAB; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it , This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) GurujiBot/1.0 ( http://www.guruji.com/en/WebmasterFAQ.html) GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html) gzip(gfe) (via translate.google.com) hdgjacfbqnupyrjavaqrngnxptq HMSE_Robot Holiday Shopping, Inc. Hotakew perljwsi nudlg HouxouCrawler/Nutch-0.8.2-dev I411Crawling/Nutch-0.9 (CrawlData; www.i411.com; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) iaskspider/2.0( http://iask.com/help/help_index.html) Ibotyze riucktv dmjy ICC-Crawler(Mozilla-compatible; http://kc.nict.go.jp/icc/crawl.html; icc-crawl(at)ml(dot)nict(dot)go(dot)jp) ICC-Crawler(Mozilla-compatible; http://kc.nict.go.jp/icc/crawl.html; icc-crawl-contact(at)ml(dot)nict(dot)go(dot)jp) ichiro/2.0 (http://help.goo.ne.jp/door/crawler.html) ichiro/3.0 (http://help.goo.ne.jp/door/crawler.html) IIITBOT/1.1 (Indian Language Web Search Engine; http://webkhoj.iiit.net; pvvpr at iiit dot ac dot in) ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company. For more information please visit http://www.ilial.com/crawler; http://www.ilial.com/crawler; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company.; http://www.ilial.com/crawler; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) IlseBot/1.1 inelibotTqopauiuehccihTeenhkau IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler) IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler/) IRLbot/3.0_space_(compatible;_space_MSIE_space_6.0;_space_http://irl.cs.tamu.edu/crawler) IUPUI Research Bot v 1.9a Java/1.4.2 Java/1.4.2_01 Java/1.4.2_03 Java/1.4.2_04 Java/1.4.2_07 Java/1.4.2_11 Java/1.4.2_12 Java/1.4.2_13 Java/1.5.0-p3 Java/1.5.0_01 Java/1.5.0_02 Java/1.5.0_09 Java/1.5.0_12 Java/1.5.0_13 Java/1.6.0-oem Java/1.6.0-rc Java/1.6.0_02 Java/1.6.0_03 Java/1.6.0_04 Java/1.6.0_05 John\'s SEO (http://www.familieboers.eu/seo/seoinfo.aspx - This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) KAIST AITrc Crawler Krugle/Krugle,Nutch/0.8 (Krugle web crawler; http://corp.krugle.com/crawler/info.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Krugle/Krugle,Nutch/0.8+ (Krugle web crawler; http://corp.krugle.com/crawler/info.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) ktbxewuqetybotfo5aevfhg LapozzBot/1.5 ( http://robot.lapozz.hu) LarbinWebCrawler This e-mail address is being protected from spam bots, you need JavaScript enabled to view it LargeSmall Crawler Lexxe/Robot libwww-perl/5.36 libwww-perl/5.43 libwww-perl/5.47 libwww-perl/5.48 libwww-perl/5.49 libwww-perl/5.50 libwww-perl/5.51 libwww-perl/5.53 libwww-perl/5.60 libwww-perl/5.62 libwww-perl/5.63 libwww-perl/5.64 libwww-perl/5.65 libwww-perl/5.68 libwww-perl/5.69 libwww-perl/5.70 libwww-perl/5.73 libwww-perl/5.75 libwww-perl/5.76 libwww-perl/5.77 libwww-perl/5.78 libwww-perl/5.79 libwww-perl/5.800 libwww-perl/5.801 libwww-perl/5.802 libwww-perl/5.803 libwww-perl/5.804 libwww-perl/5.805 libwww-perl/5.806 libwww-perl/5.807 libwww-perl/5.808 libwww-perl/5.810 libwww-perl/5.811 libwww-perl/5.812 LocalcomBot/1.3.0 ( http://www.local.com/bot.htm) LTI/LemurProject Nutch Spider/Nutch-1.0-dev (lti crawler for CMU; http://www.lti.cs.cmu.edu; changkuk at cmu dot edu) LTI/LemurProject Nutch Spider/Nutch-1.0-dev (Research spider using Nutch; http://lucene.apache.org/nutch/bot.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) LTI/LemurProject Nutch Spider/Nutch-1.0-dev (Research spider using Nutch; http://www.lemurproject.org; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) lwbotuTafkqpjfpwhjlvmukjaolmamcnk Lycos_Spider_(modspider) McBot/5.001 (windows; U; NT4.0; en-us) Mediapartners-Google Mediapartners-Google/2.1 medrabbit/0.1beta (Crawling for health research; http://www.medrapid.info/bot.htm; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Missigua Locator 1.9 MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php? ) MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+) MJ12bot/v1.2.0 (http://majestic12.co.uk/bot.php? ) MJ12bot/v1.2.0 (http://majestic12.co.uk/bot.php?+) MLBot (www.metadatalabs.com) MLBot (www.metadatalabs.com/mlbot) Moreoverbot/5.00 ( http://www.moreover.com) Mozilla/2.0 (compatible; Ask Jeeves/Teoma) Mozilla/2.0 (compatible; Ask Jeeves/Teoma; http://about.ask.com/en/docs/about/webmasters.shtml) Mozilla/2.0 (compatible; Ask Jeeves/Teoma; http://sp.ask.com/docs/about/tech_crawling.html) Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml) Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html) Mozilla/4.0 (compatible; Vagabondo/4.0Beta; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/; http://www.wise-guys.nl/) Mozilla/4.0 (compatible; GoogleBot 1.2) Mozilla/4.0 (compatible; MSIE 5.01; Windows 95; MSIECrawler) Mozilla/4.0 (compatible; MSIE 5.0; YANDEX) Mozilla/4.0 (compatible; MSIE 6.0 compatible; Asterias Crawler v4; http://www.singingfish.com/help/spider.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ); SpiderThread Revision: 3.11 Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Win 9x 4.90; MSIECrawler) Mozilla/4.0 (compatible; MSIE 6.0; Windows CE; IEMobile m.n) libwww-perl/5.805 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Google Wireless Transcoder;) Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; MSIECrawler Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1),gzip(gfe) (via translate.google.com) Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ),gzip(gfe) (via translate.google.com) Mozilla/4.0 (compatible; MSIE is not me; DAUMOA/1.0.1; DAUM Web Robot; Daum Communications Corp., Korea) libwww-perl/5.808 Mozilla/4.0 (compatible; piCrawler) Mozilla/4.5 RPT-HTTPClient/0.3-2,gzip(gfe) (via translate.google.com) Mozilla/5.0 (compatible; archive.org_bot/1.13.1x +http://crawler.archive.org) Mozilla/5.0 (compatible; Ask Jeeves/Teoma; http://about.ask.com/en/docs/about/webmasters.shtml) Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml) Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html) Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html) Mozilla/5.0 (compatible; MSIE 6.0; Podtech Network; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Mozilla/5.0 (compatible; Yahoo! DE Slurp; http://help.yahoo.com/help/us/ysearch/slurp) Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html) Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp),gzip(gfe) (via translate.google.com) Mozilla/5.0 (compatible; Yahoo! Slurp;http://help.yahoo.com/help/us/ysearch/slurp) Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/; ) Mozilla/5.0 (compatible; zermelo; http://www.powerset.com) [email: This e-mail address is being protected from spam bots, you need JavaScript enabled to view it , This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ] Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html) Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12,gzip(gfe) (via translate.google.com) Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12,gzip(gfe) (via translate.google.com) Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13; Google-TR-5.1.707.23222-en) Gecko/20080311 Firefox/2.0.0.13 Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/20070515 Googlebot 2,1 Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5,gzip(gfe) (via translate.google.com) Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:2.0.0) Gecko/20050511 Firefox/2.0.0.1,gzip(gfe) (via translate.google.com) Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.5) Gecko/20070713 Firefox/2.0.0.5,gzip(gfe) (via translate.google.com) Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.1.12; Google-TR-5.1.706.29690-en-GT) Gecko/20080201 Firefox/2.0.0.12 Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12,gzip(gfe) (via translate.google.com) Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5,gzip(gfe) (via translate.google.com) Mozilla/5.0 (Windows;) NimbleCrawler 2.0.1 obeys UserAgent NimbleCrawler For problems contact: This e-mail address is being protected from spam bots, you need JavaScript enabled to view it Mozilla/5.0 (Yahoo-Test/4.0 mailto: This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Mozilla/5.0 (Yahoo-Test/4.0; mailto: This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Mozilla/5.0_space_(compatible;_space_Yahoo!_space_Slurp;_space_http://help.yahoo.com/help/us/ysearch/slurp) Mozilla/5.0_space_(Twiceler-0.9_space_http://www.cuill.com/twiceler/robot.html) Mplfsqik yjatgh mcavperlo MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://vwbot.cs.uiuc.edu; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Mrwhebot bmlk yfkcrhsw msnbot-media/1.0 ( http://search.msn.com/msnbot.htm) msnbot-media/1.0 (+http://search.msn.com/msnbot.htm) msnbot/0.9 ( http://search.msn.com/msnbot.htm) msnbot/0.9 (+http://search.msn.com/msnbot.htm) msnbot/1.0 ( http://search.msn.com/msnbot.htm) msnbot/1.0 (+http://search.msn.com/msnbot.htm) msnbot/1.0+(+http://search.msn.com/msnbot.htm) msnbot/1.1 ( http://search.msn.com/msnbot.htm) MSR-ISRCCrawler MSRBOT (http://research.microsoft.com/research/sv/msrbot) MSRBOT (http://research.microsoft.com/research/sv/msrbot/ multicrawler ( http://sw.deri.org/2006/04/multicrawler/robots.html) mxdw ijboTgfrnTghmTwudn my-heritrix-crawler( http://mywebsite.com) NameOfAgent (CMS Spider) NASA Search 1.0 NetResearchServer/4.0(loopimprovements.com/robot.html) NextGenSearchBot 1 (for information visit http://about.zoominfo.com/About/NextGenSearchBot.aspx) NextGenSearchBot 1 (for information visit http://www.zoominfo.com/About/misc/NextGenSearchBot.aspx) nicebot nKolcutqchh4Klyi4afbotvKsxK4ons cc Nokia6610/1.0 (3.09) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible;YahooSeeker/M1A1-R2D2; http://help.yahoo.com/help/us/ysearch/crawling/crawling-01.html) Nokia6680/1.0 ((4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) ) Nokia6680/1.0 ((4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) ) Nokia6680/1.0 ((4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html) ) Nokia6680/1.0 ((4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (for mobile crawler) ) Nokia6682/2.0 (3.01.1) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 configuration/CLDC-1.1 UP.Link/6.3.0.0.0 (compatible;YahooSeeker/M1A1-R2D2; http://help.yahoo.com/help/us/ysearch/crawling/crawling-01.html) Nokia6682/2.0 (3.01.1) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 configuration/CLDC-1.1 UP.Link/6.3.0.0.0 (compatible;YahooSeeker/M1A1-R2D2; http://help.yahoo.com/help/us/ysearch/crawling/crawling-01.html) Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; http://www.google.com/bot.html) Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html) noxtrumbot/1.0 ( This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) nrsbot/5.0(loopimprovements.com/robot.html) nrsbot/5.0(loopip.com/robot.html) Nutch/Nutch-0.9 (Eurobot; http://www.ayell.eu ) nutchbot/Nutch-1.0-dev NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) NutchCVS/0.7 (Nutch; http://lucene.apache.org/nutch/bot.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) nyac libmhq afJproJnnpcbotircgukoepykhc nyjfbbotery dpvhcctvrplgfxykEgccxpx oBot Openfind data gatherer, Openbot/3.0 ( This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ; http://www.openfind.com.tw/robot.html Openfind data gatherer, Openbot/3.0+( This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ;+http://www.openfind.com.tw/robot.html OpenPlaces/Nutch-1.0-dev (OpenPlaces Content Crawler; http://www.openplace.com; dnadeau at openplaces.com) Os xsxbotjpnbfbn wtm oq owsBot/0.1 (Nutch; www.oneworldstreet.com; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Oxford_AI_BOT_V1.0 (compatible; MSIE 6.0;) PCbot/3.0 Pete-Spider Light/1.2 pgyxeXbgqin8vbotkophhoXpnptgfvl8ulolryv PHP version tracker (http://www.nexen.net/phpversion/bot.php) Pingdom.com_bot_version_1.4_(http://www.pingdom.com/) PlantyNet_WebRobot_V1.9 This e-mail address is being protected from spam bots, you need JavaScript enabled to view it POE-Component-Client-HTTP/0.65 (perl; N; POE; en; rv:0.650000) psbot/0.1 ( http://www.picsearch.com/bot.html) psbot/0.1 (+http://www.picsearch.com/bot.html) pythonic-crawler ( This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) qi8asakoakbotfmvi mltoR8xav Rankivabot/3.2 (www.rankiva.com; 3.2; vzmxikn) RedBot/redbot-1.0 (Rediff.com Crawler; redbot at rediff dot com) Retriever/Nutch-0.9 (Crawler for Balihoo.com search engine - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com) Robotgenius crawler/Nutch-1.0-dev (http://robotgenius.net; misc at robotgenius dot net) RSS-SPIDER (http://www.rss-spider.com/submit200709.php) sbotcpkwujpdumgnbugnLgqljdxbicvmsx schibstedsokbot (compatible; Mozilla/5.0; MSIE 5.0; FAST FreshCrawler 6; +http://www.schibstedsok.no/bot/) ScSpider/0.2 Seekbot/1.0 (http://www.seekbot.net/bot.html) HTTPFetcher/2.2 Sensis Web Crawler (search_comments\\at\\sensis\\dot\\com\\dot\\au) ShablastBot 1.0 ShopWiki/1.0 ( http://www.shopwiki.com/wiki/Help:Bot) Snapbot/1.0 ( http://www.snap.com) Snapbot/1.0 (+http://www.snap.com) Snapbot/1.0 (Snap Shots, http://www.snap.com) Snapbot/1.0 (Snap Shots, +http://www.snap.com) sogou develop spider Sogou web spider/3.0( http://www.sogou.com/docs/help/webmasters.htm SouFZ-Spider[SouFZ.COM] Speedy Spider (Entireweb; Beta/1.1; http://www.entireweb.com/about/search_tech/speedyspider/) Speedy Spider (Entireweb; Beta/1.2; http://www.entireweb.com/about/search_tech/speedyspider/) Speedy Spider (Entireweb; Beta/1.3; http://www.entireweb.com/about/search_tech/speedyspider/) Speedy Spider (http://www.entireweb.com/about/search_tech/speedyspider/) Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/) Spock Crawler (http://www.spock.com/crawler) Steeler/3.3 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/) Steeler/3.4 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/) SurveyBot/2.3 (Whois Source) Technoratibot/0.7 Teemer (NetSeer, Inc. is a Los Angeles based Internet startup company.; http://www.netseer.com/crawler.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Teemer (NetSeer, Inc.; http://www.netseer.com/crawler.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) TestCrawler/Nutch-0.9 (Testing Crawler for Research ; http://chitchit.org/TestCrawler.html; amitjain at spro dot net) ThaiSUE-Crawler Theme Spider ( http://www.themespider.com/spider.html) TinEye/1.1 (http://tineye.com/crawler.html) TMCrawler tmvmbmObsojgfamqqbOthlvdibxObugyvy Trailfire-bot/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Trailfire-bot/0.7.1 (Trailfire page content analyzer; http://trailfire.com; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Trailfire-bot/Nutch-1.0-dev (Trailfire page content analyzer; http://trailfire.com; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) Trailfire/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) travel-spider/Nutch-0.9 TRAVEL.com - travel-search engine (see http://www.travel.com/spiders.html) TurnitinBot/2.1 (http://www.turnitin.com/robot/crawlerinfo.html) TwengaBot/1.1 ( http://www.twenga.com/bot.html) Twiceler www.cuill.com/twiceler/robot.html Twiceler-0.9 http://www.cuill.com/twiceler/robot.html UogU0bxemmuf fyaprbot0 vbvhajwnUbaa User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1),gzip(gfe) (via translate.google.com) VadixBot VadixBot_Exp VisBot/2.0 (Visvo.com Crawler; http://www.visvo.com/bot.html; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) voyager-hc/1.0 ( http://www.kosmix.com/crawler.html) voyager-hc/1.0 (+http://www.kosmix.com/crawler.html) vspider VSynCrawler/1.0 VWBOT/Nutch-0.9-dev (VWBOT Nutch Crawler; http://vwbot.cs.uiuc.edu; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) w5vbotjovngrklftksnNvsxlimb srslooygel5 WebAlta Crawler/1.3.33 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU) WebAlta Crawler/1.3.34 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU) WebAlta Crawler/2.0 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU) webbot( http://webbot.com/bot.htm) WebRankSpider/1.37 ( http://ulm191.server4you.de/crawler/) WebRankSpider/1.37 (+http://ulm191.server4you.de/crawler/) Wells Search II WEP Search 00 WinHTTP Robot/1.0 woriobot ( http://www.worio.com/) woriobot (+http://www.worio.com) woriobot (+http://www.worio.com/) Wujugo/Nutch-1.0-dev (Wujugo Content Crawler; http://www.wujugo.org; {dnadeau,srainville} at brightspark.com) Xgpbotvu umkbwtq hmvykj XwSpider yacybot (i386 Linux 2.6.15-29-386; java 1.6.0; Europe/de) http://yacy.net/bot.html yacybot (i386 Linux 2.6.9-023stab044.12-smp; java 1.6.0_03; Europe/en) http://yacy.net/yacy/bot.html yacybot (i386 Linux 2.6.9-023stab044.12-smp; java 1.6.0_04; Europe/en) http://yacy.net/yacy/bot.html yacybot (i386 Linux 2.6.9-023stab044.4-smp; java 1.6.0_02; Europe/en) http://yacy.net/yacy/bot.html yacybot (x86 Windows 2003 5.2; java 1.6.0_03; Europe/de) http://yacy.net/bot.html yacybot (x86 Windows 2003 5.2; java 1.6.0_04; Europe/de) http://yacy.net/bot.html yacybot (x86 Windows XP 5.1; java 1.6.0_01; Europe/de) http://yacy.net/yacy/bot.html Yahoo Pipes 1.0 Yahoo! Mindset Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com) YahooFeedSeeker/2.0 (compatible; Mozilla 4.0; MSIE 5.5; http://publisher.yahoo.com/rssguide) Yandex/1.01.001 (compatible; Win16; H) Yandex/1.01.001 (compatible; Win16; I) Yeti/0.01 (nhn/1noon, This e-mail address is being protected from spam bots, you need JavaScript enabled to view it , check robots.txt daily and follow it) Yeti/1.0 ( http://help.naver.com/robots/) yoofind/yoofind-0.1-dev (yoono webcrawler; http://www.yoono.com ; This e-mail address is being protected from spam bots, you need JavaScript enabled to view it ) YooName/Nutch-1.0-dev (YooName Content Crawler; http://www.yooname.com; pythonner-at gmail . com) Hits: 2992 Comments (0) Subscribe to this comment's feed Write comment Name Website Title Comment smaller \| bigger I have read and agree to the Terms of Usage.

< Prev		Next >