Googlebot
Baiduspider
BaiDuSpider
msnbot 或 MSNBOT MSNBot
Isaac Ding
P.Arthur
Yahoo! Slurp
sohu-search
SpiderMan =(yahoo)
Gaisbot
psbot
MSIECrawler
SBL-BOT
Mediapartners-Google
BecomeBot
8fang.net
Gigabot
Spider.NET
Wget
HaoRanSoft
Swoogle
Ultraseek
Java
LinkWalker
Openfind 或Openbot
AIBOT
Nutch
Lycos
Robozilla
crawler
Web Indexing Robot
SurveyBot 2.3 (Whois Source)
FAST-WebCrawler
Scooter
Slurp
Alexa
WISENutbot
IBM_Planetwide
Turn It In
larbin
Jeeves
crawl
Voila
appie
Google AdSense
spider
robot
BBot
NewzCrawler
=============================
以下未验证
--------------------------------------------------------------------------------
AbachoBOT=Abacho.com
abcdatos_botlink=Abcdatos.com
http://www.abcdatos.com/botlink/=Abcdatos.com
AESOP_com_SpiderMan=Aesop.com
ah-ha.com crawler (crawler@ah-ha.com)=ah-ha.com
ia_archiver=Archive.org
Scooter=Altavista.com
Mercator=Altavista.com
Scooter2_Mercator_3-1.0=Altavista.com
roach.smo.av.com-1.0=Altavista.com
Tv_Merc_resh_26_1_D-1.0=Altavista.com
AltaVista-Intranet=Altavista.co.uk
jan.gelin@av.com=Altavista.co.uk
FAST-WebCrawler=alltheweb.com
crawler@fast.no=alltheweb.com
Acoon Robot=acoon.de
antibot=antisearch.net
Atomz=atomz.com
Buscaplus Robi=buscaplus.com
CanSeek/=canseek.ca
support@canseek.ca=canseek.ca
ChristCRAWLER=christcrawler.com
Crawler=crawler.de
admin@crawler.de=crawler.de
DaAdLe.com ROBOT/=daadle.com
RaBot=daum.net
Agent-admin/=daum.net
phortse@hanmail.net=daum.net
contact/jylee@kies.co.kr=kies.co.kr
DeepIndex=deepindex.com
DittoSpyder=ditto.com
Jack=domanova.co.uk
Speedy Spider=entireweb.com
ArchitextSpider=excite.com
ArchitectSpider=excite.com
Arachnoidea=euroseek.net
arachnoidea@euroseek.net=euroseek.net
EZResult=ezresults.com
Fast PartnerSite Crawler=fastsearch.net
FAST Data Search Crawler=fastsearch.net
KIT-Fireball=fireball.de
FyberSearch=fybersearch.com
GalaxyBot=galaxy.com
geckobot=geckobot.com
GenCrawler=gendoor.com
GeonaBot=geona.com
Googlebot=Google.com
googlebot@googlebot.com=Google.com
google=Google.com
moget/2.0=goo.ne.jp
moget@goo.ne.jp=goo.ne.jp
Aranha=girafa.com
Slurp.so/1.0=Yahoo
slurp@inktomi.com=Yahoo
Slurp/2.0j=Yahoo
www.inktomisearch.com=Yahoo
Slurp/2.0-KiteHourly=Yahoo
Slurp/2.0-OwlWeekly=Yahoo
spider@aeneid.com=Yahoo
Slurp/3.0-AU=Yahoo
Toutatis 2.5-2=hoppa.com
Hubater=hubat.com
IlTrovatore-Setaccio=iltrovatore.it
IncyWincy=incywincy.com
UltraSeek=infoseek.com
InfoSeek Sidewinder=infoseek.com
Mole2/1.0=intags.de
webmaster@intags.de=intags.de
MP3Bot=mp3bot.de
C-PBWF-ip3000.com-crawler=ip3000.com
ip3000.com-crawler=ip3000.com
kuloko-bot/0.2=kuloko.com
LNSpiderguy=lexis-nexis.com
NetResearchServer=look.com
MantraAgent=looksmart.com
NetResearchServer=loopimprovements.com
Lycos_Spider_(T-Rex)=lycos.com
JoocerBot=joocer.com
HenryTheMiragoRobot=mirago.co.uk
mozDex/=mozdex.com
MSNBOT/0.1=MSN
Gulliver=northernlight.com
ObjectsSearch/0.01=objectssearch.com
PicoSearch/=picosearch.com
PJspider=portaljuice.com
DIIbot=powerinter.net
nttdirectory_robot=navi.ocn.ne.jp
super-robot@super.navi.ocn.ne.jp=navi.ocn.ne.jp
griffon=super.navi.ocn.ne.jp
griffon@super.navi.ocn.ne.jp=super.navi.ocn.ne.jp
Spider/maxbot.com=maxbot.com
admin@maxbot.com=maxbot.com
gazz/1.0=Unknown Spider
gazz@nttrd.com=Unknown Spider
NationalDirectory-SuperSpider=nationaldirectory.com
dloader(NaverRobot)/=naver.com
dumrobo(NaverRobot)/=naver.com
Openfind piranha=openfind.com
Shark=openfind.com
robot-response@openfind.com.tw=openfind.com.tw
Openbot/=openfind.com.tw
psbot=picsearch.org
CrawlerBoy=pinpoint.com
ip3000.com=petersnews.com
AlkalineBOT=AlkalineBOT
Fluffy the spider=searchhippo.com
info@searchhippo.com=searchhippo.com
Scrubby/=scrubtheweb.com
asterias=singingfish.com
speedfind ramBot xtreme=speedfind.de
Kototoi/0.1=s.u-tokyo.ac.jp
Searchspider/=searchspider.com
SightQuestBot/=sightquest.com
Spider_Monkey/=spidermonkey.ca
Surfnomore Spider v1.1=surfnomore.com
Robot@SuperSnooper.Com=supersnooper.com
teoma_agent1=teoma.com
teoma_admin@hawkholdings.com=teoma.com
Teradex_Mapper=mapper.teradex.com
mapper@teradex.com=mapper.teradex.com
ESISmartSpider=travel-finder.com
Spider TraficDublu=traficdublu.ro
Tutorial Crawler=tutorgig.com
UK Searcher Spider=uksearcher.co.uk
Vivante Link Checker=vivante.com
appie=walhello.com
Nazilla=websmostlinked.com
www.WebWombat.com.au=webwombat.com.au
marvin/infoseek=webseek.de
marvin-team@webseek.de=webseek.de
MuscatFerret=webtop.com
WhizBang! Lab=whizbanglabs.com
ZyBorg=wisenut.com
WIRE WebRefiner=wire.co.uk
WSCbot=worldsearchcenter.com
Yandex=yandex.com
Yellopet-Spider=yellowpet.com
文章整理：西部数码--专业提供域名注册、虚拟主机服务
http://www.west263.com
以上信息与文章正文是不可分割的一部分,如果您要转载本文章,请保留以上信息，谢谢!
