|
|
|
|
List of search engine Robots :
| Home page/search engine | Robot identifier | IP address(es) |
|---|---|---|
| www.abacho.com | AbachoBOT | srv-ze-robot1.tricus.com |
| www.abcdatos.com |
abcdatos_botlink http://www.abcdatos.com/botlink/ |
217.126.39.167 |
| www.aesop.com | AESOP_com_SpiderMan | 209.189.115.49 |
| www.ah-ha.com | ah-ha.com crawler (crawler@ah-ha.com) | c7pub-216-250-141-186.center7.com |
| www.alexa.com | ia_archiver |
green.alexa.com sarah.alexa.com |
|
www.altavista.com |
Scooter Mercator Scooter2_Mercator_3-1.0 roach.smo.av.com-1.0 Tv<nn>_Merc_resh_26_1_D-1.0 |
|
| www.altavista.co.uk |
AltaVista-Intranet jan.gelin@av.com |
host-119.altavista.se |
| www.alltheweb.com |
FAST-WebCrawler crawler@fast.no |
209.67.247.154 |
| www.fast.no/faq/faqfastwebsearch/faqfastwebcrawler.html | ||
| Wget | ext-gw.trd.fast.no | |
| www.acoon.de | Acoon Robot | 194.231.42.178 |
| www.antisearch.net | antibot | 62.210.155.50 |
| www.atomz.com | Atomz |
router-sc.atomz.com index.atomz.com |
| www.axmo.com | AxmoRobot | 194.248.208.82 |
| www.buscaplus.com |
Buscaplus Robi http://www.buscaplus.com/robi/ |
|
| www.canseek.ca |
CanSeek/ support@canseek.ca |
216.168.111.111 |
| www.christcrawler.com/search.cfm |
ChristCRAWLER http://www.christcrawler.com/ |
207.191.111.231 |
| www.clush.com |
Clushbot http://www.clush.com/bot.html |
209.249.80.242 |
| www.crawler.de |
Crawler admin@crawler.de |
crawlit.crawler.de |
| www.daadle.com | DaAdLe.com ROBOT/ | 216.12.213.32 |
|
www.daum.net |
RaBot Agent-admin/ phortse@hanmail.net contact/jylee@kies.co.kr |
210.183.28.46 211.50.57.6 |
|
RaBot Agent-admin/ webmaster@kisco.go.kr |
202.30.94.34 | |
| www.en.deepindex.com | DeepIndex | deepindex.net1.nerim.net |
| www.ditto.com | DittoSpyder | 65.169.94.188 |
| domanova.co.uk | Jack | |
| www.earthcom.info | EARTHCOM.info | 194.108.39.74 |
| www.entireweb.com | Speedy Spider | 62.13.25.209 |
| www.excite.com | ArchitextSpider | |
| (excite) | ArchitectSpider |
crimpshrine.atext.com ichiban.atext.com |
| www.eurip.com | EuripBot | 81.169.172.30 |
| www.euroseek.net |
Arachnoidea arachnoidea@euroseek.net |
212.209.54.134 |
| www.ezresults.com | EZResult | 216.28.23.59 |
|
www.fastsearch.net |
Fast PartnerSite Crawler FAST Data Search Crawler FAST Data Search Document Retriever |
psprdcrw001.sac2.fastsearch.net 65.198.110.185 69.38.159.128 |
| www.fireball.de | KIT-Fireball | ???? |
| http://france.misesajour.com/ | france.misesajour.com | 66.98.210.71 |
| www.fybersearch.com | FyberSearch | 69.49.241.9 |
| www.galaxy.com |
GalaxyBot http://www.galaxy.com/galaxybot.html |
63.121.41.175 |
| www.geckobot.com | geckobot | ???.rdc1.az.coxatwork.com |
|
www.gendoor.com (Genealogical Search Engine) |
GenCrawler | ???? |
| www.geona.com | GeonaBot | 69.59.142.17 |
| www.getrax.com | getRAX | 81.169.156.246 |
| www.google.com |
Googlebot googlebot@googlebot.com http://googlebot.com/ |
c<nn>.googlebot.com |
| www.goo.ne.jp |
moget/2.0 moget@goo.ne.jp |
202.229.31.13 |
| www.girafa.com | Aranha | Aranha.girafa.com |
|
(inktomi) |
Slurp.so/1.0 slurp@inktomi.com |
q2004.inktomisearch.com j5006.inktomisearch.com |
|
(inktomi) |
Slurp/2.0j slurp@inktomi.com www.inktomisearch.com |
202.212.5.34 goo313.goo.ne.jp |
| (inktomi) |
Slurp/2.0-KiteHourly slurp@inktomi.com; www.inktomi.com/slurp.html |
y400.inktomi.com |
| (inktomi) |
Slurp/2.0-OwlWeekly spider@aeneid.com www.inktomi.com/slurp.html |
209.185.143.198 |
| (inktomi) |
Slurp/3.0-AU slurp@inktomi.com |
j6000.inktomi.com |
|
http://hoppa.com/ (need V5 browsers to view) |
Toutatis 2.5-2 | tisnix.xs4all.nl |
| www.hubat.com | Hubater | 209.114.176.250 |
|
www.almaden.ibm.com (research centre) |
http://www.almaden.ibm.com/cs/crawler | wfp2.almaden.ibm.com |
| www.iltrovatore.it | IlTrovatore-Setaccio | 213.26.21.8 |
| www.incywincy.com | IncyWincy | 64.81.243.66 |
|
www.infoseek.com |
UltraSeek InfoSeek Sidewinder |
cde2c923.infoseek.com cde2c91f.infoseek.com cca26215.infoseek.com |
| www.intags.de |
Mole2/1.0 webmaster@intags.de |
217.160.75.10 |
| http://mp3bot.de/ | MP3Bot | <..> |
| www.ip3000.com |
C-PBWF-ip3000.com-crawler ip3000.com-crawler |
www.ip3000.com |
| www.istarthere.com |
http://www.istarthere.com spider@istarthere.com |
66.220.24.80 |
| www.knowledge.com | Knowledge.com/ | 213.170.2.69 |
| www.kuloko.com | kuloko-bot/0.2 | 66.90.81.41 |
| www.lexis-nexis.com | LNSpiderguy | firewall5.lexis-nexis.com |
| www.linknz.co.nz | Linknzbot | 202.191.32.67 |
| www.look.com | lookbot | magma.com |
| www.looksmart.com | MantraAgent | fjupiter.looksmart.com |
|
www.loopimprovements.com (see also www.incywincy.com) |
NetResearchServer www.loopimprovements.com/robot.html |
leg-64-133-109-250-STK.sprinthome.com |
| www.lycos.com | Lycos_Spider_(T-Rex) |
bos-spider<n>.bos.lycos.com 216.35.194.188 |
| www.joocer.com | JoocerBot | 80.46.38.169 |
| www.mirago.co.uk | HenryTheMiragoRobot | 194.202.39.46 |
| www.mojeek.com | MojeekBot | ??? |
| www.mozdex.com | mozDex/ | (within comcast.net) |
| http://search.msn.com/ |
MSNBOT/0.1 http://search.msn.com/msnbot.htm) |
131.107.163.47 |
| www.navadoo.com | Navadoo Crawler | ??? |
| www.northernlight.com | Gulliver |
marvin.northernlight.com taz.northernlight.com |
| www.objectssearch.com | ObjectsSearch/0.01 | 68.88.244.177 |
| www.picosearch.com | PicoSearch/ | pipe.picosearch.com |
| www.portaljuice.com | PJspider | timber.nextopia.com |
|
www.powerinter.net but it won't let us in :-( |
DIIbot | node-d8e93393.powerinter.net |
|
http://navi.ocn.ne.jp/ |
nttdirectory_robot super-robot@super.navi.ocn.ne.jp griffon griffon@super.navi.ocn.ne.jp |
lilis00.navi.ocn.ne.jp lilis04.navi.ocn.ne.jp |
| www.maxbot.com |
Spider/maxbot.com admin@maxbot.com |
search.wport.com |
| ??? | various (fakes agent on each access) | pool0058.cvx2-bradley.dialup.earthlink.net |
|
??? |
gazz/1.0 gazz@nttrd.com |
deleuze.infobee.ne.jp derrida.infobee.ne.jp |
| ??? | ??? | search-8.xift.com |
| www.nationaldirectory.com | NationalDirectory-SuperSpider |
spider.nationaldirectory.com 209.116.58.143 |
| www.naver.com |
dloader(NaverRobot)/ dumrobo(NaverRobot)/ |
211.218.151.209 |
|
www.openfind.com (Chinese language) |
Openfind piranha,Shark robot-response@openfind.com.tw Openbot/ |
??? abovenet4.openfind.com |
| www.picsearch.org |
psbot www.picsearch.org/bot.html |
217.75.104.26 |
| www.pinpoint.com | CrawlerBoy Pinpoint.com | nitrogen.pinpoint.com |
| www.petersnews.com | user<n>.ip3000.com | news<n>.petersnews.com |
| www.qweery.nl |
QweeryBot http://qweerybot.qweery.com) |
84.82.133.41 |
| www.vestris.com/alkaline | AlkalineBOT | host130.uv-ray.com |
| www.seznam.cz | SeznamBot | 212.80.76.87 |
| www.search-10.com | Search-10 | 82.41.144.99 |
| www.searchhippo.com |
Fluffy the spider info@searchhippo.com) |
208.148.122.27 |
| www.scrubtheweb.com | Scrubby/ | 208.145.190.254 |
| www.singingfish.com | asterias | grouper.singingfish.com |
| www.speedfind.de | speedfind ramBot xtreme | BWEB.highway.telekom.at |
| www.s.u-tokyo.ac.jp | Kototoi/0.1 | crawler-red3.is.s.u-tokyo.ac.jp |
| www.searchbyusa.com | SearchByUsa | ??? |
| www.searchspider.com | Searchspider/ | 24.90.243.203 |
| www.sightquest.com |
SightQuestBot/ http://www.sightquest.com/bot.htm |
64.49.245.212 |
| www.spidermonkey.ca | Spider_Monkey/ | 66.163.18.197 |
| www.surfnomore.com | Surfnomore Spider v1.1 | 165.90.194.245 |
| www.supersnooper.com | Robot@SuperSnooper.Com | 207.8.212.162 |
| www.teoma.com |
teoma_agent1 teoma_admin@hawkholdings.com |
63.236.92.148 |
| http://mapper.teradex.com |
Teradex_Mapper mapper@teradex.com |
65.110.6.26 |
| www.travel-finder.com | ESISmartSpider | 202.46.33.15 |
| www.traficdublu.ro | Spider TraficDublu | 81.196.*.*, 193.16.218.66 |
| www.tutorgig.com |
Tutorial Crawler http://www.tutorgig.com/crawler |
216.40.225.75 |
| www.updated.com |
updated/0.1beta crawler@updated.com |
38.119.96.107 |
| www.uksearcher.co.uk | UK Searcher Spider | - |
|
www.vivante.com (coming soon) |
Vivante Link Checker | 216.93.167.106 |
| www.walhello.com | appie | uses an address at planet.nl, a Dutch ISP |
| www.websmostlinked.com | Nazilla | - |
| www.webwombat.com.au | www.WebWombat.com.au | 202.139.99.131 |
| www.webseek.de |
marvin/infoseek marvin-team@webseek.de |
arthur4.sda.t-online.de |
| www.webtop.com | MuscatFerret | ferret<nn>.webtop.com |
| www.whizbanglabs.com | WhizBang! Lab | 216.250.143.108 |
| www.wisenut.com |
ZyBorg (info@WISEnut.com) |
- |
| www.wire.co.uk |
WIRE WebRefiner: webrefiner@wire.co.uk |
brighton.wire.co.uk |
| www.worldsearchcenter.com | WSCbot | ??? |
| www.yandex.com | Yandex | ya.yandex.ru |
|
www.yellowpet.com pet-based search engine |
Yellopet-Spider | 212-82-36-23.ip.zeitraum.com |
| <client sites> | libwww-perl | www.linpro.no/lwp/ |
| http://verno.ueda.info.waseda.ac.jp/ | ||
| Iron33 | 207.18.183.251 | |
bravenet.com