Add your Web Site - Search Engines - Robots Add Search to your Site Search for: Internet Entire Directory Directory - Robots Results per page: 10 20 50 Output format: Long Short URL Match: All Any Boolean Pages from: All Countries Afghanistan American Samoa Albania Algeria Andorra Angola Anguilla Antigua and Barbuda Antarctica Argentina Armenia Aruba Australia Austria Azerbaijan Bahamas Bahrain Bangladesh Barbados Belarus Belgium Belize Benin Bermuda Bhutan Bolivia Bosnia and Herzegovina Botswana Bouvet Island Brazil British Indian Ocean Territory Brunei Darussalam Bulgaria Burkina Faso Burundi Cambodia Cameroon Canada Cape Verde Cayman Islands Central African Republic Chad Chile China Christmas Island Cocos (Keeling) Islands Colombia Comoros Congo Cook Islands Costa Rica Cote D'Ivoire (Ivory Coast) Croatia (Hrvatska) Cuba Cyprus Czech Republic Czechoslovakia (former) Denmark Djibouti Dominica Dominican Republic East Timor Ecuador Egypt El Salvador Eritrea Estonia Ethiopia Equatorial Guinea Falkland Islands (Malvinas) Faroe Islands Fiji Finland France France, Metropolitan French Guiana French Polynesia French Southern Territories Gabon Gambia Germany Georgia Ghana Gibraltar Great Britain (UK) Greece Greenland Grenada Guadeloupe Guam Guatemala Guinea Guinea-Bissau Guyana Haiti Heard and McDonald Islands Honduras Hungary Hong Kong Iceland India Indonesia Iran Iraq Ireland Israel Italy Jamaica Japan Jordan Kazakhstan Kenya Kiribati Korea (North) Korea (South) Kuwait Kyrgyzstan Laos Latvia Lebanon Lesotho Liberia Libya Liechtenstein Lithuania Luxembourg Macau Macedonia Madagascar Malawi Malaysia Maldives Mali Malta Marshall Islands Martinique Mauritania Mauritius Mayotte Mexico Micronesia Moldova Monaco Mongolia Montserrat Morocco Mozambique Myanmar Namibia Nauru Nepal Netherlands Netherlands Antilles Neutral Zone New Caledonia New Zealand (Aotearoa) Northern Mariana Islands Nicaragua Niue Niger Nigeria Norfolk Island Norway Oman Pakistan Palau Panama Papua New Guinea Paraguay Peru Philippines Pitcairn Poland Portugal Puerto Rico Qatar Reunion Romania Russian Federation Rwanda Saint Kitts and Nevis Saint Lucia Saint Vincent and the Grenadines Samoa San Marino Sao Tome and Principe Saudi Arabia Senegal Seychelles Sierra Leone Singapore Slovak Republic Slovenia Solomon Islands Somalia South Africa S. Georgia and S. Sandwich Isls. Spain Sri Lanka St. Pierre and Miquelon St. Helena Sudan Suriname Svalbard and Jan Mayen Islands Swaziland Sweden Switzerland Syria Taiwan Tajikistan Tanzania Thailand Togo Tokelau Tonga Trinidad and Tobago Tunisia Turkey Turkmenistan Turks and Caicos Islands Tuvalu Uganda Ukraine United Arab Emirates United States Uruguay US Minor Outlying Islands Uzbekistan Vanuatu Vatican City State (Holy See) Venezuela Virgin Islands (British) Virgin Islands (U.S.) Viet Nam Wallis and Futuna Islands Western Sahara Yemen Yugoslavia Zaire Zambia Zimbabwe        Show pages from IntraNet   Show all pages  DIRECTORY/Computers/Internet/Searching/Search Engines/Robots (11) ACAP - Automated Content Access Protocol - http://www.the-acap.org/ - Standard being developed on behalf of content publishers to communicate permissions information more extensively than is the case with robots.txt. Project documents, implementation and background information.   All About Search Indexing Robots and Spiders - http://www.searchtools.com/robots/ - Search Tools Consulting explains how the search engine programs called "robots" or "spiders" work, and reviews related sites.   Bots vs Browsers - http://www.botsvsbrowsers.com - This large database lists user agents in categories and distinguishes between robots and browsers.   HTTP User Agent Index - http://www.siteware.ch/webresources/useragents/db.html - An alphabetical list of user agents and the deployer behind them, compiled by Christoph Rüegg.   List of Robot Agent Strings - http://www.pgts.com.au/pgtsj/pgtsj0208d.html - A list from PGTS of Web robots with the identifying data they leave in Web site logs.   Robot IP Address - http://www.briandunning.com/seo/ - Brian Dunnintg provides a list of all the major search engine robot IP addresses, by full class C only.   Robotstxt.org - http://www.robotstxt.org/ - Information on the robots.txt Robots Exclusion Standard and other articles about writing well-behaved Web robots.   Search Engine IP Addresses - http://www.iplists.com/ - Lists IP addresses of search engine spiders. Can be searched by IP address. Also links to resources on spiders.   Search Engine Robots and Other User Agents - http://www.jafsoft.com/searchengines/webbots.html - John A. Fotheringham presents data in tabular form on the robots sent by search engines and other sites to read and index Web pages: their origins, names and IP addresses.   User Agent String - http://user-agent-string.info/ - Tool from ASAP Consulting s.r.o. for detailed user agent string analysis using an online form. Includes databases of browsers and robots.   User-Agents.org - http://www.user-agents.org/ - Large list of search engine spiders, similar web robots, and Web browsers: their web-log identification and links to their originators.    E-Mail: webmaster@teradex.com Copyright © 2007 Aaex Corp. all rights reserved.

Help build the largest human-edited directory on the web. Submit a Site - Open Directory Project - Become an Editor This site is presented in modified Open Directory form