Web Robot that Gather Web-page Information


Internet Bot
A Bot is common parlance on the Internet for a software program that is a software agent. A Bot interacts with other network services intended for people as if it were a real person. One typical use of bots is to gather information. The term is derived from the word "robot", reflecting the autonomous character in the "virtual robot"-ness of the concept.
The most common bots are web agents that interface with web pages. Web crawlers or spiders are web robots that recursively gather web-page information.
From Wikipedia, the free encyclopedia.

Zeusbot is Ulysseek's web-crawling robot. It collects documents from the web to build a searchable index for the Ulysseek search engine.

Information for webmasters
Zeusbot obeys the robots.txt exclusion standard, described at http://www.robotstxt.org/wc/exclusion.html#robotstxt. Thus to prevent Zeusbot agent to crawl pages from your site, place the following in your robots.txt file:
User-agent: Zeusbot
Disallow: /
You can submit your site for inclusion by clicking here.

Visit Odyssearch.com to find websites organized into categories.

Link to Us - Add to Bookmarks

Copyright © 2005 Zeusbot. All rights reserved