* 'Intellectual' indexes, where individual web sites are classified
by hand according to various classificatory schemes. Usually, such indices only
provide the title, URL[20] and perhaps a brief
description of each site indexed. Yahoo![21]http://www.yahoo.com/]
is a well known example of a general intellectual index of the web (ie one
which is not law-specific).
* `Robot' / automated indexes, where a program (variously called a
`web robot' or `web spider') traverses the web, downloading every page it
encounters, so that every word on every page can be indexed by a remotely
located search engine. When the search engine displays a URL as a result of a
search, that URL is to the original site, not to a mirror on the remote site.
Alta Vista[22] is perhaps the best
known general example of such an `internet-wide' search engine that searches an
index created by a web spider. The principle advantage of this approach that it
is possible to search every word that has been indexed, not just the titles and
brief summary of what is on the site.
[20] `Universal Resource Locator' or internet address of a web page
[22] http://www.altavista.digital.com/