Unlike web directories, which are maintained only by human editors, search engines also maintain real-time information by running an algorithm on a web crawler.Prior to September 1993 the World Wide Web was entirely indexed by hand.

The program downloaded the directory listings of all the files located on public anonymous FTP (File Transfer Protocol) sites, creating a searchable database of file names; however, Archie Search Engine did not index the contents of these sites since the amount of data was so limited it could be readily searched manually.

The rise of Gopher (created in 1991 by Mark Mc Cahill at the University of Minnesota) led to two new search programs, Veronica and Jughead.

Like Archie, they searched the file names and titles stored in Gopher index systems.

Veronica (Very Easy Rodent-Oriented Net-wide Index to Computerized Archives) provided a keyword search of most Gopher menu titles in the entire Gopher listings.

A web search engine is a software system that is designed to search for information on the World Wide Web.

The search results are generally presented in a line of results often referred to as search engine results pages (SERPs).

The information may be a mix of web pages, images, and other types of files.

Some search engines also mine data available in databases or open directories.

Jughead (Jonzy's Universal Gopher Hierarchy Excavation And Display) was a tool for obtaining menu information from specific Gopher servers.