The History of Search Engines

Many years back, search engines were super basic, but today they are crawled by complex spiders, which are also referred to as web crawlers, or bots.

That said, there is no doubt that the development of various search engines has changed how information is crawled, indexed, retrieved and ranked. Google, Bing, and Yahoo are key players as far as Search Engine Optimization (SEO) is concerned.

In this article, we are going to delve deeper into how search engines work plus the history and evolution of search engines.

 

How Have Search Engines Changed Over the Years?

1990:   Archie 

Archie was the first search engine. It originated from the term “archives”, only that it does not have the letter “v”. Archie was created in 1990 by Alan Emtage, who was by then a student at the McGill University in Montreal.

The role of Archie search engine program was to download and index directory listings of all the public File Transfer Protocol (FTP) servers it could locate. Archie created a database of searchable filenames, which made it easier to find files.

1991: Gopher Protocol

Gopher protocol was created in 1991 by a team from University of Minnesota namely Mark McCahill, Paul Lindner, Farhad Anklesaria, Bob Alberti, and Daniel Torrey.  

As a communication protocol, its main role was to distribute, search and retrieve documents in internet protocol networks. It was a form of anonymous FTP. Gopher is famous for having been the first search engine to utilize hypertext paradigm.

1992: Veronica

Veronica search engine was named after Veronica Lodge. It was created in 1992 by Steven Foster and Fred Barrier who were at the time based at the University of Nevada, Reno.

Its role was to search files stored in Gopher servers. Also, it was used to perform keyword search, particularly for Gopher menu titles found in Gopher listings. Veronica’s backronym was (Very Easy Rodent-Oriented Net-wide Index to Computerized Archives).

1993: Jughead

The Jughead search engine was designed in 1993 by Rhett Jones who was at the time based at the University of Utah.

Jughead, (Jonzy’s Universal Gopher Hierarchy Excavation And Display) was a tool used to retrieve menu information from different Gopher servers.

1993: World Wide Web Wanderer

WWW Wanderer was the first robot designed in 1993 by Matthew Gray who was at time an MIT student. The role of Wanderer was to count web servers in order to determine the size of the web. However, it was at some point used to find URLs and form a database of websites referred to as Wandex.

1993: ALIWEB

Also known as Archie-Like Indexing of the Web, ALIWEB was created by Martijn Koster in 1993. It helped users submit their web pages for indexing. This search engine was based on automated meta-data collection.

1993: Excite

Excite search engine was introduced in 1993 by a group of six students from Stanford University. Excite helped to make the search process more efficient by incorporating statistical analysis of word relationships. Excite officially went online in 1995.

1994: Yahoo

Yahoo was started by Jerry Yang and David Filo in 1994.  Initially, the two had created Yahoo to help them keep track of their favorite web pages, and it would also entail descriptions for the different pages.  Two years later, the two received funds, which enabled them to officially launch Yahoo as a corporation.

1994: JumpStation

JumpStation was a crawler-based search engine, but it is no longer existent. It was created by Jonathan Fletcher who was at the time based at the University of Stirling in Scotland.

The main role of JumpStation was to use web crawlers to develop a searchable index of webpage titles and headings.

1994: World Wide Web Worm 

This search engine was a web crawler. It was developed by 1994 by Oliver McBryan who was at the time a professor at the University of Colorado. In December 1999, World Wide Web Worm was acquired by GoTo.com, and it is today part of Overture.

1994: WebCrawler

WebCrawler was developed in 1994 by Brian Pinkerton who was at the time based at the University of Washington. Brian was the first person to ever develop a full-text search engine, which helped to index text on each and every page on the Internet.

1994: Infoseek

Infoseek was developed in 1994 by Steve Kirsch. It was operated by Infoseek Corporation. Infoseek was not only a full-text search engine, but it was also used to categorize lists of web pages. In 1999, Infoseek was acquired by the Walt Disney Company.

1994: Lycos

Lycos was developed in 1994 by Michael Mauldin who was at the time based at Carnegie Mellon University. Lycos was famous for its huge catalog of over 57 million indexed documents. Also, it was famous for not only relevance retrieval, but also for word proximity, and its ability to match prefixes.

1995: Alta Vista

Alta Vista was a search engine developed in 1995 by DEC. This search engine enabled people to utilize a natural language user interface, as well as advanced search techniques. Additionally, Alta Vista allowed people to search for music, videos, as well as photos.

1996: Inktomi

Inktomi was developed in 1996 by Eric Brewer and Paul Gauthier during an experiment at the University of California at Berkeley. The crawler was famous for its ability to search more than 48 million pages on the internet. In 1999, Inktomi launched a directory search engine using the concept induction technology. It was acquired by Yahoo in 2003.

1996: HotBot

This application was developed in 1996 by Inktomi in collaboration with HotWired. It was famous for its ability to provide customized website pages and ads based on the user’s type of browser.

1996: Backrub

Backrub was developed in 1996 by Larry Page and Sergey Brin who were at the time students of Stanford University. The key role of Backrub search engine was to rank websites based on the relevancy of their inbound links, as well as their popularity. Backrub was given its name because it analyzed backlinks to websites. Later, BackRub became known as Google.

1996: LookSmart

LookSmart was developed in 1996 by Tracey Ellery and Evan Thornley. Initially, LookSmart was known as NatGet. LookSmart was created as a directory service. Today, LookSmart was the most notable paid insertion directory on the World Wide Web.

1996: Netscape

Netscape was originally known as Mosaic Communications, which was founded by Jim Clark and Marc Andreessen of the University of Illinois at Urbana–Champaign. Mosaic was Andreessen’s idea, and he started working on the search engine in 1993 when he was still at the University.

Netscape was acquired by AOL in 1998. Today, Netscape works in partnership with LookSmart, Lycos, Overture, Ask Jeeves and Google.

1997: Search Engine Watch

Search Engine Watch was developed in 1997 by Danny Sullivan. The key role of this online search engine magazine was to offer critical information regarding the search industry, how search engine work and how to search the website.  How to rank websites was also part of the information offered by Search Engine Watch.

Search Engine Land 

 was another brainchild of Danny Sullivan which is still alive and thriving today even after Sullivan left the company in 2017.

1997: Ask Jeeves

Askjeeves was developed in 1997 by Garrett Greuner and David Warthen who were at the time based in Berkeley, California. When Garrett and David created Askjeeves, they wanted to help people find answers for their most pressing issues more so because they were aware that not everyone knew about Boolean search and syntax.

That said, they came up with Ask Jeeves, a search engine that featured Natural Language Processing, or NLP. Ask Jeeves, now Ask.com, enabled its users to search for answers using their day-to-day language including asking questions.