In this time and age, it is difficult to think of a time when we did not have search engines or browsers, or more specifically (since it is the most popular), Google. I mean, of course there was a time when mobile phones were still starting to pick up, and landlines and telephone booths were the common communication channels (And post offices were still largely in use, even when it wasn’t K.C.S.E. or K.C.P.E. period). Back to a time when people actually used encyclopedias and telephone directories were functional; when dictionaries were very much of service, and libraries would be flooded with people; at a time when people would actually and happily search for knowledge within the pages of books.
Don’t get me wrong; people still do read. However, as the years progressed we were able to get a ground breaking and time changing discovery by the name of a search engine. It was probably a mere concept and rather unheard of at the time of its inception; but the search engine has grown to become a tool in most people’s day-to-day lives and indeed one of the most used inventions to date.
But if you think about it, how many people have ever wondered how this great tool came into being? It is usually so easy to just use a common item each day and not even know how it began or who thought of it. So brace yourself for this great series, where we will dive deeper into the innovation of the search engine, how it came about, its history and future if we can. Let’s find answers about the answer machine.
First of all, let me mention that I love how ironic it is that I had to find answers and content for this article by using a search engine. Because of that, is it wrong to say that the search engine gave me an interview? Told me its life story?
Anyway, I’ve always been curious about the person who invented the search engine. It’s always interesting for me to try and figure out whether it was laziness, being futuristic or creativity that drove this individual to have such an idea (I have seen laziness lead to people coming up with brilliant innovations, so I’ll place my bet on laziness).
This history goes back to the 1900s.
In 1945, an American engineer, inventor and science administrator called Vannevar Bush, had an article called ‘As We May Think’ published that started the great conversation on search engines. Bush urged scientists to come together to build a body of knowledge for all mankind, since he strongly believed in storing data.
Gerard Salton, a Professor of Computer Science at Cornell University, became the father of modern search technology by founding SMART (Salton’s Magic Automatic Retriever of Text) informational retrieval system. Ted Nelson then coined the term hypertext in 1963, and most of his work provided the inspiration for creating WWW. Advanced Research Projects Agency Network (ARPANet), which led to invention of the internet, came next.
Search Engines vs. Web Directories
As much as both search engines and directories provide answers to questions posted on the web, there are some differences in their operation. Search engines use internet bots to search the web index documented. This means that search engines basically search the database for any information that seems to answer best what you wish to show, and display those websites on the search engine results page. Directories operate by having owners of websites submit their websites to that directory, either paid or not, in order to appear in searches on the directory.
Finally… Search Engines!
WHOIS was said to be the first search engine-like invention, whose work started in the 1970s, was a protocol that stored and delivered data like domain names from a database.
Archie was the first well documented search engine. Archie, originally ‘archives’ but then became shortened to Archie (You see, laziness had to come in at some point!), came into being in 1990. Alan Emtage, who was a student at McGill University, Canada, and worked as a system administrator for the school of computer science at the school, gave birth to the idea of Archie. Archie started as a project when Emtage and two colleagues were to connect the school of computer science to the internet (which at that time was a rather new creation).
Archie set the ball rolling for more inventions. And in 1991, Mark McCahill from the University of Minnesota, came up with Gopher. Gopher was an advancement of Archie. Gopher led to the development of Veronica and Jughead. Very Easy Rodent-Oriented Net-wide Index to Computerized Archives (V.E.R.O.N.I.C.A.) worked in a similar way to Archie, though it worked on text files. Jonzy’s Universal Gopher Hierarchy Excavation and Display (J.U.G.H.E.A.D.), on the other hand, obtained information from specific Gopher servers. (Side note: Theses acronyms I thought were just incredible! So creative.)
Tim Berners-Lee, while working as an independent contractor for CERN in 1980, proposed a project on the hypertext. When he later joined the company as a fellow, he used the opportunity to connect hypertext to the internet. The first website was then built at http://info.cern.ch/ containing information on what the World Wide Web was. He managed the websites, as well as creating the Virtual Library (VLib), the oldest catalogue of the internet.
In 1993, Matthew K. Gray produced the first web-bot. Initially created to measure the size of the web, the World Wide Web Wanderer later developed into a database of the web. Later in the same year, the first web-based search engine, W3Catalog, was launched. W3Catalog went through the list of websites in its search.
Now not to carry on with this boring science class, the year 1993 seemed to be teeming with inventions. Later on in the year, the second web-based search engine was formed, Aliweb, as well as the first web browser to use a crawler, JumpStation. Note as well, that a crawler is an internet bot that helps in web indexing.
1994 saw the launch of Yahoo! as a new web directory. What distinguished it from the Wanderer was that it provided a description with each URL, which made it easier to understand. Other web based search engines that came about that year were such as Infoseek, Web Crawler, World Wide Web Worm and Lycos.
Yahoo! then launched their search engine in 1995, Yahoo! Search Engine. AltaVista search engine was also launched later in the year, with an almost unlimited bandwidth, and the first search engine that allowed natural language querying.
Larry Page and Sergey Brin began working on the algorithm that was later to be used by Google, BackRub in 1996. HotBot search engine as well premiered that year. In 1997, Ask Jeeves search engine was released as a natural language search engine, and it was later known as Ask.com.
The domain name Google.com came about in 1997, and went public in 1998. In 1998 as well, the search engine Inktomi was formed, and it later had its name changed to Bing in 2009 after a deal with Microsoft. AllTheWeb was a search engine also released in 1999. Other search engines such as Yandex, DuckDuckGo and Cuil were also formed over the years.
G oo gle
Over the years, several more search engines continued being formed and evolving, but one particular one stood out: Google. After going public in 1998, Google proceeded to be selected as a search partner by AOL in 1999 (Yahoo! followed in 2000). From the early 2000s, Google started launching vertical search services and other information such as Google News, Google Maps, Google Scholar, Google Email (Gmail), Google blog search, just to name but a few.
In addition to that, Google also enhanced its user experience by starting programs like Google Suggest (integrated to system in 2008) and Google Instant (2010), as well as enabling using web histories to help in searches as from 2005.
To maintain their systems, Google release updates to their search algorithm, in order to maintain relevance. Some of their updates are such as: Google Panda (updates released from 2011 until 2014), Google Penguin (updates released from 2012 to 2014), Google Hummingbird (released in 2013), and Google Pigeon (update made in 2014). (Again, named after birds, some genius is naming stuff so well, and I am here for it!)
Challenges, of course, are a handful as well, and they strive to counter them. An example is the bug reported in 2019 that was dropping pages out of index, which is that they were not being located in the database anymore. However, keep calm, Google were able to resolve it. Google also faced several other bugs over the years, and they continue to strengthen their security and enforce tighter measures to ensure safety of data.
Other companies such as Yahoo! and Microsoft have also been major players in the history of search engines and have made great advancements. In the present day and age, search engines are almost a basic tool for a majority of people on the planet. From finding the spelling of a word you are uncertain of; to looking up that recipe you have always wanted to try; from looking for pictures of your celebrity crush or lyrics to your favourite song, to looking up the scores of the sports team you are so loyal to and getting tickets to a concert you have really longed for. Search engines have proved to always have the answer to it all.
Without a doubt, we are more than grateful to Vannevar Bush, Alan Emtage, Mark McCahill, and all those who started and carried out this idea of search engines to the point it is now. Some people owe their degrees and jobs to this brilliant invention! And as time progresses, search engines seem only to be getting smarter and better. It seems that with time, this tool we use every so often may very well take over our human communication, books, and all other literary forms. After all, search engines provide all the answers, right?
The history of search engines – A timeline. (n.d.). WordStream: Online Advertising Made Easy. https://www.wordstream.com/articles/internet-search-engines-history
The history of web search engines: What came before Google? (2020, April 7). WhoIsHostingThis.com. https://www.whoishostingthis.com/resources/history-search-engines/
Media, F. N. (2008, July 17). What’s the difference between a directory and a search engine? takeflyte. https://www.takeflyte.com/whats-the-difference-between-a-directory-and-a-search-engine
(n.d.). Search Engine History.com. https://www.searchenginehistory.com/
Search Engines – Information, People, and Technology. (n.d.). The Pennsylvania State University – Open Resource Publishing. https://psu.pb.unizin.org/ist110/chapter/2-1-search-engines/
Search Engines. (n.d.). RxJS, Ggplot2, Python Data Persistence, Caffe2, PyBrain, Python Data Access, H2O, Colab, Theano, Flutter, KNime, Mean.js, Weka, Solidity. https://www.tutorialspoint.com/internet_technologies/search_engines.htm