How Do Search Engines Work?

Since the publication of the World Wide Web in 1991 people have been using search engines to obtain their information (Berners-Lee). These sources of information have greatly evolved over the past two decades and are continuing to become more efficient. Even though most any person with a computer uses a search engine, many do not know how it works. For starters, there are two main types of web searches: crawler based and human powered. Google is a prime example of a crawler or “spider” based engine; it gets information from sites automatically by following a preset algorithm. These algorithms are massive preset formulas that sort information and metadata from sites automatically according to the user’s input. Additional coding tells the main system to refresh the sites and information found periodically as desired by the programmer. After obtaining data from the Hypertext Markup Language (HTML), which consists of the symbols or codes that comprise the webpage, the algorithm then translates the information into an index structure usable by the search engine. This is similar to how a businessman would organize information in a filing cabinet for later retrieval. Usually these index structures take keywords from the text, measure size and type of the webpage, and determine whether it is commercial, a news article or just entertainment. Then these newly created index structures continue to pull various details from the page such as color and other seemingly trivial features to build a searchable product (Sullivan). On the other hand, a search engine such as Yahoo! is human powered, thus does not rely on automated algorithms to file data. Instead, human powered engines are based entirely off of user-submitted data. In doing this, information can be more specific and carefully filed, usually resulting in a better search. Unfortunately, this kind of engine does come with