Understanding Search Engine Models

To understand search engines and search engine marketing, one must first understand the search engine model. There are two fundamentally different types of search engine back ends: site directories and spidering search engines. Site directory databases are built by a person manually inputting data about websites. Most directories include a site’s url, title, and description in their database. Some directories include more information, such as keywords, owner’s name, visitor rankings and so on. Some directories will allow you to control your website’s information yourself others rely on editors that write the information to conform to the directory standards.

It is important to note that most directories include directory listings as an alterative to the search box for finding websites. A directory listing uses hierarchal groupings from general to specific to categorize a site.


Spidering search engines take a very different approach. They automate the updating of information in their database by using robots to continually read web pages. A search engine robot/spider/crawler acts much like a web browser, except that instead of a human looking at the web pages, the robot parses the page and adds the page’s content it’s database.

Many of the larger search engines will have both a directory and spidering search engine, e.g. yahoo.com, google.com, and allow visitors to select which they want to search. Note that many search engines do not have their own search technology and are contracting services from elsewhere. For example, Google’s spider SE is their own, but their directory is and Open Directory; additionally aol.com and netscape.com both use Google’s spider SE for their results.

There are a few other search engine models of interest. There are some search engines that combine results from other engines such as dogpile.com and mamma.com. There are also search engines that add extra information to searches such as Amazon’s alexa.com, which uses Google’s backend but adds data from its search bar regarding tracking traffic to the site.