Internet Search Engines

Search is the most popular activity online second only to e-mail. If you want to find a product or service, where else can you go? The answer is Internet search engines.

This term is often used generically to describe both true search engines and directories. They are not the same...

Search engines vs. directories

Internet search engines (SEs), such as Google, create their listings automatically. They crawl the Web and maintain gigantic databases (indexes) of hundreds of millions of Web pages from millions of websites.

If you change your Web pages, Internet search engines eventually find these changes, and that can affect how you are listed. Page titles, meta tags, body copy and other elements all play a role.

Directories, such as Yahoo, are nothing more than gigantic bookmark lists, organized into logical categories and subdivided into hundreds of sub-categories. Directories depend on human editors. You submit a short description to the directory for your entire site, or editors write one for sites they review.

While you can search for "keywords" in directories, the search looks for matches only in the descriptions submitted. So they usually end up using a third party SEs to search the Web. The advantage of the directories is that the sites listed are usually of a higher, more uniform quality.

Changing your Web pages has no effect on your listing. Things that are useful for improving a listing with SEs are completely useless for improving a listing in directories. The only exception is that a good site, with good content, might be more likely to get listed than a poor site.

How Internet search engines work

They have three major elements...

First is the spider, also called the crawler or robot. The spider scurries around the Web, visits pages, reads them, and then follows links to other pages within the site. This is what it means when someone refers to a site being "spidered" or "crawled."

Everything the spider finds goes into the second part of SEs, the index, also called the catalog. The index is like a giant book containing a copy of every Web page that the spider finds. If a Web page changes, then this book is updated.

Sometimes it can take a while for new pages or changes that the spider finds to be added to the index. Thus, a Web page may have been spidered but not yet indexed. Until it's indexed it's not available to searching.

Search engine software is the third part of Internet search engines. This program sifts through the millions of pages recorded in the index to find matches to a search and rank them in order of what it believes is most relevant.

Search engine positioning

Building a website without consideration for how it'll rank in the major SEs is folly in today's marketplace. Search engine positioning is a critical component of success for your website.

A top ranking will generate more traffic than any other advertising campaign, and it's both free and targeted. About 95% of the search engine traffic to most websites comes from 8 to 10 major Internet search engines (see Collection of Free Search Engines for the list).

But the competition on this over-congested Web is now fiercer than ever. Searches for even two and three word phrases often return millions of Web pages, but studies show searchers still do not go past the first three pages of search results.

If you're listed but not within the first two or three pages of results, you lose, no matter how many engines you submitted your site to, or how much you spent building an attractive website.

Factors that influence search engine ranking

1) Good, solid content is the basis of search engine optimization. Create your pages as if you're writing a focused article about one particular keyword phrase. For more information, see Building: Search Engine Optimization.

2) Don't make the mistake of picking the wrong keywords. Don't optimize your pages for words and phrases that nobody uses and nobody wants. For more information on finding the correct keywords, see Building: Site Content: Choosing Keywords.

3) Though META tags have become less and less important, it's still essential to create effective title, description, keywords, and other tags. For more information on META tags, see Building: Search Engine Optimization: Meta Tags.

4) Internet search engines like simplicity, and so do Web users. Stay away from Flash, Java, scrolling text, heavy graphics and image maps, and lengthy JavaScript. Don't try to impress anyone with your Web building skills.

5) If you use frames, that can cause problems with submissions. For the solution to these problems, see Advanced: HTML Frames.

6) If you use any questionable techniques that might be considered as spamming (i.e., excessive repetition of keywords, same color text as background, redirects, meta refresh tags, etc.), Internet search engines may ignore or reject your submissions. If you're having trouble getting indexed in the expected amount of time, make sure your site is spam-free.

7) Many SEs consider pages containing special symbols (ampersand "&", percent sign "%", equals sign "=", dollar sign "$", question mark "?") in the URL to be dynamic pages and may ignore them.

8) If your site has a slow connection or the pages are very complex and take a long time to load, it might time out before the spider can index all the text. For the benefit of your visitors and the SEs, limit your page size to less than 50-70K.

9) Ensure that all your pages are within two clicks of your home page since many spiders will not index any further than that. You need to submit pages individually that appear further down into your site or create more direct links from the home page.

10) Make sure that all submitted pages are linked directly or indirectly with the home page, otherwise, some SEs may not give a good ranking to them.

11) Some Internet search engines such as Google may refuse to index websites that don't have any other sites linking to them. Link popularity is one of the criteria used by nearly all engines when ranking websites. For more information, see Website Link Popularity.

12) Many engines don't index pages from free websites or they limit the number of pages they will index from these hosts. It's always best to buy your own domain name and place it on a respected, paid service to avoid being discriminated by both the SEs and the human visitors.

13) If your website shares the same IP address as many other websites on your host's Web server and these sites use any questionable techniques, either submitting too frequently, optimizing for irrelevant keywords, etc., SEs may block all of the websites hosted on that IP address. So, when you select a Web hosting service provider, ensure that they give you unique IP address.

---

For more tips on submitting your site to the Internet search engines, see Search Engine Submissions.

• Overview
• Search Engines:
• Search Engine Submissions
• Collection of Free Search Engines
• Website Link Popularity
• Web Directories
• PPC Advertising
• Optin Email List