Thinking Australia - Internet Management

Web Sites | Web Squads | Web Tools | Web Hooks | destination-e

Search Information

1. What is a search engine?

Search engines utilise indexing software agents often called robots or spiders. These agents are programmed to constantly "crawl" the Web in search of new or updated pages.

When visiting a Web site, an agent will record the full text of every page (home and sub-pages) within the site. It will then continue on to visit all external links.

Robots will then revisit your site periodically to refresh the information they already have in their database.

top

2. How do you get listed in a Search Engine

Each search engine looks at different elements of your page, and some or all are important to each search engine.

a. Use keywords in the <TITLE> of your document making it as descriptive as possible. When visiting your site, an agent will go first to the <TITLE> tag, (when you open a browser it displays the title in the top bar). Search engines will display the text located between the <TITLE> tags when your web page is listed in a search. By making your <TITLE> descriptive, you'll be better off than those who only have keywords within the text of their page. It will also be helpful when people bookmark your web site. If a more descriptive name appears in a person's hotlist, it will be easier to find your site at a later date.

For example, instead of using <TITLE>Thinking </TITLE> as the title of Thinking's home page, <TITLE>Thinking Internet Management</TITLE> would be much more descriptive. It would also place greater emphasis or relevancy on "Internet Management" when calculating keywords.

The HTML title is the link name of the page as it appears on lists of results. "No Title" doesn't really attract people to click to go to your page. And having multiple pages with the same title is confusing and looks unprofessional.

In the ranking rules that determine which pages will appear near the top of a list of matches, the HTML title is the most important element of the page.

When people search for your kind of information, they are most likely to use those words that belong in your HTML title. When you leave the title blank, use unimportant words, or use the same title on many different pages, you've just lost an opportunity of attracting people to you site.

b. Descriptive Page Text Search engines assign greater relevancy to text located at the top of a page than to text located in the middle or at the bottom of the page. The search engines assume that web page authors will present their most important information first. If your page has a main graphic at the top, you should place some descriptive text either underneath or beside the image. The search engines will index this text and assign it a high level of relevancy.

c. Use <META> tags which allow you to provide even more detail about your Web pages and thereby gain greater control over how your pages are indexed. Not all search engines make use of <META> tags, but adding these tags to your pages will make them more accessible to the search engines that do.

<META> tag codes are inserted within the <HEAD> </HEAD> tags. The basic syntax is:

This will control what appears as the summary of your Web page and will be displayed after the title of your document in the index listing. The content of the description should clearly convey what one can expect to find when linking to your site.

This will allow you to provide extra information about your page to the search engines without it being visible to the reader. While search engines do take these keywords into account when indexing your page, they are still going to index the entire contents of your page as many sites do not include <META> tags.

Beware: do not, excessively repeat keywords in a keyword <META> tag, as search engines may penalise you for this. InfoSeek and Lycos are two such examples and others may adopt similar policies in the future. The penalty will most likely be the spider disregarding the <META> tag and extracting keywords from the content of your page.

d. If your site uses frames, you should be aware that search engines treat frames as if they are links within your main page. As a result the engines will review and index your main page and, at a later date, return to index each individual frame just as it will return to index all other internal links within your web site. Therefore, in order to have your main page indexed accurately and efficiently, we recommend that you add some descriptive text between the <noframes> and </noframes> tags of the HTML source coding of your main page. The noframes tags are usually placed below your frame set information. The frame set information is designated by <frameset> and </frameset>. This text should include your most important keywords and keyword phrases. Adding this text will provide the search engines with content from which to derive keywords for indexing. After this change has been made to your Web site, the page itself will appear exactly the same to anyone using a browser that supports frames. However, users of browsers that do not support frames (i.e. Netscape 1.0 or lower) will now be able to successfully view your home page.

top

3. Indexing Characteristics of Individual Search Engines

Excite
At the present time, Excite does not make use of <META> tags. Since keywords and summaries are automatically generated by Excite, you have less control over their creation. However, there are still a few things you can do.

Excite's software looks for common words or themes within a page. It then selects sentences for the summary that either contain these words or convey the overall theme. The words within these sentences are also used as keywords for which the site can be searched.

Especially at the beginning of your page, be as concise as possible and limit non-descriptive sentences. If the Excite robot comes across a number of ambiguous phrases, it will have to look deeper and deeper into your site to determine its theme and site summary. Along the same line, too little text will also force the robot to travel further into the site for more information in order to establish a theme.
Excite's indexing software places preference on complete, punctuated sentences. If you have content, such as a quote, at the top of your page that you do not want Excite to include in your site summary, do not display it as a complete sentence. This will lessen the chance that the quote will be included, but will not guarantee its exclusion.

HotBot / Inktomi

HotBot supports both the keywords and description <META> tags.
If you strongly believe that your site was not ranked as high as you thought it deserved in a search query, HotBot allows you to send them an email to bugs@hotbot.com. Be sure to include the URL of the search page.

InfoSeek

InfoSeek supports both the keywords and description <META> tags. Your description can include up to 200 characters of text and the keywords can include up to 1000 characters of text. Do not repeat versions of a keyword more than seven times. If you do, InfoSeek will disregard the entire keyword list.
If you do not make use of the description <META> tag, InfoSeek's agent will simply insert the first 200 characters after the <BODY> tag as the web page description. Hence, if your Web document does not contain <META> tags, at least try to make your first 200 words accurately describe your Web site.
InfoSeek also indexes the ALT attribute in the <IMG> tag. If the majority of your home page consists of graphics, you can describe your page with the ALT attribute.

The syntax for an <IMG> tag code is:

InfoSeek Ultra
InfoSeek Ultra will make use of <META> tags allowing you to control the description that appears in a search result as well as guide its web indexing in the selection of your site's keywords. If you do not make use of <META> tags, Ultra will simply use the first few words it comes across as your Web site summary.

Lycos
Lycos creates Web site titles and descriptions from the text of your Web page. Lycos' search agent selects a portion of the site that well represents its theme. It then displays this section as the site's description.

The keywords are also chosen via artificial intelligence by Lycos' spider. With this in mind, do not open your page with an image map. If you do, Lycos will not be able to take an abstract for your document.

WebCrawler
WebCrawler relies on the statement within the <TITLE> tag to use for the name of your page. While other search engines will derive a summary from the <BODY> text of the document, Web Crawler will default to the URL if you fail to include a title.

In conclusion, do not make it your goal to appear in the top ten list of every search engine. This would not only be a very difficult task, but would most likely end in disappointment. You are virtually guaranteed to have varying success rates with different search engines due to the collection of variables that play a role in every search result. These variables include size of database, method used for determining relevancy, policy on spamming, use of <META> tags and more. Hence, optimise your listings by capitalizing on the indexing criteria shared by search engines. If you have a favorite or preferred search engine that you feel strongly about being yielded as high as possible in a search, customise your Web pages accordingly. Just keep in mind that this might lessen your perceived relevancy on another.

top

4. How long does it take a Search Engine to list my site?

Approximate times:
1-2 weeks: Altavista, Infoseek
2-4 weeks: Excite, HotBot, Lycos, Webcrawler
6-8 weeks: Yahoo

If after the time listed in the chart above has elapsed and you are still not finding your site listed, you should re-submit your URL to the search engines that do not have your listing. It often takes more than one submission to get the best results.

Search engines receive tens of thousands of submissions every day. On days when the submission level is above normal, a certain number of submissions can be received properly but still not get listed. If your listing is not appearing on a certain search engine, you should resubmit it.

The following links take you to the Submit pages of the top 10 search engines used by Australians.

Altavista
Excite
Lycos
MSNSearch
Looksmart
Google Search
NineMSN
WebWombat
Yahoo

top

5. Should I submit inside pages of my site?

When considering how many pages of your site to promote, it is important to remember directories generally accept one listing per company. This is usually your home or main page. Search engines index multiple pages within a site, but you cannot control which pages they will index unless you submit the individual URLs (pages).

The critical pages to submit are your major topic pages, pages with unique content, or pages that describe a specific product or service. For example, a sports store will have separate pages on basketball, baseball and football which should all be submitted. Another benefit to submitting multiple pages in this example is that the basketball page will appear higher in a search for "basketball" than the sports site home page. These pages should also include descriptive title tags and meta tags. In order for site visitors to easily navigate your site, these major topic pages should provide clear links to other topic pages and your home page.

top

6. When will I need to inform search engines and directories of updates to my Web pages?

Here are some general rules and indicators for knowing when you need to make updates to your listings. For more detailed information on the process involved in making updates to each directory, consult the Submission Hints and Tips accessible within the Submit It! Gold and Pro versions.

For search engines: Make updates to listing:

If your URL changes, you will need to resubmit. Some search engines have Dead Link forms for you to fill out. Those that do not will drop the old URL from its records the next time it tries to visit your site at the old address and is unable to find it.
If the content of your site changes so extensively that the abstract no longer adequately describes your site. These changes, however, will be taken into account when the indexing agent next visits your site to refresh its information. Resubmitting will hasten this process.

For directories: Make updates to listing:

If your Web site address (URL), description, or category classification changes, you will need to inform the webmaster of that directory.
If your category classification has changed and you are now inaccurately located within the directory's index.

Do not make updates to listing:

If there has been minor changes made within the site which do not alter the site's description. If the content or purpose of the site remains the same, you are not misleading anyone who chooses to visit your site based on the original description.

top

7. "Non-Netiquette" Things

There are several "tricky" things you can do to try to get your URL listed higher or more times on a search engine results page. We are listing these alternatives, but are in no way recommending them as these Web site promotion methods may be viewed as "cheating" by some members of the Internet community. In fact, some of the search engines are starting to penalise people that use these tricks. The reason why we list these tricks is to help you understand why some sites always show up multiple times or always at the top.

Placing mass amounts of hidden keywords <!software manufacturer, software manufacturer,...> at the bottom of your document. Search engines calculate keywords by how many times they appear on a page. It is important to be aware that search engines may penalise you for excessively repeating keywords. At present, InfoSeek and Lycos are two examples and others may adopt similar policies in the future.
Changing your name to A1 Enterprises or !Rob's Restaurant in order to appear at the top. Our suggestion is that you pick something descriptive instead. You might appear at the top by changing your name, but be skipped over because the new one does not evoke a professional or trustworthy company image.
Changing your <TITLE> regularly so when robots revisit your site to refresh their information, they will interpret the new title to mean the existence of a new site. The result being your page will be listed more than once in a search.

top

8. Query Relevance

There are hundreds of millions of Web pages, so almost any query is likely to have a huge number of matches. For search results to be useful, search engines must rank more highly pages that are most likely to have relevant information. Search Engines formula for doing that is a closely kept secret (like the formula for Coca-Cola), and is subject to continuous fine-tuning. But an understanding of the main ingredients can help you build pages that will be valued by search engines and hence found by people who use them.

Content counts; content near the top of a page counts more than content at the end. In particular, the HTML title and the first few lines of text are the most important part of your pages. If the words and phrases that match a query happen to appear in the HTML title or first lines of text of one of your pages, chances are very good that that page will appear high in the list of search results.

Say you want to put your resume on the Web. Keep this rule in mind: Don't put your name first. You aren't trying to be found by people who already know you. You want to be found by people who have never heard of you. So don't waste any letter in the HTML title on your own name. The first word should be "resume." After that, list your main qualifications and the kinds of jobs that you are looking for. Put the same kinds of things in the first lines of text. That's what will come up as the default as the description in match list, and it's also an important position for ranking.

top

9. Spamming

Some barriers to being indexed are due to the misbehavior of a handful of webmasters who have tried to fool search engines into ranking their pages high on lists of matches and including them as matches to queries they aren't appropriate for. This is one kind of behavior that is known as "spamming." Spamming degrades the value of the index and is a nuisance for all.

The logic that leads people to try such tricks is rather bizarre. "I figure everybody searches for the word 'sex.' I don't have any sex at my site, but I want people to stumble across my site. So I'm going to put the word 'sex,' three thousand times as comments. And any time that anybody searches for 'sex,' my pages will show up first."

People have actually tried that. They have tried doing the same kind of thing in the backgrounds of their Web pages. They have also created page after page of text that is in the same color as the background color so visitors won't see the words, but search engine crawlers will. They have tried everything imaginable to fool search engines.

If being found via search engines is important to your business, be very careful about where you have your pages hosted. If the hosting service also hosts spammers and pornographers, you could wind up being penalised or excluded simply because the underlying IP address for that service is the same for all the virtual domains it includes.

top

Search Squad
Topics