This article answers common questions about search engines. Check back next month for a companion article discussing the myths of website traffic.
What is a Search Engine?
Google is a search engine. It just happens to be the best and most popular. A search engine is just a website that you can type search words into. Magically, it responds milliseconds later with a list of useful websites. I say magically because Google claims 3.3 billion sites in their index - and somehow they search all of them in less than a second. It's even more amazing when you realize that thousands of other people are searching at the same time.
It takes lots of computers and some pretty smart people to build a search engine. Listing pages for people is only the last step of the process. Most of a search engine's work is done ahead of time visiting sites and building its index. It stores a copy of each page and makes a note on when it should visit again. The more often a page changes, the more often it gets visited. This is one reason you really want to change your pages frequently. An active site will be visited more than once a week. Most pages will be visited less than once a month.
What Are Keywords?
The words a person types into a search engine are your keywords. In the early days of the Internet, they were special tags programmed into the page. Like submission forms, it was abused. So now, most search engines evaluate your site based on the content. The "magic keywords" of yesterday just don't work anymore.
There are tools available to webmasters to evaluate what keywords people are looking for. It is also useful to determine what words people use to find your site. You can do this by analyzing the traffic logs.
It's a good idea to make a list of words that your potential clients will type to find your site. Then create pages that talk about these words. You should also write about questions your clients ask. Not only will those pages be searchable; your site will be providing useful content.
How Do I Get Listed?
The easy answer is do nothing - they'll find you soon enough, and there's really nothing you can do to get listed faster. Just make sure a site that gets indexed frequently has a link to your site. Note: every search engine has a way for you to submit a new address. These frequently get abused by webmasters and thus aren't very useful.
Some people think it's a good idea to submit a site every day or 1000 times. They'll even charge you money to do it. Google has been known to blacklist sites that use these types of tactics. So, no, it's not a good idea to pay for the service. Submit it once if you must and simply be patient.
You can often buy ads in the search engines. Google has adwords, Inktomi has a paid indexing service, etc. The rules are likely to change over time - and really you need to just stay current or work with someone that does. Regardless, ads are a good way to find traffic in the search engines - though that's a whole different article.
How To Tell When You Were Last Indexed?
If you search for your website's address (ex: www.invisiblegold.com) it won't show the cache link. You'll need to choose words that the page contains instead. Also, your site won't come up if the site hasn’t been indexed yet. It takes 4-6 weeks for a new website to show up in the index.
How Do I Get A Good Rank?
Most people won't look past the first 10 or 20 sites returned, so you need to be on that first page. There are fortunes won and lost in this game and enough "magic" solutions to drain your bank account. You're best off hiring a professional. However, first of all, you need to make sure your website is optimized for search engine traffic. Most editable websites are not. Actually most websites in general are not. Invisible Gold was designed from the start to work well with Search Engines so even if you do nothing else, you will probably do quite well in the search engines. (Contact us if you would like help in planning a search engine campaign.)
Second, you need to really focus on your content. Choose a good title for each page, mention your keywords several times and make sure there is enough content for someone to sit down and learn something. Try to minimize banner ads, popups, hidden text, or any other of the tricks. Search Engines are pretty smart and will just lower your rank if they notice you trying to be sneaky.
How Many Search Engines Are There?
We are often misled by the offers to submit a website to hundreds or thousands of search engines. In reality, there are just 3-5 that people use on a regular basis (ex: in order of popularity: Google, Yahoo, MSN, AOL, and Ask Jeeves). When you factor in the fact that Yahoo and AOL use Google’s index, you understand its importance.
Is There Anything Else I Can Do?
While you are waiting for the major search engines to index your site, you should submit it to DMOZ.org. It's really a directory or list of websites - but for every submission, someone actually reviews the site. The backlog is long: 3-4 months. Major search engines use it as a reference and it costs nothing so go ahead and submit a listing to it. Be sure to get the text right the first time, as it will be very difficult to get it updated.
Yahoo has something similar, however they charge you quite a bit of money to get listed. People debate whether it is worthwhile. If you have the money to spend, it makes more sense to hire a search engine expert to explore your goals and recommend site updates. They might even submit it to Yahoo as part of the process.
What About Links and Link Popularity?
One of the ways search engines evaluate your site is by counting the number of links pointing to it. You've probably seen offers where people link to your site or trade links, etc. The smart search engines punish websites that are listed on "link farms". These are websites that seem to exist for no other reason than to link to other sites. A rare few can be useful if they send visitors your way. However, if it is just to increase search engine rank, then there is almost no reason to be listed - and definitely no reason to pay.
There are several free online services that calculate your link popularity. One of the easiest methods is to simply go to Google and type in “link: and your website”. (ex: link:www.invisiblegold.com) Then look for the count count of pages returned. marketleap.com has a nice tool for comparing your link popularity with your competitors' sites.
Just keep in mind that your usually better off focusing on your content than worrying about things like link popularity.
Food For Thought
A recent report found that “499 out of 500 sites” on the Internet is invisible to the search engines (often called the Invisible Web). These are primarily database-backed and editable sites and the problem is in the way they use addresses.
Search engines shy away from addresses with question marks and thus these sites just won't show up.
Are Search Engines Worth It?
Some people ask whether they should even bother to get listed in the search engines. It's actually harder to not be listed. Regardless, it is a good question. Most websites will never get the level of traffic they expect. Many people think that all they have to do is create a website selling something and there will be instant clients. The truth is that it is more like tossing a couple of brochures into the ocean. Your website is there and it can provide instant value for anyone that knows to type the address in and visit.
For most businesses, you are better off treating your website as the core of a complete marketing campaign. Use ads, sales, and networking to drive people to your site. Put useful and relevant information on it and give people a reason to come back. If a search engine sends visitors your way; even better.