208621 Resource 76407 24 24 1215376038444 1215376092040 Controlled Deep and Thorough Indexing is Critical For Custom Search Engines

Google custom search performance is dependent on how well your content is indexed in Google.

Recently, a client with an integrated Google CSE contacted me and said - "Hey, my search engine is busted; it won't bring up a post that's in one of my blogs!". This client is using MyST Blogsite as well a few other blogging platforms and he also uses our newest product MyST/VS™, integrated custom search solution powered by Google. It turns out his custom search engine is fine; but what isn't fine is the content he was searching for is not in the Google index. Many people believe that the simple exercise of creating a custom search engine (CSE) for a blog or collection of web sites automatically causes all content in the domains of the CSE to be included in search results. This is not the case at all; a Google custom search engine can only perform as well as the content that exists in the Google index itself. As many of you know, I'm a big proponent of a <a href="http://blogsite.com/public/blog/198211" title="Index Footprint: Does Size Matter?" target="_blank">large and dominant footprint in Google</a>. I won't rehash my viewpoints on this matter except to point out that with the increasing importance of second-tier search, deep and thorough indexing of our content is even more important. If every bit of your content isn't in the Google index, you will experience an ever-growing competitive disadvantage. Many people have challenged my philosophy about thorough indexing in the past; they handily dismissed the benefit of a large Google footprint as a business requirement. With the importance of comprehensive second-tier search, this subjects emerges again; the consequences of a partially indexed body of content are clear - visitors aren't given accurate search results. Missing pages in search results are simply lost opportunities to provide a complete visitor experience that offers up all your content. An incomplete picture of your marketing message may mean the difference between acquiring a new customer, and allowing a prospect to wander off. Can you Overcome This Problem? Yes. Fortunately, Google engineers count on the possibility that your content architecture isn't designed well enough to achieve deep and thorough indexing. You can use the Google Sitemap capabilities to submit your unindexed pages directly into your CSE. But, this requires that you know all the URL's of your unindexed pages - a task that could take a fair amount of time to complete. This is not ideal though because you'll have to babysit this process time-and-again to make sure it's kept up to date.MyST Products Work Together To Solve This Problem MyST/VS™ solves the the issue of incomplete indexing because it has the ability to locate and automatically include unindexed pages left behind by Google's crawlers. And it does this regardless of the blog tool or content management system used. MyST Blogsite™ itself works in concert MyST/VS™ to provide the most comprehensive indexing possible for Google crawlers. There are some other tricks that can help, so if you're interested in learning more, feel free to <a href="mailto:bfrench@myst-technology.com" target="_blank">contact me</a>. 10 application/xml false CSE custom search engine Google indexing MyST Blogsite? MyST/VS? A Google custom search engine can only perform as well as the content that exists in the Google index itself. Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9) Gecko/2008061004 Firefox/3.0 127.0.0.1 127.0.0.1 bfrench 76.120.48.245 false true false false true 174050 false false true false false false true 76407 blog blogsite/Blogsite/web