Is it always possible to balance your customers requests with a spider friendly website? In most cases, it is easy to make sure that a site has great navigation and flows well. Certain website features are known to cause problems with spiderability, but they can be the same features that customers insist on using. Splash pages, image maps, JavaScript effects, Ajax, and Flash elements are all very attractive. They might help a human visitor to stay tuned to a website longer, but those same things can prevent a spider from crawling your website properly.

There are ways to implement these features and technologies while keeping your website spider friendly that include text-based navigation menus and simple hyperlinks. If you must add a splash page that uses Flash, be sure to provide an alternate link just below the movie to enter the rest of the site. If you are using Ajax, the content that gets displayed is buried behind JavaScript or ECMAScript which spiders cannot see. If Ajax is used for navigation or a substantial portion of the pages content, add a direct link to the page without using Ajax. Provide an alternate page for the same content. This makes linking to the page as easy as adding a basic hypertext link… one that spiders can follow!

With a little effort, it is easy to turn non-spiderable web page elements into sources of potential content and search fodder. While it is better to avoid certain things for the sake of search engines, it is also possible to provide the “fun stuff” for the human visitors while keeping your site spider-friendly. This also serves to keep your pages accessible to users who can not use or view javascript, images, Flash or other advanced browser technologies, but that’s an entirely different post for a later time!