Fascinating and the robot-itization of the web has implications for website design and conversion. Here is how to make sure your website "appeals" to the good bots:

* QDF - Quality Deserves Freshness so build ways for your pages to change frequently.

* Avoid "Spider Traps" Where Bots Go In, But They Don't Come Out.

* Limit Links - good idea anyway, easier the crawl the better.

* Pay attention to your DIVs (feed the spider differently than the way the page appears to visitor, same content just different order and use "read more" inline teases to hide too many words).

* One inside baseball secret I picked up was get your website certified as accessible (for handicapped) can function as a way to check your code's Google acceptability.  Accessible certification is rigorous so if you pass that you are probably good to go with SEO.