Scraping AJAX sites with Scrapy | datawarehouse | Scoop.it
A common question in the Scrapy community is how to scrape AJAX sites. As Mark Ellul pointed out in the scrapy-users mailing list, there are two basic types of AJAX requests that web sites make use of. These are: "static" requests which their parameters (URL, post data) doesn't change, and "dynamic" requests that use some variables based on properties from the current page.
Via Mathieu D.