Smithsonian Now Using Archive-It to Crawl Websites | Smithsonian Institution Archives | The Information Professional | Scoop.it

by Jennifer Wright:

Smithsonian Institution Archives announces its use of Archive-It to crawl websites.

In September 2012, the Smithsonian Institution Archives began using Archive-It, a service of the Internet Archive, to crawl its almost 250 websites.  Archive-It is "a web archiving service to harvest and preserve digital collections" that is used by more than 200 organizations.


This is exciting news for us as archivists.  While Archive-It uses the same software for crawling and viewing websites as we had been using for the past three years, we have been plagued with hardware issues and have not been able to keep our software up-to-date.  We now have access to software updates as soon as they are available.  The processes of setting up a crawl and reviewing it afterwards are also more user-friendly with Archive-It.  In addition, we now have the benefit of support from both the Archive-It staff and the larger Archive-It user community for those times when we just cannot figure out why a crawl is not working.