The Wayback Machine's First Crawl 1996 -- The Internet Archive

 

https://archive.org/details/wayback-machine-1996/The+Wayback+Machine's+First+Crawl+1996.mp4

Quoting from the Internet Archive article, "The Wayback Machine's First Crawl 1996," 

In October of 1996, engineers at the the San Francisco-based Internet Archive launched their first web crawlers, taking snapshots of web pages. At the time, the World Wide Web was only 2.5 terabytes in size. In 1996, it was still impossible to predict how large the World Wide Web might become.

Even in those early days of the Web, broken links (404 errors) were a growing problem, and it was clear that most Web pages were short-lived. Brewster Kahle and Bruce Gilliat invented a system for archiving Web pages before they vanished. The tools for this project were not terribly sophisticated; they were essentially PC applications built to capture entire websites by following the links from the main page.

Most of the genealogy community is probably not aware of the Internet Archive (Archive.org) or the Wayback Machine. The Wayback Machine--which first launched as a public search engine of web pages in 2001--has preserved some 588 billion web pages by working with 800+ partners around the world. These webpages are searchable with some limitations on the Internet Archive. 

I find this website to be essential for doing research in many places in the world.