Moving Information -- A Commentary

The last time I backed up my files to a new hard drive, it took over eight hours to move the information using USB 2 from device to device. OK, I know there are faster ways to move information, such as Firewire or Thunderbolt, or even faster versions of USB, but you are, in a sense, stuck with whatever is the slowest link. This is especially true if you are trying to use WiFi or some other type of connecter. You can download or move files no faster than the slowest connection.

This basic fact of life, the limit of the slowest connection or device, is a physical limitation of the system. One problem with this limitation is that you can spend a lot more money and time and still not experience faster connections. It turns out that even the quality of the connecting wires can affect the speed of transmission. This problem has a name. It is called Amdahl's Law which states that the speed-up that can be achieved by distributing a computer program over p processors cannot exceed 1/{f + [1 - f)/p]}, where f is the fraction of the work of the program that must be done in serial mode.

Wikipedia:Amdahl's Law.

Another common way of expressing Amdahl's Law is to say that the performance improvement to be gained from using some faster mode of execution is limited by the fraction of the time the faster mode can be used. Therefore, like I said above, the speed of any computer system is limited by the speed of its slowest component if that component is used all of the time. If I am copying data from one device to another, the speed at which this can be accomplished is limited to the slowest data transfer rate of any of the components whether that is the wire, the programs, the computer, the hard drive or whatever.

This fact of life is definitely not comforting. OK, the results of this whole issue is simple. I keep looking for the solution by purchasing faster computers, faster and bigger hand drives, using faster connections, and so forth. At the same time the files I am working with get larger and larger so that despite the increases in speed, I am still faced with waiting for data transfers based on the larger file sizes. My little 200K bit mapped images of a few years ago are now replaced by 16 or 20 MB or larger images stored as Digital Negatives. The fact is that the older systems would not even accommodate the larger file sizes. 

Genealogy files are mostly text based so they are relatively very small. They began to grow exponentially when we started to attach media to the files. At one point in time, I could back up my genealogy files onto CDs. Then I had to move to DVDs. Finally, when it took eight or more DVDs to make one backup, I moved entirely to hard drives. Online storage is a wonderful idea, but the file sizes allowed for storage, even from the companies with the largest allowable storage is still way too small to allow me to use an online service for all of my files. So online storage is relegated to backing up only my most valuable files. 

As my files get larger and larger, the time it takes to make a complete backup gets larger also. I have already far exceeded any internal memory storage capacity I might have purchased with my computer and have to rely solely on external drives. 

Will my data needs increase? Tanner's Law of Disk Storage says that my data will increase to fill up any possible and theoretical data storage capability and I don't expect things to change.