Sharing Data Files or What Happened to GEDCOM?

Let's suppose that after a lifetime of research, your Aunt Madge finally agrees to share all of her files with you. In a brief discussion, you know that she is using an antiquated computer running Windows 98 and you have had nightmares about the computer crashing and losing all of her data. But you are grateful that she has at least added all of her sources into her version of Personal Ancestral File (PAF).

After some long telephone conversations, you convince her to send you a GEDCOM file. The fact that the file arrives at your home on a series of floppy disks is a minor problem, but you get online and find a USB floppy disk drive and are finally able to copy the GEDCOM files to your own computer. Using the most current version of a popular genealogical database, you can easily import the entire GEDCOM file and magically, all of the 23,987 names appear in a new file you created to receive the information. As you begin to examine the file, you are mortified to see that all of your Aunt's sources are in the notes. The few sources that do come across, are all combined on one line and have a lot of strange characters. You begin to realize the huge task you now face to "clean up" the old PAF file.

In desperation, you look for further help online and discover that some of the current programs will read PAF files directly. Thinking this might be a solution, you purchase Ancestral Quest, the program claiming the most PAF compatibility. After another round of telephone calls to your Aunt, you obtain a copy of her PAF file and open it in Ancestral Quest. Unfortunately, the PAF file itself is too large to copy to floppy disks, so you have to buy your Aunt a hard drive and figure out how to get the information onto the drive, when her computer has no USB port. After a huge amount of technical difficulties, you obtain a copy of her PAF file.

All of the weird character problems are solved, but the sources in the notes are still in the notes. You wonder if there is a way to convert the sources in the notes into sources in your program. You would like to move the Ancestral Quest file over to your current genealogy program but when you try to do so, you find you have to export the file as a GEDCOM and then import the file into your own program. Putting the file through the GEDCOM conversion still adds all the weird characters.

Ultimately, you resign yourself to a perhaps years of cleaning up all of the data, not to mention the time it will take to check your Aunt's sources and move them all to the source fields in your own program.

This common example of problems in sharing data between computers with different operating systems and using different genealogy programs illustrates the issue of file sharing in the modern genealogical computer world. If you think your troubles are over when you finally "clean up" the file. You are just beginning your trouble. You have taken the time to add media, i.e. video, audio and images to your file. When you try to share your file with others, you find out that none of this media comes across exactly right. In fact, your biggest challenge is the fact that all of the media needs to be linked to the people in your file and when you move the data to another computer, it all has to be relinked because the program you are using lost the links in the transfer. You further realize that the amount of information you have attached to your file is huge and you have no practical way, other than purchasing a very large hard drive, to transfer all of your attached information to your family members.

Sharing files, especially very large files is a major challenge and the current status of GEDCOM and the almost non-existent file sharing capabilities of the most current versions of the genealogy programs do not make life any easier. There are some file sharing schemes out there in the form of utility programs that claim to transfer files between programs, but if your program and the program you are transferring the file to do not match the programs supported, you are still out of luck.

Here is what needs to happen for a file to be transferred from one program to another. I call this the "Fat Man's Pass" problem. The data fields in one program have to correspond exactly with the same field descriptions in the target program or the data goes into la-la land and is shaved off of the transfer. So data is lost in each transfer. GEDCOM is like the narrow Fat Man's Pass, only a certain amount of data will transfer properly from one program to another.

This is a real and very serious issue in the genealogical community. One reason I work with several popular programs at the same time on my computers is to make sure I know the difficulties and quirks of moving files from one program to another. One solution, of course, is to have everyone in your family use the same program. But that entails a whole new set of challenges and problems. For example, Aunt Madge's computer and operating system will not support any of the newer programs and she certainly does not have the storage capacity to include media with a file.

Some time ago, two separate organizations began an effort to address a few of these issues. With an almost universally recognized need to upgrade GEDCOM. There are currently three separate organizations working on the problem. That is the good news. The bad news is the apparent lack of progress in coming to a common consensus of how and what needs to be done to implement some workable way to share and transfer files.

Just for the record, without a lot of comment, the three organizations are as follows:

• BetterGEDCOM
• The Family History Information Standards Organization (FHISO)
• GEDCOM X

From time to time, these organizations surface at various conferences and discuss their mutual problems and goals, but very little progress has been made to resolve the perceived differences between the different proposals. If you watch carefully as new versions of the popular genealogy programs are introduced, you will notice a complete lack of reference to GEDCOM issues at all. What these organizations have mostly created is a huge amount of online posts and social networking discussions.

So what do most of the genealogists think about this major issue of file compatibility and file transfers? Until they are faced with the problem in a very personal way, most genealogists are not even aware that there is an issue.

Do I have a solution? Yes, but no one really wants to listen to specifications for image transfer and technical problems with text file formatting. Interesting in a world where stories and photos are taking such an important part in the world of genealogy. One small example will suffice, most online family tree programs allow uploading images. Nearly all of these images are in the lossy JPEG file format with some few exceptions. Are we creating a dead-end file sharing environment by allowing JPEGs to become the de facto standard file sharing format?