Genealogy and Complexity

There are two major types of complexity: complexity of numbers and complexity of concepts. There are also different areas of study that are labeled with the term "complexity." One common usage of the term "complexity theory" is used in computer science in the area of computational complexity theory that focuses on defining the classes of complex computational problems and developing organizational solutions through assigning problem solving algorithms. A type of complexity theory is also used in organizations and is also called complexity strategy and is used in strategic management and organizational studies.

Genealogical research is a complex activity on both levels of complexity. On one hand, genealogists are faced with the reality of an exponential growth in the number of their potential ancestors. Although the reality is that due to intermarriage among any person's ancestors (pedigree collapse) that the theoretical number is never actually reached, the number of ancestors and their potential descendants can be very large and much larger than any one person can comprehend. At the other end of the complexity spectrum, genealogists also face complexity in the number and types of records used to research relationships. This is not just a complexity of the number of records, but also the difficulty in obtaining, reading, evaluating and extracting information from records that are almost chaotic in nature.

Historically, very few genealogical researchers reached the level of involvement with their research to become embroiled in complexity issues. However using today's online tools coupled with the vast increase in the availability of digitized records, a genealogist can easily be catapulted into a complexity unimaginable just a few year ago. Traditional genealogical organizational systems based on color coding, filing systems, notebooks and research logs fail to address the basic issues and in most cases, simply add more complexity. In my experience, most researchers avoid the problem of confronting the overall complexity of genealogical research by ignoring the problem and focusing on a tiny part of their pedigree. You will hear comments such as "I am working on such and such a line" that indicate that the researcher is avoiding the complexity issues.

Even if a researcher focuses on a single line, the complexity is still there. Every generation on that line is also subject to the ever present geometric increase complicated by the natural decline in the availability of records as we move further into the past. Ultimately, the researcher is faced with the inevitable "end of line" or so-called "brick wall" problems when the difficulty in finding records exceeds the researcher's resources. All the time this is happening today, the researcher is being bombarded with messages from various sources extolling the virtues of this or that software product or system that will "solve' the problems and make them go away. This is partially supposed to be solved by collaborating with other members of the researcher's family who are, for the most part, entirely clueless about the problems faced by the researcher and usually uninteresting solving them.

Let me give one example at this point. Many of the newer strategies from the large, online genealogically database programs involve automated systems of supplying source records suggested by complex algorithms. These records are directed at the computational complexity of handling billions of records and billions of people in the largest of these online resources. The effect of these programs is to move more people faster into the realm of complexity. Instead of starting with a few manageable ancestors, the neophyte is faced with hundreds of potential ancestors long before they have the mental tools to handle the complexity both of record evaluation and sheer numbers. At the same the time they are being told that genealogical research or even genealogy is no longer necessary. The reality of the online record hinting programs is that they break down in exactly the same places that the competent researchers have reached in their approaches to the problems. I do not find "record hints" for those people I am really most interested in researching.

The main advantage of the record hinting technologies is to add more names or complexity to the more recent ancestral lines. For example, I have literally thousands of potential record hints from several large online database companies. If I focus on one of my "end of line" situations developed over years of research, I almost immediately find that there are no more record hints for the people I am researching. The algorithms have no more resources than those I have already been able to research. The computers cannot go out into the world and find records that their owners have not already incorporated into their databases. If the records are not there, they are not there and all the computer power in the world cannot solve the problem and I am back to my traditional methods of doing research.

In other words, the complexity of the ultimate genealogical research problems still exceed any existing algorithmic solutions. What the programs do is to increase the numerical complexity faced by the researcher by automatically adding records that the researcher would have traditionally ignored or avoided.

Meanwhile to add to the complexity, we have an additional wave of people who are not really interested in or educated in genealogy per se, but now have access to tools that supply a seeming endless number of "sources" and "names" to add to the monumentally large online family tree structures. We also have an emphasis on quasi-scientific "advances" such a DNA research that add yet another level of complexity by indicating previously unknown potential relationships. Those who request DNA tests do so, in almost all cases, without any of the research tools they need to pursue the information they receive from the tests.

What is needed? First and foremost, those who are attempting to resolve this complexity should try to avoid marginalizing and preventing the genealogists who have the education and sophistication from doing their research. There is nothing wrong at all from encouraging someone without any involvement in "family history" from beginning to accumulate a pedigree but this should not be done by denigrating the efforts of those who created the the original organizational structure. The ease at which most people can discover their first few generations of ancestry should not be held as the norm because of the complexity that this initial research generates as those same people attempt to move back in each succeeding generation.

Genealogy has moved into the mainstream of those major areas that confront different levels of complexity. The noise level in the entire genealogical system is rising to almost impossible levels. To paraphrase John Donne, no one individual is an island in genealogy. We are all ultimately related and this level of complexity cannot be ignored with impunity.

If genealogy were being analyzed by systems engineers in same way any other data transmission system was being analyzed, then the engineers would be seeing an unacceptable level of noise interfering with the operation of the system. Right now, the record hints, emphasis on the ease of adding sources and records, the DNA tests, the invitation to neophytes to work on complex problems without understanding the issues and many other factors have raised the noise level of the system to the point where is is nearing a breakdown. We need to address the organization of the entire system from a complexity standpoint and stop treating it as if we were all working on separate, independent simple, easy and fun issues.

[Closing note: I do not know why I try to address these issues when nobody seems to be interested in pursuing the real issues and addressing potential solutions. Anybody out there?]