Cluster Research

Very few genealogists realize the importance of cluster research. It seems counterintuitive to spend time researching people who are not obviously related to you in any way. The idea is that information that is entirely missing from your ancestors' records may have been preserved by neighbors. Cluster analysis is very common in a number of disciplines. Here is a definition from Wikipedia:

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique forstatistical data analysis, used in many fields, including machine learning,pattern recognition, image analysis, information retrieval, and bioinformatics.

One common way of establishing a cluster is to look for class members who live in proximity to each other. From an historical basis, the clusters should be defined by sampling the area surrounding the target ancestral family to establish homogeneity. One of the most common characteristics of the selected class is that they all have same immigrant origin. The affinity of the class can be based on a similar religion, national origin, language or economic level. One study of the subject from Harvard University is entitled, "Determinants of Immigrant ClusteringEvidence from New York City in the late 20th century" by Kris Bartkus.

This is an instance of a very powerful analytic tool that is very rarely employed to solve genealogical problems. As the study undertaken by Kris Barkus indicates:

This paper will explore how a number of macro-level indicators impact immigrant clustering rates of immigrants from various countries. Some of these indicators would appear to be cultural, but we can use basic economic theory to predict how each of these indicators would impact clustering rates. We then test these predictions against the data to see which of these indicators actually have a significant impact on clustering rates. Unlike other studies, which either study clustering on the state rather than neighborhood level or look at only the characteristics of the neighborhoods themselves when studying determinants of clustering (which raises endogeneity concerns), this study will look at clustering on a micro level, which is more accurate since clustering occurs on a neighborhood scale, while using macro level panel data to explain variation in clustering. The paper will also comment on differences in clustering between immigrants and first-generation Americans. 

A simplistic way of examining clusters is to plot the addresses of groups with similar countries of origin in close proximity as indicated by the U.S. Federal Census Returns. It is helpful to use a spreadsheet, such as Microsoft Excel, to enter the data. To do this, you have to examine the original U.S. Census Enumerators Maps. For example, FamilySearch.org has the United States Enumeration District Maps for the Twelfth through the Sixteenth US Censuses, 1900-1940, in its digitized Historical Record Collections. Older enumeration maps are also available but are more difficult to locate. Genealogist Michael Hait, CG has compiled a valuable reference entitled the, "United States Federal Census Pathfinder."

Clusters can also be created by using alternative search terms in programs such as FamilySearch.org, Findmypast.com and Ancestry.com. In some instances the search results can be copied or exported to a spreadsheet for further analysis. I recently employed a modified form of cluster research with searches in the Findmypast.com program to break through an end-of-line problem in England that had baffled researchers for over 100 years. In this case, I searched for all of the individuals with the target surname in an entire county and then reviewed the list by locality within the parish.

Each particular cluster search will entail its own unique parameters. But, generally speaking, the concept is to extend the research to a defined geographic area surrounding the target family and then examine the resultant population for similarities that may be applied to further delineate the origin or even the identity of the target family.

New Statistics from the Family History Library in Salt Lake City, Utah

This new infographic from FamilySearch.org gives some insight into the technological changes that have occurred over the past few years. The Family History Library in Salt Lake City, Utah is celebrating its 30th anniversary in the "new" building on West Temple. In a blog post dated 23 October 2015 entitled, Mormon Family History Library Still Connecting Generations of Families after 30 Years" FamilySearch.org updates several statistics that illustrate the dramatic growth of the Library's worldwide impact due to technological change.

Diane Loole, the Director of the Library, is quoted as giving the current number of Family History Centers worldwide as 4883. This is an increase from the estimated numbers that have been used recently in most references to this vast system of family history support facilities. The infographic shown above also shows some of the dramatic changes in the Library itself due to technological changes. Quoting from the post:

In this age of 24/7 access to information and growing thirst for digital services, libraries across the nation are evolving to meet the changing demands of the communities and patrons they serve, and the Family History Library is no exception. 

About 25 percent of the 2.4 million rolls of microfilm stored at the Granite Mountain Vault have been digitally published online. The Family History Library itself has about 1.5 million rolls on site. As physical films are digitized, they are removed from the library. Insofar as possible, the records teams plan on digitally publishing all of the microfilm online for 24/7 access. 

In 1985 family history research was a very individual experience requiring each person interested in a specific record to scroll through microfilm or search microfiche. In 1985 over 600 microfilm and fiche readers were housed in the Library. Though microfilms and fiche still play an important, though less frequently used role, a large portion of today’s research is now computer-based. Today the Family History Library boasts 550 Internet-enabled patron computers while still providing access to over 200 film and fiche readers. The Library also offers free access to film, book, and photo scanning equipment to help patrons digitally preserve and share family records.

The number of microfilms that have been digitized of the total available is lower than some of the recent estimates I have heard.  But the number highlights the extensive work that still needs to be done to digitize all of the records. As the post points out all of the records are being digitized. However, many of the records being added to the FamilySearch.org website are coming from the ongoing effort to gather records, now being digitally recorded, from all over the world. Here is a quote about this effort:

The library continues to move with digital innovations, benefiting from the latest technology to preserve and provide access to the world’s genealogical records and increase the success of personal discovery. Progress in gathering, copying, and making records available has been dramatic and fast. Over 300 camera teams are digitally preserving historic records worldwide—over 100 million images per year—that are published directly online.

The post also gives some historical perspective. Here are some further quotes:

It appears the masterminds behind its construction had a vision of future demands. Plans that seemed almost grandiose when construction of the edifice was announced in 1983 have not only materialized, but have also led the way through the years to accommodate ever-improving research and information gathering options. It has come a long way since its humble beginnings in 1894 as a one- room repository of the Genealogical Society of Utah, just around the corner and up the street in a small building called the Church Historian’s Office at 58 E. South Temple.

This historical note contrasts with the situation today.

Visitors to the Family History Library find an amazing collection of resources collected over 120 years and hosts of friendly people with expertise available to help them. The Library delivers with an impressive cadre of 45 full and part-time staff, and perhaps unprecedented for libraries, 550 full- and part-time volunteers or “missionaries.”  The volunteers hail from all over the world, many of them dedicating up to 18 months—at their own expense—to help patrons make successful discoveries. 

The main floor of the library is specifically designed to assist inexperienced patrons in getting started. The floor has been outfitted with computers supported by volunteers trained to assist beginners. Volunteers and expert reference staff are also available for more in-depth research on the other floors dedicated to records from certain areas of the world. 

On its lower level, for example, is found the largest number of Chinese clan genealogies outside Mainland China.  This level is also used for storing family histories, and overflow films, and books available by request.  Requests for digitalization of these and other personal books can be requested here, and is  done at another facility in Salt Lake or at many of the Family History Centers and affiliate libraries. 

“The library is not a repository for original documents as is the case with specialized archives; it is not an archive in that sense,” noted David Rencher, chief genealogy officer for FamilySearch.  “But it accepts donations of published works of genealogical significance.”  Books and serials are continually added to the library’s shelves—over 600,000 in fact—and the library is heading up an initiative with other public libraries to digitally publish historic books of genealogical relevance online—over 225,000 have been digitally published online to-date. 

I spent two days this week at the Family History Library and I will be posting some of my own impressions of the changes and the status of the Library today in future posts.

Historic Maps of Europe and the British Isles

Europa in Geologischer Beziehung nach den Hauptmassen der Gebirgs-Formationen. 3te Abtheilung: Geologie, No. 4. Gotha, bei J. Perthes. 1843. David Rumsey Maps

One of the most challenging aspects of genealogical research can be determining the country of origin of an immigrant. In beginning such a search, commonly the place recorded as the birth location either no longer exists or the place names are so painfully distorted as to be unrecognizable. It is an historical fact that the boundaries and even the names of the countries in Europe and even the parishes in Great Briton have changed over time due to political action or wars. In some parts of Eastern Europe, the names of the places could have been recorded in German, Russian or Polish.

These name changes often reflect differences in where the records about the ancestral family may be found.

As part of the methodology for solving these difficult immigration problems, maps are a good starting point for not only finding obscure locations but also for providing a perspective of the country boundaries at the time your immigrants left the Old Country for the new. However, depending on the type of map, the information may be vague or very specific.

Identifying the place where your ancestors lived is crucial in separating them from people with the same of similar names. Sometimes the identification needs to be as specific as the house or farm where they lived. Some historic maps have that degree of detail.

One type of map with detail down to the identify of individual houses or farms are the ordnance maps of Great Britain. Here is an example from Scotland:

Sheet 32 - [Edinburgh]
Publication date: 1857

Ordnance maps were prepared for military purposes but the value of this type of map to genealogists is the detail of the entries. The names of the farmhouses are included. There are also databases and gazetteers that list place names, but with a map you can see approximate distances and surrounding communities. One way to identify family groups is to plot all the places listed as births and marriages on a map and calculate the distances involved. The older the records, the more likely that extreme distances did not happen and that despite similar names the people may not be related.

The determination of which jurisdiction was in power at the time of your ancestor's event may be crucial in determining if the information is at all reasonable. Maps provide the insight into the lives of the ancestor and his or her family. It is important to remember that as you go back in time, the distances people could travel every day had to decrease.

Here is an map of Europe from the early 1800s.

Faden, William (1749-1836)Composite Map: Map of the Ce…1816

You may have to click on the map to see some of the detail or you can also click on the caption and look at the original. In the original map from the David Rumsey Map Collection, you can zoom in on the map and see the boundaries of the various countries that existed in that time period and identify the country where you ancestor may have lived. The names of the countries have all changed, some many times, since 1816. Just as a suggestion, look at this map and try to find a country named "Germany." In the U.S. Federal Census Records the country of origin is sometimes listed as Germany even during time periods when there was no country with that name.

Another valuable website with access to tens of thousands of historical maps is OldMapsOnline.org. Here are some of my earlier posts that have references to map resources:

• Maps of German Empire -- Always Finding New Things Online
• Major Improvements to OldMapsOnline
• Using Pictorial Maps
• Use Cadastral Mapping to find your ancestors
• 8000 Historic Norwegian Maps Online
• Online Digital Map Collections by State

Statistics and Genealogy

My son, the Astrophysicist, recently wrote a blog post entitled, "Is Salt Lake City America's Vainest City? Correcting Forbes." This article is an excellent example of why you do not want to get into a dispute with someone from my family. On the other hand, it is also a very good example of why people and genealogists in particular, need to be aware of the pitfalls of relying on popular media articles quoting statistics of any kind whatsoever.

Sticking to the Original Records – What is an original record?

When I first started practicing law, now about forty or so years ago, I frequently had to worry about producing the "original" document as "evidence" in a hearing or trial. For example, if the dispute was about ownership of a piece of real property, I had to find and produce either the original deed or a certified and sometimes verified copy of the original. A mere copy of the deed was not acceptable and would be subject to the objection of the opposing attorney. After having produced the original and once it was "entered" into evidence, I could make a motion to the court (Judge) to substitute a "true" copy of the original rather than lose the original to the court's records of the trial.

It was not unusual to have a controversy over whether or not the document produced was actually an "original" document. As time passed, the need to produce an original document slowly disappeared. By the time I retired, the issue of an original only came up in very limited cases. One notable exception was the production of an "original" signed and witnessed will in a probate action. No copy was ever accepted by the court (Commissioner or Judge).  Likewise, if there was dispute over whether or not an original document existed or over the accuracy of a copy, the original document had to be produced.

Genealogist live in a world of documents. Some of these are "originals" in the sense that they were created at or near the time of an event, but most, are really just copies of these originals, if an original document could be said to exist at all. For the purposes of the court, we had to show that the document contained either an "original signature" or that the document had a seal and stamp from some government agency certifying that it was a true copy of the original in the agency's files. The reason why this early rule was enforced was to ensure that the original document had not been altered by making a copy. As document reproduction technology developed, this need to see an actual original document almost disappeared, as I noted.

When I read or listen to genealogists talk about documents, they often refer to the distinction between examining the original record as opposed to a copy. The idea is that the original will somehow always be more reliable than the copy. Let's think about that for a minute. For example, an index is a copy of a document. Unless the index clearly indicates the source of the information and identifies the original documents used to compile the index, there is no way to tell whether or not the index is an accurate representation of the information contained in the original. I often see an online database program where the information provided comes from an index and there is no link or citation to where I might find the original record. Here is an example from Ancestry.com. This is a screenshot of the California Death Index 1940 - 1997. It has over 9 million individual records.

Here is the citation provided by Ancestry.com.

Ancestry.com. California, Death Index, 1940-1997 [database on-line]. Provo, UT, USA: Ancestry.com Operations Inc, 2000.
Original data: State of California. California Death Index, 1940-1997. Sacramento, CA, USA: State of California Department of Health Services, Center for Health Statistics.

In other words, the Ancestry.com index is a copy of the California Index. This may or may not be a problem, but we certainly do not know the relationship between the two indexes. Ancestry.com does provide some additional information.

About California, Death Index, 1940-1997

Indexing death records in California between 1940 and 1997, this database is an invaluable tool for those researching in the state of California. 

Vital records in California have been kept by the state registrar of vital statistics since 01 July 1905. This database is an index to the death records in California from 1940 through 1997. The database may provide such valuable information as first, last and middle names of the decedents, birth dates, mother's maiden name, father's last name, sex, birth place, death place, residence at time of death, death date, social security number (when available), and the age of the individual at the time they became deceased. The information provided in this database is sure to prove useful to those researching in the state of California. 

Use the information below to obtain a copy of a death certificate from California. 

California Department of Health Services
Office of Vital Records
M.S. 5103
P.O. Box 997410
Sacramento, CA 95899-7410

Please visit the following web site to obtain the necessary forms and current pricing information: http://www.cdph.ca.gov/programs/CHS/Pages/default.aspx 

Why can’t I see the Social Security Number? 

If the Social Security Number is not visible on the record index it is because Ancestry.com does not provide this number in this database for any person who has passed away within the past 10 years.

I am guessing the nearly all the users of the California Death Index on Ancestry.com, simply enter the dates into their family tree and ignore all the supplementary information. More sophisticated users might follow up and try to order a copy of the death certificate.

Why do I bring up this issue? The answer goes back to my initial discussion about the need to produce an "original" document. This rule existed in the court system until copies of the original became reliable enough to use in the ordinary course of litigation. But in some instances, even a photocopy was not sufficient to satisfy the court that the document was reliable. This is still a very good criteria for use by serious researchers. Here, using the California example, the accuracy of the index cannot be determined because the original supporting documents are not readily available. In these cases it is necessary to follow up and obtain access to the most reliable documents, most original, that can be obtained. Any information obtained from an index where the original documentation cannot be examined, must be considered to be tentative and unreliable.

Genealogists sometimes couch the distinction between a copy and the original in the terms of "direct" vs. "indirect" evidence. Sometimes the terminology is between a "primary source" and a "secondary source." These distinctions are only partially helpful. Is the California Death Index a primary source? Is it direct or indirect evidence? If I were in court, I could argue that the California Death Index was a government record, kept in the course of the government's normal activities and should be accepted as reliable. Just because a document may fall into the category of an original, a primary document or even direct evidence, does not guarantee its accuracy. Even if I took the time and spent the money to get a copy of the "original" death certificate, there is no guarantee that the information on the death certificate would be accurate.

What is the poor genealogist to do? How do we know what is correct and what is not? That is the core issue of any historical research. All conclusions are tentative. All "proofs" are subject to revision. We may become convinced that we are right, but it is always a good idea to remind ourselves of this fact from time to time.

What is "Indirect" Evidence?

From time to time I hear genealogists banter about using direct or indirect evidence to prove their conclusions. Of course these are legal terms created in whole cloth using the terms that are defined in the Rules of Evidence. Unfortunately, nearly all those who use the terms are not lawyers or even paralegal assistants to lawyers. So how do they know what the two terms mean? Where did they get this amazing insight into complicated legal jargon?

First a word about the Rules of Evidence. The Federal Rules of Evidence govern the admissibility of evidence in federal trials. Each state in the United States has its own version of the rules, most of which are based on the Federal Rules. Trial practice and evidence are complex courses studied in law schools across the country. The strategy necessary to have a crucial piece of evidence admitted in a trial can become one of the most challenging aspects of trial practice. Many cases are decided on a decision by the Judge to admit or exclude a single piece of evidence. I have seen an attorney break down into tears in frustration at not being able to ask questions in a way to overcome objections to evidence.

Legal terms are rigorously defined by thousands of law cases and hundreds of years of accumulated legal experience. Genealogists who use those terms have no idea what they are talking about. I was recently listening to a presentation by a genealogist who used a lot of legal terminology. My silent comment was that I would love to have him as an opponent in a trial because he would lose no matter what client or issue he was representing. Merely using legal terminology does not confer on the speaker the ability to understand and appreciate what those words mean.

There are many examples of where the use of legal terms by those who do not understand the meaning of the words causes great harm and injustice. In genealogy the misuse of legal terminology leads to confusion, arrogance, apparent superiority, and becomes an obstacle to communication. Now, there are some with a strong legal background that can use the terms in an appropriate way, but those lawyers who do use the terms correctly are apparently forgetting that there are no genealogical judges and juries. Likewise, there are no opposing parties represented by their own legal jargon spewing genealogical attorney. Do we really want genealogists to be forced to follow the Rules of Evidence? Do we really want to create our own ad hoc rules of evidence?

Now, back to direct and indirect evidence. In law, the term "indirect evidence" is usually called "circumstantial evidence." The issue in a court case concerning direct and circumstantial evidence is addressed in Rule 401 of the Federal Rules of Evidence. Here is Rule 401:

Evidence is relevant if:
(a) it has any tendency to make a fact more or less probable than it would be without the evidence; and
(b) the fact is of consequence in determining the action.

Here is an additional quote from the Notes of the Advisory Committee on Proposed Rules. These notes are often used to amplify or explain the rules.

Relevancy is not an inherent characteristic of any item of evidence but exists only as a relation between an item of evidence and a matter properly provable in the case. Does the item of evidence tend to prove the matter sought to be proved? Whether the relationship exists depends upon principles evolved by experience or science, applied logically to the situation at hand. James, Relevancy, Probability and the Law, 29 Calif.L.Rev. 689, 696, n. 15 (1941), in Selected Writings on Evidence and Trial 610, 615, n. 15 (Fryer ed. 1957). The rule summarizes this relationship as a “tendency to make the existence” of the fact to be proved “more probable or less probable.” Compare Uniform Rule 1(2) which states the crux of relevancy as “a tendency in reason,” thus perhaps emphasizing unduly the logical process and ignoring the need to draw upon experience or science to validate the general principle upon which relevancy in a particular situation depends.

If this loses you, you are certainly normal. Many experienced attorneys avoid trial practice because it is too stressful and difficult. However, the genealogists who spout this quasi-legal jargon are completely unaware of the difficulty of living in a trial environment.  Can you tell from the quote of Rule 401 and the partial explanation contained in the note that they are talking about direct and indirect evidence (circumstantial evidence)?

In some cases, legal terminology has become part of our common speech. We use the terms in a general unsophisticated and certainly not legal way. I can say something, such as, "I found some evidence that my grandfather came from Germany" and no one misunderstands what is being conveyed. Here, the term "evidence" means merely that you found a document or other record indicating your grandfather's origin. Detective fiction is full of references to legal terms as are fictional accounts about lawyers and court cases. I was a great fan of Perry Mason in my youth and probably watched every episode. Now, after years spent in trial practice in real courts, Perry Mason looks ridiculously naive and he is commonly shown to violate almost every rule of evidence.

Is there a definition for "indirect evidence?" Yes, you can start by reading these books.

• Twining, William. Theories of Evidence: Bentham and Wigmore. Stanford, Calif: Stanford University Press, 1985.
• Imwinkelried, Edward J., Richard D. Friedman, and John Henry Wigmore. The New Wigmore: A Treatise on Evidence : Evidentiary Privileges. Austin: Wolters Kluwer Law & Business, 2010.

Then you might want to read about two or three hundred cases about evidence. That would be a good start.

More insight into the Scottish contract with CACI

In a very recent blog post I wrote about the fact that the new contract for 2016 for the ScotlandsPeople.gov.uk website from the Government of Scotland had been awarded to an international technology company called CACI. The present contract is with Findmypast.com.  I surprised to run across another article mentioning CACI. But for the previous mention, I would not have noticed the name of this company.

The newer article has been reported hundreds of times in the online press and is various titled. One Associated Press article is entitled, "Close calls with drones spur federal registration of owners." The last three paragraphs of the article are interesting given the future relationship of a major online genealogical database company. Here is the quote which is essentially the same in hundreds of other articles.

Said Sen. Edward Markey of Massachusetts: “We have rules of the road, but in this brave new world, now we need rules of the sky.” 

There is virtually no difference between model aircraft and the drones used by many commercial operators. Model aircraft also vary widely in size, weight and capability. Some are larger than the 55 pounds that the FAA uses to define small drones, and can fly thousands of feet high. 

The FAA signed an agreement last month with CACI International Inc., an information technology company in Arlington, Virginia, to test technology that could locate the operators of small drones that are flying illegally near airports. The technology would let the government track radio signals used to operate drones within a 5-mile radius and identify the operator’s location.

Do you see anything that might have caught my interest between the two articles?