Does your genealogy qualify as microhistory?

The use of the term "microhistory" has come up several times recently. I had an interesting comment from Adrian Bruce that said:

FamilySearch's FamilyTree is a major innovation technically - but how much of an innovation is it in *genealogical* terms? I think the phrase "One step forward; two steps back" applies. Take its inability to record Notes in *all* the places that GEDCOM based programs can (against the individual; against an individual's events / attributes; shared between lots of individuals and events ...) And what there is for Notes, only came in in October. How can one write micro-history for people without notes against specific events? 

And so far as I know, none of the major software platforms allow multi-person events. Why is this important? Take an emigration. Why should I enter the same event with the same descriptive Note against all members of my family on that ship? Or even copy the same stuff? And then how about the further linking of the emigration to the micro-history of the ship?

In addition, Tony Proctor in Ireland, sent me a link to a post he wrote entitled, "The Future Representation of the Past." If you review Tony's post, you will see that he is also involved in the concept of creating a context for microhistory. I usually have something to say about nearly every topic and this is no exception.

First, a definition of "microhistory." This term did not spring from the imaginations of the genealogists trying to legitimize genealogy as history, but it may well have come from the historians trying to grapple with family specific histories and not wanting to appear as "genealogists." So is the term "microhistory" just another euphemism for genealogy such as the use of the term "family history" in an attempt to sanitize the pursuit? I think the term is more of an attempt to justify an interest in history as a more detailed level than has been generally accepted in the past. But isn't this already being done in the area of biographies and autobiographies? The analysis could go on and on, but essentially the term "microhistory" is used as the opposite of "macrohistory.

To some extent, microhistory is an European phenomenon. See Giovanni Levi, “On Microhistory,” in Peter Burke, ed., New Perspectives on Historical Writing (University Park, Pa.., 1991) - See more in the article "What is Microhistory?" There is also a website called Microhistory.org from the Center for Microhistorical Research is an independent research institute at the Reykjavik Academy (www.akademia.is). However it should also be noted that there are opinions that use of the term is limited as can be seen from the statements made by Jill Lepore in an article entitled, "Historians Who Love Too Much: Reflections on Microhistory and Biography." Quoting from that article concerning a reticence of American historians to use the term microhistory:

One explanation for this reticence is that microhistory was established among historians of early modern Europe, where it has thrived, but there is no American school of microhistory, only practitioners who rarely if ever call themselves microhistorians and who, at any rate, may only occasionally dabble in microhistory.

Backing off from this type of discussion, I would think that the use of the term is more likely a way to approach individualized histories from the standpoint of "history" rather than an appropriate way of inducing genealogists to look at more than names and dates. For whatever reason, whenever some historian or other person wishes to cast a pejorative slant on genealogy, they take the position that genealogy is nothing more than the collection of names and dates. This is certainly true for some genealogists, but do I really become a historian merely by labeling my study "microhistory" simply to avoid the connotations that arise from labeling it as genealogy? I think not. On the other hand, much of the work done by serious genealogists is certainly on par with what has been produced by "historians."

Returning to the Lepore article, she offers the definition of microhistory as:

The recent microhistory conference's call for papers defined microhistory, in part, as the history of "hitherto obscure people" that "concentrates on the intensive study of particular lives" to reveal "the fundamental experiences and mentalités of ordinary people." 

Doesn't that kind of definition really describe what genealogists attempt to do in many cases?  The main issue then returns to what I have pointed out in the past; the lack of academic rigor and reliability in much of what passes for genealogy. Perhaps all the hand-wringing and angst (including my own) by some genealogists over the issue of the academic acceptability of genealogy is misplaced. Perhaps the legitimization of genealogy in academic circles comes from producing academically acceptable work rather than trying to insinuate ourselves into the hostile environment of academic history. Maybe it is a better idea to concentrate on raising genealogical standards and simply ignore historians but not ignore history?

Of course, both of my commentators cited at the beginning of this post as somewhat concerned with structuring the processes of recording genealogy to allow for the expansion into an area that could be called microhistory, but I am more concerned with the content of what we already produce as genealogists. As long as we are buried in the trash pile of unsourced and unreliable family trees, we will assume the smells and stains of what we have collectively created for the past hundreds of years as genealogists. We need to move on from on from our concerns with acceptance and become acceptable.

I do acknowledge that achieving any degree of acceptability is likely impossible as long as genealogy is viewed an open pursuit which can be done by anyone with a piece of paper and a pencil regardless of education, structure and intent. But rather than try to change all of the sloppy practitioners and make them over into academics, why not simply work towards including more people into the process of improving the general level of work done who are willing to improve?

How valuable are indexes to source records?

I am ambivalent as to the need for indexes. Don't get me wrong, indexes are a great labor saving device. But the very nature of indexes discourages in-depth searches into the original records that likely contain just the information you are looking for even though it does not appear in the index. Statements about indexes have the tendency to portray them as the ultimate solution to research problems. It is true, that much of the easy research is aided by indexes. It is equally as true that indexes sometimes prevent the very research they are supposed to assist.

The tragedy in genealogical research occurs when the neophyte researcher relies entirely upon the index and assumes that the records is missing rather than the fact that the index is simply defective. It is a fact of life that when an index is off even one letter the resultant entry can be completely unconnected to the original. When I say this, think what happens if the initial letter is misread. One example from my own genealogy involved the indexing of "Tanner" as "Danner." An index-only search in that database fails to return even one "Tanner" entry.

When you begin your research you may think that your ancestors have simple surnames but your quickly disabused of this belief by the variety of ways the original names were recorded in the source records and even more by the variety of index entries that have misread the originals. There is always a point in any researcher's experience with genealogy that spelling variations in names becomes an issue and indexes do not handle name variations well. If your ancestors spelled their name "Smith" as "Smythe" on occasion, then a very good index would move that spelling completely out of the "Smith" returns to a place where you might not think to look.

I could write a very, very long post about all the incidents of spelling variations and all the people who got hung up on the "way my name is supposed to be spelled" but the idea here is that all these variations show up in indexes by moving the entries from one results list to another. Some of the changes come from the way the individual spelled his or her name in the first place. You have a very hard life as a genealogist if you don't make allowances for spelling variations at all levels of record keeping. But all of these variations make indexes less than perfect. How many "brick wall" situations are simple spelling issues? I would guess quite a few.

The idea here is not that indexes per se are bad, but that relying on them exclusively is foolhardy. The classic example is the ancestor who seems to disappear from the U.S. Census in certain census years. The vast majority of these disappearances turn out to be spelling based errors occurring somewhere in the process of making an index. As some of my friends would say about now, "Well, James why don't you tell us what you really think about indexes?" No, I am not advocating a wholesale abandonment of the indexing industry, I just got fed up with bad entries lately and needed some space to think about the subject.

Back to the issue at hand. Some people seem to think that indexes are proper in all instances, that is, every single source record should appear in an index. I disagree with that broad brush approach to indexing. Most of the indexing efforts pick and choose which parts of the original records are included in the indexing and sorting process. Valuable information in original records can be and often is, completely obscured by failing to examine the original and relying on an index. There is often an excessive reliance on surname indexing that is inappropriate with certain naming conventions. For example, Spanish language surnames may contain two or more "surnames" such as "Maria Gonzalez y Rodrigues." So is this indexed under "G" or "R" or both? What if one of my ancestors used the name "Tanner-Roberts" or some variation with a hyphen, where does the name fall in an index? Is it under "Tanner" or "Roberts" or both? It is my experience that most indexing programs or procedures would not include the name twice.

I am always amazed at how many people, when using Ancestry.com, stop at the page where the summary of the record is given and fail to examine the image of the original record closely. How much of the original is in the summary? Yes, of course, the original record could also be a mess, but how do you know that if you rely exclusively on indexes? Some original records should not be indexed in my opinion. Indexing is a waste of time and misleading. For example, Spanish and English language parish registers. I have yet to see an index that includes an index entry for the witnesses to baptisms and marriages. They might exist, but most of the indexes are doing good to index the main name of the people baptized or married. Leave this type of record alone and let the researchers search through the entire record themselves. How many children and others are missed in families simply because the index is faulty?

My conclusion is don't believe everything you see and read. Take time to think about alternatives and don't get hung up on spelling variations. Finally, don't rely too heavily on indexes, they lie.

Finding what needs to be researched on your FamilySearch Family Tree: Puzzilla.org revisited

In a previous post I wrote about a new utility program from Puzzilla.org. The basic idea of this program is to provide a visual representation of the FamilySearch.org Family Tree in way that shows the descendants of any of your direct line ancestors. The advantage of the visual representation is that it shows any areas of the descendants of your ancestors where information is lacking, i.e. where the lines end. In essence the program graphically portrays all of the descendants of any of your ancestors. This is explained in two videos on the website.

It is very interesting to see how little information exists on various family lines. As you would expect, some lines have been extensively researched, while others have been neglected to extinction. Some genealogical researchers arbitrarily draw lines around who they consider to be part of their "family." These decisions are made primarily based on the idea that "those people are not my relatives." I guess I sometimes do the same thing, but it does give me pause to wonder why I make those distinctions. From one viewpoint, there is no real end to who you can or should research. Part of the problem arises from the old, tired issue of ownership. People, genealogists in particular, think they own their family and its records. I cannot tell you how many times I have heard the phrases "my research," "my family," and "my records" in conjunction with concerns about someone else adding to or taking away from "their research." Maybe a program such as Puzzilla.org really points to the fact that there is no real rational end to anyone's family.

I don't really know who the person or people are who are developing this particular program, but I can say that this type of tool is an eye opening experience and this is something we all need to see and be using to show what we have and have not done as genealogists and family historians.

When was the last time you checked primary sources?

I was looking at the website for our local City of Mesa Public Library and I noticed the following description of services:

• Reference and research assistance on Mesa topics
• Access to archival manuscripts and oral histories documenting a diverse citizenry and prominent Mesans; personal papers and biographies of founding families; archival records of Mesa community organizations and businesses; current information on Mesa schools, businesses, and organizations; maps of Mesa, past and present; and photographs of Mesa and its peoples
• Access to current publications, as well as permanent historic records created by Mesa municipal agencies
• Programs, workshops, and outreach to help Mesans learn about their heritage and how to preserve it
• Consulting for individuals, agencies, and organizations on preservation of historical records and disaster mitigation and recovery

I began wondering how many times genealogical researchers seek out original source material stored in libraries, archives, historical societies and other repositories around the world? I am certain that this is not a common as you might suppose. I mentioned before that we have an active organization of "professional" genealogists in our state, but I have never seen or met any of these people at the Mesa FamilySearch Library except those who actually work there. This has often caused me to wonder whether or not the genealogical community, for the most part, relies on the "easily" obtainable records online and ignores records that may not be indexed or organized?

I ran across this website from the University of Idaho called Repositories of Primary Sources. The site has been maintained since at least 1995 and has over 5000 repositories listed worldwide. This type of list is valuable for the reason that it gives a graphic example of the places genealogists hardly use or look at. What is important to realize about these locations is the amount of material that is unindexed, unorganized and simply sitting in boxes or on shelves. I thought I might give an example of what I am talking about by listing all of the items cataloged on one of my ancestors sitting in the University of Utah Special Collections Library in Salt Lake City, Utah. I am adding that list to the end of this post.

As you look at the list below, think about the number of documents in this one collection in one library in one city in a state where I do not live. Then look at the website referenced above and click on the list of repositories in the state of Massachusetts or New York. You might want to look at Pennsylvania or Ohio. How long would it take you to determine that these repositories did not have any information about your family? You might be able to understand why I think it is ludicrous when some one tells me that they have searched everywhere to find an ancestor. That type of statement tells me the person has no idea what types and quantity of records are sitting in libraries and other repositories across the country and around the world.

When you look at the list below, think about all the other names of people mentioned in this one individual's "papers." How many of the descendants of these people know that there is information sitting in boxes and folders in the University of Utah Special Collections Library under my ancestor's name? Notice the list of diaries and other materials from various people in this one file.

Sometimes I just sit there staring at my computer screen and feeling the huge weight of all the records I will never have time to review or organize during my lifetime. I feel so sorry for people who waste time researching without knowing where or how to look for information. I also feel sad when I talk to people who give up looking because the don't want to take the time to do systematic actual research into what documents and sources are available.

One observation, note that there is a 24 hour notice requirement and that parts of these records might be restricted. Why?

Well, here is the list. Just think about what might be located in a library where your ancestors lived, waiting to be examined:

Collection Overview +/-

Title:	John Hamilton Morgan papers
Dates:	1845-1986 (inclusive)
Collection Number:	Accn1465
Summary:	The John Hamilton Morgan papers (1845-1986) detail the personal, professional, and religious activities of John Morgan (1842-1894), a noted Utah educator and Church of Jesus Christ of Latter-Day Saints (LDS) leader.
Repository:	J. Willard Marriott Library, University of Utah
Address:	Special Collections 295 South 1500 East Salt Lake City, Utah 84112 801-581-8864

Collection Inventory +/-

box 1: Journals and Letters

folder 1-16: Typescript of John Morgan's Journal (1875-1892)

folder 17: Handwritten Letters from John Morgan (1874-1888)

folder 18: John Morgan, Doctrines of the Church of Jesus Christ, Its Faith and Teachings

volume 1: "Volume 1, Letters and Articles on the Missionary Labors of Pres. John Morgan" (1872-1879)

volume 2: "Volume 2, Letters and Articles on the Missionary Labors of Pres. John Morgan" (1879-1894)

box 2: Letters and Personal Papers

folder 1: Miscellaneous Typescript Letters From John Morgan (1863-1886)

folder 2-12: Typescript Letters from John Morgan to Church Leaders (1878-1887)

folder 13: Miscellaneous Letters to John Morgan (1877-1878)

folder 14: Report of Southern Emigrants (1877)

folder 15: John Morgan, "Riding the Denver and Rio Grande Railroad in 1878"

folder 16: Typescript of Speeches and Services Given by John Morgan (1888-1893)

folder 17: Quotations from the Patriarchal Blessing of John Morgan (1875)

box 3: Missionary and Government Work

folder 1: List of Missionaries and Dates of Service

folder 2: Missionary Records for the Southern States Mission (1875-1888)

folder 3: List of the Converts of John Morgan on First Mission (1876)

folder 4: List of Meetings Attended by John Morgan on Second and Third Missions (1878-1880)

folder 5: List of Conferences Established by John Morgan while Organizing the Southern States Mission (1882)

folder 6: Gospel Messages

folder 7: Missionary Names and Addresses

folder 8-9: Typescript of Southern States Mission Reports (1875-1887)

folder 10-16: "Settlement of San Luis Valley, Colorado by the Mormon People in 1878"

folder 17: Articles Concerning the San Luis Valley Settlements

"The Founding of Mormon Settlements in the San Luis Valley, Colorado," by Andrew Jenson; "The San Luis Valley, Colorado. The Beginning of its Early Colonization by Mormons," by D. H. Christensen.

folder 18: San Luis Valley, Colorado (1878-1888)

Items contain geographic and demographic information.

folder 19: Governor and Appropriation Bill of 1884, Utah Territorial Legislature

folder 20: John Morgan's Reaction to the Shooting of an Elder (1884) (1886)

folder 21: Salt Lake Academy and the Chambers of Commerce (1881-1894)

folder 22-25: Historical Monument Dedications and Corporation Beginnings (1848-1898)

Organizations and monuments included are Salt Lake Theater Company, Bowery, Pioneer Park, Liberty Park, Fort Douglas, Salt Lake schools, the Old and New Tabernacle, Utah Affair, Zion's Corporation Mercantile Institution, Union Academy, and others.

folder 26: Presidents and First Counselors to the President of the LDS Church

folder 27-42: Church Business and Decisions (1867-1887)

box 4: Research Material

folder 1-32: Typed Version of John Morgan's Journal Told in the Third Person (1875-1892)

folder 33: Notes for Corrections of John Morgan's Journal

box 5: Research Materials

folder 1-10: Narrative Based on the Diaries of John Morgan

folder 11: Historic Events in the Life of John Morgan

folder 12-13: Research Material on his Soldier Career

folder 14-16: Information, Photographs and Maps of the Civil War

folder 17: Information and Photographs about John Morgan and his Family

folder 18: Letters Written About John Morgan (1845-1914)

folder 19: An Article Written About John Morgan (1982)

folder 20-21: Miscellaneous Biography Research Material

folder 22: John Morgan "Experiences," Published Version (1964)

folder 23-25: John Morgan "Experiences," Drafts

folder 26-27: Photographs of Missionary Work in Manassa, Colorado (1877-1910)

folder 28: Photographs of Missionary Work in Pueblo, Colorado (1970-1986)

folder 29: Information and Photographs about the Death of John Morgan and Other Church Members

folder 30: "In Memoriam" and Other Tributes to John Morgan (1947-1959)

box 6: Research Materials

folder 1: Rough Draft of John Morgan Biography

folder 2-4: Final Draft of John Morgan Biography

folder 5-6: Matthias F. Cowley and John W. Taylor, Correspondence (1877-1882)

folder 7: History of Matthias F. Cowley

folder 8: Diary of James Thomas Lisonbee (1876-1877)

folder 9: History of Ellen Amelia Johnson Lisonbee

folder 10: History of Annie Catherine Christensen

folder 11: History of the Kirtland Family (1944)

folder 12: "Louis M. Peterson in Connection with Hans Jensen Mission"

This article was taken from "Histories of the Old Council House of Manti" (1935), and this copy was made July 10, 1947.

folder 13: Miscellaneous Papers Pertaining to John Morgan

Content Description +/-

The John Hamilton Morgan papers (1845-1986) detail the personal, professional, and religious activities of John Morgan (1842-1894), a noted Utah educator and Church of Jesus Christ of Latter-Day Saints (LDS) leader. The collection includes letters (1875-1894) and journals (1875-1892) written while Morgan served as a Union soldier in the Civil War; while he conducted LDS Church business as a missionary leader and a ranking member of the LDS Church hierarchy; and while he acted on political issues. Official papers concerning LDS Church affairs and Morgan's government involvement, as well as biographical and religious articles written by and about John Morgan, also appear in the collection. Much of the collection's information concerns the settlement of the San Luis Valley, Colorado, where many LDS converts settled after arriving there (under Morgan's leadership) from the newly-formed Southern States Mission. While most of the material originated in Morgan's lifetime, collections of his inspirational stories, tributes to his memory, and a biography were produced posthumously.

Collection Use +/-

Restrictions on Access:

Twenty-four hours advance notice encouraged. Access to parts of this collection may be restricted under provisions of state or federal law.

New Landing Page for New.FamilySearch.org

In an announcement dated 20 November 2013, FamilySearch notified users that the link to New.FamilySearch.org will be modified to include the above notice as part of the landing page. What this means in English is that when you go to New.FamilySearch.org, instead of going directly to the program, you will see the above screen. This is already in effect. Here is the explanation of what is happening from the blog post:

Recently, we have announced that new.familysearch.org will become a read-only website by next month. Once this change takes place, each time you log onto new.familysearch.org you will be guided to a new landing page. From here, you can either proceed to new.familysearch.org or you can go to Family Tree.

One small step for a man and sort-of small step for mankind.

Is Handwriting Recognition the Holy Grail of Genealogy?

"Massachusetts, Plymouth County, Probate Estate Files, 1686-1915," images, FamilySearch (https://familysearch.org/pal:/MM9.3.1/TH-1961-26979-18706-60?cc=1918549&wc=M9S5-2FG:n612103331 : accessed 14 Oct 2013), Plymouth > Case no 21895-21929 Warren, Nathan-Washburn, Benjamin > image 23 of 533.

For me seeking Holy Grails and Lineages back to Adam fall into similar categories, but the idea of a blockbuster jump in technology is something I have lived with most of my life. For me, there are two areas of computer technology that fall into the category of game changing or earth shattering or whatever analogies you would like to use. These two are voice and text recognition.

I have written recently about my experiences with voice recognition. With advanced software technology and computers with virtually unlimited computational speed, voice recognition (VR) is a reality. It has gone past the interesting experimental stage into a practical daily-use type tool. Presently, my iPhone and other devices talk to me on a regular basis and they haven't carted me away yet to the funny farm.

What about optical character recognition or OCR? I haven't said much about OCR for a while, mainly because it is ubiquitous. I guess I have taken it for granted for some time now. I can take almost any page of printed text and turn it into a computer text file in just a few seconds. As genealogists, we live with massive amounts of OCR text in the form of digitized books and newspapers. Most large businesses use some form of OCR to handle routine mail.

But what about handwriting? Some handwriting recognition software already is in use on a daily basis by such entities as the U.S. Postal Service's system for reading handwritten addresses and ZIP codes. But reading a document such as the probate inventory shown above is still an unattained goal. But it might be getting closer. I received a short note from Cliff Shaw at Mocavo.com. Apparently, Mocavo.com is working on some breakthroughs in handwriting recognition. Here is a quote from Cliff to me:

A little over a year ago, Mocavo acquired ReadyMicro and the incredible mind known as Matt Garner. One of Matt’s lifelong passions and curiosities is to enable computers to read historical handwritten documents to bring genealogy search to the next level. It’s well known in the genealogy industry that historical handwriting recognition is the Holy Grail – the single largest technological advancement that would enable more content to become accessible online (except for maybe the invention of the Web). For the past year, we’ve joined with Matt to tackle this very hard problem, and have finally made enough progress that we can begin to report on it.

 Mocavo.com is making an announcement today about their progress and I will let you read what they have to say in the complete press release or post.

I know they are not the only ones working on this technology, but they may be the only ones in the genealogy community that are pushing this technology in a practical way. For my part, I hope they keep at the problem and solve it. I have tens of thousands of pages of handwritten letters that could benefit from this technology.

Why are there no new genealogical database programs?

You may or may not have noticed that the current crop of genealogical database programs are all into higher numbers in their versions. For example, Family Tree Maker from Ancestry.com is up to Family Tree Maker 2014, sort of like the new model cars coming out a calendar year early. Most of the other products that are on version numbers are also crawling up there, such as:

• RootsMagic 6
• Ancestral Quest 14
• Reunion 10
• MacFamilyTree 7
• Heredis 2014

When I am talking about genealogical database programs, I mean those programs that you purchase for use on your personal computer. Of course, there are a multitude of choices for parking your family tree information online. But most serious genealogists still believe that having your own program on your own computer is preferable to relying on online sources.

In analyzing the genealogical database program market worldwide there seems to be a core of programs that have wide enough circulation and sales to appear as main participants in the genealogical community. There are, of course, dozens of other programs, mainly developed by individual programmers, or developed in the open source world of genealogy. Whenever I write about software programs, I always get comments about some of these other programs and how wonderful they are. One of these that is often mentioned as the program Gramps. But even that program is in version 4.0.2. 

I guess my question is why there appears to be so little innovation in new programs for genealogy? it seems to me that genealogy is reached the same plateau as word processing programs, spreadsheet programs, presentation programs, and all of the other categories of software that seem to have a few entrenched software products with no realistic challengers. In some of these areas it is obvious that any challenger to the leading products would have to have a massive amount of financial backing to go up against software providers such as Microsoft and Apple. But with the exception of offerings from Ancestry.com and MyHeritage.com, most of the genealogical database programs come from smaller software development companies.

In the area of Apple compatible genealogy software, the whole genre is vastly underrepresented. None of the Apple products for genealogy are actively promoted at conferences or in the genealogical publications. Even in the Windows realm, very few of the existing genealogy programs are actively promoted. Even at a big genealogy conference such as RootsTech, there are only a small handful of genealogy program vendors.

One argument for the reason that there are so few new programs is that the existing programs are more than adequate and there is simply no room in the market for new programs. However, what I think is more likely the case is that the innovations in the genealogy software industry are currently almost entirely directed towards online programs.

There are certainly some areas of the genealogy programs that need innovation. For example, in my experience none of the merge functions in the programs that I have used work more than marginally well. That is just one example of a long list of functions that are in most cases adequate but could be improved. Data entry and almost all of the programs is painfully slow and as I have recently pointed out over and again, moving data from one program to another relies on antiquated procedures.

My observations are not intended to be a criticism of the existing programs, some of which are extraordinarily useful and have some extremely beneficial features. But it is interesting to contemplate what might happen if someone approached the issue of recording family history from a holistic standpoint with economic backing to actually move into the forefront of the community.

It was not my goal in writing this commentary to include an exhaustive list of every genealogy software program available, so please do not feel offended if I did not mention your favorite program.