Handwriting Recognition, Ancestry.com, and FamilySearch.org, and Gemini 3

 

This post used AI to generate the above image and to support the research herein. Further, all the information in this post is supported by the short bibliography of sources at the end of the article. 

As a long time genealogist I have spent countless hours trying to decipher old handwriting, some dating back to the 15th Century and before. I have also studied Paleography and many different languages. I am painfully aware of the challenges faced by genealogists to decipher handwritten, valuable genealogical documents. I am now thankful to have lived long enough to see this perpetual obstacle to research partly resolved. 

Let's go back, with the assistance of Google Gemini 3, to a short history of handwriting recognition. 

1950s - Origins: The concept began with "Optical Character Recognition" (OCR) for machine-printed text. Early attempts to read handwriting (like the "Perceptron" neural networks) were largely theoretical or limited to single, clearly written digits.

1989 - The Neural Network Pivot: Yann LeCun and colleagues at AT&T Bell Labs developed the first effective Convolutional Neural Network (CNN) to read handwritten zip codes for the US Postal Service. This paved the way for modern deep learning.

1990s - The PDA Era: Devices like the Apple Newton and Palm Pilot popularized "online" handwriting recognition (tracking pen movement in real-time), though accuracy was often the subject of ridicule.

2010s - Deep Learning: The shift to deep neural networks allowed computers to recognize "offline" static handwriting (images of text) with much higher accuracy, enabling the transcription of historical documents.

What happened to supercharge this long history? The answer is the real breakthrough in 2025. The new Gemini 3.0 model has demonstrated a massive leap in handwriting recognition, achieving a 0.56% Character Error Rate (CER) on 18th-century manuscripts. This is considered "expert human" level and is a significant improvement over previous models (which hovered around 4% error rates).

Both Ancestry.com and FamilySearch.org have been implementing advanced handwriting technology. Once again, here is a summary to their attempts from Gemini 3. 

The application of AI in genealogy has been defined by two distinct but complementary approaches led by Ancestry.com and FamilySearch.

Ancestry.com focused its efforts on solving the problem of scale and speed, particularly for the release of the 1950 US Census. To handle over 150 million names, they developed a proprietary AI trained on documents that were digitally "aged" with synthetic coffee stains and rips, teaching the machine to read through historical damage. This system utilized a confidence score approach: the AI transcribed the clear text, and only flagged ambiguous entries for human review. This technology has since evolved into a consumer-facing "Transcribe" feature, allowing users to apply the same powerful recognition tools to their personal family letters and recipes.

In contrast, FamilySearch has focused on depth and context through their "Full-Text Search" experiment. Moving beyond traditional indexing—which only extracts specific details like names and dates—FamilySearch’s AI transcribes every single word on a document page. This shift fundamentally changes research by allowing users to search for occupations, descriptions, or incidental mentions (such as "blacksmith shop") rather than just names. This method is currently being aggressively expanded to unlock complex, unstructured documents like land deeds and wills, which have historically been the most difficult for computers to parse.

 These development add another layer of complexity to the role AI plays in actual genealogical research. AI is a tool, not a fad or a gimmick. However, I can only be useful if the user knows how to ask it questions and interpret the answers. The obvious extraction of this principle is that people who cannot read and write cannot use AI effectively and I would add that those people who lack advanced computer skills are also unable to use AI for accurate historical and genealogical research. This is not an elitist statement it is reality. 

Here are some supporting sources for the information above. 

“A Journey Through History: The Evolution of OCR Technology.” Accessed November 26, 2025. https://www.docsumo.com/blog/optical-character-recognition-history.

“Ancestry® Proprietary Artificial Intelligence-Powered Handwriting Recognition Technology Processes Over 150 Million Records from the 1950 U.S. Census in Only 9 Days | Ancestry Corporate.” Accessed November 26, 2025. https://ancestry-prod2.adobecqms.net/content/ancestry-corp/en-us/blog/ancestry-proprietary-artificial-intelligence-powered-handwriting-recognition-technology.

“Ancestry® to Apply Handwriting Recognition Artificial Intelligence to Create a Searchable Index of the 1950 U.S. Census.” Accessed November 26, 2025. https://ancestry-prod2.adobecqms.net/content/ancestry-corp/en-us/blog/ancestry-apply-handwriting-recognition-artificial-intelligence-create-searchable-index-1950-us.

“BREAKING NEWS Ancestry Released NEW Handwriting Recognition & Transcription from YOUR IMAGES.” Genealogy TV, August 27, 2025. https://genealogytv.org/breaking-news-ancestry-released-new-handwriting-recognition-transcription-from-your-images/.

FamilySearch Full-Text Search: A Genealogy Game Changer - Genealogy Bargains. Bargains. August 21, 2025. https://genealogybargains.com/familysearch-full-text-search/.

Guinness World Records. “First Neural Network to Identify Handwritten Characters.” Accessed November 26, 2025. https://www.guinnessworldrecords.com/world-records/760232-first-neural-network-to-identify-handwritten-characters.html.

“Performance and Its Stunning,” Google’s Large Model Rarely Creates “Momentum” before Its Release, Will Gemini 3.0 Debut This Week? “‘Performance and Its Stunning,’ Google’s Large Model Rarely Creates ‘Momentum’ before Its Release, Will Gemini 3.0 Debut This Week?” Accessed November 26, 2025. https://longbridge.com/enhttps://longbridge.com/news/266095506.

“The Release of Google’s Gemini 3.0 Pro Model Card Demonstrates Significantly Enhanced Multimodal Capabilities, Surpassing Competitors.” Accessed November 26, 2025. https://news.futunn.com/en/post/65096729/the-release-of-google-s-gemini-3-0-pro-model.

“The Writing Is on the Wall for Handwriting Recognition.” October 10, 2025. https://newsletter.dancohen.org/archive/the-writing-is-on-the-wall-for-handwriting-recognition/.

Wikipedia. “Handwriting recognition.” October 3, 2025. https://en.wikipedia.org/w/index.php?title=Handwriting_recognition&oldid=1314898544.

The Fatal Flaw of AI Chatbots

Even when AI chatbots are trained to do highly technical and accurate research such as that done by the recent release of Google Gemini 3, they all fall victim to a fatal flaw: they cite and trust online documents that are of themselves inaccurate and flawed. This flaw extends to other areas of chatbot response including the reflection of prejudices and bias from the LLMs used to train them. An image, such as the one above reflects a common prejudice and bias. The AI chatbot in a generated image is a humanoid robot. AI is not a robot, humanoid or otherwise. Here is a typical technical definition of a robot:

A robot is technically defined as an automatically operated machine capable of carrying out complex tasks, often substituting for human effort, and typically programmable to perform a variety of functions autonomously or semi-autonomously. Robots are composed of mechanical, electrical, and computer engineering systems that enable them to sense, process, and act within their environment. See https://www.meegle.com/en_us/topics/robotics/robotics-engineering

 Now, here is typical technical definition of a chatbot:

A chatbot is a computer program or software application designed to simulate human conversation through text or voice interactions, often using natural language processing (NLP) and artificial intelligence (AI) to understand user queries and respond appropriately. Chatbots can be rule-based, handling scripted responses, or AI-driven, learning from user interactions to provide increasingly accurate and context-aware answers.​ See https://www.ibm.com/think/topics/chatbots

Depicting a chatbot as a robot means that the viewer will associate the AI technology with robots and all their online and video history from "Metropolis" (1927) which featured the Maschinenmensch and was influential in popularizing the robot concept worldwide to I, Robot (2004) inspired by Isaac Asimov's stories.  I, Robot depicts a future where household robots rebel against humanity, driven by their interpretation of their core programming. Robots are consistently, with a few exceptions, depicted as untrustworthy and even dangerous. AI chatbots are NOT robots. 

I suppose that as AI continues to evolve at an ever increasing rate, the fatal flaw will be overcome by additional programming that alerts the AI chatbot to verify every source. When that happens, the chatbot will be able to do "real" research but not before. 

As the number of tasks mastered by the AI chatbots increases, their functionality as an advanced tool also increases. One of things we can do now is ask the AI chatbots if they can determine when a source website or record in wrong, misleading, or fabricated. Here is one response to that question or prompt from Google's Gemini 3. 

The short answer is no, not definitively. An AI chatbot cannot determine "truth" in the way a human expert or a court of law does. It does not have access to physical reality to verify if a document actually exists or if an event truly happened.

However, it can be a powerful tool for identifying inconsistencies, anomalies, and signs of unreliability.

Here is a breakdown of what AI can and cannot do regarding false or misleading records.

---

1. The Core Limitation: No "Ground Truth"

AI models are "probabilistic," not "deterministic." They predict the next likely word in a sentence based on patterns they learned during training.

• It doesn't "know" facts: If an AI reads 1,000 articles saying the moon is made of cheese, and only 1 saying it is rock, a basic model might conclude it is cheese.

• It cannot visit the source: An AI cannot walk into a courthouse to pull a physical will, nor can it call a witness to verify a quote. It relies entirely on digitized text and data provided to it.

2. What AI Can Detect (The "Red Flags")

While it cannot guarantee truth, an AI is excellent at spotting patterns that usually indicate a record is wrong or fabricated.

A. Logical Inconsistencies

AI is highly capable of analyzing data points for mathematical or chronological impossibility.

• Genealogy Example: "This record states the mother was born in 1900 and her child was born in 1905. This is biologically impossible."

• History Example: "This article claims Napoleon used a telephone. Telephones were invented after his death."

B. Stylistic & Linguistic Analysis

AI can analyze the tone of a source to determine if it is misleading.

• Sensationalism: Detecting highly emotional language or clickbait titles often associated with fake news.

• Formatting Errors: Identifying malformed URLs or poor grammar typical of phishing sites or hastily fabricated records.

C. Cross-Referencing (If Web-Enabled)

If the AI has access to the live internet (like Gemini or ChatGPT with browsing), it can compare the claim against trusted databases.

• If Source A makes a bold claim, the AI can check Source B (Wikipedia), Source C (Encyclopedia Britannica), and Source D (Academic Journals). If A is the outlier, the AI can flag it as "likely inaccurate."

---

3. Where AI Fails (The Dangers)

Critical Warning: AI can suffer from "The Echo Chamber Effect." If a fabricated story has been repeated enough times on the internet, the AI may accept it as fact because the data volume supports the lie.

• Fabricated Citations (Hallucinations): Sometimes, when you ask an AI to verify a source, it will invent a fake verification. It might say, "Yes, this is confirmed by the New York Times," when no such article exists.

• Deepfakes and High-Quality Forgeries: If a digital image of a document is perfectly forged (correct fonts, seals, and aging), the AI will likely accept it as valid. It cannot put the document under a microscope to check ink density or paper fiber.

• Data Poisoning: Malicious actors can intentionally feed bad data into open datasets. If the AI was trained on that poisoned data, it will confidently defend the falsehood.

---

4. Summary: AI vs. Human Verification

Feature	AI Capability	Human Capability
Spotting logical errors	High (Very fast at math/dates)	Medium (Prone to fatigue)
Verifying physical existence	None (Cannot leave the digital realm)	High (Can visit archives)
Understanding Context/Nuance	Medium (Misses sarcasm or cultural nuance)	High
Detecting "Vibe" or Bias	High (Can flag subjective language)	High
Susceptibility to Hallucination	High (Will invent facts to please user)	Low (Usually knows what they don't know)

How to use AI for Verification

Do not ask: "Is this record true?"

Instead, ask: "Please analyze this record for chronological inconsistencies, logical errors, or signs of bias."

Can all this be overcome? We will have to wait and see. Meanwhile, the above is a good way to judge that accuracy of any generated response.