The Fatal Flaws of AI Chatbots

 

Even a diamond can have its flaws. Artificial Intelligence (AI) is big in the news right now but for all the hype, there is a growing undercurrent of realization that it has some perhaps fatal flaws. Unless these flaws are fixed the utility of the programs will remain superficial and only marginally useful for pursuits such as historical or genealogical research. Here are two examples of the flaws. 

Flaw #1: Lack of citations to the source for the information supplied.

This isn't important to people who are not doing historical research, but when the chatbot gives you a date and description for a historical event, you need to know it is accurate. 

Flaw #2 The citations, when present are superficial and sometimes inaccurate information and not to original records or sources. 

The only chatbot that attempts to give source citations in the form of footnotes is Microsoft Copilot. Even then, the citations are mostly heavily dependent on Wikipedia.com and other websites heavily covered in ads. So, the chatbot is merely acting as a search engine with a Natural Language Interface. Let me show an example using Google Gemini to ask a question about the history of Apache County, Arizona. 

Prompt: Tell me about the history of Apache County, Arizona including the names of some of the original settlers and the settler's background. 

Here is a screen shot of the response. You might note that there are only two dates: 1879 and 1882. You can click on the image to get a larger copy.

If you take the time to read this response, you might want to know a few other details that are missing. First, for the northeast area of Arizona, the first county created was Yavapai in 1864. This was followed by Apache County in 1879. Navajo County was created in 1897. The earliest European settlements in this part of Arizona (excluding Native Americans that date back thousands of years) were of the towns of Brigham City, Joseph City (Allen's Camp), Obed, and Sunset in 1876. Joseph City (originally Allen's Camp and then St. Joseph) is the only surviving town. Both Snowflake and Taylor were established in 1878. 

The earliest or oldest continually occupied is the town in Arizona is Old Oraibi, more than 800 years old. Here is one reference book about the colonization. 

Tanner, George S., and J. Morris Richards. Colonization on the Little Colorado: The Joseph City Region. 1st ed. Flagstaff: Northland Press, 1977.

AI misses too much to be really reliable with historical or genealogical research. That might change.