One Website is dedicated to locating these errant ancestors. Appropriately named, BlacksheepAncestors.com is described as follows:
Welcome to BlackSheepAncestors.com! Here you will find genealogical prison records and insane asylum records for the US as well as genealogical prison records for the UK. You will also find numerous links to historical court records, execution records and biographies of famous outlaws & famous criminals across the US, UK and Canada.The categories of records include prisons and convicts, outlaws and criminals, court records and executions searches. The site also has an extensive list of pirates, buccaneers and privateers.
Sources such as the Family History Library have extensive records of Civil War Prisoners, but lack records such as those of the Arizona State Prison, as an example. The Black Sheep Website claims to have the Arizona Inmate Register of over 30,000 prisoners between 1872 and 1972. This Website may just solve one of those persistent mysteries in your pedigree.