It is with sorrow that I announce the shuttering of Decoding the Civil War. As many are aware, Decoding the Civil War has been on pause since January 2018. We first attempted to build a new crowdsourcing interface, called Phase 2, building on the hard-earned transcripts from you, our faithful volunteers. This interface asked that volunteers work with individual telegrams, identifying specific metadata such as the sender, recipient, date sent, time received, etc.
However, this tagging of individual telegrams ran into significant technical difficulties. Therefore, in respect to our volunteers, and after time and effort had been spent trying to fix the technical issues, it was decided this past June follow a new path. We will instead extract through text mining the data required. We are optimistic about the results as we are using the clean transcriptions you valiantly produced in Phase 1.
All our volunteers have done great work, and that work will continue to help and inform researchers for some time to come. You can see your hard work on the Huntington Digital Library in the Thomas T. Eckert Papers at: https://hdl.huntington.org/digital/collection/p16003coll11.
I am very happy to have known and worked with you, our volunteers. Your fervor, your determination, your industry, made Decoding the Civil War a great success.
One last time, three cheers for our volunteers!
Huzzah! Huzzah! Huzzah!
Decoding the Civil War Lead
It is that time of year again! Here is my New Year’s resolution: To make sure that the next meeting of the Symposium on Cryptological History features a panel on the Civil War codes.
The American Historical Association’s Zoë Jackson reports on this year meeting in the latest issue of the Perspectives:
“the two-day symposium reflects a greater truth about intelligence work: it is deeply indebted to the work of the code breakers who came before, and that cryptologic history should inform intelligence work being done today.”
It is about time that our good friends the cipher men of the United States Military Telegraph get the recognition they so obviously deserve!
Happy New Year!
Alternate history is fun. It is very satisfying to imagine what would have happened if Alexander the Great had not died young, or Napoleon had succeeded in cobbling together a world empire. It is no wonder that alternative history has evolved into a rich genre that allows its practitioners to indulge in historical fantasies or offer dire warnings.
Since the early 1900, the American Civil War has been occupying a sizeable chunk of this literary real estate, producing its own genre of Civil War alternate history (CWAH) –see Renee de Groot’s wonderful article Divided We Stand: “Confederate” and Civil War Alternate Histories.
This CWAH narrative admittedly is a bit repetitive as it tends to hinge on the Confederacy winning the war. This “what-if-ism” occasionally takes somewhat amusing forms: in his Man of War: My Adventures in the World of Historical Reenactment (Penguin, 2013), journalist Charlie Schroeder recounts his stint at the two-day annual reenactment the Brooksville Raid: on Saturday the Union troops win, but on Sunday, it is the Confederates turn, because, as one reenactor told journalist, the event’s coordinator, a Baptist minister, “wants the Confederates to win on the Lord’s day.”
CWAH traditionally entailed a rosy picture of the flourishing independent South inhabited by benevolent planters, graceful and tenderhearted females, and happy slaves. This fantasy was not the sole domain of the exponents of the “Lost Cause.” In his (exceedingly boring) novel Hallie Marshall: A True Daughter of the South (1900), Frank Purdy Williams, a progressive reformer, held up this imaginary “Southland” to contrast with the horrible labor conditions in the Gilded Age America. Ernest Howard Crosby, wrote “If the South Had Been Allowed to Go,” (1903) to speculate that if it had not been for the imperialist North, slavery would collapse in the South organically, without “the evils which we have entailed upon ourselves by the manner of its abolition.” Winston Churchill offered that if the Confederacy had triumphed, it would forge a “new fundamental relationship between master and servant” instead of the effort to “graft white democratic institutions upon the simple, docile, gifted African race belonging to a much earlier chapter in human history.”
At the turn of the last century, the picture of a Confederate victory took a darker turn. Harry Turtledove’s The Guns of the South (1992) has a supply of AK-47 delivered to the struggling Confederacy by a time-traveling group of South African racists. It became even more jarring as historians dispelled the once common assumptions that that the Union victory was preordained or that slavery would have ended on its own. CWAH embraced and rather realistic image of a United States resting on slavery and white supremacy, most vividly and chillingly portrayed by Ken Willmott’s in his mockumentary CSA: The Confederate State of America (2004) or in Ben Winters Underground Airlines (2016).
CWAH tends to think grand, largely in terms of great battles and great men. But what if we think on a less grandiose scale? For example, what if Anson Stager’s parents had decided to move South and the inventor of the USMT code had been born in Virginia rather than New York?
On one hand, he might not have even made it in telegraphy. He would not have been apprenticed to Henry O’Reilly at the Rochester Daily Advertiser, and quite possibly had no one to introduce him to telegraph business. Stager certainly would have never made it as the first general superintendent of Western Union.
On the other hand, had Stager been born in Virginia, entered telegraphy, and shown the same promise, he could have risen to the top of the American Telegraph Company which had lines in the North and South. (Western Union lines lay entirely within the Union). He could have associated with Dr. William Sylvanus Morris (1823-1893) of Lynchburg, Va., and, like him, become a Southern director for the company. Stager could have been present at the emergency meeting of the company’s executives on May 12, 1860 which split the American Telegraph Company into Southern and Northern branches. And, as a superintendent or even president of the Southern Telegraph Company, he would have taken an oath of loyalty to the Confederacy.
And, of course, it could have been the governor of Virginia John Letcher rather than Ohio’s William Dennison to ask Stager to develop the codes for the Confederate army. In the fall of 1861, Stager could have used his organizational talents to build up the Confederate Military Telegraph that would report directly to Jefferson Davis and his Secretary of War Judah P. Benjamin.
With the Confederate command in possession of superior telegraph communications and unbreakable ciphers, its chances of winning would have increased exponentially. It would certainly have minimized a possibility of Union soldiers coming across battle plans wrapped around some cigars, as it happened with Robert E. Lee’s orders to General D.H. Hill before the battle of Antietam. More importantly, it would have afforded unprecedented level of battlefield and logistical communications not to mention infinite possibilities to mess with the increasingly bitter partisan divisions in the North.
Could a simple accident of birth have changed the course of history? Is it impossible to imagine that should Stager’s life have taken a different turn, alternative historians would now be speculating what would happen if the North had won the war? Is it inconceivable that we could be now living in a world where Willmott’s CSA was an honest to goodness documentary?
Olga Tsapina, Norris Foundation Curator of American History, Huntington Library.
On June 21, 2016 we embarked on Decoding the Civil War, an ambitious project to transcribe 12,921 handwritten pages from 35 ledgers. These ledgers were from the archive of Thomas T. Eckert, head of the Washington office of the United States Military Telegraph, Assistant Quartermaster of U.S. Army, and Assistant Superintendent of the U.S. Military Telegraph.
The 35 ledgers record the telegrams sent and received by the War Department during the American Civil War. About one-third of the recorded messages in the ledgers were written in code, and another third may have never been published in the The War of the Rebellion: A Compilation of the Official Records of the Union and Confederate Armies (often simply abbreviated to OR), and there are more than 100 communiques from Lincoln himself.
As of this morning, November 15, 2017, we can announce that all the ledgers have been transcribed by our Volunteer Corps! All the difficult work of parsing sentences, figuring out 19th Century hand and spelling, and even dealing with difficult to read, smeared, pages has been completed in under 17 months. An incredible achievement. It would have taken 2 staff members working nearly full time 3 to 4 years to reach this goal!
We have begun to publish those ledgers on the Huntington Digital Library; 16 are already available for researchers. As we work though the huge amount of data provided by our Volunteer Corps, the remaining volumes will be published. We are excited to provide this new trove of American Civil War material to the Public, Researchers, and Students. The work completed has already been used to create educational materials that will be published shortly.
We began this project hoping to be finished in six months to a year. We learned quickly that it would take longer, that the task was full of challenges, and that tools needed to be built to overcome some of those challenges. Through it all our loyal volunteers stayed with Decoding the Civil War. As we move forward with the next phase of our project we will call again on those volunteers to help.
Right now, those volunteers can rest. All who have worked on Decoding the Civil War deserve thanks.
Strike up the Band! A lively polka is in order!
Thank you also to the National Historical Publications and Records Commission (NHPRC), which funded a two year grant for this project and to our institutions who have provided support throughout: The Huntington Library, Art Collections, and Botanical Gardens; North Carolina State University; and Zooniverse with its spectacular team at the University of Minnesota. Finally, a thank you to Daniel W. Stowell, whose advice, support and enthusiasm has been fundamental since the beginning.
After a year of hard work by our volunteers on Decoding the Civil War, Phase 1, we are about ready to launch Phase 2, the marking of metadata within specific telegrams. There are two work flows to this task. The first work flow, Code Words, is marking the arbitraries, or code words, for those messages in code. These coded telegrams will then be fed into Phase 3, the final decoding of the telegrams. Having the marked arbitraries should make the process of decoding much faster, possibly aided by computer algorithms.
The second work flow, Metadata, is a little more ambitious and complex, as we are asking our volunteers to work with individual telegrams, identifying specific metadata such as the sender, recipient, date sent, time received, etc. We are asking for metadata for a total of 10 fields; most telegrams have only a few; rarely do they have all 10. What we wish to accomplish is a way to provide simple metadata that will enable researchers to find all the telegrams to, say, Secretary of War Edwin M. Stanton, no matter whether it is in Ledger 2, or 6, or 22.
But did not Phase 1 enable full-text searching? Yes it did, and it is wonderful, but the transcriptions are accurate to the text as written in the ledger. Keeping with Stanton, if you typed in “Stanton” in the search box, you would get those pages where “Stanton” matches the search. But what if the telegram begins or ends with “EMS” or “Stantin” or “the Secretary of War”? The full-text search would ignore those pages. Furthermore, such a search looks at the whole message and returns results for any mention of “Stanton,” including other people named Stanton or places named Stanton. What if you want to look for Stanton only as the recipient? A search in a specific metadata field for “recipient” would enable that search and give you the correct results.
To aid in that search we will take the metadata tagged by the volunteers in Phase 2 and standardize the terms. So, continuing with Stanton, if the recipient is “EMS” and it is tagged as a sender or recipient, we will be able to take the consensus term and edit it to the standardized form of “Stanton, Edwin M. (Edwin McMasters), 1814-1869.” Once all the telegrams are tagged and the fields edited, if you do a specific search for “Recipient” as “Stanton, Edwin M. (Edwin McMasters), 1814-1869.” you will only get those telegrams to Stanton, not from or about him, and you will have those whether they are sent to him as “EMS,” “Stanton,” or “Stantin.”
The tagging of individual telegrams in the Phase 2 Metadata workflow will eventually enable specific searches to be done across the almost 16,000 telegrams. It will enable users to look for individuals or places or dates in specific fields. And the tagging of code words (arbitraries) in the Code Word work flow will help round out this project with the final decoding of encoded telegrams. An incredibly useful archive has been made available in Phase 1 of Decoding the Civil War. Help us leverage and categorize that hard-earned knowledge in Phase 2 to aid in the discovery of the American Civil War.
From Washn. Oct. 31’62
For Geo H Bangs Adams HdQrs
Madame Williams wife to the Clerk
of Major Cameron Paymaster now with
Army of Potomac passing off is
a South Carolinian was sent for
to join her husband & information
says has letter for the south
quilted in her chemise or skirts
is to go through the lines
& will also Carry intelligence she
ought to be found & searched
As also her husband signed E
J Allen cut open her things
What makes a war civil? The Romans who invented the term ‘civil war’ (bellum civile) defined it as a war among fellow citizens (cives). The same Latin root element that produced the adjective describing the worst and most barbaric type of warfare also created the notions of “civility” and “civilization.”
With the outbreak of the Civil War, these last notions seemed to come crashing down. The values that Americans had long cultivated – national unity, moderation, compromise, aversion to secrecy and subterfuge, and what Abraham Lincoln called the “mystic cords of memory,” were swept away under the combined onslaught of sectional loyalties, disputes over slavery, and military necessity.
Yet some foundations held. The shared bonds of personal friendship, marriage, and family, transcended sectional divides and political interests. Officers and soldiers on both sides held the ties of manly friendship sacrosanct, and both Union and Confederate authorities felt compelled to stick to the norms of “civilized” warfare or at least aspire to it. Among these norms were the respect for the obligations imposed by marriage and family and the codes of behavior dictated by ideals of gentlemanly manliness and feminine gentility.
Thus, despite ever growing concern about smuggling and espionage, neither side attempted to close off the borders and ban traveling and communications between divided families. Regulations put in place by the beginning of 1862 by both sides allowed family members to cross the lines, provided that they obtained a pass from a provost marshal or the Secretary of War. The standards of vetting the applicants were rather loose; often they did not extend beyond having the applicant to give his or her word not to do anything untoward.
The word of a lady carried additional weight. As historian Amy Murrell Taylor (The Divided family in Civil War America, University of North Carolina Press, 2005) shows, women could count on preferential treatment. No honorable man could ignore, or decline, a plea of a dutiful wife, devoted sister, or a loving mother. Treated as “the weaker sex” and urged not to bother their pretty little heads with the filth of politics, ladies could also plead detachment from the war which was, after all, a man’s business. As one woman with admitted Confederate sympathies remarked: “I think inasmuch as ladies did not make this war, they are silly in the extreme to mix themselves up in it.” Being a lady was synonymous with integrity and trustworthiness.
A lady was entitled a respect that no gentleman, Union or Confederate, could fail to honor. Yet there was growing evidence that ladies were deeply involved in the conflict. In early 1862, the New York Tribune, fuming over the fact that “nearly every instance” of a pass granted to a lady ended up in her carrying “letters and other documents” hidden in her clothes, called for a complete ban on the passes issued to women. The Union authorities, however, never went as far as to dismantle the system. The most they could do was to issue fewer passes, even despite what Halleck lamented in 1864, as “a superabundance of female spies among us now.”
With the exception of such celebrities as Rose O’Neal Greenhow, Belle Boyd, Elizabeth Van Lew, or Pauline Cushman, the information about these female spies is hard to come by. A historian is often left with limited information offered by applications for passes, arrest records (in case a woman spy was ever caught), or an occasional mention in a diary of a private letter. For example, a letter from the Huntington’s collection of papers of New Yorker John Burrud relates of “one woman” who was “allowed to go with the army. She is a Capt. Wife on Gen. Sheridan’s staff. She acts as a spie. She is very successful in her business. She will go all through the Rebel army, and is of great service to the Union cause. This must be kept in secret of course.” (John Burrud collection, mssHM 75242 1864, Sept. 23).
The archive of the United States Military Telegraph offers a new and untapped resource for historians. To wit: the above telegram.
The message is addressed to George H. Bangs (1831-1883), a superintendent of the Pinkerton Detective Agency. The sender, “J. Allen,” was Bangs’ boss, Allan Pinkerton. Contrary to Pinkerton’s later claims, he did not head the United States Army Secret Service. Rather, much like Anson Stager, Pinkerton was an entrepreneur contracted to perform a vital service for the Army of the Potomac. In the first months of the war, his old friend George B. McClellan asked him to head his intelligence operations. Because Pinkerton’s name was so familiar, he operated under the pseudonym Major E.J. Allen.
Pinkerton turned out to be patently unqualified to conduct military intelligence. As a private detective, however, he proved very adept at exposing spies and saboteurs. It was he who exposed Rose O’Neal Greenhow as a Confederate spy, a feat that he accomplished by utilizing his surveillance skills honed working divorce cases.
“Madame Williams,” a mere paymaster’s clerk wife, is, of course, harder to place than the Washington socialite. She would have received her pass from the office of the Provost Marshal of Washington D.C., and it’s possible that her application file could still be located among the papers of the Provost Marshal at the National Archives and Records Administration (RG-110.2)
The office was then headed by Lafayette C. Baker who also presided over the National Detective Bureau, another private outfit employed by the War Department. It is possible that it was Baker who had alerted Pinkerton to the danger posed by a clerk’s wife.
“Madame Williams” indeed could do considerable damage. Paymasters, i.e. military accountants charged with distributing soldiers’ pay, reported to the office of Quartermaster General. They were not attached to particular regiments, but rather shuttled between Washington and the various army units. Since her husband’s boss, Major Bruce Cameron, was a paymaster assigned to the Army of the Potomac, “Madame Williams,” a “South Carolinian,” would have ample opportunity to gather and transmit intelligence, as she accompanied her husband on his errands.
No wonder Pinkerton raised the alarm an instructed his employee to dispense with the norms of civility due to a lady and “cut open her things,” including her “chemise and skirts.”
Two weeks later, Pinkerton, who was fiercely loyal to George B. McClellan, would leave the army, to protest his friend’s removal from command.
Thank you Decoding the Civil War Volunteer Stork for finding this gem!
Olga Tsapina, Norris Foundation Curator of American History, Huntington Library.
Olga Tsapina, Norris Foundation Curator of American Historical Manuscripts.
On 10 a.m. July 4, 1863, the United States Military Telegraph transmitted this notice:
“The President announces to the country that news from the Army of the Potomac, up to 10 p.m. of the 3rd. is such as to cover that Army with the highest honor, to promise a great success to the cause of the Union, and to claim the condolence of all for the many gallant fallen. And that for this, he especially desires that on this day, He whose will, not ours, should ever be done, be everywhere remembered and reverenced with profoundest gratitude. ABRAHAM LINCOLN.”
This was the President’s press-release of the victory at Gettysburg. Two hours later, Ulysses S. Grant accepted the surrender of the Confederate army at Vicksburg, Miss. The double victory of Gettysburg and Vicksburg, made July 4, 1863, in the words of historian James McPherson, “the most memorable Independence Day in American history since the first one four score and seven years earlier.”
Yet, apart from the traditional celebrations on the White House grounds, there were few signs of jubilation elsewhere in the Capital. The immediate reaction to the battle of Gettysburg was a mixture of relief and frustration at the failure to capture Lee’s army. It would take three days for the news of the fall of Vicksburg to reach Washington.
The work of the United States Military Telegraph proceeded at a frantic pace, as a team of operators were busy receiving, sending, and routing messages. On 6 p.m., for example, one operator entered a message in his ledger (EC 6), reporting the progress (or rather lack of thereof), General William F. Smith’s unsuccessful pursuit of Lee’s army.
6 P.M. Harrisburg Pa. July 4, 1863
For SecWar Genl Smith on his
march met a flag of truce
two miles from Mt. Holly on
the Bendersville road with two thousand
prisoners which he recd & sent
the escort back trusting that the
exaggerated idea which they had of
his numbers would have a good
effect he delayed his march for
for two hours as his route was
thus discovered he says with
references to news “nothing unfavorable”
Signed L. Thomas Adjut Genl
Sent to Fredk & Baltimore 7 P.M.
At the same time, another operator, his own ledger (EC 8) on his desk, was receiving this message from Dana Couch to General Meade:
6 PM Harrisburg Penna July 4,1863
for Genl Meade Genl Smith’s
advance in the mountain
passes beyond Mount Holly
met two thousand paroled
prisoners from your army
under Escort period Smith
being discovered recd the
prisoners I will send
them to Camp at
West Chester Signed D. N. Couch
Not sent to Meade. One from Thomas gives same information.
Thirty minutes later, Thomas T. Eckert transmitted this cipher from the Secretary of War Edwin M. Stanton (Arabia) to William H. Ludlow, the Union commissioner for exchange.
Sheldon Ft Mon. Wash DC July 4, 1863
Rosetta Venus Lud-low you will
if it has not already been
done forward to Adorn by Express
the copy of Jeff Davis’ dispatch
sent him today also my telegram
of this Evening and until you
receive the Adam’s instruction hold no
communication with Mr Stephens or Mr
Ould nor permit either of them
to come within our lines period
Our Victory is complete Lee in
full retreat Arabia Bully
The blotted out “Bully,” (meaning “Well done!”), at the end of the message is one of the few signs of celebration at the War Department.
Yet the telegram showed that the victory at Gettysburg had already born results. Alexander H. Stephens, the vice-president of the Confederate States of America and Robert Ould, chief agent for prisoner exchange of the Confederate Army, were envoys of the President of the Confederate States of America. Jefferson Davis dispatched the two on July 2, in anticipation of Lee’s successful invasion of Pennsylvania. Lee’s victory was expected to force Lincoln’s hand not only in the negotiations over cartel of the prisoner exchange — suspended in retaliation for the harsh treatment of African American prisoners of war — but in recognizing the independence of the Confederate States of America.
Stephens acted as the bearer of the official communication addressed to Lincoln and intended to start international diplomatic negotiations “which public law recognizes as necessary and proper between hostile forces.” The communication was filled with complaints about “numerous difficulties” in “the execution of the cartel of exchange,” for which Davis, unsurprisingly, squarely blamed the Union side. More ominously, Davis accused Lincoln of war crimes: some of “your officers” asserted “a right to treat as spies” and execute prisoners of war, i.e., “the military officers and enlisted men under my command who may penetrate into States recognized by us as our allies.” Davis referred to the execution of William Orton Williams, a staff officer in Braxton Bragg’s command, and a cousin-in-law of Robert E. Lee, and his adjutant, Walter G. Peters. Captured by in Tennessee, while impersonating Union officers, Williams and Peters were hanged on June 9 on the order of William S. Rosecrans.
Rosecrans had acted in accordance with the Instructions for the Government of Armies of the United States in the Field, a.k.a. Lieber Code, the first United States Army field manual, (and a foundation of the Hague Conventions of 1899 and 1907 as well as the 1949 Geneva Conventions). The Code distinguished between a prisoner of war, “a public enemy armed or attached to the hostile army for active aid, who has fallen into the hands of the captor, either fighting or wounded, on the field or in the hospital, by individual surrender or by capitulation” (Section III, Article 49), and a spy defined as “a person who secretly, in disguise or under false pretense, seeks information with the intention of communicating it to the enemy.” (Section IV, Article 88). While a prisoner of war was “subject to no punishment,” nor any acts of “the intentional infliction of any suffering, or disgrace, by cruel imprisonment, want of food, by mutilation, death, or any other barbarity,” a spy was “punishable with death by hanging by the neck, whether or not he succeed in obtaining the information or in conveying it to the enemy.” The execution of Williams and Peters was, in fact, the lead item in the July 4, 1863 issue of the Harpers’ Weekly:
Stephens’ mission hinged on the success of Lee’s invasion of Pennsylvania. But when Stephens and Ould arrived to Fort Monroe in the afternoon of July 4, they were greeted by the news of the Union victory at Gettysburg. When Stanton found out about their arrival, the Secretary of War “swore and growled indignantly.” He then fired off the above telegram, instructing Ludlow to forward Davis’ message to John A. Dix (Adorn) and refrain from any communication with the envoys until he received Lincoln’s instructions.
Lincoln’s instructions were telegraphed a couple of hours later. The President of the United States refused to meet with Stephens. Stating that all questions having to do with prisoner exchange were to be resolved through “Military channels,” he added that “nothing else, will be received by the President, when offered, as in this case, in terms assuming the independence of the so-called Confederate States.” (Collected Works of Abraham Lincoln, vol. 6, p. 315)
On July 7, after the news of the fall of Vicksburg reached Washington, a procession with bands of music marched to the Executive Mansion. The crowd cheered as the President marveled at the double victory in Pennsylvania and Mississippi marking the “birthday of the United States of America. It was no accident that “the cohorts of those who opposed the declaration that all men are created equal” should “turn tail and run” on the very same day when, “eighty odd years” ago “for the first time in the history of the world a nation by its representatives, assembled and declared as a self-evident truth that ‘all men are created equal.'” Lincoln then added as an afterthought: “Gentlemen, this is a glorious theme, and the occasion for a speech, but I am not prepared to make one worthy of the occasion.” He finally delivered that speech on the Gettysburg battlefield on November 19, 1863.