# The Bixby letter: who composed it

Report: https://historical-mysteries.netlify.app/bixby-letter/report (copy in `report.html`; the sections it was assembled from are in `report_sections/`).

## The mystery, in brief

In September 1864 a Boston widow, Lydia Bixby, showed William Schouler, the Adjutant General of Massachusetts, five letters, each reporting the death of one of her sons in the Union army. Governor John A. Andrew asked the War Department for a letter from the President. On 21 November a letter of 139 words signed "A. Lincoln" was sent to her, and on 25 November the Boston Evening Transcript printed it. It has been reprinted and anthologised ever since, and it was read aloud in the film Saving Private Ryan (1998). Its facts were wrong: two of the five sons had been killed. The error was Schouler's.

The manuscript has never been found. The newspaper text is the only authority for the wording, and the "facsimiles" sold from 1891 are forgeries. Lincoln signed the letter. His 26-year-old private secretary John Hay (later Secretary of State, 1898 to 1905) drafted many letters for Lincoln's signature, and after Hay's death several men said he had told them in confidence that this one was his. None wrote it down in his lifetime; the earliest record is from 1922. Hay himself wrote in 1904 that "the letter of Mr. Lincoln to Mrs. Bixby is genuine" and told Robert Lincoln, the President's eldest son, that he knew nothing of the letter's source.

The sides. For Hay: drafting was his job; "beguile", the letter's rarest word, is his and never Lincoln's; a clipping of the letter is pasted in two of his scrapbooks. The case was built by Michael Burlingame, author of the fullest modern life of Lincoln (1995, 1999), and seemed settled in 2019 when Jack Grieve, a linguist at the University of Birmingham, and six colleagues applied n-gram tracing, a technique that counts how many of a text's short strings of letters and words occur in each candidate's known writing, and found Hay "almost certainly" the author. For Lincoln: he wrote his own condolences (both that survive from his presidency are in his hand); the request came through the Secretary of War with a governor's wish that the President write; and Hay's own statements point away from Hay. The case was argued by F. Lauriston Bullard, a Boston editor (1946), by Joe Nickell, an investigator of questioned documents, with the linguist Jean Pival (1989), and by Jason Emerson, a historian of the Lincoln family (2006, 2008). Roy P. Basler, editor of the standard edition of Lincoln's writings (1953), printed the letter as Lincoln's and called the claim for Hay "unproved".

How it was judged. The printed record was read in full, every reported claim was dated and traced from mouth to print, and the documents of the week were counted from Basler's handwriting notes. The 2019 technique was rebuilt from the paper and checked against the paper's results (they reproduce), then tested in three ways the paper did not (date, kind of writing, manuscript against print), with a second, independently written implementation of the tests as a check. The technique and three other standard methods were then tried on the known letters Hay wrote for Lincoln's signature; none detects them. With the stylometry treated as neutral, the documentary items were weighed as explicit likelihood ratios, each with a stated weight and a sensitivity table.

## Verdict and confidence

The question is unsettled, and the evidence leans narrowly to Lincoln. Our judgement: Lincoln composed the letter, about 53 percent; John Hay, about 42; another hand in the office, about 5. Lincoln against Hay alone: 56 to 44.

Every percentage is conditional. It comes from the documentary items weighed as explicit likelihood ratios (report, section 5), and the weights are judgments. Changing any one weight moves Lincoln between 26 and 75 percent; taking the 2019 stylometry at face value puts him at 11. The sensitivity table is printed beside the verdict. Nothing in the report excludes Hay, and nothing in it establishes that Lincoln wrote the letter.

Label: Unsettled. No answer is clearly ahead of the others; the report still says which one it favours.

## How it stands to earlier views: shifts, provisionally

The question was unsettled before this report and is unsettled after it. Basler's 1953 headnote called the claim for Hay "unproved"; Jason Emerson (American Heritage 2006; Lincoln Herald 2008) and the Lincoln scholar Harold Holzer (review on the history website Emerging Civil War, 23 July 2026) still call the question open. Within that unsettled question the balance had tilted to Hay since 2019, on Michael Burlingame's articles ("New Light on the Bixby Letter", JALA 1995; "The Trouble With the Bixby Letter", American Heritage 1999) and on the n-gram study by Grieve, Clarke, Chiang, Gideon, Heini, Nini and Waibel ("Attributing the Bixby Letter using n-gram tracing", Digital Scholarship in the Humanities 34, 2019), which called Hay "almost certainly its primary author". On the other side stood F. Lauriston Bullard (Abraham Lincoln and the Widow Bixby, 1946), Joe Nickell with Jean Pival (Lincoln Herald 1989) and Emerson.

What shifts is the tilt. The 2019 attribution reproduces, but its validation does not carry over to texts of the letter's period and register: under the paper's design between 19 and 30 of 63 register-matched Lincoln pieces go to Hay, the Gettysburg Address among them, and the design cannot detect Hay writing as the President. Two implementations of these tests, written independently of each other, agree on the direction (19 of 63 and 30 of 63 on the same pieces; on 97 pieces the second implementation chose itself, 27 by its own code and 28 to 40 by the first's, depending on the pool). With the 2019 study treated as neutral, what remains is documentary and near a balance, leaning to Lincoln on judgment weights.

Why provisional. Both implementations reconstruct the paper's design on corpora the paper did not use (its own corpus could not be obtained); the size of the register effect depends on the Lincoln pool; and no one else has rerun the tests. A rerun on the paper's own corpus by a third party would settle the rate. If it found no register effect, the label should return to "confirms" and the 2019 tilt to Hay would stand.

## The 2019 study: three separate findings

1. The original result reproduces, in both implementations. Under the study's own design (Lincoln before 18 May 1860 against all of Hay) the letter goes to Hay at all seventeen string lengths, as the paper reports.
2. The study's own validation reproduces. In the first implementation every one of 1,011 known texts of 50 to 400 words (423 Lincoln, 588 Hay) is attributed correctly by the majority vote of the 4- to 10-character analyses. In the second, 1,380 of all 1,397 texts in its pools are, with the paper's accuracy curve (its F1 score) at every length; the misses are Lincoln's verse, his 1842 eulogy, his temperance address and reported speech.
3. Under tests the paper did not run, the attribution is sensitive to period, register and edition. Period: Lincoln's own-hand wartime texts of 100 to 200 words go to Hay 14 percent of the time under a pre-1860 Lincoln pool (18 percent at 50 to 100 words), and the two Lincoln periods are separable at 100 percent. Register: 30 of 63 register-matched Lincoln pieces (about 140 words each, from 30 documents of 1861 to 1865) go to Hay under that design in the first implementation and 19 of 63 in the second; the own-hand design (Lincoln's writing in his own hand from 1860 to 1865 against Hay's letters and diary of the same years) misses 17 of 63 and none of Hay's 57. Edition: 95 Lincoln texts present in both Basler (manuscript) and Lapsley (print) sit 0.010 to 0.016 nearer Hay in the print version at every string length.

The original result reproduces, and the added tests show that its attribution is sensitive to period, genre and editorial differences. That is a robustness result, found twice in the same direction. It is not an established reversal.

Why the two rates differ (report, section 4). The gap is in the Lincoln pool, not in the code. The first implementation's code on the second's exact pool gives 19 of 63 under the paper's sample rule with 50 random sequences and under the first's rule with 20, and 20 with 10; the second's code on the first's pool gives 26; the first's pool under three seeds gives 28 to 30. The one difference that matters is 19 items from the third and fourth volumes of Arthur Brooks Lapsley's 1905 edition of Lincoln's writings, 83,000 words of Lincoln's 1858 campaign speeches and his side of the debates with Stephen A. Douglas, which the second implementation dropped because Lapsley prints a few hundred words of Douglas inside some of them. Without those items the first implementation's code gives 18 to 20 of 63; with them restored to the second implementation's pool (Douglas's words cut out), the second's code gives 27 of 63 (`independent-rerun/results/summary_debates.txt`). The mechanism is the method's equal-size sampling: the debates dilute Lincoln's sample with oratory (24 against 20 wrong at an equal 159,000 words) and enlarge Hay's sample by the same amount (29 at 243,000); Hay's sample enlarged alone sends 54 of 63 to Hay. The paper's own Lincoln corpus (1,085 texts, 400,747 words, speeches included; samples of 261,000 words) is nearer the larger pool than the smaller, so 30 of 63 is the nearer reconstruction of the paper's design, but neither pool is the paper's corpus. The range 19 to 30 of 63 (30 to 48 percent) is what the evidence supports. Runs: `analysis/diag_gap.py`, `analysis/results/diag/` (README.txt there explains the file names; `converse_their_code_our_pool/` holds the second implementation's code run on the first's pool).

Limits of the added test set. The 63 Lincoln pieces come from 30 documents. The 57 Hay pieces come from three documents: "Ellsworth" (Atlantic, July 1861), "Colonel Baker" (Harper's, December 1861) and the 55-word Leonard condolence of 1864. Document-level validation (leave-one-source-out) is good, but on the Hay side it tests only three sources, and a Hay eulogy is not a Hay condolence. The narrow Hay range limits how far the register result generalizes. The second implementation's own set (97 pieces from 38 Lincoln documents, 19 of them not among the first's 30; Hay's two essays cut afresh; 185 pieces of Hay's 1861 to 1865 letters and diary) has the same Hay limit.

Credit. The concern about register was the authors' own. Their first presentation (Aston University, LSS Seminar Series, 13 April 2016; slides in `dossier/grieve2016_lss_bixby_slides.pdf`) says they had "not looked at detail at the effect of ... register variation" and "also need to test sensitivity to register variation", and its last slide lists testing "across different registers" as work to do. The paper says "a systematic analysis of the writings of many authors and many registers is needed". The new contribution is the specific test, not the concern.

## What this report adds

No new evidence. No document was found that was not already in print. Everything new is analysis of evidence already published.

- The first replication of the 2019 study and of its validation, now done twice.
- Three tests of that design's sensitivity (period, register, edition), which its authors had named as untested. The register test exists in two independent implementations, which agree on the direction and differ on the rate, and the cause of the difference is traced.
- A corpus design built from Basler's source codes, so that Lincoln's side holds only what survives in his hand from 1860 to 1865 in the three volumes of Basler's edition that could be read, IV, VII and VIII (1,887 items, 125,600 words), and a register-matched Hay set from his 1861 war prose (attribution confirmed from both magazines' indexes).
- The detection test. Under the own-hand design 18 of the 19 documents in Basler's edition of 50 to 200 words that survive in Hay's hand, three of Hay's four known letters for Lincoln's signature (to the poet George H. Boker, 1863; to the abolitionist William Lloyd Garrison, Congressman John F. Driggs and Charles Butler of New York, 1864 to 1865) and Hay's own condolence of 1864 are called Lincoln by the n-gram vote; the calibrated classifier (a statistical classifier trained on short character strings) and Burrows's Delta (a distance measure based on the most common words) call all five Lincoln. The second implementation finds the same under period-matched pools of its own (all four letters to Lincoln, 7-0, 7-0, 6-1, 7-0; the condolence to Lincoln). The methods cannot detect Hay writing as the President, so a result on the letter under them is not evidence for either man. Stylometric likelihood ratio taken as 1 (range 1:3 to 3:1).
- Under the own-hand design the letter itself splits eight lengths to Hay and nine to Lincoln (the 4- to 10-gram vote is Hay, six to one); the calibrated classifier gives P(Hay) 0.29 to 0.41 (Hay's own register pieces average 0.66); Delta says Lincoln. Under the second implementation's period-matched pools it splits 5 to 2 for Lincoln on the character vote and 9 to 8 for Hay over all 17.
- A word and phrase table across four corpora: "beguile" is Hay's (0 in 338,000 words of Lincoln); "tendering to you", "the files of the War Department" and "should attempt to" occur only in Lincoln's own hand; "the altar of", "solemn pride" and "gloriously" only in Hay, mostly after 1865; "tender" as a verb is Lincoln's (22 to 0 in the wartime corpora).
- The documentary record read in full (Bullard's book; Sherman Day Wakefield's pamphlets; the memoirs of Nicholas Murray Butler and John Morley; Kendall Banning's 1922 article; Schouler's history; the 1908 Hay letters and diary; Thayer's biography of Hay; Burlingame 1995 and 1999; Emerson 2006 and 2008; Nickell 1989) and weighed as likelihood ratios with a sensitivity table.
- A count of Basler's items for every week of the autumn of 1864: the Bixby week has 15 against a thirteen-week mean of 21, 23 in election week and 35 in the week of the annual message. Bullard's answer to the "too busy" argument, with a number.
- The observation that the Phillips letter of 21 November 1864, kept beside the Bixby clipping in Hay's Library of Congress scrapbook, survives only in "an unidentified handwriting" that is not Hay's (Basler, CW VIII, 118 n.), which cuts against reading the scrapbook as a record of Hay's own compositions. Not found pointed out before.

## What it confirms

The hearsay character of every reported Hay claim (five chains, none first-hand, none written down before 1922: Bullard 1946, Emerson 2008); Hay's letter to Chandler of 19 January 1904 and what he told Robert Lincoln (Emerson 2008); Hay's practice of writing routine letters for Lincoln's signature, including four in the Bixby week (Basler); that the 2019 result reproduces under its own design and passes its own validation; that the method's sensitivity to register needed testing (Grieve and colleagues, 2016).

## What would overturn it

Toward Hay: a draft or claim in Hay's hand; the manuscript in Hay's hand over Lincoln's signature; a pre-1864 Hay use of "beguile you from the grief", "the altar of freedom", "solemn pride" or "assuage the anguish"; a first-hand record from Morley, Page or Brownell dated before 1922; the Phillips copy identified as Hay's hand; a method that still says Hay after validation on Hay's letters for Lincoln's signature; a rerun of the register test, on the paper's own corpus, that finds no register effect. Toward Lincoln: the manuscript in his hand with corrections; a statement by Hay that Lincoln wrote it; a Lincoln use of "beguile".

## Where it falls short

- The added tests exist in two implementations, written independently of each other, and no one else has rerun them; neither used the paper's own corpus, which could not be obtained (the University of Michigan site refused the connection).
- The size of the register effect depends on the Lincoln pool (19 to 30 of 63); the report gives the range and its cause, not a single figure.
- The Hay side of the register set is three documents, and Hay's condolence writing of the 1860s is one 55-word letter. A proper test of the hypothesis needs Hay condolences that do not exist in print.
- No archive was visited and no new document was found (Hay Papers, Morley papers, Bixby family papers unsearched).
- Basler's volumes II, III, V and VI (1848 to 1863) were lending-only and are absent from the Lincoln corpus; the McCullough condolence enters the register set from a printed text.
- Burlingame's 2000 chapter in At Lincoln's Side (the fullest case for Hay) and Edward Steers's Lincoln Legends (2007) were not available; the 2000 chapter is cited through Emerson and Burlingame's later summaries. Hay's letter to Spencer of 25 November 1864 is quoted from Burlingame's quotations.
- The impostors method (a check that adds many unrelated writers to the comparison) did not finish and is not in the report; `analysis/results/other/imp_ownhand.json` does not exist.
- The likelihood ratios are judgments; a reader who changes several at once can end the other side of 50.
- Searches for post-2019 work could not reach Project MUSE, JSTOR, HathiTrust or Google Books; no replication, re-analysis or rebuttal was found elsewhere (dossier/since_2019.md).

## Files

- `report.html`: the published report. `report_sections/`: the HTML fragments it was assembled from (template CSS shared with the other reports in the collection).
- `figures/`: the two charts in section 4 of the report, with the plotted values beside them. figure1.svg and .png (the letter's margin at each string length under the two designs) are drawn from `analysis/results/grieve_specials_50/results.json` and `analysis/results/ownhand_specials_50/results.json`, values in figure1_data.csv; figure2.svg and .png (the method's accuracy by string length on ordinary Lincoln texts and on the register-matched pieces) from `analysis/results/grieve/summary.json`, `grieve_register/summary.json`, `ownhand/summary.json` and `ownhand_register/summary.json`, values in figure2_data.csv. Both are drawn by `analysis/make_figures.py` (`python3 analysis/make_figures.py analysis/results figures`).
- `independent-rerun/`: the second implementation's own README, report, code, data and results, written by its authors, not edited here.
- `dossier/`: the documentary notes with page citations: hay_claims.md (the five chains), hay_own_statements.md (everything Hay wrote about the letter, with the Chandler letter in full), route_of_letter.md, lincoln_calendar_nov1864.md (every Basler item 17 to 28 November and the weekly counts), hay_nov1864.md, lincoln_case.md, hay_case.md, since_2019.md, errors_and_disagreements.md (twenty factual conflicts among the sources); grieve2016_lss_bixby_slides.pdf (Aston, 13 April 2016) and grieve2017_cl_bixby_slides.pdf (Corpus Linguistics 2017, 26 July 2017), the study's authors' two presentations.
- `corpus/`: lincoln_basler.jsonl (3,535 items from Basler vols I, IV, VII, VIII with source codes and hand flags), hay.jsonl (895 items, 502,000 words), controls.jsonl (20 writers, 1,180,000 words), hay_warprose.jsonl (the 1861 essays), README.md (every source, parsing rule and count), scripts/ and raw downloads.
- `analysis/`: ngt.py (n-gram tracing), designs.py (the corpus designs), run_ngt.py, build_register.py and register.json (the register-matched set), build_paired.py and paired_tests.json (the edition test), run_other.py with delta.py, clf.py, impostors.py (other methods), features.py (Pival's markers), bixby_phrases.py (the word table), lr.py, register_summary.py, make_tables.py, diag_gap.py (the gap between the two implementations); results/ (final runs; results/diag/ the gap runs), results_v1/ and results_v2/ (earlier pool versions, kept for the record), logs/.
- Reproduce: `python3 run_ngt.py ownhand --nseq 10 --loo --out results/ownhand` (also grieve, ownhand_all); `--nseq 50 --tests specials` for the letter; `--tests register` for the register set; `BIXBY_EXTRA_TESTS=paired` for the edition test; `python3 run_other.py ownhand delta,clf`; `python3 register_summary.py`; `python3 make_tables.py results`; `python3 diag_gap.py theirs --frac 1.0 --nseq 50` (and the other variants in results/diag/README.txt) for the gap. Python 3.11 with numpy, scipy and scikit-learn; the leave-one-out runs take about 40 minutes each on four cores, the diag runs four to ten minutes each.