# Beale-Ciphers Reproducible Analysis [![Cryptologia DOI](https://img.shields.io/badge/Cryptologia-10.1080%2F01611194.2026.2698071-blue)](https://doi.org/10.1080/01611194.2026.2698071) [![Zenodo DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21193924.svg)](https://doi.org/10.5281/zenodo.21193924) **Published:** Hsieh, M.-Y. (2026). *A reproducible re-examination of Beale ciphers B1 and B3: systematically closing the composite-cipher hypotheses, with methodological cautions.* **Cryptologia.** **Archive:** A permanent snapshot of the code and data corresponding to the published article is preserved at Zenodo: ### Cite this work > Hsieh, M.-Y. (2026). A reproducible re-examination of Beale ciphers B1 and B3: systematically closing the composite-cipher hypotheses, with methodological cautions. *Cryptologia*. https://doi.org/10.1080/01611194.2026.2698071 To cite the code and data archive specifically: > Hsieh, M.-Y. (2026). *beale-cipher-analysis* (v1.0.0) [Data set and code]. Zenodo. https://doi.org/10.5281/zenodo.21193924 --- This repo runs an end-to-end statistical analysis of the three Beale ciphers (Ward, 1885) and converges on the verdict summarised in [MEMO.md](MEMO.md): cipher 2 is a real Declaration-of-Independence book cipher; ciphers 1 and 3 are hoaxes with high posterior probability. For the full project narrative — how each conclusion was reached, what surprises emerged, what would change the verdict — see [NARRATIVE.md](NARRATIVE.md) (English) or [NARRATIVE_zh-TW.md](NARRATIVE_zh-TW.md) (繁體中文). **Headline results:** | cipher | best key tried | best decode bigram | hoax posterior | |---:|---|---:|---:| | 1 | Declaration (base) + 26 Caesar shifts + 33 Vigenère keywords + transposition (periods 2–25) + all 157,248 Hill 2×2 matrices | −3.55 (Caesar, k=4 transposition) | **≈ 99 %** | | 2 | Declaration of Independence | **−2.82 (real English)** | < 0.1 % (it is a real cipher) | | 3 | Wirt's *Patrick Henry* (best of 10 candidates, plus transposition + Hill 2×2 on top 5 keys) | −3.46 (Declaration + transposition k=9) | **≈ 88–92 %** | For comparison: real English bigram log-prob ≈ −2.6 (KJV-trained model). The cipher-2 anchor at −2.82 is the only point in the whole landscape that touches English. ## Quickstart ```bash # Smoke test — no MATLAB required, finishes in < 1 s python3 scripts/smoke_test.py ``` ```matlab % Full pipeline — requires MATLAB R2024a+ (verified on R2025b). % Runs all phases from a clean state in ~5 min and writes results/*.csv. cd path/to/repo run_all runtests('tests') % regression suite ``` ### MATLAB and the `**` path bug MATLAB's name resolver fails when the working-directory path contains the glob meta-character `**`. If this repo lives inside an iCloud Drive folder whose path includes `**`, every script will error out with `Unrecognized function or variable 'phase_…'`. `run_all.m` works around this by rsync-mirroring the repo to `/tmp/beale_repo` before running. Set `SKIP_MIRROR = true` in your MATLAB workspace before calling `run_all` if your clone path is `**`-free. If you prefer to manage the mirror yourself: ```bash ./mirror_to_tmp.sh # then in MATLAB: cd /tmp/beale_repo SKIP_MIRROR = true run_all ``` ## Layout ``` repo/ ├── README.md ← you are here ├── MEMO.md ← 1-page executive summary ├── NARRATIVE.md ← long-form project walkthrough (~5k words) ├── NARRATIVE_zh-TW.md ← 繁體中文對照版 ├── run_all.m ← top-level MATLAB driver ├── mirror_to_tmp.sh ← workaround for iCloud `**` path bug ├── LICENSE ├── .github/workflows/ci.yml ← GH-Actions smoke test on every push │ ├── data/ ← raw + cleaned cipher arrays │ ├── cipher{1,2,3}_raw.txt │ └── cipher{1,2,3}.json │ ├── keys/ ← candidate book-cipher key texts (cleaned) │ ├── declaration_body.txt (Project Gutenberg PG #1) │ ├── body_pg_constitution.txt (PG #5) │ ├── body_pg_articles_confed.txt (PG #13) │ ├── body_pg_common_sense.txt (PG #147) │ ├── body_pg_federalist.txt (PG #1404) │ ├── body_pg_kjv_bible.txt (PG #10) │ ├── body_pg_shakespeare.txt (PG #100) │ ├── body_pg_watts_hymns.txt (PG #13341, Phase 5b) │ ├── body_pg_wirt_henry.txt (DocSouth, Phase 5b) │ ├── body_pg_beale_pamphlet.txt (Wikisource, Phase 5b) │ ├── pg2048_irving_sketchbook.txt (PG #2048, used by audit) │ └── pg2275_cooper_pioneers.txt (PG #2275, used by audit) │ ├── scripts/ ← MATLAB pipeline + Python smoke test │ ├── phase1_clean.py ← raw text → cipher{1,2,3}.json │ ├── phase1_decode_cipher2.m ← positive control │ ├── phase3_scan.m ← 7 keys × 3 ciphers, simple book cipher │ ├── phase3_gillogly.m ← alphabet-run anomaly test │ ├── phase4_scan.m ← Caesar + Vigenère composites │ ├── phase5b_newkeys.m ← 3 Phase-5 Text-Hunter candidates │ ├── phase5b_kasiski_ioc.m ← Vigenère-period falsification │ ├── phase5c_transposition.m ← columnar transposition sweep │ ├── phase5c_hill.m ← Hill 2×2 brute force │ ├── phase5_audit.m ← bigram model + null robustness audit │ ├── phase5_cipher_tester.m ← long-Vigenère + autokey + Beaufort │ └── smoke_test.py ← Python regression for CI │ ├── tests/test_beale.m ← MATLAB unittest suite (4 test cases) │ ├── reports/ ← human-readable phase write-ups │ ├── 00_project_plan.md │ ├── 01_phase12_stats_report.txt │ ├── 02_phase3_report.md │ ├── 03_phase4_report.md │ ├── 04a_phase5_text_hunter.md │ ├── 04b_phase5_cipher_tester.md │ ├── 04c_phase5_audit.md │ ├── 04d_phase5_skeptic.md │ └── 05_phase5_synthesis.md │ └── results/ ← regenerable CSV / MAT output ├── cipher2_decoded.txt ├── phase2_stats_summary.csv ├── phase3_results.csv ├── phase4_caesar.csv ├── phase4_vigenere.csv ├── phase5_long_vigenere.csv ├── phase5b_newkeys.csv └── phase5c_hill_top10.mat ``` ## Adding a new candidate key text The Phase 3 / 5b harness is parameterised over `keys/`. To test a new candidate: 1. Drop a plain-text body into `keys/` named `body_pg_.txt`. Strip Project Gutenberg front/back matter manually if it has them. 2. Edit `scripts/phase3_scan.m` and add the filename to the `keyFiles` cell. 3. Re-run `run_all` (or just `phase3_scan` alone). 4. Compare the new row in `results/phase3_scan.csv` to the cipher-2 anchor: bigram ≥ −3.0 AND z ≥ +5 = English. Anything else is noise. Same procedure works for `phase5b_newkeys.m` if you want to keep the candidate set isolated from the original seven. ## Methodology summary The pipeline answers two questions about each cipher × key pair: 1. **Is the decode English?** Mean per-letter bigram log-probability under a KJV-trained model, plus common-word substring rate, plus χ² of letter frequencies. Cipher 2 / Declaration scores bigram = −2.82 (real English) and serves as the **anchor**: any new candidate must reach roughly this magnitude to be taken seriously. 2. **Is the signal larger than the background?** A null distribution of 200 random integer sequences indexed into the *same* key, then run through the *same* transformation, gives a per-(cipher, key, transform) z-score. The decision rule throughout is **bigram ≥ −3.0 AND z ≥ +5**. (z alone misleads — see Phase 5 audit; the absolute bigram floor is required.) Phase 5 added a 4-agent adversarial review (Text Hunter / Cipher Tester / Statistical Auditor / Skeptic) which caught one real bug (a uniform-null artifact in the Phase 4 writeup) and surfaced one genuinely unfalsified counter-hypothesis (Pelling's double-encipherment). The Phase 5b/5c follow-ups falsified the specific Vigenère-, transposition-, and Hill-flavoured versions of that counter-hypothesis, leaving only exotic composites (Hill 3×3, key-derived irregular transposition) which have very low prior given Gillogly's construction signature. ## Sources - Ciphers and Ward narrative: *The Beale Papers* (Ward, 1885) — public domain. Transcription from Wikisource compiled view. - Key texts: Project Gutenberg (license: public domain US), Documenting the American South (UNC), Wikisource. Project Gutenberg IDs noted inline in the layout above. - Reference cryptanalysis: Hammer (1971); Gillogly (1980) *Cryptologia*; Pelling (CipherMysteries.com 2010–2018); Campanelli (*Cryptologia*, 2022); Wassmer (IACR ePrint, 2024); NSA FOIA Beale Papers documents. ## License [MIT](LICENSE) for the code and reports. The cipher arrays themselves and the historical key texts are public-domain works of unknown or 18th–19th-century authorship and are not subject to copyright.