A Reversible, ASCII-7, Diacritic-Free Romanisation of the Brahmi, Perso-Arabic, and Northwest Semitic Script Families

Abstract

We establish a constraint-satisfaction theorem for a reversible romanisation φ : 𝒮\ → Σ_R\ whose domain 𝒮 unifies three script families — Brahmi (60+ scripts), Perso-Arabic (Arabic plus ~10 language extensions), and the Northwest Semitic abjad family (Hebrew, Syriac, Samaritan, Mandaic, Imperial Aramaic, Nabataean, Palmyrene, Hatran, Phoenician). On the orthographic kernel of each script the mapping simultaneously satisfies six constraints: ASCII-7 closure, case independence, diacritic independence, C-identifier legality, reversibility, and linear-time computability. No prior published romanisation satisfies this conjunction on this domain. Over the full arbitrary input space [a-z]⁴ the system is measured at 98.68% reversible-or-canonicalizable with a 1.31% irreducible-ambiguity floor. The result is enabled by a two-layer architectural separation (script-system identity; canonical-to-ASCII reduction) and has been continuously implemented in deployed code since 15 August 2004 (Hindawi Programming System).

1. Introduction

Multilingual computing meets a language through its script's byte encoding. Existing romanisation standards each optimise for one property at the expense of others: IAST and ISO 15919 are reversible but diacritic-dependent and not ASCII-7; Harvard-Kyoto, ITRANS, SLP1, and WX are ASCII but case-dependent or not identifier-legal; none unifies abugida and abjad families under one reversible ASCII-7 scheme. We state and prove that a single mapping satisfies all six constraints across three script families, and we report its measured reversibility over an exhaustive small-string domain.

The contribution is the conjunction, not any single property, and the architectural separation that makes the conjunction extensible. We place the work in the Pāṇinian–Sībawayhi formal-grammatical lineage, for which reversible faithful representation is the foundational concern.

2. The mapping

Domain 𝒮 = Brahmi ∪ Perso-Arabic ∪ NW-Semitic. Codomain Σ_R = {a–z, _, ^, |, space}, a 30-character subset of ASCII-7. The forward map φ composes Layer 1 (canonical-form projection) with Layer 2 (ASCII reduction); ρ is the reverse. _ carries retroflexion and homophone disambiguation (legal in C identifiers, unlike the diacritics ṭ ḍ ḷ ṣ it replaces); | is the daṇḍa; ^ is the matra sigil. Uppercase never serves as a phonological distinguisher: Latin keywords pass through the lex front-end verbatim and never collide with the lowercase Romenagri output.

3. The theorem

Theorem. For every s ∈ 𝒮 and every well-formed w in the orthographic content of s: 1. φ(w) ∈ Σ_R\* (closure); 2. φ(w) contains no uppercase, no diacritic, no byte > 0x7E, and tokenises legally under the C identifier grammar [a-z_][a-z0-9_]*; 3. ρ(φ(w)) = w on the orthographic kernel (reversibility); 4. φ, ρ compute in O(n) time, O(1) space per byte.

Proof. By construction of the 84-entry phonological basis (acii.h) for Layer 2 and the per-family canonical-form projections for Layer 1; by composition (each family reaches a canonical form — Devanagari for Brahmi, Arabic for Perso-Arabic and NW-Semitic — then the fixed Layer-2 reduction applies uniformly); and by empirical verification (§5). The reverse parser (rmn2acii.c) is recursive-descent with LL(2) lookahead in the disambiguation production (level2c), giving O(n). ∎

4. Prior art

| System | ASCII-7 | Case-indep | Diacritic-free | C-ident-legal | Reversible | Multi-family | |---|:-:|:-:|:-:|:-:|:-:|:-:| | IAST | ✗ | ✓ | ✗ | ✗ | ✓ | ✗ | | ISO 15919 | ✗ | ✓ | ✗ | ✗ | ✓ | partial | | Harvard-Kyoto | ✓ | ✗ | ✓ | ✗ | ✓ | ✗ | | ITRANS | ✓ | ✗ | ✓ | ✗ | partial | ✗ | | Velthuis | ✓ | ✓ | ✗ | ✗ | ✓ | ✗ | | SLP1 / WX | ✓ | ✗ | ✓ | ✗ | ✓ | partial | | Aksharamukha / BrahmiNet | ✓ | partial | ✓ | ✗ | partial | within-Indic | | Romenagri/ILM | ✓ | ✓ | ✓ | ✓ | ✓ kernel | ✓ 3 families |

5. Empirical verification

Exhaustive small-string domain. Over [a-z]^1..4 (475,254 strings; fresh compile of the 2003–04 GPL C), classifying each string as exact round-trip, normalized (stable canonical fixed point), or irreducible-unstable: exact + normalized = 98.68%; irreducible floor = 1.31% (all in length 3–4); residual crash = 0.005% (25 strings — a buffer overrun beyond the known NULL-on-lookup bug). The dominant non-identity class is normalization (c→ch, schwa-completion) to a stable fixed point — canonicalization, not loss. (Generated by az_reversibility_experiment.sh.)

Natural prose. 15 sentences of running Hindi: 1186 bytes UTF-8 → 461 ISCII (md5 47788fe4…) → 740 Romenagri (0 non-ASCII, 0 uppercase) → 461 ISCII (md5 identical). Byte-identical round-trip.

Cross-family, end-to-end, byte-clean ASCII-7. Hebrew שלום→shalauma; Phoenician 𐤁𐤏𐤋→ba_aeela; Syriac ܫܠܡܐ→shalamaa; Imperial Aramaic 𐡌𐡋𐡊→malaka; Urdu بھارت→bhaara_ta. Aggregate: 0 non-ASCII, 0 uppercase, all C-identifier-legal.

6. Residue classes

Six characterised residue classes (avagraha; ISCII numeral/letter byte collision; LLLA missing entry; schwa-deletion tokenization artifact; abjad vowel underspecification — a property of the source orthography, not of φ; conjunct-letter homograph). Each admits a constructive patch; none is a structural obstruction to the theorem.

7. The two-layer architecture

Layer 1 maps many graphemes denoting one phonological unit (ক/க/క/ਕ/क → /ka/; א/ܐ/ا/𐤀 → glottal stop) to one canonical form. Layer 2 reduces the canonical form to Σ_R reversibly. New scripts join by adding Layer-1 table rows; the Layer-2 algorithm is fixed. Unicode collapses both into codepoint identity; CLDR into script-to-script projection; BPE eliminates Layer 1 — the structural origin of Latin tokenization bias (see companion paper on efficiency).

8. Priority

HPS v1 public release 15 Aug 2004; CSI National Young IT Professional Award 2005; Sarai/CSDS FLOSS Fellowship 2005–06; GNU Savannah registration 2006; TDIL (Govt of India) recognition; continuous public maintenance 2004–2026. Every post-2004 multilingual-NLP system is post-Romenagri; none cites it, solves the cross-family unification, or provides the layer separation.

9. Conclusion

A single reversible mapping satisfies six simultaneous constraints across three script families, with measured reversibility on an exhaustive small-string domain and byte-identical round-trip on natural prose. The two-layer separation makes the result extensible by table, not by algorithm.

*References: Daniels & Bright 1996; Coulmas 2003; Meletis 2020; Sproat 2000; Kunchukuttan et al. 2014; Sennrich et al. 2016; Kudo & Richardson 2018; Pāṇini, Aṣṭādhyāyī; Sībawayhi, al-Kitāb; Choudhary 2004 (HPS v1). Copyright © 1993–2026 Abhishek Choudhary.*