freococo/pali-words-myanmar-script

Pali Words in Myanmar Script (Master Index)

This dataset is a master index of 220,252 unique Pali words written in Myanmar (Burmese) Unicode script, intended for reuse across linguistic, religious, and computational workflows.

Dataset Description

• —Creator: freococo
• —Source: Tipitaka Pali–Burmese Dictionary data extracted from the open-source repositories maintained by pndaza
• —Total Entries: 220,252 unique words
• —Format: JSONL (JSON Lines)
• —License: CC0 1.0 Universal (Public Domain)

Data Fields

Each record contains:

• —`word_id`: A stable, sequential integer identifier.
• —`pali_word`: A Pali lexical item rendered in Myanmar Unicode script.

Data Processing Methodology

The dataset was constructed using the following steps:

• —Lexical Extraction: Isolating Pali headwords from raw dictionary HTML.
• —Deduplication: Removing repeated headwords appearing across multiple dictionary volumes.
• —Script Normalization: Fixing Myanmar-script artifacts, including erroneous leading tone marks (e.g., incorrect leading ့).
• —Alphabetical Sorting: Ordering entries according to Myanmar/Pali lexicographic conventions.
• —Index Stabilization: Assigning persistent identifiers (word_id) to support future linked datasets.

Intended Use Cases

• —Buddhist Digital Studies: Lexical indexing and vocabulary analysis of Tipitaka texts.
• —Natural Language Processing: Tokenizer training, normalization, spell-checking, and lexicon-based language models.
• —OCR Post-Processing: Dictionary-based correction, validation, and normalization of OCR outputs from scanned Pali–Myanmar texts.
• —Digital Humanities: Construction of interoperable dictionaries and language tools.

License

This dataset is released under the CC0 1.0 Universal (Public Domain Dedication), permitting unrestricted use, modification, and redistribution.