themechanism/script-fidelity-benchmark

Script fidelity benchmark Anonymous supplement for the paper "Script collapse in multilingual ASR: A reference-free metric and 100-pair benchmark." Script Fidelity Rate (SFR) measures the fraction of ASR hypothesis characters that belong to the expected target script. WER measures word edits, while SFR checks whether the output is written in the target orthography. Related resources: PyPI package: https://pypi.org/project/script-fidelity/ Hugging Face Evaluate metric:… See the full description on the dataset page: https://huggingface.co/datasets/themechanism/script-fidelity-benchmark.