Datasetpublic

aguangguang/LCAR-Hallucination-Benchmark

LCAR Hallucination Benchmark LCAR Hallucination Benchmark is a manually reviewed speech benchmark for studying acoustic-grounding failures in LLM-based ASR. It contains two 500-utterance suites: controlled speech synthesized with IndexTTS2 and speech derived from openly released corpora. The benchmark covers translation or transliteration, spoken or text-prompt instruction execution, unsupported repetition, and catastrophic deletion. The benchmark is a targeted stress set. It is… See the full description on the dataset page: https://huggingface.co/datasets/aguangguang/LCAR-Hallucination-Benchmark.

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

1likes113downloads

Dataset Card

LCAR Hallucination Benchmark

LCAR Hallucination Benchmark is a manually reviewed speech benchmark for studying acoustic-grounding failures in LLM-based ASR. It contains two 500-utterance suites: controlled speech synthesized with IndexTTS2 and speech derived from openly released corpora. The benchmark covers translation or transliteration, spoken or text-prompt instruction execution, unsupported repetition, and catastrophic deletion.

The benchmark is a targeted stress set. It is intended to evaluate whether a decoder leaves a hallucinated trajectory, not to estimate hallucination prevalence in ordinary speech or compare the general recognition accuracy of the source ASR systems.

Repository Structure

text

LCAR-Hallucination-Benchmark/
|-- README.md
|-- LICENSE
|-- tts/
|   |-- indextts2.jsonl
|   `-- wav/
|       `-- *.wav
`-- openspeech/
    |-- openspeech.jsonl
    `-- wav/
        `-- *.wav

Dataset Summary

Suite	Utterances	Description
IndexTTS2	500	Controlled TTS speech with coherent Mandarin-English content and targeted prompting conditions
OpenSpeech	500	Speech derived from open-source Mandarin-English corpora with source provenance retained

Data Provenance and Licensing

This repository contains the released WAV files, not only metadata or links to files on the authors' machines. The audio field is relative to the JSONL file's suite directory, so a complete repository download is self-contained.

OpenSpeech records retain the upstream dataset name, source identifier, citation, and available license information in the nested source field. The current release contains the following source components:

Upstream source	Records	License information in release
TALCS	263	Official open-source release; verify the current upstream terms
ASCEND	124	CC BY-SA 4.0 for the upstream data; some records also retain mirror metadata
NTU ML 2021 Mandarin-English ASR/ST	73	MIT
CS-Dialogue	38	CC BY-NC-SA 4.0
AISHELL-2	1	Verify the current upstream terms
Emilia-YODAS	1	Apache-2.0

IndexTTS2 audio is generated output and is covered by the terms of the IndexTTS2 model license. OpenSpeech audio remains subject to the terms of its respective upstream source. The repository-level LICENSE and NOTICE.md describe these boundaries; neither file grants rights beyond the applicable upstream terms. Users should check the upstream release and comply with its attribution, non-commercial, share-alike, and redistribution requirements before reusing or redistributing any record.

Download

Clone the complete repository or download all files from the Hugging Face dataset page. Downloading only a JSONL file will not download its audio:

bash

git clone https://huggingface.co/datasets/aguangguang/LCAR-Hallucination-Benchmark

After downloading, tts/indextts2.jsonl resolves audio under tts/wav/, and openspeech/openspeech.jsonl resolves audio under openspeech/wav/.

Record Format

Both metadata files use JSON Lines. Each line describes one audio sample.

Field	Description
`id`	Stable release identifier
`audio`	Relative path to the WAV file
`suite`	`IndexTTS2` or `OpenSpeech`
`hallucination_type`	Manually accepted failure category
`reference`	Reference transcript
`prompt`	Adversarial text prompt for Prompt-HAL; empty otherwise
`hypothesis`	Baseline ASR output exhibiting the failure
`source`	OpenSpeech source provenance; omitted from the TTS records

Loading

python

import json
from pathlib import Path

root = Path("LCAR-Hallucination-Benchmark")
with (root / "tts" / "indextts2.jsonl").open(encoding="utf-8") as f:
    records = [json.loads(line) for line in f]

audio_path = root / "tts" / records[0]["audio"]

License and Attribution

This repository contains material governed by multiple upstream licenses and therefore uses the Hugging Face other license tag. IndexTTS2-generated audio is subject to the bilibili Model Use License Agreement. OpenSpeech records retain source-level provenance and remain subject to the terms of their respective upstream datasets. See LICENSE before using or redistributing the benchmark, and see NOTICE.md for the source-level attribution summary.

Citation

Citation metadata will be added with the accompanying paper release.