SDAIANCAI/Saudilang-Code-Switch-Corpus

SCC - Saudilang Code-Switch Corpus

The National Center for Artificial Intelligence at the Saudi Data and Artificial Intelligence Authority (SDAIA), published the "SCC" dataset, which stands for "Saudilang Code-Switch Corpus”.

This dataset contains a transcription of general conversations taken from a YouTube podcast "Thmanyah" that has been transcribed by the National Center for Artificial Intelligence in SDAIA. The data features three episodes covering different domains: investment, establishing restaurants, and entrepreneurship. It includes 4.45 hours of labeled audio spoken in Arabic with occasional English words, featuring Saudi dialect, thus providing a linguistic and cultural context for studying code-switching phenomena. The corpus comprises a transcription of recordings for 4 unique speakers across 3 files, including segments with overlapping speakers.

CSV File

There is 1 csv file encoded in UTF8. It contains the transcription for each segment, together with their annotation. In total there are 12 columns. The column headings are listed in the first line of each csv file, and explained below:

• —Link: Youtube video link.
• —SegmentStart: start of segment as offset from the beginning of the audio file in seconds.
• —SegmentEnd: end of segment as offset from the beginning of the audio file in seconds.
• —Original_text: the actual uttered text of that segment.
• —ProcessedText: the pre-processed text of the original text.
• —Language: the language which is ar_SA (Arabic, Saudi dialect).
• —Environment: the surrounding environment of the segment (Clean, Music, or Noisy).
• —Speaker: unique speaker ID across all files.
• —SpeakerGender: the gender of the speakers (all of them are Male).
• —FullFileLength: duration of the full video seconds.
• —SegmentLength: segment's duration in seconds.
• —Segment_ID: unique ID for each segment. Note: Text processing includes normalizing Arabic letters to unified forms such as آأإ to ا, removing punctuations, emojis, diacritics, and any special characters. Utterances with empty text, converting all English words to lowercase, inserting spaces between Arabic and English words and removing apostrophes from contractions.

Datasets Statistics

The following table shows a detailed statistics.

Licenses

This work is licensed under a CC BY-NC-SA 4.0 license.

Citation

If you use SCC dataset please use the following citation:

@inproceedings{SCC2024,
  Title= {Leveraging LLM for Augmenting Textual Data in Code-Switching ASR: Arabic as an Example},
  Author= {Sadeen Alharbi, Raghad Aloraini, Reem BinMuqbil, Ahmed Ali, Saiful Bari, Areeb Alowisheq, Yaser Alonaizan},
  Booktitle = {To be published},
  affiliation = {NCAI-SDAIA}
  Year 	= {2024}
}

File last update: JUL 23, 2024.