userdavek/Amharic_news_Normalized

Dataset Name

Amharic news dataset

Dataset Details

It is a non-normalized version of news dataset crawled from Amharic news websites and from researchers provided in their works.

Dataset Description

<!-- Provide a longer summary of what this dataset is. --> The dataset is collected from different news websites and from different researchers crawled Amharic news dataset from different NLP downstream tasks. News sites like FanaBC, EthiopianReporter, Zehabesha,Esat Amharic, BBC Amharic are the sources for these news data. Tasks like news classification and summarization are the related NLP applications where some of the data in this dataset are used from.

• —Curated by: Dawit K. <!-- - Funded by [optional]: [More Information Needed] --> <!-- - Shared by [optional]: [More Information Needed] -->
• —Language(s) (NLP): Amharic
• —License: MIT

<!-- ### Dataset Sources [optional]

<!-- Provide the basic links for the dataset. -->

<!-- - Repository: [More Information Needed]

• —Paper [optional]: [More Information Needed]
• —Demo [optional]: [More Information Needed] -->

Uses

<!-- Address questions around how the dataset is intended to be used. --> The dataset can be use in the area of NLP/AI/Data science to build models, improve or finetune previously built models.

<!-- ### Direct Use -->

<!-- This section describes suitable use cases for the dataset. -->

<!-- [More Information Needed] -->

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. --> Please use this data only for research and educational purpose only. <!-- [More Information Needed] -->

Dataset Structure

<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. -->

<!-- [More Information Needed] --> The dataset has two fields namely "Text" and "Summary" which can also be "Article" and "Headline" or "Content" and "Title" as per your need of naming the fields. The dataset has three splits as "train", "test", and "eval" (validation) with as percentage split of 80:10:10 respectively.

Dataset Creation

Curation Rationale

<!-- Motivation for the creation of this dataset. --> The dataset is intended to be used in the task of news summarization and reverse summarization (News article generation). <!-- [More Information Needed] -->

Source Data

<!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). --> The data is prepared from Amharic news websites.

Data Collection and Processing

<!-- This section describes the data collection and processing process such as data selection criteria, filtering and normalization methods, tools and libraries used, etc. --> After the data is collected from different sources, different stages of processing are done on it. primarily the data gets integrated and filtered null rows, duplicates and removed none amharic scripts. so, in this dataset there are only Amharic/ge'ez scripts to make easy for the model on bias issues. the other processing tasks are abbreviation expansion, and Normalization. This data is the normalized version where all homophones and unliablized letters are as written on the news websites. the non-normalized version of this dataset is available at this Dataset <!-- [More Information Needed] -->

<!-- #### Who are the source data producers?

<!-- This section describes the people or systems who originally created the data. It should also include self-reported demographic or identity information for the source data creators if this information is available. --> <!-- [More Information Needed] -->

<!-- ### Annotations [optional]

<!-- If the dataset contains annotations which are not part of the initial data collection, use this section to describe them. --> <!-- #### Annotation process

<!-- This section describes the annotation process such as annotation tools used in the process, the amount of data annotated, annotation guidelines provided to the annotators, interannotator statistics, annotation validation, etc. --> <!-- [More Information Needed] -->

<!-- #### Who are the annotators?

<!-- This section describes the people or systems who created the annotations. --> <!-- [More Information Needed] -->

<!-- #### Personal and Sensitive Information

<!-- State whether the dataset contains data that might be considered personal, sensitive, or private (e.g., data that reveals addresses, uniquely identifiable names or aliases, racial or ethnic origins, sexual orientations, religious beliefs, political opinions, financial or health data, etc.). If efforts were made to anonymize the data, describe the anonymization process. --> <!-- [More Information Needed] -->

<!-- ## Bias, Risks, and Limitations -->

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

<!-- [More Information Needed] -->

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users should be made aware of the risks, biases and limitations of the dataset.

<!-- ## Citation [optional] -->

<!-- If there is a paper or blog post introducing the dataset, the APA and Bibtex information for that should go in this section. -->

<!-- BibTeX: -->

<!-- [More Information Needed] -->

<!-- APA: -->

<!-- [More Information Needed] -->

<!-- ## Glossary [optional] -->

<!-- If relevant, include terms and calculations in this section that can help readers understand the dataset or dataset card. -->

<!-- [More Information Needed] -->

<!-- ## More Information [optional] -->

<!-- [More Information Needed] -->

<!-- ## Dataset Card Authors [optional] -->

<!-- [More Information Needed] -->

<!-- ## Dataset Card Contact -->

<!-- [More Information Needed] -->