Datasetpublic

german-tokenizer-benchmark/mobie

DFKI MobIE: Tokenized Sentences This datasets hosts a sentence-tokenized version of the DFKI MobIE dataset. Creation The following script can be used to reproduce the creation of the dataset: import json from flair.datasets import NER_GERMAN_MOBIE corpus = NER_GERMAN_MOBIE() with open("./train.jsonl", "wt") as f_out: for sentence in corpus.train: current_example = { "text": sentence.to_tokenized_string() }… See the full description on the dataset page: https://huggingface.co/datasets/german-tokenizer-benchmark/mobie.

sourceHugging Facecc-by-4.0updated 11mo agoView on Hugging Face

0likes7downloads

Dataset Card

DFKI MobIE: Tokenized Sentences

This datasets hosts a sentence-tokenized version of the DFKI MobIE dataset.

Creation

The following script can be used to reproduce the creation of the dataset:

python

import json

from flair.datasets import NER_GERMAN_MOBIE

corpus = NER_GERMAN_MOBIE()

with open("./train.jsonl", "wt") as f_out:
    for sentence in corpus.train:
        current_example = {
            "text": sentence.to_tokenized_string()
        }
        f_out.write(json.dumps(current_example) + "\n")

The extracted dataset has 6,900 sentences.