Datasetpublic

german-tokenizer-benchmark/biofid

BIOfid: Tokenized Sentences This datasets hosts a sentence-tokenized version of the BIOfid dataset. Creation The following script can be used to reproduce the creation of the dataset: import json from flair.datasets import NER_GERMAN_BIOFID corpus = NER_GERMAN_BIOFID() with open("./train.jsonl", "wt") as f_out: for sentence in corpus.train: current_example = { "text": sentence.to_tokenized_string() }… See the full description on the dataset page: https://huggingface.co/datasets/german-tokenizer-benchmark/biofid.

sourceHugging Facecc-by-4.0updated 11mo agoView on Hugging Face

0likes11downloads

Dataset Card

BIOfid: Tokenized Sentences

This datasets hosts a sentence-tokenized version of the BIOfid dataset.

Creation

The following script can be used to reproduce the creation of the dataset:

python

import json

from flair.datasets import NER_GERMAN_BIOFID

corpus = NER_GERMAN_BIOFID()

with open("./train.jsonl", "wt") as f_out:
    for sentence in corpus.train:
        current_example = {
            "text": sentence.to_tokenized_string()
        }
        f_out.write(json.dumps(current_example) + "\n")

The extracted dataset has 12,668 sentences.