NataliaH/TransformerDecoderModel

tags:

• —language-model
• —transformer-decoder
• —tiny-shakespeare license: mit datasets:
• —tinyshakespeare modeldescription: | This is a small autoregressive language model based on the Transformer architecture trained on the Tiny Shakespeare dataset.

## Model Description The model is a custom implementation of a TransformerDecoderModel, which uses a decoder-only architecture similar to GPT-2. It was trained on the Tiny Shakespeare dataset to generate text in the style of William Shakespeare.

## Training Details The model was trained and tracked using Weights & Biases.

## How to Use To generate text with this model, you can load it and the tokenizer as follows:

    from transformers import AutoTokenizer
    from transformers import GPT2LMHeadModel

    # Load the model and tokenizer
    model = GPT2LMHeadModel.from_pretrained('NataliaH/TransformerDecoderModel')
    tokenizer = AutoTokenizer.from_pretrained('NataliaH/TransformerDecoderModel')

    # Provide input text and generate output
    input_text = 'To be or not to be'
    inputs = tokenizer(input_text, return_tensors='pt')
    outputs = model.generate(**inputs)
    print(tokenizer.decode(outputs[0], skip_special_tokens=True))