hoangkha1810/bart-mathematics

README: Chatbot Training with BART

Overview

This project trains a chatbot using the facebook/bart-large-cnn model from Hugging Face's Transformers library. The chatbot is trained on a dataset of question-answer pairs and is capable of generating responses to user queries.

Dependencies

Ensure you have the following libraries installed before running the script:

pip install transformers datasets torch

Dataset

The chatbot is trained on a CSV dataset (dataset.csv) containing two columns:

• —question: The input question.
• —answer: The corresponding answer.

The dataset is loaded using the Hugging Face datasets library.

Training Process

1. 1.Tokenization:
2. 2.Uses AutoTokenizer to process text.
3. 3.Truncates and pads input to a maximum length of 256 tokens.

1. 1.Data Splitting:
2. 2.The dataset is split into a training set (80%) and an evaluation set (20%).

1. 1.Training Configuration:
2. 2.Uses Trainer API for fine-tuning.
3. 3.Trains for 10 epochs with a batch size of 12.
4. 4.Saves checkpoints every epoch.
5. 5.Loads the best model at the end.

1. 1.Model Saving:
2. 2.The trained model and tokenizer are saved in ./saved_model.

Inference (Generating Responses)

After training, you can generate responses using the generate_text() function. It supports parameters like:

• —temperature: Controls randomness of responses.
• —top_p: Nucleus sampling for response diversity.
• —repetition_penalty: Prevents excessive repetition.

Interactive Chatbot Mode

The script includes an interactive mode where users can input queries:

python chatbot.py

To exit, type exit.

Model Storage

• —Trained model is stored in ./saved_model.
• —Training logs and checkpoints are stored in ./results and ./logs.

Future Improvements

• —Train on a larger dataset.
• —Use a larger model like facebook/bart-large-xsum.
• —Integrate a web-based frontend.

Author

This project was created for research and development in chatbot training using transformer-based models.