AndyChiang/Pre-CoFactv3-Question-Answering

Pre-CoFactv3-Question-Answering

Model description

This is a Question Answering model for AAAI 2024 Workshop Paper: “Team Trifecta at Factify5WQA: Setting the Standard in Fact Verification with Fine-Tuning”

Its input are question and context, and output is the answers derived from the context. It is fine-tuned by FACTIFY5WQA dataset based on **microsoft/deberta-v3-large** model.

For more details, you can see our paper or **GitHub**.

How to use?

1. 1.Download the model by hugging face transformers.

from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline

model = AutoModelForQuestionAnswering.from_pretrained("AndyChiang/Pre-CoFactv3-Question-Answering")
tokenizer = AutoTokenizer.from_pretrained("AndyChiang/Pre-CoFactv3-Question-Answering")

1. 1.Create a pipeline.

QA = pipeline("question-answering", model=model, tokenizer=tokenizer)

1. 1.Use the pipeline to answer the question by context.

QA_input = {
    'context': "Micah Richards spent an entire season at Aston Vila without playing a single game.",
    'question': "Who spent an entire season at aston vila without playing a single game?",
}
answer = QA(QA_input)
print(answer)

Dataset

We utilize the dataset FACTIFY5WQA provided by the AAAI-24 Workshop Factify 3.0.

This dataset is designed for fact verification, with the task of determining the veracity of a claim based on the given evidence.

• —claim: the statement to be verified.
• —evidence: the facts to verify the claim.
• —question: the questions generated from the claim by the 5W framework (who, what, when, where, and why).
• —claim_answer: the answers derived from the claim.
• —evidence_answer: the answers derived from the evidence.
• —label: the veracity of the claim based on the given evidence, which is one of three categories: Support, Neutral, or Refute.

Fine-tuning

Fine-tuning is conducted by the Hugging Face Trainer API on the Question Answering task.

Training hyperparameters

The following hyperparameters were used during training:

• —Pre-train language model: microsoft/deberta-v3-large
• —Optimizer: adam
• —Learning rate: 0.00001
• —Max length of input: 3200
• —Batch size: 4
• —Epoch: 3
• —Device: NVIDIA RTX A5000

Testing

We employ BLEU scores for both claim answer and evidence answer, taking the average of the two as the metric.

Other models

AndyChiang/Pre-CoFactv3-Text-Classification

Citation