anmol-unitmole/longformer-qasper-document-qa
Longformer QASPER Extractive Document QA
This model card is the upload template for the checkpoint produced by notebooks/complete_longformer_training_evaluation_pipeline.ipynb.
Base model
valhalla/longformer-base-4096-finetuned-squadv1
Project fine-tuning
The checkpoint is further fine-tuned by this project on a documented subset of QASPER v0.3 containing only answerable questions with one contiguous extractive span that can be located in the reconstructed paper text. Free-form, yes/no, unanswerable, unresolved, and multi-span annotations are excluded because this model predicts one contiguous span.
Evaluation
The model was evaluated on 200 contiguous-extractive examples from the QASPER validation set.
Training configuration
- Training examples: 803
- Validation examples: 419
- Evaluation examples: 200
- Maximum training length: 3,072 tokens
- Sliding-window stride: 384 tokens
- Training epochs: 2
- Learning rate: 1e-5
- Precision: BF16
- GPU: NVIDIA GeForce RTX 5090
- Fine-tuned by this project: Yes
Performance interpretation
The QASPER-fine-tuned Longformer outperformed both the truncated BERT baseline and the original SQuAD-fine-tuned Longformer checkpoint across Exact Match, token-level F1, evidence recovery, and evidence-token recall.
The model remains imperfect. Exact Match is a strict metric, and QASPER questions frequently contain long, technical, and semantically complex answers. The reported values should not be interpreted as production-level reliability.