Team Ai
Modelpublic

Sujith2121/flan-t5-multitask-svd

sourceHugging Faceupdated 6mo agoView on Hugging Face
0likes
Model Card

Model Card for FLAN-T5 Multitask SVD Adapter

Model Details

Model Description

This model is a multitask fine-tuned version of google/flan-t5-base that can perform both:

  • —Text Classification
  • —Text Summarization

It is built using parameter-efficient fine-tuning (LoRA, QLoRA, DoRA) and combines multiple adapters into a single unified adapter using Singular Value Decomposition (SVD).

  • —Developed by: Yenugu Sujithreddy
  • —Funded by: Self / Academic Project
  • —Shared by: Yenugu Sujithreddy
  • —Model type: Seq2Seq Transformer
  • —Language(s): English
  • —License: Apache-2.0
  • —Finetuned from model: google/flan-t5-base

Model Sources

  • —Repository: https://huggingface.co/your-username/flan-t5-multitask-svd
  • —Paper: N/A (Project-based implementation)
  • —Demo: N/A

Uses

Direct Use

This model can be used directly with prompt-based instructions:

Classification Example:

Task: classify
Text: Apple launches a new product

Summarization Example:

Task: summarize
Text: Long paragraph...

Downstream Use

  • —Chatbots with multitask capability
  • —News classification systems
  • —Content summarization tools
  • —AI assistants

Out-of-Scope Use

  • —Medical or legal decision-making
  • —Safety-critical systems
  • —Real-time high-risk applications
  • —Tasks without clear prompt instructions

Bias, Risks, and Limitations

  • —May inherit biases from training datasets
  • —Performance may degrade slightly after SVD merging
  • —Requires explicit task prompting
  • —Summaries may sometimes copy input text
  • —Limited generalization beyond trained domains

Recommendations

  • —Always validate outputs before use
  • —Use clear prompts (Task: classify / Task: summarize)
  • —Fine-tune further for production-grade applications
  • —Evaluate on domain-specific datasets

How to Get Started with the Model

python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model = AutoModelForSeq2SeqLM.from_pretrained("your-username/flan-t5-multitask-svd")
tokenizer = AutoTokenizer.from_pretrained("your-username/flan-t5-multitask-svd")

input_text = "Task: classify\nText: Tesla stock rises"

inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

  • —Custom classification dataset (news categories like Sports, Business, Science)
  • —Custom summarization dataset (text-summary pairs)

Training Procedure

Preprocessing
  • —Text cleaning
  • —Prompt formatting (task-based input)
  • —Tokenization using Flan-T5 tokenizer
Training Hyperparameters
  • —Training regime: fp16 mixed precision
  • —Learning rate: 2e-4
  • —Batch size: 8
  • —LoRA rank: 8
  • —Optimizer: AdamW

Evaluation

Testing Data, Factors & Metrics

Testing Data
  • —Held-out classification samples
  • —Sample summarization inputs
Factors
  • —Task type (classification vs summarization)
  • —Input length
  • —Domain variation
Metrics
  • —Classification: Accuracy
  • —Summarization: Qualitative + ROUGE (manual check)

Results

  • —Good classification accuracy on known categories
  • —Summarization produces concise outputs
  • —Slight performance trade-off due to adapter merging
Summary

The model successfully performs multitask learning using a single adapter, demonstrating the effectiveness of SVD-based adapter merging.

Model Examination

  • —No formal interpretability methods applied
  • —Behavior verified through manual testing

Environmental Impact

  • —Hardware Type: GPU
  • —Hours used: ~4–6 hours
  • —Cloud Provider: Local / Colab
  • —Compute Region: N/A
  • —Carbon Emitted: Not measured

Technical Specifications

Model Architecture and Objective

  • —Base: FLAN-T5
  • —Adapter Type: LoRA
  • —Merge Method: SVD
  • —Objective: Multitask learning (classification + summarization)

Compute Infrastructure

Hardware
  • —NVIDIA GPU (recommended)
Software
  • —PyTorch
  • —Hugging Face Transformers
  • —PEFT

Citation

BibTeX:

@misc{flant5_multitask_svd_2026,
  title={FLAN-T5 Multitask Model using SVD Adapter Merging},
  author={Yenugu Sujithreddy},
  year={2026}
}

APA: Sujithreddy, Y. (2026). FLAN-T5 Multitask Model using SVD Adapter Merging.

Glossary

  • —LoRA: Low-Rank Adaptation
  • —QLoRA: Quantized LoRA
  • —SVD: Singular Value Decomposition
  • —Multitask Learning: Training a model for multiple tasks

More Information

This project demonstrates an advanced fine-tuning pipeline combining multiple PEFT techniques and adapter merging.

Model Card Authors

Yenugu Sujithreddy

Model Card Contact

For questions or collaboration, please open an issue on the Hugging Face repository.