tuanio/binary-bert-for-sequence-classification

1---2license: apache-2.03base_model: facebook/wav2vec2-xls-r-300m4tags:5- generated_from_trainer6metrics:7- wer8model-index:9- name: 1-epochs5-char-based-freeze_cnn-dropout0.110  results: []11---12 13<!-- This model card has been generated automatically according to the information the Trainer had access to. You14should probably proofread and complete it, then remove this comment. -->15 16# 1-epochs5-char-based-freeze_cnn-dropout0.117 18This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on an unknown dataset.19It achieves the following results on the evaluation set:20- Loss: 0.124521- Wer: 0.086522 23## Model description24 25More information needed26 27## Intended uses & limitations28 29More information needed30 31## Training and evaluation data32 33More information needed34 35## Training procedure36 37### Training hyperparameters38 39The following hyperparameters were used during training:40- learning_rate: 2e-0541- train_batch_size: 1042- eval_batch_size: 243- seed: 4244- distributed_type: multi-GPU45- num_devices: 446- total_train_batch_size: 4047- total_eval_batch_size: 848- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-0849- lr_scheduler_type: linear50- lr_scheduler_warmup_ratio: 0.151- num_epochs: 552 53### Training results54 55| Training Loss | Epoch | Step  | Validation Loss | Wer    |56|:-------------:|:-----:|:-----:|:---------------:|:------:|57| 2.8545        | 0.37  | 2500  | 2.8872          | 1.0    |58| 0.7012        | 0.74  | 5000  | 0.3473          | 0.2840 |59| 0.46          | 1.11  | 7500  | 0.2032          | 0.1510 |60| 0.3848        | 1.48  | 10000 | 0.1668          | 0.1194 |61| 0.3535        | 1.85  | 12500 | 0.1518          | 0.1086 |62| 0.3667        | 2.22  | 15000 | 0.1442          | 0.1019 |63| 0.3058        | 2.59  | 17500 | 0.1381          | 0.0961 |64| 0.3026        | 2.96  | 20000 | 0.1327          | 0.0924 |65| 0.2891        | 3.33  | 22500 | 0.1326          | 0.0917 |66| 0.294         | 3.7   | 25000 | 0.1278          | 0.0894 |67| 0.2846        | 4.07  | 27500 | 0.1257          | 0.0885 |68| 0.259         | 4.44  | 30000 | 0.1244          | 0.0874 |69| 0.2348        | 4.81  | 32500 | 0.1245          | 0.0865 |70 71 72### Framework versions73 74- Transformers 4.34.075- Pytorch 2.0.176- Datasets 2.14.577- Tokenizers 0.14.178