fifadxj/tiny-bert-sequence-classification

1---2library_name: transformers3license: apache-2.04base_model: google-bert/bert-base-chinese5tags:6- generated_from_trainer7metrics:8- accuracy9- precision10- recall11- f112model-index:13- name: tiny-bert-sequence-classification14  results: []15---16 17<!-- This model card has been generated automatically according to the information the Trainer had access to. You18should probably proofread and complete it, then remove this comment. -->19 20# tiny-bert-sequence-classification21 22This model is a fine-tuned version of [google-bert/bert-base-chinese](https://huggingface.co/google-bert/bert-base-chinese) on an unknown dataset.23It achieves the following results on the evaluation set:24- Loss: 0.337525- Accuracy: 0.946726- Precision: 0.949127- Recall: 0.942728- F1: 0.945929 30## Model description31 32More information needed33 34## Intended uses & limitations35 36More information needed37 38## Training and evaluation data39 40More information needed41 42## Training procedure43 44### Training hyperparameters45 46The following hyperparameters were used during training:47- learning_rate: 2e-0548- train_batch_size: 6449- eval_batch_size: 6450- seed: 4251- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments52- lr_scheduler_type: linear53- lr_scheduler_warmup_ratio: 0.154- num_epochs: 1055- mixed_precision_training: Native AMP56 57### Training results58 59| Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1     |60|:-------------:|:-----:|:----:|:---------------:|:--------:|:---------:|:------:|:------:|61| 0.3578        | 1.0   | 150  | 0.2034          | 0.9242   | 0.9169    | 0.9309 | 0.9238 |62| 0.1677        | 2.0   | 300  | 0.1731          | 0.9442   | 0.9747    | 0.9106 | 0.9416 |63| 0.0932        | 3.0   | 450  | 0.1917          | 0.9442   | 0.9696    | 0.9157 | 0.9419 |64| 0.0555        | 4.0   | 600  | 0.1904          | 0.9517   | 0.9652    | 0.9359 | 0.9503 |65| 0.0312        | 5.0   | 750  | 0.2270          | 0.95     | 0.9635    | 0.9342 | 0.9486 |66| 0.0218        | 6.0   | 900  | 0.2619          | 0.9525   | 0.9653    | 0.9376 | 0.9512 |67| 0.014         | 7.0   | 1050 | 0.3316          | 0.9467   | 0.9616    | 0.9292 | 0.9451 |68| 0.0094        | 8.0   | 1200 | 0.3375          | 0.9467   | 0.9491    | 0.9427 | 0.9459 |69 70 71### Framework versions72 73- Transformers 4.57.174- Pytorch 2.8.0+cu12675- Datasets 4.0.076- Tokenizers 0.22.177