Team Ai
Modelpublic

sickcell/pythia-70m-deduped-finetuned-github_cybersecurity_READMEs

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face
0likes32downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

pythia-70m-deduped-finetuned-githubcybersecurityREADMEs

This model is a fine-tuned version of EleutherAI/pythia-70m-deduped on an unknown dataset. It achieves the following results on the evaluation set:

  • —Loss: 7.1003
  • —Accuracy: 0.0669

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 3e-05
  • —trainbatchsize: 32
  • —evalbatchsize: 32
  • —seed: 42
  • —gradientaccumulationsteps: 4
  • —totaltrainbatch_size: 128
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: linear
  • —lrschedulerwarmup_steps: 1000
  • —num_epochs: 100

Training results

Training LossEpochStepValidation LossAccuracy
No log0.971433.17510.0595
No log2.02932.96040.0635
No log2.974332.70280.0655
No log4.05832.35670.0674
No log4.977227.94920.0686
No log6.0876.44750.0665
No log6.971015.72080.0645
No log8.01165.48070.0690
No log8.971305.30240.0670
No log10.01455.12000.0640
No log10.971595.00310.0850
No log12.01744.90630.0845
No log12.971884.84880.0849
No log14.02034.79950.0827
No log14.972174.73930.0830
No log16.02324.68670.0812
No log16.972464.63460.0809
No log18.02614.58730.0801
No log18.972754.54350.0793
No log20.02904.49550.0780
No log20.973044.45050.0770
No log22.03194.40440.0760
No log22.973334.32580.0782
No log24.03484.29260.0760
No log24.973624.23530.0769
No log26.03774.21570.0751
No log26.973914.17050.0752
No log28.04064.13100.0754
No log28.974204.09810.0752
No log30.04354.09090.0733
No log30.974494.02910.0743
No log32.04644.07610.0721
No log32.974783.97940.0727
No log34.04933.95210.0733
8.048434.975073.94210.0733
8.048436.05223.93100.0727
8.048436.975363.91420.0728
8.048438.05513.93380.0723
8.048438.975653.91890.0716
8.048440.05803.91860.0718
8.048440.975943.92160.0722
8.048442.06093.89440.0718
8.048442.976233.90380.0705
8.048444.06383.93710.0707
8.048444.976523.87160.0714
8.048446.06673.91530.0705
8.048446.976813.95400.0703
8.048448.06963.99730.0706
8.048448.977104.00110.0701
8.048450.07254.05470.0696
8.048450.977394.18990.0693
8.048452.07544.12400.0707
8.048452.977684.24800.0699
8.048454.07834.29860.0691
8.048454.977974.20610.0695
8.048456.08124.36890.0695
8.048456.978264.41210.0688
8.048458.08414.45000.0686
8.048458.978554.60040.0686
8.048460.08704.63570.0680
8.048460.978844.84640.0684
8.048462.08994.68060.0687
8.048462.979134.83740.0682
8.048464.09284.86530.0679
8.048464.979425.04240.0680
8.048466.09575.15180.0680
8.048466.979715.12400.0683
8.048468.09865.16610.0678
1.955968.9710005.39920.0687
1.955970.010155.48760.0680
1.955970.9710295.56090.0683
1.955972.010445.67070.0679
1.955972.9710585.75510.0667
1.955974.010735.90360.0675
1.955974.9710876.13550.0665
1.955976.011026.29950.0661
1.955976.9711166.25460.0677
1.955978.011316.31690.0672
1.955978.9711456.33770.0669
1.955980.011606.49690.0673
1.955980.9711746.66360.0664
1.955982.011896.75500.0672
1.955982.9712036.70440.0661
1.955984.012186.77130.0669
1.955984.9712326.85950.0668
1.955986.012476.92190.0663
1.955986.9712616.91740.0666
1.955988.012766.91580.0667
1.955988.9712906.97440.0670
1.955990.013056.93750.0669
1.955990.9713196.99470.0668
1.955992.013347.04210.0671
1.955992.9713487.02400.0666
1.955994.013637.04800.0669
1.955994.9713777.06790.0668
1.955996.013927.10260.0670
1.955996.5514007.10030.0669

Framework versions

  • —Transformers 4.40.0.dev0
  • —Pytorch 2.2.1+cu121
  • —Datasets 2.18.0
  • —Tokenizers 0.15.2