Delta-Vector/Pascal-Base-12B

Hparams:

• —LR: 1e-5
• —Weight Decay: 0.02
• —grad-clip: 0.2
• —R64, A32, Rslora, 0.1 drop out, 16bit lora (qlora is cope.)

pretrained on a buncha LNs, books, etc. Used for a base for Pascal.