Team Ai
Apppublic

JulianKrgd/Julian-600M-Base

sourceHugging Faceapache-2.0updated 8mo agoView on Hugging Face
0likes
App README

Julian-600M-Instruct Demo

Try Julian-600M-Instruct, a 600M parameter instruction-tuned language model trained from scratch on Google Cloud TPUs.

Bilingual: English & French | Format: ChatML | Training: 39B tokens + 100K steps SFT on 2.47M instructions

Features

  • —Bilingual text generation (English & French)
  • —Streaming inference with ChatML formatting
  • —Adjustable generation parameters (temperature, top-k, top-p, repetition penalty)
  • —GPU-accelerated inference

Benchmark Results (zero-shot)

BenchmarkScore
HellaSwag53.5%
PIQA66.8%
LAMBADA37.3%
ARC-Easy54.0%
WinoGrande52.7%
BoolQ57.9%
Julian-600M outperforms OPT-1.3B (41.5% HellaSwag) despite having less than half the parameters.

Usage

  1. 1.Enter a question or instruction (English or French)
  2. 2.Adjust generation parameters if needed
  3. 3.Click "Generate Response"
  4. 4.Try the example prompts for inspiration!

Model Details

ParameterValue
Parameters600M
ArchitectureLLaMA-style (RoPE, SwiGLU, RMSNorm)
Pretraining39B tokens (70% EN, 30% FR)
Fine-tuning100K steps on 2.47M instruction pairs (ChatML)
Context Length2048 tokens

Links

Trained with Google TPU Research Cloud (TRC) program