knpatil/laya-browser-agent-base

Laya Browser Agent Base (149M) — ModernBERT Knowledge Distillation

`knpatil/laya-browser-agent-base` is a high-speed, sub-30ms browser decision agent distilled from `knpatil/laya-browser-agent` (ModernBERT-large, 421M).

Designed specifically for real-time in-browser agent decision loops (such as the BroPilot Chrome Extension), this model predicts immediate next-step browser actions (CLICK, TYPE_TEXT, SELECT, SCROLL_DOWN, WAIT, DONE, BLOCKED), element targets, and goal completion criteria in 29.89 ms on Apple Silicon Metal (MPS).

🚀 Key Performance Highlights

• —Ultra-Low Latency: 29.89 ms median forward pass (p95: 33.37 ms) on Apple Silicon M2 Max (MPS) — 28x faster than commercial cloud APIs (TypeSafe Jev: 841.8 ms).
• —High Decision Accuracy: 96.94% decision accuracy across 70 standard web automation workflows (vs TypeSafe Jev: 86.89%, Teacher: 82.14%).
• —Compact Footprint: 164.0M total parameters (149M backbone) with a 596 MB disk size (-61.1% parameter reduction from 421M).
• —Exceptional Calibration: Brier score of 0.0025 (Platt-scaled temperature: choice: 1.0381, score: 1.0000, noul: 1.0650).
• —Zero Cloud API Charges: 100% on-device private execution with zero browser session exfiltration.

📊 Comprehensive Benchmark Results

Evaluated across the 70 benchmark scenarios (244 structured decisions) in data/browser_test_cases.jsonl:

Accuracy by Sub-Decision Primitive

• —`operation` (7-way Action Choice): 100.0% (70/70)
• —`action_type` (Navigation Intent): 100.0% (70/70)
• —`is_goal_satisfied` (Binary Goal Check): 100.0% (70/70)
• —`click_target` (Element Selection): 85.7% (60/70)
• —`type_text_target` (Input Field Selection): 95.2% (40/42)

🧠 Distillation Architecture & Training

The student model was trained using knowledge distillation from knpatil/laya-browser-agent:

• —Teacher: Frozen ModernBERT-large (421M params, 28 layers, d=1024, 16 attention heads).
• —Student: ModernBERT-base (149M params, 22 layers, d=768, 12 attention heads).
• —Loss Function: Multi-task joint loss: L = α_KD · τ² · L_KD(σ(z_S/τ), σ(z_T/τ)) + α_CE · L_CE(z_S, y) with distillation temperature τ = 2.0, α_KD = 0.6, α_CE = 0.4.
• —Optimizer: AdamW (lr=2.5e-5) with Cosine Annealing learning rate schedule.
• —Hardware: Trained natively on Apple Silicon Metal (MPS).

💻 Quickstart & Inference

Using the Python Client

import laya

# Initialize the distilled 149M model on Apple Silicon MPS or CUDA
agent = laya.Agent("knpatil/laya-browser-agent-base", device="mps")

state = {
    "page": {
        "url": "https://huggingface.co/models",
        "title": "Hugging Face Models",
        "text": "Explore over 1M open-source AI models and datasets."
    },
    "elements": [
        {"index": "1", "role": "textbox", "label": "Search models, datasets, users..."},
        {"index": "2", "role": "link", "label": "Tasks"},
        {"index": "3", "role": "link", "label": "Libraries"}
    ]
}

questions = {
    "action_type": {
        "type": "choice",
        "instructions": "Given the goal 'Search for ModernBERT models', what immediate browser action should be taken?",
        "criteria": {
            "click": "Click a visible link, button, or tab",
            "type": "Enter search text into an input field",
            "scroll": "Scroll down to reveal more content",
            "wait": "Wait for dynamic content to load"
        }
    }
}

prediction = agent.predict(state, questions)
print("Action Decision:", prediction["answers"]["action_type"]["choice"])
print("Confidence:", prediction["answers"]["action_type"]["confidence"])

📜 Citation & Credits

Developed as part of the BroPilot autonomous browser companion project.

• —Backbone: ModernBERT by Answer.AI & LightOn.
• —Teacher Checkpoint: `knpatil/laya-browser-agent`.