Lexmount/WebJev-35B-A3B

WebJev-35B-A3B

WebJev-35B-A3B is a decision model built to drive browser agents on real websites.

At each step of a web task, WebJev reads the agent's view of the page and makes the step's two decisions:

• —which action to take next;
• —which of up to 255 on-page elements to take it on.

The page view is its URL, its visible text, its actionable elements and the recent actions.

WebJev is fine-tuned from Qwen3.5-35B-A3B-Base as a discriminative decision model: instead of generating text, it learns to score the options of a question. Its training combines two sources:

• —tens of thousands of execution-verified decisions that browser agents made on live websites;
• —a broad mixture of typed decisions: classification, routing, tool choice, policy and evidence checking, and knowledge-intensive multiple choice.

The result is precise element grounding on long, cluttered pages, reliable next-action choices and strong general structured decisions. Each decision takes a single forward pass and returns a full probability distribution over the options.

**Code** · **Dataset** · **Training**

[image]

Highlights

• —Next-action prediction and element grounding. WebJev decides directly from the agent's page state:
• —the action: click, type, select, scroll, wait, submit, dismiss, go back, finish or give up;
• —the element to act on, among up to 255 candidates on long real-world pages.

It learns both from execution-verified decisions on live websites (Lexmount/WebJev).

• —Stronger agents on the live web. In the same browser agent, with the same tasks and budget, WebJev completes 38.5% of 125 live-website tasks. jev-1.13 completes 16.7%, so WebJev solves 2.3× as many. It leads on all three task collections, with four times the success rate on WebGym.
• —General structured decisions. WebJev leads jev-1.13 on JevBench public (87.9 against 85.7) and on multi-class classification (86.3 against 83.3). It is ahead on Nimble evidence checking and customer-ticket triage too.
• —Fast and exact. Each decision is one forward pass with no decoding: a web-page decision takes about a third of a second on one A100. The answer is always one of the listed options.

Model overview

WebJev scores the options of each question about a state. It never generates free text, so there is no output parsing and no answer outside the options you list.

[image]

Evaluation

All numbers are measured with the released BF16 weights served by vLLM, at temperature 1.0. The jev-1.13 numbers were measured through its official API on the same inputs.

End-to-end web tasks

WebJev serves as the decision component of the same browser agent on 125 real-website tasks: 75 from Online-Mind2Web, 26 from WebGym and 24 from WebVoyager. Each run has a budget of 900 seconds and 60 actions. A deterministic grader checks the final page state and answer.

Success rate = solved ÷ evaluable tasks; tasks lost to browser infrastructure or grader errors are excluded.

General structured decisions

Eight benchmarks of structured decisions, beyond the web. Each item gives a state and a set of candidate answers, and the model's choice is correct when it equals the reference label. The table reports accuracy.

Speed

Measured on one NVIDIA A100 80GB with vLLM and BF16. Requests are sent one at a time; latency covers all questions about one state.

How to use

With transformers

The model needs a transformers version with Qwen3.5 MoE support (qwen3_5_moe_text, 5.15 or newer) and one GPU with at least 80 GB of memory.

import json, torch
from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "Lexmount/WebJev-35B-A3B"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="cuda").eval()

state = {"page": {"url": "https://www.spanishdict.com/", "title": "SpanishDictionary.com", "text": "…"},
         "elements": [{"index": "9", "role": "combobox", "label": "Translate Spanish or English", "value": "spring"},
                      {"index": "14", "role": "option", "label": "spring"}],
         "recent_actions": [{"action": "Translate Spanish or English", "kind": "fill", "text": "spring"}]}
options = ["CLICK", "TYPE_TEXT", "SCROLL_DOWN", "PRESS_ENTER", "DONE", "BLOCKED"]
prompt = ("Context:\n" + json.dumps(state, ensure_ascii=False) +
          "\n\nQuestion: Search SpanishDict for 'spring'. Which operation should the agent perform next?\nOptions:" +
          "".join(f"\n({chr(65 + i)}) {o}" for i, o in enumerate(options)) + "\nAnswer: (")

inputs = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
with torch.no_grad():
    logits = model(**inputs).logits[0, -1]
label_ids = [tok.encode(chr(65 + i), add_special_tokens=False)[0] for i in range(len(options))]
probs = torch.softmax(logits[label_ids].float() / 1.0, -1)       # temperature from decider_config.json
print(dict(zip(options, probs.tolist())))

For questions with more than 10 options, labels continue as single tokens (K … Z, then AA, AB, …). Build those prompts with decider/prompt.py from this repository, which renders every label as one token.

With vLLM

Start a server that returns the logits of the allowed tokens:

vllm serve Lexmount/WebJev-35B-A3B --dtype bfloat16 --max-model-len 34816 \
    --logprobs-mode processed_logits --max-logprobs 256

Then score a prompt: one generated token, restricted to the option labels.

import math, requests

ids = tok(prompt, add_special_tokens=False)["input_ids"]          # the prompt above, ending with "Answer: ("
out = requests.post("http://127.0.0.1:8000/v1/completions", json={
    "model": "Lexmount/WebJev-35B-A3B", "prompt": ids, "max_tokens": 1, "temperature": 1.0,
    "logprobs": len(options), "allowed_token_ids": label_ids, "return_tokens_as_token_ids": True}).json()
top = out["choices"][0]["logprobs"]["top_logprobs"][0]           # {"token_id:<id>": logit}
logits = [top[f"token_id:{i}"] for i in label_ids]
z = [math.exp(x - max(logits)) for x in logits]
print(dict(zip(options, [x / sum(z) for x in z])))

How it works

The input is Context: {state}, followed by the question, the lettered options (A) … (B) … and the answer position Answer: (. At that position, the model's hidden state is projected onto the LM-head rows of the option labels only and normalized with a softmax over the valid labels. The labels are never generated.

• —Several questions about one state are scored as separate prompts that share the state as a prefix.
• —Option order is part of the input. The model was trained with shuffled options, so it conditions on the candidates' content rather than their position.
• —Inference settings are stored in decider_config.json: temperature 1.0, up to 255 options per question, state before question.

Model architecture

Training

• —Recipe, data mixture and scripts: github.com/lexmount/WebJev/tree/main/train.
• —Web-agent decision data: Lexmount/WebJev.

Intended uses

Intended uses.

• —The decision component of browser agents: next-action prediction and element grounding.
• —Typed decisions in software pipelines: routing, classification, extraction choices, policy and evidence checks. Each is a question with an explicit option list.

Out of scope.

• —Free-form generation, chat and open-ended question answering.
• —Decisions whose options are not listed in the input.

Usage notes.

• —Knowledge-intensive decisions. WebJev decides from the state, so include the relevant facts in it.
• —Confidence. Probabilities use temperature 1.0 and rank options reliably. To act on confidence thresholds, check calibration on your own labels.
• —Input and hardware. Inputs of up to 16,384 tokens. One GPU with 80 GB of memory runs the BF16 weights.
• —Live websites change over time, so end-to-end results on live tasks vary between runs.

License

WebJev-35B-A3B is released under the Apache License 2.0. You may download, use, fine-tune and redistribute the weights, including for commercial use.

• —The model is a fine-tuned derivative of Qwen3.5-35B-A3B-Base, released under the Apache License 2.0.
• —The prompt builder in decider/ derives from the open-source Decider package, under the same license.

Citation

@misc{lexmount2026webjev,
  title        = {WebJev-35B-A3B: A One-Pass Decision Model for Web Agents},
  author       = {{Lexmount}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Lexmount/WebJev-35B-A3B}},
  url          = {https://github.com/lexmount/WebJev}
}

Contact

Lexmount, via the Lexmount organization on Hugging Face.