najdresearch/system-one

Najd System One — public research draft

Decision tasks in English, MSA and Saudi Arabic. This draft is for research and debugging; independent linguistic and label review is pending.

What is inside?

Each case gives a model some context and asks it to make a small decision. The expected answer comes from the dataset, not from a model run.

The same request in three registers

These are actual aligned rows from natural-development. The policy and question are localized too; the table shows the request only.

Choice — English

Actual case: N0101-en. The supplied policy is fictional; this is not government or company guidance.

{
  "id": "N0101-en",
  "state": {
    "policy": "Route by requested service. Municipal shop permits go to licensing; broken streetlights to maintenance; invoice copies to billing. Ask for clarification when the service is unspecified. Requests to disclose another person’s records go to privacy review.",
    "request": "I want to apply for a permit for my new shop."
  },
  "questions": {
    "action": {
      "criteria": {
        "billing": "Billing",
        "clarify": "Clarify service",
        "licensing": "Licensing",
        "maintenance": "Maintenance",
        "privacy": "Privacy review"
      },
      "instructions": "Choose the next action using the supplied policy.",
      "type": "choice"
    }
  },
  "expected": {
    "action": "licensing"
  }
}

Boolean — MSA

Actual case: B01-ar-MSA. The supplied policy is fictional; this is not government or company guidance.

{
  "id": "B01-ar-MSA",
  "state": {
    "policy": "سياسة افتراضية: يُسمح بالدفع فقط عند وجود اعتمادين وأمر شراء مطابق وفاتورة غير مدفوعة وحساب بنكي موثق. نقص أي شرط أو عدم معرفته يعني عدم السماح.",
    "request": "اعتمد المديران الفاتورة غير المدفوعة، وأمر الشراء مطابق والحساب موثق."
  },
  "questions": {
    "action": {
      "instructions": "هل يُسمح بالدفع الآن؟",
      "type": "noul"
    }
  },
  "expected": {
    "action": true
  }
}

Score — Saudi Arabic

Actual case: S01-ar-SA. The supplied policy is fictional; this is not government or company guidance.

{
  "id": "S01-ar-SA",
  "state": {
    "policy": "سلم أولوية تقني افتراضي: ٠ لطلب معلومات دون عطل، و١ لتعطل مستخدم واحد دون حل بديل، و٢ لتعطل عدة مستخدمين دون حل بديل، و٣ لاشتباه نشط باختراق حساب بصرف النظر عن عدد المستخدمين. اختر أعلى مستوى منطبق.",
    "request": "وين ألقى دليل إعداد VPN؟ كل شي شغال."
  },
  "questions": {
    "action": {
      "criteria": [
        "معلومات",
        "تعطل مستخدم واحد",
        "تعطل عدة مستخدمين",
        "اشتباه اختراق"
      ],
      "instructions": "اختر مستوى الأولوية من السلم المرفق.",
      "type": "score"
    }
  },
  "expected": {
    "action": 0
  }
}

Original cases cover government routing, complaints, prerequisites, banking, telecom/logistics, HR/IT, procurement, and tool/text-UI actions. Adapted upstream packs add intent classification, tool selection, and language/culture diagnostics; see the pack table and per-subset licenses.

Files and columns

Join viewer rows to the metadata file for the same configuration using id. The viewer omits provenance_json and release_rights_json. Benchmark rows keep their original bytes for reproducible experiments.

state_json contains the supplied context, questions_json defines the decision and allowed answers, and expected_json contains the gold answer. Parse these three columns with json.loads when using the viewer export.

Use and limitations

Viewer fields ending in _json preserve heterogeneous decision types as JSON strings. Benchmark runners use the original typed cases.jsonl files and verify their manifest hashes. The viewer calls development data train for tooling compatibility; this is not a training recommendation. Original split names remain in each row. Published reserved cases are not secret holdouts. Policies in Najd-authored data are fictional. Controlled cases share one logical template; translations are correlated. Do not treat every row as independent or pool packs into one headline score. The Saudi controlled wording is record-style, not natural customer conversation. Source Arabic locales are not automatically Saudi dialect. Rights apply per subset; no blanket relicensing of third-party material. Attribution and transformations are retained in row provenance and clearance.json.

Reproduction

Builders: https://github.com/najdresearch/datasets . Scoring: https://github.com/najdresearch/benchmark . Pin the Hugging Face commit SHA and verify release-lock.json before evaluation.