Datasetpublic

Hzou9/PCBSchemaGen-Benchmarks

PCBSchemaGen Benchmarks Two benchmark suites for LLM-driven PCB schematic synthesis, from the paper PCBSchemaGen: Reward-Guided LLM Code Synthesis for Printed Circuit Board (PCB) Schematic Design with Structured Verification. Correctness in this domain is not defined by unit tests: there are no per-task golden references, and SPICE does not validate schematic-level correctness. Instead, each task is scored by a deterministic structural verifier against real-IC pin- and… See the full description on the dataset page: https://huggingface.co/datasets/Hzou9/PCBSchemaGen-Benchmarks.

sourceHugging Facemitupdated 3mo agoView on Hugging Face

0likes47downloads

Dataset Card

PCBSchemaGen Benchmarks

Two benchmark suites for LLM-driven PCB schematic synthesis, from the paper *PCBSchemaGen: Reward-Guided LLM Code Synthesis for Printed Circuit Board (PCB) Schematic Design with Structured Verification*.

Correctness in this domain is not defined by unit tests: there are no per-task golden references, and SPICE does not validate schematic-level correctness. Instead, each task is scored by a deterministic structural verifier against real-IC pin- and topology-level constraints (no LLM, no API key, no cached results). The verifier and knowledge graphs live in the GitHub repository.

Suites

Config	Tasks	Easy / Medium / Hard	Real ICs	Domains
`pcbbench`	62	17 / 28 / 17	41 commercial	22
`open_schematics`	165	67 / 63 / 35	439 commercial	22
Total	227	—	480	22

Load

python

from datasets import load_dataset

pcbbench = load_dataset("Hzou9/PCBSchemaGen-Benchmarks", "pcbbench", split="test")
ose      = load_dataset("Hzou9/PCBSchemaGen-Benchmarks", "open_schematics", split="test")
print(pcbbench[0])

Fields

Both suites share these 10 core fields:

Field	Description
`id`	Task identifier
`level`	Difficulty: `Easy` / `Medium` / `Hard`
`type`	Circuit type (e.g. Sensing, Power, Digital)
`task`	Natural-language design specification
`input_nodes`	Required input net(s)
`output_nodes`	Required output net(s)
`input_voltage`	Input voltage / range
`output_voltage`	Output voltage / range
`components`	Required component set (real ICs and/or passives)
`sub_module_name`	Canonical sub-module name for the target circuit

open_schematics additionally carries two optional ground-truth fields, present on a subset (20/165) of tasks and null elsewhere:

Field	Description
`n_gt_components`	Ground-truth component count for the reference schematic
`gt_component_types`	Number of distinct ground-truth component types

Evaluation

A candidate is a SKiDL (Python) schematic-generation program. Run it, then score the resulting schematic with the deterministic 5-layer verifier in the GitHub repo (framework/topo/), which checks pin-level assignments and topology against constraints induced from real IC datasheets. See the repo README for the runner and per-task configuration.

License

Citation

If you use PCBBench or Open-Schematics-Eval, please cite:

bibtex

@article{zou2026pcbschemagen,
  title   = {PCBSchemaGen: Reward-Guided LLM Code Synthesis for Printed Circuit Board (PCB) Schematic Design with Structured Verification},
  author  = {Zou, Huanghaohe and Han, Peng and Nazerian, Emad and Zhang, Mafu and Guo, Zhicheng and Huang, Alex Q.},
  journal = {arXiv preprint arXiv:2602.00510},
  year    = {2026},
  url     = {https://arxiv.org/abs/2602.00510}
}