HypoAgent/HypoAgent-SFT

HypoAgent-SFT: study description → prespecified statistical analysis plan

54,163 supervised fine-tuning pairs that teach a model to read a study registration and write the analysis plan that belongs to it. The input is a study description — objective, design, population, exposure, comparator, outcome, timing, whatever the registry record actually contains. The target is a prespecified hypothesis-testing and statistical-analysis plan built for that design.

This is the curated split used to train [HypoAgent/HypoAgent-Mixtral-8x7B-Instruct-SFT](https://huggingface.co/HypoAgent/HypoAgent-Mixtral-8x7B-Instruct-SFT), which on the held-out test split below beats base Mixtral-8x7B-Instruct in 93.6% of judged comparisons and cuts plans containing a critical statistical error from 33.8% to 14.4%.

[image]

Contents

data/sft/curated/train.csv    48,218 rows · 18,699 study families
data/sft/curated/test.csv      5,945 rows ·  2,315 study families

Two files, nothing else. The split is grouped by `study_family_id`: every task derived from one registry entry lands in the same split, so train/test family overlap is exactly 0. A family yields between one and five examples depending on how much the source record supported.

Of the 48,218 training rows, the Adaption run that produced the model consumed 47,700 after platform-side validation.

A companion Kaggle dataset, alikhajegilimirabadi/adaption-hypoagent-sft, holds the file the platform actually trained on. It is not a copy of these CSVs: each record there pairs the corpus prompt and plan with a platform-enhanced rewrite of both, and the released adapter's output style follows the enhanced rewrite rather than the fixed template used here. Use these files for the corpus and its metadata; use the Kaggle file to see what the training run consumed.

from datasets import load_dataset
ds = load_dataset("HypoAgent/HypoAgent-SFT")
print(ds)  # train: 48218, test: 5945

The text columns contain commas, quotes and newlines — read them with a real CSV parser, never by splitting on commas.

Record schema

The five tasks

A_full_plan targets run to twenty numbered sections: research question, hypothesis, estimand, null/alternative, primary endpoint, analysis units, primary method, why that method, assumptions, diagnostics, effect size, uncertainty, alpha, sample size, missing data, multiplicity, subgroups, sensitivity, decision rule, limitations. Median target length is 2,391 characters; the 95th percentile is 6,705.

The 6,816 D_critique prompts each carry exactly one deliberately planted error, drawn from nine failure modes that recur in real analysis plans:

How the targets were produced

Every assistant response is synthesized from extracted structured facts, not copied or paraphrased from source text. The pipeline pulls fields out of the registry record — design, randomization unit, arm count, outcome scale, timing, reported design parameters — and a rule-based reasoning engine keys the method off the combination of outcome distribution × dependence structure × unit of analysis × estimand × comparison type. A cluster-randomized trial with a continuous endpoint gets a linear mixed model with a random cluster intercept, plus a sentence explaining that clustering — not the raw participant count — drives precision. A single-arm study with a censored endpoint gets Kaplan-Meier description against a prespecified performance criterion, not a two-sample test.

Two consequences follow, and both matter:

• —No verbatim source text appears in any target. Only extracted structured fields are reused, which is what makes the AEA and OSF portions distributable at all.
• —These are not real statistical analysis plans. No trial statistician wrote them. They are internally consistent, design-appropriate plans generated from a rule set. A model trained on them learns correct structure and method selection, not the judgement of an experienced SAP author.

Curation ran as hard eligibility (outcome determinable, minimum length, bucket-adaptive quality floor) → keep_score ranking → budgeted allocation with source quotas, a per-family cap and a 20% ceiling on any single design. That rebalancing is why non-clinical studies make up 33.8% of these files against 21.4% of the 96k pre-curation pool, and why single-arm designs sit at 13.2% instead of 17.3%. Without it, two-arm clinical RCTs with continuous endpoints would swamp everything else.

Evaluation slices

challenge_sets marks examples belonging to a hard slice, so you can score a model on the structures it is most likely to get wrong rather than on the corpus average. Slices overlap; membership is pipe-separated.

Sources and licensing

The AEA and OSF portions are what make the corpus non-clinical: economics field and lab experiments, psychology and ecology preregistrations, online A/B tests. They are also the portions with unresolved licence terms, tagged human_review_status=license_review on every row. Filter on license or human_review_status to keep only the public-domain ClinicalTrials.gov subset if your use case needs clean terms.

Treat the corpus as research use. No verbatim source text is reproduced, but the licence posture of two of the three sources is unresolved rather than permissive. Attribute the underlying registries when you use the corresponding subsets; source_ids and source_urls resolve every row back to its originating record.

Example

A cluster-randomized ICU study, ctgov_NCT05144607::A_full_plan, abridged:

user

STUDY DESCRIPTION
Objective: Patient-ventilator asynchronies can occur as a result of a mismatch between
neural (patient) and ventilator inspiratory and expiratory phases. [...] The impact of
the display of Pmus together with the other ventilator waveforms on the ability of
health-care professionals to identify asynchronies has not been tested so far.
Domain: clinical / Mechanical Ventilation Complication
Design: cluster_randomized (randomized, parallel, masking: none)
Population / eligibility: Inclusion Criteria: * Healthcare professionals (physicians and
respiratory therapists) who work in intensive care units [...]
Intervention/exposure: Muscle Pressure curve (Pmus)
Comparator: placebo/standard care
Primary outcome: Ability of ICU health care professionals to detect patient-ventilator
asynchrony
Primary timing: Immediately after the completion of the test sessions
Number of groups/arms: 2
Planned enrollment: 105 (actual)
Reported design parameters: power 0.9, alpha 0.05

Task: Produce a rigorous, prespecified hypothesis-testing and statistical-analysis plan
for this study.

assistant (sections 6–8 of 20)

## 6. Study and analysis units
- Randomization/assignment unit: cluster
- Observation unit: participant
- Analysis unit: cluster
- Dependence structure: Participants within a cluster are correlated (intracluster
  correlation), so a random cluster intercept is included; ignoring it would understate
  standard errors and inflate the type I error.

## 7. Primary statistical method
Linear mixed model with a random cluster effect. Model: Linear mixed model: outcome ~
treatment (+ baseline, stratification factors) with a random intercept for cluster.
Primary test: Fixed-effect test of the treatment term.

## 8. Why the method is appropriate
Because randomization is at the cluster level while the outcome is measured on
individuals, observations are not independent within clusters. A linear mixed model with
a random cluster intercept (or GEE with robust standard errors) propagates the
intracluster correlation into the inference so the effective sample size, not the raw
participant count, drives precision.

Section 14 shows the other half of the design. The record reports 90% power at α = 0.05 with n = 105 but no ICC, so the target does not invent one — it states that the power calculation must match the mixed-model analysis and that a design/analysis mismatch invalidates the nominal power. Unreported inputs are named as prespecification decisions throughout rather than filled in with plausible numbers.

What this corpus is not

• —Not authoritative SAPs. Synthesized targets, described above.
• —Not a benchmark with ground-truth labels. The test split is held-out training-distribution data. Use it to compare models against each other, as the model card does; a target here is a reasonable plan, not the only correct one.
• —Not clinically validated. Nothing here has been reviewed by a trial statistician or a regulator.
• —Not multilingual. English only.

For questions, use the repository discussion page.