MyHeartCounts/OpenMHC-leaderboard-data

OpenMHC Leaderboard Data

Per-user substrate behind the OpenMHC wearable-health benchmark leaderboard. Each file is one method's reduced per-user, per-task values for one track; the leaderboard recompute consumes these to produce paired skill scores, cross-method ranks, and fairness skill scores.

This repo holds reduced metrics / predictions keyed by pseudonymous participant id — not raw sensor data.

Layout

<track>/<method>.parquet           e.g. downstream/xgboost.parquet, imputation/locf.parquet, forecasting/seasonal_naive.parquet
<track>/bootstrap/draws.parquet    per-draw bootstrap reference for the CIs

Tracks: downstream (Track 1), imputation (Track 2), forecasting (Track 3) — all live.

Schema

See `SCHEMA.md` for the full column spec per track (and each <track>/SCHEMA.md for track-specific submission details). In brief: each row is a per-user value for one task cell, evaluated on the canonical sharable_users_seed42_2026 test split. The tracks differ in what each row holds:

• —Track 1 (downstream / predictive tasks) stores the raw per-user prediction pairs (y_true, y_pred, y_proba) — its cohort-level metrics (AUPRC / Spearman / Pearson) don't decompose into a per-user error.
• —Track 2 (imputation) stores the per-user error E_per_user (MAE / 1 − AUC).
• —Track 3 (forecasting) stores the raw per-user metric_value (so one file serves skill, rank, and fairness — each reducer converts/uses it on load).

Why per-user (not aggregate)

The skill score is a paired per-user statistic against the track baseline (linear for downstream, locf for imputation, seasonal_naive for forecasting), and the rank is a per-user rank across all methods — so both require each method's per-user values, paired on user_id, not per-task aggregates.

Usage

import glob, os
import pandas as pd
from huggingface_hub import snapshot_download

root = snapshot_download("MyHeartCounts/OpenMHC-leaderboard-data", repo_type="dataset")
# one track at a time (schemas differ per track)
frames = [pd.read_parquet(p) for p in glob.glob(os.path.join(root, "downstream", "*.parquet"))]
df = pd.concat(frames, ignore_index=True)

Methods are uploaded with tools/upload_leaderboard_substrate.py; the per-track bootstrap references with tools/upload_leaderboard_bootstrap.py, both in the code repo.

Provenance

Generated by the OpenMHC evaluation harness. Baselines: linear (Track 1), locf (Track 2), seasonal_naive (Track 3).