ZipLime/commitments-of-traders

Commitments of Traders

Who was long and who was short in every US futures market — and, for once, when anyone could actually see it.

421 223 market-weeks · 211 071 point-in-time rows · 2 585 weekly releases · 762 markets · 2010-01-05 to 2026-09-08

The pipeline lives in `recipe/` at the same revision as the data. See PIPELINE.md for the method.

The data is from Tuesday. It comes out on Friday.

A COT report is taken as of the close on Tuesday and published at 3:30 p.m. Eastern on the Friday — three days later. Almost every chart of this data is drawn against the Tuesday, and almost every backtest built on it therefore acts on positioning three days before it existed.

This dataset carries both dates and never confuses them:

The median gap is 3 days. It is not always three: 273 of 2 585 releases came out later than that.

The gap is not a constant, and that is the point

Federal holidays push the release into the following week. And when the CFTC stops publishing altogether, the gap stops being days at all. The autumn of 2025, from the agency's own catch-up schedule:

A rule would have dated the 30 September report to 3 October. Seven weeks of positioning that nobody had would have been sitting in the backtest, in the middle of a period when it mattered.

Where each timestamp comes from

release_source says, and the four values are in descending order of how directly the CFTC stated it:

The rule is not a guess. It reproduces the CFTC's published 2026 schedule exactly — all fifty-two dates, including the six the agency marked as moved by a holiday — and tests/test_schedule.py asserts that on every one of them. Where a derived instant could still be wrong, it errs late: being late costs a backtest some edge, being early invents information.

What is in it

The three are not versions of one report: they split the same open interest among different categories, for different halves of the market. A physical commodity is in the disaggregated report and a financial future in the traders-in-financial-futures one, so a market has a legacy row and at most one detailed row — which is what lets pit put them side by side, one row per market per week.

Net positions (net_managed_money, net_leveraged_funds, net_commercial and the rest) are long minus short, computed here so that it is computed the same way everywhere. Long and short are kept alongside.

Corrections leave one trace, and it is in the data

The CFTC revises past weeks and republishes only the corrected state, so the archive cannot show what a week used to say. But every report prints how much open interest moved since the week before, computed at publication. Where that figure no longer matches the two weeks as the archive now stands, one of them was changed afterwards.

1 082 market-weeks carry that mark, out of 412 594 that can be checked. change_reconciles is false on them.

Using it

import polars as pl

pit = pl.read_parquet("data/pit/*.parquet")

# Managed money in gold, as it was actually published
gold = pit.filter(pl.col("market_name") == "GOLD").select(
    "event_date", "knowledge_date", "net_managed_money", "open_interest"
)

# Everything a strategy could have known on a given morning
known = pit.filter(pl.col("knowledge_date") <= pl.lit("2025-11-01").str.to_datetime(time_zone="UTC"))

That last filter is the one that matters. Run it against 1 November 2025 and the September and October reports are correctly absent — because they had not been published yet.

Known gaps

• —Futures-only, not futures-and-options combined. The combined reports exist and are a known gap; publishing both under the same market names would double every series.
• —No vintages before the archive's current state. The CFTC keeps one state per week and corrects it in place, so what a week said when it was first published is not recoverable for history. change_reconciles is the only surviving trace. Vintages accumulate going forward.
• —Backfill starts in 2010, because that is where the disaggregated and financial archives begin. The legacy report reaches 1986 and is one configuration change away.
• —Three consolidated index markets do not add up. DJIA, NASDAQ-100 and S&P 500 Consolidated combine contracts of different sizes, so their position columns do not sum to their open interest. 564 rows of 39 766 in the financial report; the arithmetic is the CFTC's, not a parsing fault.
• —Two disruptions are reconstructed from prose. The 2018-19 lapse in appropriations and the 2023 ION incident were announced in words rather than in a table. Those rows carry release_source = suspension and release_estimated = true.

Provenance

Every release row carries the archive it was read from, the sha256 of those bytes, and the parser version. Sources: the CFTC's annual archives, release schedule and historical special announcements. US Government works, public domain. Rebuilt on Hugging Face Jobs after each Friday release, with a sweep for the weeks that move.