Datasetpublic

Omarrran/StackPulse_778K_QnA_Code_dataset

💻 StackOverflow-778K: Multi-Year Developer Q&A Dataset Dataset Summary A large-scale Stack Overflow question dataset containing 778,929 unique questions sampled across 7 years (2015–2022). Each question includes the raw HTML body, plain-text version, tags, score, view count, answer count, and a rich set of derived features for immediate ML use. Collected across 8 sampling runs on Feb 27 2026, deduplicated to 778,929 unique questions with only 2 duplicates… See the full description on the dataset page: https://huggingface.co/datasets/Omarrran/StackPulse_778K_QnA_Code_dataset.

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

0likes182downloads

Dataset Card

💻 StackOverflow-778K: Multi-Year Developer Q&A Dataset

Dataset Summary

A large-scale Stack Overflow question dataset containing 778,929 unique questions sampled across 7 years (2015–2022). Each question includes the raw HTML body, plain-text version, tags, score, view count, answer count, and a rich set of derived features for immediate ML use.

Collected across 8 sampling runs on Feb 27 2026, deduplicated to 778,929 unique questions with only 2 duplicates removed.

📁 Files in This Dataset

File	Format	Rows	Description
stackoverflow778kfull.csv	CSV	778,929	Complete dataset
stackoverflow_unanswered.csv	CSV	224,733	Questions with 0 answers
stackoverflowwithcode.csv	CSV	595,679	Questions containing code blocks
stackoverflowhighquality.csv	CSV	20,205	Score ≥ 5 and answered
stackoverflow_python.csv	CSV	107,083	Python-tagged questions
stackoverflow_javascript.csv	CSV	87,367	JavaScript-tagged questions
stackoverflow_java.csv	CSV	54,077	Java-tagged questions
stackoverflow_csharp.csv	CSV	40,420	C#-tagged questions
stackoverflow_android.csv	CSV	42,004	Android-tagged questions
*.jsonl versions	JSONL	same	HuggingFace-native format for all above

🏗️ Schema Reference

Column	Type	Description
id	int64	Unique Stack Overflow question ID
title	string	Question title
question_body	string	Raw HTML body (includes `<pre><code>` blocks)
body_text	string	Plain text body (HTML stripped, code replaced with [CODE])
tags	string	Pipe-separated tags e.g. `python\	pandas\	dataframe`
score	int64	Net upvotes (can be negative)
creation_date	string	ISO 8601 UTC creation timestamp
year	int32	Year extracted from creation_date
month	int32	Month (1–12)
hour	int32	Hour of day (0–23, UTC)
dayofweek	int32	Day of week (0=Monday, 6=Sunday)
view_count	int64	Total question views
answer_count	int64	Number of answers received
body_len	int64	Plain text body character length
title_len	int64	Title character length
tag_count	int64	Number of tags (1–5)
codeblockcnt	int64	Number of `<pre>` code blocks in body
has_code	bool	True if question contains at least one code block
is_unanswered	bool	True if answer_count == 0
is_popular	bool	True if view_count > 95th percentile (~2,599)
is_viral	bool	True if view_count > 99th percentile (~11,513)
ishighlyvoted	bool	True if score >= 10
is_negative	bool	True if score < 0
score_bucket	string	"negative" / "zero" / "low" / "medium" / "high"

📈 Dataset Statistics

Overview

—Total questions: 778,929
—Date range: 2015-02-14 → 2022-09-25
—Missing years: 2019, 2021 (sampling gaps)
—Unique tags: 41,754
—Zero nulls in all core columns (2 questions have empty tags)

Score Distribution

—Mean: 0.69 | Median: 0 | Std: 4.77
—Range: -27 to 1,061
—Negative score: 63,415 (8.14%)
—Zero score: 461,977 (59.31%) — majority never upvoted
—Score ≥ 10: 7,431 (0.95%)
—Score ≥ 100: 246 (0.032%)

View Count Distribution

—Mean: 794 | Median: 66 | P95: 2,599 | P99: 11,513
—Max: 915,870 views
—Popular (>P95): 38,944 (5.00%)
—Viral (>P99): 7,789 (1.00%)

Answer Count

—Unanswered: 224,733 (28.85%)
—1 answer: 387,399 (49.73%)
—2+ answers: 166,797 (21.42%)
—Max answers on a single question: 36

Question Body

—Has code block: 595,679 (76.47%)
—Avg code blocks per question: 1.56
—Avg body length: 557 chars | Median: 447 chars
—Avg title length: 59 chars

Questions Per Year

—-1: 2,694
—2015: 99,664
—2016: 120,887
—2017: 20,791
—2018: 20,096
—2020: 99,676
—2022: 415,121 (Note: 2017/2018 low counts reflect sampling focus; 2022 dominates at 53%)

⚠️ Known Issues & Caveats

1.YEAR GAPS: Years 2019 and 2021 are absent — this is a sampling artifact, not a gap in SO activity. Do not use for temporal trend analysis without noting this.

1.2022 DOMINANCE: 415,121 questions (53%) are from 2022. The dataset skews heavily toward recent questions. Stratify by year if balance matters.

1.RAW HTML: question_body contains raw HTML including <, >, <pre><code> blocks. Use body_text for NLP tasks. Use question_body for HTML-aware or code-extraction tasks.

1.SCORE SKEW: 59.3% of questions have score=0. Mean (0.69) is misleading. Use score_bucket or is_highly_voted for classification tasks.

1.VIEW COUNT SKEW: Mean (794) is 12× the median (66) due to viral questions. Use log-transformed view_count for regression tasks.

1.PIPE-SEPARATED TAGS: The tags column uses | as delimiter e.g. python|pandas|dataframe. Split with str.split("|") before use.

1.CODE PLACEHOLDER: In body_text, all <pre>...</pre> blocks are replaced with the token [CODE]. The original HTML is preserved in question_body.

1.DUPLICATE IDs: 2 exact duplicates were found and removed during processing.

🚀 Quick Start

pandas

python

import pandas as pd

# Full dataset
df = pd.read_csv("data/stackoverflow_778k_full.csv")

# High quality only (score >= 5, answered)
hq = pd.read_csv("data/stackoverflow_high_quality.csv")

# Python questions only
py = pd.read_csv("data/stackoverflow_python.csv")

# Unanswered questions (good for difficulty modeling)
ua = pd.read_csv("data/stackoverflow_unanswered.csv")

# Split tags into list
df["tag_list"] = df["tags"].str.split("|")

# Filter by year
df_2022 = df[df["year"] == 2022]

# Log-transform view count for regression
import numpy as np
df["log_views"] = np.log1p(df["view_count"])

HuggingFace datasets