Lelonthecodeur/web-research-coding-5m

Web Research + GitHub + Website Coding Dataset

Version: 1.1.0

Total examples: 5,000,000

Splits

• —train: 4,750,000
• —validation: 125,000
• —test: 125,000

Core capabilities

• —Web search
• —Web research
• —Evidence extraction
• —Fact verification
• —Multi-hop research
• —Multi-layer technical analysis
• —Architecture analysis
• —Root-cause analysis
• —Security analysis
• —Performance analysis
• —UX analysis
• —Design analysis
• —Refactoring
• —Code review
• —Debugging
• —Website coding
• —Design systems
• —Developer experience
• —Developer onboarding
• —API developer experience
• —Documentation experience

Source diversity

• —Hugging Face: 20%
• —GitLab: 10%
• —Codeberg: 5%
• —Stack Exchange: 15%
• —MDN: 15%
• —PyPI: 10%
• —npm: 10%
• —Official web documentation: 10%
• —GitHub: 5%

Multi-layer reasoning structure

Each deep analysis can cover:

1. 1.Problem
2. 2.Context
3. 3.Evidence
4. 4.Architecture
5. 5.Implementation
6. 6.Quality
7. 7.Security
8. 8.Performance
9. 9.UX
10. 10.Design
11. 11.Developer Experience
12. 12.Validation
13. 13.Trade-offs
14. 14.Final synthesis

Rate limits

GitHub is intentionally limited to a small anchor pool. The 5,000,000 examples are generated locally from the collected source anchors and therefore do not require 5,000,000 API requests.

Data note

The dataset consists primarily of generated examples grounded in public technical source metadata and URLs. It is not intended to be a bulk reproduction of third-party website contents.