helloadhavan/python-docstrings

Python Docstring Diff Dataset

This dataset contains training samples for models that generate Python documentation patches. Each example provides a Python source file with its docstrings removed and a corresponding unified diff patch that restores the documentation.

The dataset is designed for training or evaluating language models that assist with:

• —Automatic code documentation
• —Docstring generation
• —Code review automation
• —Developer tooling
• —Dataset Structure

Each entry contains the following fields:

Field Description ------------------- instruction| Task instruction given to the model code| Python source code with docstrings removed response| A unified diff patch that adds the correct docstrings file| Original file path from the source project

Task Format

The model receives a Python file missing its documentation and must produce a unified diff that adds appropriate docstrings.

Example input:

def load_json(path):
    with open(path) as f:
        return json.load(f)

Example expected output:

--- a/file.py
+++ b/file.py
@@
 def load_json(path):
+    """Load JSON data from a file path."""
     with open(path) as f:
         return json.load(f)

Data Sources

The dataset was generated by scanning Python packages in github. Docstrings were extracted from functions, classes, async functions, methods, and modules using Python's AST parser. Low-quality documentation was filtered out using heuristics such as:

• —Minimum docstring length
• —Removal of TODO or placeholder documentation
• —Deduplication of similar examples

Intended Use

This dataset is useful for training models that perform:

• —automatic docstring generation
• —documentation patch creation
• —codebase documentation improvement tools
• —AI-assisted code review systems

License

This dataset is released under the MIT License.