ASSERT-KTH/stack-smol-docstrings

Stack-Smol-Docstrings

This dataset contains Python functions extracted from the-stack-smol, filtered for high-quality docstrings and implementations. Each sample includes the function's docstring, implementation, and a masked version of the code where the function is replaced with a comment.

The dataset is designed for code completion tasks where a model needs to restore a function that has been replaced with a comment. The model is provided with:

1. 1.The full file context with the function replaced by a comment
2. 2.The docstring of the function
3. 3.The function name

The model's task is to generate code that replaces the comment with a proper implementation of the function based on the docstring and surrounding context.

Dataset Structure

Each sample contains:

• —function_name: Name of the function
• —docstring: The function's docstring
• —masked_code: The full file with the function replaced by a comment
• —implementation: The original function implementation
• —start_line: The starting line number of the function in the original file
• —end_line: The ending line number of the function in the original file
• —file_content: The full original file content

Quality Filtering

Functions are filtered based on:

• —Docstring quality (length, structure, descriptiveness)
• —Implementation quality (no SQL strings, reasonable number of variables, sufficient complexity)