Jerry-lin23/deepseek-leetcode-p3

DeepSeek-Coder-6.7B LeetCode Fine-Tuned

A fine-tuned version of DeepSeek-Coder-6.7B-Base specialized for solving LeetCode-style algorithmic problems in Python.

Model Details

Performance

Evaluated on 100 LeetCode problems with automated code execution:

Key Findings

• —Significant gains on Easy and Hard problems — model learned both fundamental patterns and complex algorithms
• —Slight regression on Medium — possible overfitting to extremes of difficulty distribution
• —Domain-specific data matters — initial training on general coding data degraded performance

Intended Use

• —Solving algorithmic coding challenges
• —LeetCode practice and learning
• —Code generation for competitive programming
• —Educational tool for understanding algorithmic solutions

Limitations

• —Optimized specifically for LeetCode-style problems, may not generalize to other coding tasks
• —Python-only (not trained on other languages)
• —May produce syntactically correct but logically incorrect solutions
• —Struggles with problems requiring complex data structure implementations (LinkedList, Trees)

How to Use

With Hugging Face Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "Jerry-lin23/deepseek-leetcode-fp16",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Jerry-lin23/deepseek-leetcode-fp16")

prompt = """### Problem:
Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.

### Starter Code:

class Solution: def twoSum(self, nums: List[int], target: int) -> List[int]:

### Solution:

"""

inputs = tokenizer(prompt, returntensors="pt").to(model.device) outputs = model.generate(**inputs, maxnewtokens=200, temperature=0.2) print(tokenizer.decode(outputs[0], skipspecial_tokens=True))

### With Ollama (Local Deployment)

1. Convert to GGUF format
2. Create a Modelfile:

FROM ./deepseek-leetcode-q8.gguf PARAMETER temperature 0.2 PARAMETER top_p 0.95

3. Import: `ollama create deepseek-leetcode -f Modelfile`
4. Run: `ollama run deepseek-leetcode`

## Training Details

### LoRA Configuration

LoraConfig( r=16, loraalpha=32, targetmodules=["qproj", "kproj", "vproj", "oproj", "gateproj", "upproj", "downproj"], loradropout=0.05, bias="none", tasktype="CAUSALLM" )

### Training Arguments

SFTConfig( numtrainepochs=3, perdevicetrainbatchsize=1, gradientaccumulationsteps=8, learningrate=2e-4, warmupratio=0.03, fp16=True, gradientcheckpointing=True, maxseqlength=2048, datasettext_field="text" )

## Citation

@misc{deepseek-leetcode-finetuned, author = {Jerry Lin}, title = {DeepSeek-Coder-6.7B LeetCode Fine-Tuned}, year = {2024}, publisher = {Hugging Face}, url = {https://huggingface.co/Jerry-lin23/deepseek-leetcode-fp16} }

## Links

- 📊 [Benchmark Repository](https://github.com/jerrylin-23/DeepSeek-LeetCode-Oriented-Training)
- 🤗 [Base Model](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base)
- 📚 [Training Dataset](https://huggingface.co/datasets/LongQ/leetcode_python)