datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
details_deepseek-ai__deepseek-coder-1.3b-instruct
Dataset Card for Evaluation run of deepseek-ai/deepseek-coder-1.3b-instruct
Dataset Summary
Dataset automatically created during the evaluation run of model deepseek-ai/deepseek-coder-1.3b-instruct on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_deepseek-ai__deepseek-coder-1.3b-instruct.details_AIGym__deepseek-coder-6.7b-chat
Dataset Card for Evaluation run of AIGym/deepseek-coder-6.7b-chat
Dataset automatically created during the evaluation run of model AIGym/deepseek-coder-6.7b-chat on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_AIGym__deepseek-coder-6.7b-chat.details_deepseek-ai__deepseek-coder-6.7b-instruct
Dataset Card for Evaluation run of deepseek-ai/deepseek-coder-6.7b-instruct
Dataset automatically created during the evaluation run of model deepseek-ai/deepseek-coder-6.7b-instruct on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train"… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_deepseek-ai__deepseek-coder-6.7b-instruct.openassistant-deepseek-coder
Chat Fine-tuning Dataset - OpenAssistant DeepSeek Coder
This dataset allows for fine-tuning chat models using:
B_INST = '\n### Instruction:\n'
E_INST = '\n### Response:\n'
BOS = '<|begin▁of▁sentence|>'
EOS = '\n<|EOT|>\n'
Sample Preparation:
The dataset is cloned from TimDettmers, which itself is a subset of the Open Assistant dataset, which you can find here. This subset of the data only contains the highest-rated paths in the conversation tree, with a total of 9,846 samples.
The… See the full description on the dataset page: https://huggingface.co/datasets/Trelis/openassistant-deepseek-coder.details_deepseek-ai__deepseek-coder-6.7b-base
Dataset Card for Evaluation run of deepseek-ai/deepseek-coder-6.7b-base
Dataset automatically created during the evaluation run of model deepseek-ai/deepseek-coder-6.7b-base on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_deepseek-ai__deepseek-coder-6.7b-base.details_AIGym__deepseek-coder-6.7b-chat-and-function-calling
Dataset Card for Evaluation run of AIGym/deepseek-coder-6.7b-chat-and-function-calling
Dataset automatically created during the evaluation run of model AIGym/deepseek-coder-6.7b-chat-and-function-calling on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_AIGym__deepseek-coder-6.7b-chat-and-function-calling.details_AIGym__deepseek-coder-1.3b-chat-and-function-calling
Dataset Card for Evaluation run of AIGym/deepseek-coder-1.3b-chat-and-function-calling
Dataset automatically created during the evaluation run of model AIGym/deepseek-coder-1.3b-chat-and-function-calling on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_AIGym__deepseek-coder-1.3b-chat-and-function-calling.details_deepseek-ai__deepseek-coder-7b-instruct-v1.5
Dataset Card for Evaluation run of deepseek-ai/deepseek-coder-7b-instruct-v1.5
Dataset automatically created during the evaluation run of model deepseek-ai/deepseek-coder-7b-instruct-v1.5 on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_deepseek-ai__deepseek-coder-7b-instruct-v1.5.java-deepseek-coder-1.3b-base-empty-10details_AIGym__deepseek-coder-1.3b-chat
Dataset Card for Evaluation run of AIGym/deepseek-coder-1.3b-chat
Dataset automatically created during the evaluation run of model AIGym/deepseek-coder-1.3b-chat on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_AIGym__deepseek-coder-1.3b-chat.java-deepseek-coder-1.3b-base-oneshot-10deepseek-coder-processeddetails_OpenBuddy__openbuddy-deepseekcoder-33b-v16.1-32k
Dataset Card for Evaluation run of OpenBuddy/openbuddy-deepseekcoder-33b-v16.1-32k
Dataset automatically created during the evaluation run of model OpenBuddy/openbuddy-deepseekcoder-33b-v16.1-32k on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_OpenBuddy__openbuddy-deepseekcoder-33b-v16.1-32k.deepseek-ai-deepseek-coder-v2-lite-instruct__llm-quality-persona-consistency-mini__019e3b6fdda4
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct on llm.quality.persona-consistency-mini (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
5
N Ok
5
Ok Rate
1
Persona Consistency Mean
0.88
Accuracy
0.88
Persona Consistency P50
0.8
Persona Consistency P95
1
Accuracy P05
0.8
Accuracy P50
0.8
Accuracy P95
1
Drift Rate
0.6
Mean Drift Turn
2.6667
TTFT P50
57.5324
ms
Total P50 Ms
2672.8064… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/deepseek-ai-deepseek-coder-v2-lite-instruct__llm-quality-persona-consistency-mini__019e3b6fdda4.python-deepseek-coder-1.3b-base-markdown-10python-deepseek-coder-1.3b-base-markdowndeepseek.coder.1.3b.base.python.mbpp.mxevaldeepseek-ai-deepseek-coder-v2-lite-instruct__code-generation-mbpp-mini__019e3b6f7d82
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct on code.generation.mbpp-mini (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
5
N Ok
5
Ok Rate
1
Pass At 1
1
Pass At 1 P05
1
Pass At 1 P50
1
Pass At 1 P95
1
Timeout Rate
0
TTFT P50
55.0369
ms
Total P50 Ms
1212.8
Tokens Out Total
807
Run configuration
Model: deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct @ unknown00
Engine: vllm… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/deepseek-ai-deepseek-coder-v2-lite-instruct__code-generation-mbpp-mini__019e3b6f7d82.deepseek.coder.1.3b.base.python.mbpp.taskdeepseek-coder-1.3b-base-empty-fnpython-deepseek-coder-1.3b-base-oneshot-10java-deepseek-coder-1.3b-base-oneshotcode-alpaca-eval-v0-deepseek-coder-7b-instruct-v1.5-annotationsjava-deepseek-coder-1.3b-base-emptyallenai_WildChat-1M-Full-neuralmagic_DeepSeek-Coder-V2-Instruct-FP8deepseek-ai-deepseek-coder-v2-lite-instruct__llm-quality-factual-mini__019e3b6f9bdd
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct on llm.quality.factual-mini (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
10
N Ok
10
Ok Rate
1
Accuracy
1
Accuracy P05
1
Accuracy P50
1
Accuracy P95
1
TTFT P50
45.0118
ms
Total P50 Ms
359.225
Tokens Out Total
373
Run configuration
Model: deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct @ unknown00
Engine: vllm vunknownQuantization:… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/deepseek-ai-deepseek-coder-v2-lite-instruct__llm-quality-factual-mini__019e3b6f9bdd.deepseek-ai-deepseek-coder-v2-lite-instruct__llm-inference-chatbot-short__019e3b6f22ca
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct on llm.inference.chatbot-short (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
TTFT P50
74.4268
ms
TTFT P99
521.1922
ms
TPOT P50
21.0371
ms
TPOT P99
23.458
ms
Total P50 Ms
2661.4077
Total P99 Ms
3136.5972
Req Per S Passing
1.0172
Req Per S All
1.1559
Compliance Rate
0.88
Ok Rate
1
Throughput Tok Per S
134.316
Power Avg W
808.3085
Power Peak W
854.62… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/deepseek-ai-deepseek-coder-v2-lite-instruct__llm-inference-chatbot-short__019e3b6f22ca.deepseek.coder.1.3b.base.python.mbpp.markdowndeepseek.coder.1.3b.base.python.mbpp.oneshotdeepseek.coder.1.3b.base.python.mbpp.simple
