preference-data
open-image-preferences-v1-flux-dev-loragemma-2-2b-it-preference_dataset_mixture2_and_safe_pku-Preferencegemma-3-1b-it-preference_dataset_mixture2_and_safe_pku-PreferenceLlama_3_2_1B_Filler_preference_based_training_data_20251210_114617_DPO_v2gemma-2-2b-it-preference_dataset_mixture2_and_safe_pku-Preference-2048Llama_3_2_1B_Filler_context_based_training_data_20251105_172125_DPO_preference_finetuned_dec10MNLP_purturbed_preference_data_relevance_rampLlama_3_2_1B_Filler_preference_based_training_data_20251210_080644_DPO
tulu-2.5-preference-data
Tulu 2.5 Preference Data
This dataset contains the preference dataset splits used to train the models described in Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback.
We cleaned and formatted all datasets to be in the same format.
This means some splits may differ from their original format.
To see the code used for creating most splits, see here.
If you only wish to download one dataset, each dataset exists in one file under the data/… See the full description on the dataset page: https://huggingface.co/datasets/allenai/tulu-2.5-preference-data.700k_Human_Preference_Dataset_FLUX_SD3_MJ_DALLE3
NOTE: A newer version of this dataset is available Imagen3_Flux1.1_Flux1_SD3_MJ_Dalle_Human_Preference_Dataset
Rapidata Image Generation Preference Dataset
This Dataset is a 1/3 of a 2M+ human annotation dataset that was split into three modalities: Preference, Coherence, Text-to-Image Alignment.
Link to the Coherence dataset: https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Coherence_Dataset
Link to the Text-2-Image Alignment dataset:… See the full description on the dataset page: https://huggingface.co/datasets/Rapidata/700k_Human_Preference_Dataset_FLUX_SD3_MJ_DALLE3.preference-dataset-qwen3preference-datasets-tulupreference_data_llama_factory_wo_checklist
Dataset Card for "preference_data_llama_factory_wo_checklist"
More Information needed
Apertus-v1.5-Preference-Data
Apertus 1.5 Preference Dataset
This is the preference dataset used for the offline DPO stage of Apertus v1.5 alignment training, applied to the 70B model.
The prompts come from Ai2's Olmo 3 Dolci-Instruct-DPO dataset. We only reuse the prompts from Dolci-Instruct-DPO; all chosen / rejected responses in this dataset were generated by us.
How this dataset was built
Prompts. Taken from Dolci-Instruct-DPO (ODC-BY).
Response generation and annotation. Every prompt was… See the full description on the dataset page: https://huggingface.co/datasets/swiss-ai/Apertus-v1.5-Preference-Data.