Personalization
PersonalizationV3PersonalizationV4
PersonalizationV4
PersonalizationV4 (PV4) is a synthetic personalization benchmark. Each user is a detailed fictional persona who has
had 200 short conversations with an AI assistant. The evaluation questions place the user in a new scenario and ask
what they would most likely do or prefer, and each one is written to require combining at least two facts about the user. A model
never sees the persona itself: it gets the user's conversations, in which those traits are shown rather… See the full description on the dataset page: https://huggingface.co/datasets/memorilla/PersonalizationV4.enron_personalization_test
Dataset Card for "enron_personalization_test"
More Information needed
recenter-personalization
Re-Centering Humans in LLM Personalization — Data
Data for the paper Re-Centering Humans in LLM Personalization
(Lechen Zhang, Jiarui Liu, Tal August). Code: https://github.com/orange0629/recenter-personalization
We frame personalization as a three-stage pipeline and collect human judgments for every stage,
grounded in real users from WildChat:
Stage
Task
Human judgments
Folder
1
Attribute extraction — is an attribute extracted from a user's history accurate?
5,949 (1… See the full description on the dataset page: https://huggingface.co/datasets/leczhang/recenter-personalization.PG-Personalization-Amazon2023PersonalizationV4
PersonalizationV4
PersonalizationV4 (PV4) is a synthetic personalization benchmark. Each user is a detailed fictional persona who has
had 200 short conversations with an AI assistant. The evaluation questions place the user in a new scenario and ask
what they would most likely do or prefer, and each one is written to require combining at least two facts about the user. A model
never sees the persona itself: it gets the user's conversations, in which those traits are shown rather… See the full description on the dataset page: https://huggingface.co/datasets/MemoryAsModality/PersonalizationV4.