datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
truth-probe-activationspythia-massive-activations
Hidden Dynamics of Massive Activations in Transformer Training
Dataset Description
This dataset contains comprehensive analysis data for the paper "Hidden Dynamics of Massive Activations in Transformer Training". It provides detailed measurements and mathematical characterizations of massive activation emergence patterns across the Pythia model family during training.
Massive activations are scalar values in transformer hidden states that achieve values orders of… See the full description on the dataset page: https://huggingface.co/datasets/Aimpoint-Digital/pythia-massive-activations.tda-activations