verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1� R � & � � � c ; � � � � �� �� <� w� �� �
� � Q� �� �� `� ɛ C� �� �� � �� ɢ !� c� Ԥ �� �� �� ߮ �� � �� p� ,� .� �� �� �� k� :� �� L� �� �� �� �� )� V� �� �� 8� � �� � �� � � , / y _ w d ` � � � � � � � ! � B " &- �- 2. �L �N }P }R �R �W Y �Z �Z \ �] ^ _ b Tb c Tc �d }e �e |f �f 1i �} �} D ̵ M� � 3� d� N� m� �� �� Q� %� �� ]� 1� � �� �� '� j� D� `� 8� Z� �� �� �� j� � �3 S � j > \ � ! P% * P+ d. W/ k@ �C QD iJ lL �L �S NU qU V �Z _ 5` �a �b �d �d �e �h =l �r b� �� �� R� q� �� � v� ۛ Z� r� ў �� �� 5� λ ^� �� �� !� � '� � 4� z� 0� �� �� �� -� �� � ]� � �� � �� �� :� �� � 6 � " �: 1@ �L N �P !Q �U �V AW �W xX Z JZ �t ]� ݲ � � | � � D � 7 � �
� � / # � � � � U � �" e# �# $ C% A& �& (( s( �5 �7 KD �D �D �D �G �e �h �i �u �v �w Ax �x �{ J� [� �� ϋ �� K� R� �� � ڑ ]� %� �� 8� �� x� f� �� � � ץ 5� ɦ ܦ �� �� � �� Ӯ d� ]� U� �� (� �� �� �� �� �� W� �� �� �� q� �� w� �� �� K� �� �� �� 7� �� �� �� T� � =� �� �� �� �� )� { �
� O � = � . _. �. �8 �9 Z> �> �> I@ �@ �E �G H �H WI �P �P �R NS �T �U �X J^ �^ \h �h �j �j �m �m �n "q ?z 8 � � g� v� �� �� � |� ]� � E� �� T� [� H� Ӡ š v� B� k� R� �� �� �� � � �� �� » �� �� � � �� �� r� �� �� Q� �� � �� �� �� G� �� �� <� b� �� �� �� -� � � � 7 }# �#
$ H( ) ) �) �) 5, �- `. F/ �/ >1 �D �Q �U �V �X �X �] �^ ?_ O`
c ~d Ce �e ul 0n �n Jo �q �q w w aw x �y Jz �z w} �} � S� � � ݊ R� n� � M� � ǒ Y� � 3� K� �� ,� � �� )� ]� T� ;� v� n� �� � *� �� i� !� �� I� �� �� -� �� m� �� �� �� [� � � �� i� �� }� L� 2� =� �� f� �� �� �� �� � �� �� �� �� � H 5 � V | � 64 l5 � � % $� 5� �� � � . Y D � � � Y 5 � � r u � � B ; �; �; �? �A gD �D "E �E �F nG �H 0U �V W �W 6X �X �Y �] �a ob �b hc �i �q �x \y �z �{ �{ -� � � Å m� �� J� �� � ,� � �� � �� � �� �� �� �� �� �� �� l� ��
� �� _� � �� � I q U( �) B, ; �; < ? �H 0U �W �_ .c �c �c �d mh Ei �i �s à s� ߋ ڎ C� � �� c� `� � &� v� B� w� ƭ � \� �� �6