verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1a � � 8 � � W ! � i� w� �� 5� �� �� 3� �� E� �� ~� � �� ɛ r� s� �� �� 7� �� {� � �� Y� �� �� � �� g� 2� �� � �� �� }� F� J� � 5� �� �� � �� �� � �� �� �� K� �� &� D� �� 1� �� � U � � 1 a 7
\
� � �$ q% �& L- �0 �4 7 Y8 �8 �8 �9 6; s< G= �> �? O@ �A ZB �F 2H ~H Gm �v �x �x �y �} �� �� �� � i� ʋ �� m� {� �� �� �� E� �� 9� ߜ � �� �� � �� � � �� � j� � � �� � #� �� � {� �� � 3� ^� �� �� A� 2� �� � ,� >� �� � e� R� P� #� �� +� �� � > � ; �
� wi Zj Ep
s Gs ڂ k� � <� Ʌ
� �� �� �� R� }� ۋ � � �� �� Ց 5� X� u� �� e� �� g� � � � ,� � �� Υ �� d� � p� o� į ð � � ״ b� ͷ ڸ �� 0� �� ~� (� I� �
�$ % �' �* �- �. e/ �/ 3 �5 V6 �8 �9 : �; �> &? V? �G �G }I BJ �J �T �Y �^ 1c _c <d �d �e �h �i �l 2p wp _x M� ݃ ,� �� � Ĩ �� !� W� � �� �� �� |� �� �� �� 7� }� �� �� �� �� �� 3� �� � � m� v� f� �� M� �� ? 3 >
� ; � H# % �& �. �3 �5 �5 ^8 �8 n< �= �= T> ? g? �? �A �C �I �I �J �K L �L �M IN hS �Y
Z �Z �Z 3[ �[ �] �b je �e f g i 7k �k >l 4� ނ � v� � �� i� x� �� �� �� � � 0� Y� � �� � d� !� � �� E� �� q� � -� �� i� �� �� g� D� P >Q AR �S >V nW ^e �e i �i Ip �p ov { �| �} Q~ � � � �� � C� 3� �� �� r� ٔ �� f� � s� H� �� �� � E� ۳ �� � �� 4� �� b� �� s� 7� � �� _� �� }� �� �� �� � F� �� |� �� �� � U � S � G � � G 7! " �" �# A& y& /) �) �- 2 �2 <4 |5 6 }7 �7 E8 �8 �A D `I �J �M -N �P *Q �Q �Q (R �S �T �T �V ^ i^ �b �f �g i 'l ct Kw x yx \z d| �} �� �� Ӎ �� �� z� H� t� ҥ [� ê v� �� �� �� �� =� @� V� � ս � ;� w� �� O� ,� �� ;� �� *� �� �� � 2� �� � v� 1� �� $� �� �� <� M� �� �� (� y� �� a� �� =� x " � A � ^ $5 � �
� � � � � � � $ r% �' �) �; }< �< �B �C 1F G zH I }Q �[ �^ �n ?p +q �q �z �{ � [� � E� ȍ t� � �� /� � Ҙ �� � '� ӟ &� �� E� ݢ �� Ϥ �� �� j� W� i� �� %� ۰ �� ˱ �� ?� �� �� ]� p� �� � <� �� �� � �� �� b� �� � �� m� � � $� �� �� Y� �� �� [� �� �� �� K� �� �� &� z� �� B� � M t y/ �/ F2 k2 �8 O; @< z= �? x@ D �D �E �E �H �I �K �K �O �� � �� �� R� � � �� e� 3� �� 4� q� �� � �� 1� 1� �� �� c� }� l� �� Q� �� �� �� K6