verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1� Z � /� �� �� y� %� �� $� '� � � � �� J� �� 8� �� �� �� �� � 2� �� � �� � $� �� !� � 1� �� �� �� �� �� � {� �� �� �� �� � � )� z i ^ � w2 : b � X � @ &
� � � � �" �# �# �) �) �, Z- /. #/ �/ 1 �1 3 �4 I9 : �< �= g? �@ UC �F �I J yJ �J aQ �W X k[ .] .f �f { �| �} �~ � � v� p� g� ʋ �� �� ۑ ؒ 7� ֯ а � L� C� ̾ �� n� �� �O fT dV �V @X J\ ] ^ �_ b �m Ao nt 9v Sy U~ �� ?� ڊ %� � � ޓ W� }� &� ^� �� Ħ l� l� �� b� �� �� �� � � � �� �� �� �� 7� @� �� � %� �� `� s� �� 2� %� >� � � � �" �$ "% S% &' ( T( ;) E* `. �. 2 z< �< s= �= 3> �N >P �P R �R �S �Z �^ �i �l �n 8o zo �q �s �s �u &v y �y �{ È Љ 6� � )� G� z� �� �� ȩ )� W� ´ 5� � � � ؼ �� �� �� @� A� �� )� � �� r� U� u� �� �� +� &� G � ( � W � 4 ? � � b � " =# \# y) =* e, �- N2 f2 �< MC 2D �P aS �S �U �W �W Y �Y [ \ �\ �] �] 8_ u` �b ]c e �e �f �f �g xh �h i 7i �i �l �o q 8w x � � ۆ g� X� � �� [� �� *� � ۫ 9� �� y� ů x� � h� V� �� �� z� E� � �� �� �� �� e� Zv Pw Iy mz ~ "� � b� �� m� �� =� �� t� �� 3� �� � �� >� [� j� @� .� �� ~� l� � � �� �� B� �� �� � ( � � 6 i � - 1 �1 �2 @4 ^6 1> 5? �? �C �E KF �F �I �J �N bO kP eR aS 1T �U HX mZ �Z ;[ S[ �[ \ �\ �] g_ �i �j k Ik l �l rm �n p Xs �s ct [u 5v 4w � � �� l� {� � �� �I6