verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1E � � P
M k � � H � � :$ �$ �% n) �+ m. Q1 �1 �5 7 V= ?B 4C �D dF LL }N �R Y A_ �_ \` �c id Ae �e �g �h �o cp x� � �� � �� O� �� a� �� �� �� � � � / O �2
+ ] ! � z � �" 6# !$ �* + �+ W, - �- �U �X �X �Y �[ �[ u^ �^ �b }c ~d �d �h �} j� v� L� �� �� �� f� �� m� W� � � �� �� �� ��
� +� �� T� � � L � G �( ]+ �+ 1, ?- �/ c0
1 _3 �6 �7 �� W� N� �� 5� 3� ̬ � J� S� Ծ q� e� � �� F� �� s� �� �� Q� �� E� m� �� m� � �� �� 7� �� �� �� �� � ]� E� f� .� �� � | ^ � d
7 ~ . \ �! ?$ % & �+ I- �/ @0 �2 C3 �5 jd �d �e Vf �f "g 0h ri �i 7k l `m n �o �p ~q �r �r �| e} �~ I � �� n� �� 9� v� <� �� S� � � �� ö �� � ѹ i� �� �� 9� � � 3� �� =� %� �� .� �� � ~� ^� +� � �� �� �# �# $ �$ K) �) �) �,
6 H6 �7 L8 9 C9 �9 �: �> @ _@ mF �H �L MP {P �P �R �U V X^ U` �c 7q �t �t �w Lx ox )y �z � ҕ �� U� �� ծ �� � -� � �� $� �� H� �� 9� V� x� <� ~� �� �� *� �� .� Z� �� � E� � 8� c� M� �� 9� �� ]� o� �� � �� �� �� =� ^� �� (� J� N� k #3 � \ � � � N * � [ �& �( ) - ;- �- ). k3 �3 x9 �9 \� �� �� w� i� �� 4� \� L� �� � 5� �� �� :� 4� �� �� K� � �� l� �� M� � d � " ( ) �) �, �- 04 :6 �6 R< �= �?
A �B {C (E �E M �N �N +P Q �R �S T
Z �] '^ m^ � w� @� �� ն � ķ �� m� l� �� �� �� �� G� {� �� � � 7� �� �� � �� o� �� �� V 4 ! � * X � U � \ �& �* k, �, �3 �6 �7 ?8 �9 �9 �: => $@ B XB �B �B �C �D dH H j �{ �| h} � �� � 1� �� t� �� r� � � {� %� �� O� ]� �� ţ F� � ¬ R� �� �� � �� �� � � Q� ҷ Y� �� l� �� 5� S� � @� x� �� H� k� �� �� j� Z� �� M� *� �� �� �� %� �� "� �� �� k� � a q S �5 � y � R i& ' �' 5( h* �4 f6 U7 �7 �7 8 < �B �F �F �G uI PJ jJ NN P �P S )S �S QV �V (X )Z Yf �m �p �r 7s lt �t :u )v w 2y �y �| u� �� (� �� �� .� K� �� �� � � t� � �� U� %� 1� �� Ö � )� j� �� 4� 3� �� A� �� Ѷ � !� � Z� .� f� j� @� �� !� [� 1� �� �� �� �� � H� � �� �� �� ?� �� �� �� T� 9� ~� | 6