verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1>� �� �� �� n� � q� U� �� (� U� ^� `� �� �� g� �� �� p� �� � R� � � |� �� �� �� G� 0 �2 �3 @
�
� � �
-( 6 �6 �8 �A 5B �C @D E oE RF JH �H nI �I lN IP +R }S :U �V �V �W �Y UZ &^ ` �` ;a �a �e If �f �f � � � � �� ʌ �� 3� �� �� i� �� h� �� �� _� �� �� � � �� ^� �� �� \� �� �� B� '� �� o� �� � \� �� m�
� 5� >� �� � � G� �� a� � �� �� �� G� ^� j � U � � � � �! �# �$ >% N' ) �; �< sG �L +S sS �� � �� ~� �� D� �� �� �� �� 2� �� �� ��
� �� �� .� �� W� �� 9� �� 7 ] � �h tm �n �n 1o ;p �v z O� � g� � Ն � �� �� M� Ó �� p� !�
� ڜ �� �� �� � � Ц m� S� �� ɰ �� �� �� �� }� � �� A� �� �� i� h� �� � � > q � � . Z � t �d Oj �j Nk �n qo Bp �r ?u �w 4x px { L{ �� ɋ �� �� � P� �� � =� ĝ � ?� o� � r� 4� �� ƿ .� o� %� � X� �� p� �� ,� o� �� � =� )� ~� �� �� �� �� r � �) �* �+ �- �3 �= H> &? �A �A �A �B C D #E KH �H �M T �X BZ [ �\ �^ c_ �d oe �f �j �m �o 0p |� z� �� � �� !� �� P� �� ҏ �� ϑ � g� x� �� �� �� <� L� �� +� � � g� �� F�
� �� �� 5� e� � �� J� ܹ �� �� ,� Q� A� �� �� �� Y� �� ?� � �� �� �� M� �� � M � n 1 � �" -& ]& �' ( �( ) �) �+ �/ 2 U7 �7 :
> e? }@ D �D �F �F tG �I �I GJ 5K �K
L M �M P "Q �S KU .W [W qX �Y Z [ i\ �_ �` jd �e [f h� �� � D� �� 9� C� � G� *� �� � I� 3� g� �� Q� .� P� ú � �� �� �� �� �� �� o� � x� J � q � � )$ �$ #, <. �/ 0 �0 �0 H1 �6 �L �L 8O �P $S T CT �Y �Z �[ �_ la �d g �g �n vp �x 6~ �~ � �� s� !� A� � � e� �� �� � Ө b� 6� �� U� �� O� h� � � �� � � p� �� �� [� �� �� �� �� x� T� �� � t� �� �� �� �� A� � Y7 ( � h - �( ) q) * �+ �, l9 �9 P; �< �> �A .C gN *Q dT �U nV �V `^ Na �b Ne re �f �v �w �x Ry �y �z u| } K} �� t� � � +� w� �� �� � Z� �� ę �� L� �� h� ʢ � *� դ � � I� 5� �� �� !� �� � �� � �� �� � J
� F � � 7 �! �# $ i$ % �& T' �' �( 6) 2* �1 32 �5 �� N� l� �� �� O� �� �� N� p� L� �� N� c� K� �� �� d� � :� c� �� Q8