verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1� � � > � v C � �- �/ #1 j2 �2 �3 5 5 �6 �6 �7 �8 �9 l? @ �@ �B F &F �H �I �I rK O XO �O _Q �T V -V #Z P\ ~] �^ �^ b ;b c �c �d Se �f g kg j
k �m �o �p q �q 5r z �� � �� N� �� 6� >� �� � C� o� �� �� }� �� L� �� T � � y i �
F - � � a �* �, :- �- �0 ~1 V2 ~4 �6 [7 �7 �8
: ;; �; < �< �= �@ �A QE F �b �b qm �n Mo �r s t :z �z 0} t} ~ �~ � � _� � ,� �� z� Ǎ �� �� �� � Μ � "� a� �� � �� �� $� � �� J� L� �� й � j� 7� G� �� �� �� X� +� S� �� E� '� <� �� �� �� i � � I � � � �2 p ] � . �! �" �# �$ �& e( + E- �- �. �3 J6 {: �: \> t> �> )@ [@ �@ �B C �K �L ^R �S cU �U }V dW Lm Cl �l �m �n wo �q �y O| �} %� V� Ȑ +� �� Ւ � O� �� Ö "� &� � \� �� w� i� �� /�
� � >� A� �� ¯ Ȳ \� 5� S� d� X� �� �� � �� �� t� X� )� �� t� �� ~� �� �� <� �� n� �� �� b� P� �� i � � � � � � � � 3 / T � � x � � \ ^ :" �( 8) �) (* + �, . �/ �0 �1 O= B> �> O 5P �R �S ET ?U �V X >X Y UY �Y Z �Z q[ -\ k] �^ %_ �_ �a _d h i J� � �� �� � -� �� �� �� �� � �� �� � �� �� 9� �� `� ,� �� �� � F� �� *� � � � � ! r& D' �' �( �( �5 A: ,O _ Hb �d �e f ؓ Л
� ޠ �� �� �� C� �� j� �� _� '� 2� �� �� �� �� 4� �� �� �� Q� 9� �� �� %� � �� 4