verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1� � � ] 6 Z � , * R � � � ~ U � 6 G y / D � � 2 ? ^ � �) 3U 3W Z Q^ �` �b �e �i 4k �l cp =q �q qr �s �v w Vy �y �z q| �} {� � �� � n� v� C� �� ��
� ² �� �� � T� G� � � �� �� 0� m� �� �� w� :� �� C� "� ��
� W� ]� �� �� *� x� t� �� .� c� -� z� @� �� t� �� �� �� > _ � % L f �4 � �! �" �% m& a, �9 �< <B �E |L �M Z� 0� �� }� 5� @� �� �� �� N� I� ~� �� �� �� � 0 s
q � ! � I � p � � � � L � 8 �" U$ �% * h* 2, �, �- �/ e0 �0 -3 �3 �5 �8 �9 ; U \U
V W �_ i` �c �e yf lh �h 4i "j sn +q eu �u �v �w y �� �� 4� ҆ �� �� � �� 0� �� �� � Ժ � ջ �� Ҿ � �� g� q� � A� � �� �� �� �� �� u� S� W� �� �� � � A� � A� �� �� �� $� �� u N
� � ] { � � � S! �! " m# ) Q* i1 �2 �4 �N ZP oR �T ] K_ �c Ep �q Xu �u �{ | � +� �� ğ � �� � � �� I� ��
� � J� *� ^� �� �� �� f� �� �� � 5� u� T� �� �� �� [ �
a �4 ,6 D7 8 �@ cA �D E �G �G �T �U *W nZ [ �^ G_ . � � � ) � v" �# 9 �9 �9 �: cE �E �I �P 6_ ` �c d Bg �l �m dp �p �q kr �s t mt Du Jv 5{ B| �~ �� =� ȧ }� � ܭ � #� m� �� � � � %� � �� 8� �� w� n� \� �� �� �� V� 6� j� �� @� � 0� N� �� � : � �
4 � 6 u � � � � �$
% C( �( �) �+ z, - �. 0 +0 �2 4 �4 >< F= �A KC �I �I �I zK �K (M �O xR tT �T [U �U p� � � ї 3� ͥ J� �� e� Ǭ � �� �� +� � � � K �# % �% r( ) 3* _* + �/ �2 �6 �7 b8 �; �= �@ �A
D 6F �M *N �O UP R �S /W \ n\ ^ [_ 0` Fa 0c Nc ~c d �e �e �h _k �k Ql ,o �t �t �u �v
w Kw �y �| J� p� � u� >� �� � Z� � �� �� �� p� � � �� 0� �� M� �� � �� � � t M � , I b" �% �& �( q+ . b. �. �. 0 ]2 4 j4 5 k5 �5 R6 �7 �8 `: �; v< �< O= -? �@ �@ (D �D �E �G J �L �Y �Z �[ �] �_ �` Fb 3c �e wk �l �m n Qn �n �o s �s Au `u av �v -z } 6 3� ǀ `� x� � H� � �� � ü �� �� k� �� �� � �� �� �� � �7