verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1@ B � , g 1 � � | � �) - j. �0 �1 �2 �2 C4 �6 �7
8 �8 �9 Q: �: �= ,? D XD �D fN �N �Q 6R JS �T IW 2Y F� �� 6� .� � � �� � � d� �� t� '� � �� �� �� �� ��
� >� �� �� 6� "� Y� _� �� c� i� �� �� �� Y� #� �� \� � 5 � k � > � j$ �+ z. �0 �0 �7 �8 �9 �: 3< �< d= B D E aE �E �E �F <G �O �Q LS U �V �W H[ �] ^ �^ c lc �d 5e ye tg �i aj �j 3k �k �m � �� �� M� ̍ �� �� �� I� Ϩ �� \� �� 4� e� �� � � {� � �� T� � � �� �� �� #� �� � �� �� �� S� � �� j� �� �� �� �� �� C� P� �� 0� � � �� �� �� �� � �� �� �� x , # X , �5 M6 �6 �R pS %U a 5b Lb @e �e �g ,h fh �m �m @o |p �p �s >t Iv �v �v @x `y �~ �� � W� 2� v� <� ~� à B� I� � 2� i� #� N� X� b� e� A� d� �� ^� A� T� 9� �� �� � � � � F K � ) �( * B, �. �/ H0 �J �J N �N qQ �R �U �W gX U^ r_ �a Yc �c 2d �r Kw �z | �� G� � U� �� s� p� �� m� Q� K� � �� K� �� �� � =� "� e� �� �� � o� � ^� �� �� G � � : " n z * �+ �. �/ �/ �/ v1 �1 3 �6 �6 �7 �? �S �\
^ G_ �_ o` (b �b *d 3i �j ,k �k �l �o �p �p
q 2� � �� �� �� 8� Ҕ �� i� � �� �� ƪ �� � O� \� κ 8� 2� R� +� R� �� I� J� � �� � e� � ;� a� �� �� 5� �� �� �� d� �� � w� �� � �� }� �� v� w� �� N & 3 � � T w g � � � H � � 0 n< �< s= �> �? �? �@ C iE ~E �F nK �M �N O �O �R S �S T �T X �X �Z �\ �\ |] �a �f h �� '� �� �� ȓ y� �� � G� �� �� 1� �� =� W� I� d� h� �� &� �� ^� �� ;� ^� �� m� R� �� � 6 w7 � L � � l � M \ � g W "& �/ 8 0< _> �A ^B �C �D yK �L �M HN O WO �T �Y 'Z Q[ �\ �] �a "c �c kf �f �h i �j #k �k Ul �o (p �q <r r �s y �y �z | �} h � i� ~� l� �� F� �� Ö s� � �� V� � �� �� r� /�
� �� �� P� �� � �� �� �� � � ]� p � @ � L � � �
h h � �) �* 0, 0 ~2 T6 �6 :7 �8 �9 c: W< v@ UB �\ w] Sa �a �c tj k �l �o v 5� �� E� ե #� �� � �� �� �� <� �� �� �� g� k� 2� �� !� �� "� =� I� � �� �8 �
l f! h# �# $ �$ U, �/ 64 �4 �4 5 �6 �6 �7 �9 �9 �: *; �G S 9V �Z �� +� �� څ j� v� � N� �� k� v� ˚ J� � u : 5 �10 �
Z � � � � � � � � �* +, �. �0 �B 8D �E "F �F P �P �T U AZ �Z �[ �\ s] �_ bl �l �m eo �o uq ou �u �u �~ � \� %� �� ͋ � J� �� "� ؗ s� �� � &� 8� �� �� �� �� ɣ �� 8� �� �� �� �� �� �� m� �� �� $� �� �� �� �� �� "� �� � B� �� �� � �
�
�11
�
�
�
�
�
�
{
�
J#
Y.
�5
t6
�7
8
~8
�8
p;
�;
<
�=
�=
�?
�D
E
�F
�I
�I
J
�J
�K
&M
�M
:N
�V
�e
�h
�h
En
�w
�w
cy
�z
{
�{
�|
�|
�}
�~
��
$�
\�
b�
��
�
�
j�
q�
î
��
�
��
��
��
��
��
��
A�
��
��
��
��
��
l�
F�
��
��
��
-�
< � R D � �12 g P �P +Q YS �T �f gg �g ~z \� c� v� �� �� M� !� �� ¥ ̮ � [� � .� � %� X� � �� � @� �� 13� *� h� �� �� �� � �� �� K� �� 6� �� $� �� ` P � � �% �% & A' @. �/ I3 �3 =7 �7 _8 +: U: �: 1= > H> A �D �E hF -I ^J �M mO P Q FT �T �U oW Z �Z ~[ \ A^ |l �l 14m #m �m _n �o �o �q �r #t K{ � d� �� � � �� �� � �� H� �� � �� �� _� �� �� �� 0� � �� I� 0� � b� �� )� �� �� i� �� �� !� �� � c� �� I� �� �� � � A - � V � � � � w � Z � � � x � � � � � . | � �! 8# �% �' �) * �* #+ l+ �/ L1 1 �1 �4 �8 �; R> �> _? QA �B !C �C *F 9L �L �c Td kh fl �n r H� A� � D� ̩ � |� q� � �� 3� G� '� �� K� �� � [� � [� �� �� � �� (� Q� �� /� �� �� *� � <� �� A� �� J� h � % ] � � Z
� � J � > D { B }+ �, �/ J0 �1 q4 �4 (6 t? @ B �B �C E �H �I �J �K �K L PL �M P Q �Q �T dW �Z �^ F_ Ld j Vj o �o 'p vr J{ �{ �� 6� w� � &� f� A� �� M� � e� �� � q �& !' �' �' �, 3 5 k9 8: �: �; �< K= }? @ .B �C YE FH JQ �Q !R \R �S �W X 7Y �Y Z �Z �[ �_ L` 1r �v |w � 2� ݀ =� � �� ͡ �� 6� �� ά � -� �� ȯ �� �� �� :� �� )� <� վ m� ؿ =� l� d� �� � �� g� �� J� �� F 15 � � f � c16 �17 �18 b � 19
� � w& �) �* `. �. �/ $0 s1 62 3 �8 �8 ^9 �9 2: l: c; �; wH �K �M @N �N wO �P �Q K[ �[ ^ �^ �_ 2` Ea �d �d $e �h ]i lk q t �u �x �y �z { �{ �| �} �} �� ]� I� N� ʄ �� C� Y� h� ȉ `� � � v� �� � � �� )� Z� � � �� �� �� �� �� � r� 5� t� �� �� y� �� R� �� t� � j� �� -� � � G� � � �� �� �� ;� �� � � � 6 � = �
&0 /1 �2 �2 �3 4 �4 �5 t7 �< = jD F WF \H PI #Q �W X �X �X �Z �b Jc �c \d 4g �k xm r vr �r �r s �s �t �u �u �v �w �{ �| �~ � *� ρ ]� �� � �� �� `� � b� `� &� C� �� Ο �� � � ȧ <� � �� � � �� $� �� L� �� +� �� 9� C� t� �� [� �� � �� �� 1� @� G� m� �� 8� �� �� E� �� �� _� �� �� >� �� � � � N � H ? M � � R x � � #! K" �" # �M qN �O �Q �R "S ST :U W �W JY v\ Y^ �_ ` &b �b c �c �d �d �� !� Њ F� �� � � �� � � �� .� �� �� [� � � �� y� ³ *� o� � =� � k� A� M� }� +� �� b� �� <� �� *� � �� �� r� p� � � :� �� d� Z� �� ߫ ;� 4� �� � � �� `� �� �� ܹ � ,� �� �� %� Y� �� �� �� �� $� r� -� 8� ~� � c� �� � r� �� F� &� x� �� �� � � �� ,� O� �� � � ^ m S � � �
' t � : �! �# �+ �- �3 )5 �5 �A B �Q �S �W .Z �\ �\ �\ ` [` �h `i <m lo �o Zp zp ?q �q &r �y z Y{ �{ � A� � q� H� Ç �� .� h� �� p� ޞ A� �� &� � #� � �� �� �� � <� h� � ҽ B� � 6� _� �� �� �� `� �� �� [� T� +� l� � �� <� .� � �� @� -� �� �� >� �� 7� �� '� �� � �20 �21 � n �- �. 9 �@ kA k � r %! A$ �& �' 2( ) �+ / �0 ~5 L6 7 |8 �8 N< �C �P �R �U �V �[ �] -^ ` Oa b Yd Bw �x
z E} �~ � �� ΅ �; x< D= ? � �� � \� � ȟ ݟ �� � ɩ %� �� Q� '� �� z� ʿ �� �� �� D� �� X� e� �� �� e� ��
� )� "� \� �� �� C� :� g� �� �� �� � k� �� �� b� n � . & "
�
/ � � P �Q aV �W Y g\ ] �p Yq �q �r 't �u Sx �� � � l� �� !� �� �� Q� �� £ ؤ �� 0� 6� /� ��
� �� � �� �� �� 2� _� �� W� :� D� � �� �� Y� �� �� �� �� %� � E 5 � � ?22 � c
) #* �1 j2 {= Z> b? @ �K �K "L �N O �S 2U �V �W X _ wc Nd 23e j �j t u ~v 'z � O� х �� *� �� � � d) o) �) ~/ �0 �1 �2 �5 F 9F DG �G |J �J [Q �Q �R �S �S jU jV �V �X �Z �[ ] �] �^ =a �b Pc 7e �h �l In �n �p r =r ~r �w �w "~ �~ dž � d� Lj r� �� g� C� -� ܰ � �� �� 1� �� � � �� ڿ �� �� �� �� � W� � �� 5� �� �� �� z� �� D� �� �� � �� X� �� �� �� �� 3� �� �� .� � _� �� �� �� . � �24 � > � P ; | � # �# �( �) : �D �D 4E �K `N �N P )R 1f �g �h �i 6k ]k �k 9l mm �n �o �v �v �x Uy �{ �| Ѐ �� �� �� M� �� �� � b� � �� � �� �� �� '� e� �� �� �� d� �� >� �� 8� (� �� � R� ! �! e! h! ! �25! �26! �! z! 27! ! ! �! a! �! 4! !! `! 3! ! � ! �$! �$! �'! �)! 9*! \*! ,! �-! .! H.! �2! H4! E6! �6! 78! }8! �9! rH! 'J! K! CN! �O! 'Q! �Q! ES! V]! �]! /^! r_! �`! |a! �b! Sc! �c! Od! �h! wi! �i! �y! |! �|! l}! �! ��! �! ��! &�! \�! �! ��! ��! �! M�! ��! l�! R�! G�! x�! ��! �! h�! w�! ��! ��! �! ��! ��! &�! \�! ��! ��! �! q�! .�! ��! h�! h�! 8�! �! ��! ��! �! ��! ��! \�! Q�! ��! �"