verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1� r l � C f .
% e s � r � � �! &( ) b- �- �. @ �I �I VJ �J �K !L %M bM �N �N wR �S T IU dU uU f[ Q_ }` �b -e �e �f �j �k �l �m {p q y �z | S~ P ]� � R� �� � ڌ � Z� u� .� �� "� 9� �� � N� �� �� Y� �� �� \� � �� �� "� � � )� N� � �� � ܻ $� A� �� �� N� �� �� }� �� -� L� � � � � U � � � � L � � / � ! � � � � /# X$ ) �2 �4 �= �? F �F �G dK 5L �T �T rV �V �W �X hY �^ �_ ~` -a �c 3p gq t �t kv w �x Ey { �~ �� �� ;� 6� � )� � � �� s� �� %� � y� f� � ¡ �� � ;� � U� n� z� M� |� Ʒ �� f� U� <� �� � T� � � M
� � D H � 0 r �! �# >$ �' - �- Q6 �6 �6 �7 `9 �9 *@ @B C �C �C �Z v` �` a �d i �i �j �n �q �q E� �� [� m� R� � �� � в � m� �� �� �� � �� � �� � � s� ?� d� t� :� �� 2� D� �� M� O� v� � J � G � � " Y � T q � � � M � - �� h� '� ]� ץ � � �� ɩ M� �� Ů �� U� �� M� � ܽ n� �� `� g� �� x� �� �� � �� H� �� � �� �� �� �� g� �� �� �� G� � : � @ � M J � � E � B J � � � � h V7 @ ^@ �@ �A XI rM P Q �R �S �S CZ �Z �\ �^ �_ ]b �b d �d �e �f wm �m Pn �r �r � � ѡ ~� {� q� ͮ � y� �� �� �� � � f� �� 0� q� K� t� �� � H� �� �� /� �� �� R� �� @� �� � �� w� N� �� (� �� �� ]� �� � �� �� �� d� � �� p� � � %� �� H s � z � - � / P $ U �- �/ ]1 �5 !7 tN kP �Q �R �V �] �^ �_ ̖ � =� K� �� � �� Ӳ � @� �� � � �� H� 4� �� �� �� n� �� [� �� � �� �� 7� �� b� ]� �� W� �� m� � �� �� �� �� �� � �� �� � �� �� Q � � X s
% 7 X & � � � �* �+ 2, �, - #- <. X. N/ ]1 �1 �F +H /K bK �K �L �M �N %O �P �P UQ e 8e !f @u �v �w �x �{ ~� Q� j� �� � H� ȏ � ~� U� E� � �� 2� �� �� ,� �� � ;� � 8� �� 9� � �� � �� {� �� �� � �� =� � 6 � =
" � N& w& �' �( 7* �1 �5 7 �7 �: �< �> �C �C �D F G qP �R �S �W �X .Y �[ �^ fh �k nq �r �r /t *� � t� dž c� ݈ �� �� �� � � �� �� � #� М � � D \ � � � ~ �= D@ �@ �A �B �B 6C {D �D �J 2K sQ �Q �R �S DT V �Z Q[ ] ^ 5a �a #i �i Tl �m �n �v :w �w bx iy _z �z b� �� � 3� � Y� � ;� !� И � y� %� r� �� �� ~� �� q� Q� �� �� 9� �� �� 0� �� �� �� D� X� � �� s x �#
% ^& 9( �) �* u6 �A �C �E �F tI �L qO �O �Q |S 2l �o Dr +t �| �| �} ؕ � �� Û �� 5� �� �� S� Ǔ )� � �� � � �� V� �� �� I� K� |� � � � �� � ͼ Ѿ � :� �� �� �� s� �� <� �� �� �� �� m� �� �� o� �� G� �� � �� x� � Q� h� �� �� 7
�
4
�
,
�
t
�
�
�
�
&#
&
u&
r'
�*
o+
�.
/
b/
�/
1
R2
L8
�9
�:
;
�;
�<
�?
@
�@
�A
H
�H
[K
NQ
J^
d_
&`
�b
�h
�h
i
�q
�q
)r
�r
�r
5s
Lu
y
~{
�{
~|
�|
/
�
8�
��
|�
��
��
�
T�
��
�
�
S�
��
��
��
6�
f�
��
�
��
��
ƞ
��
Z�
�
��
X�
�
p�
�
5�+ �+ ��+ �+ F�+ ��+ �+ + 9�+ �+ 4�+ ��+ M�+ ��+ ��+ ��+ 4, �, k, , �, �, s, �, A, ], �, �, 9, %, 7, �%, \', �3, F5, 8, &9, b9, %;, �;, �=, C, �F, CI, �I, �n, �o, �{, C}, j}, ~, Ń, ��, ܊, (�, ۋ, ��, :�, ��, ��, �, ݯ, �, �, ��, v�, }�, ��, ��, �, �, a�, ��, ��, ��, ��, %�, P�, ��, -�, �, ��, ��, ��, ��, ��, �- V- r- - �- f- �5- }- �- �- �
- - �- �- �- �- �- �- o- 0- f- T - !(- +- �=- �>- QA- sN- MO- �X- ^- d_- �c- �d- �e- �g- 'j- �p- Pq- ur- �s- �P0 �P0 �Q0 ��0 �0 *�0 O�0 ��0 -�0 �0 �0 G�0 ��0 Q�0 ��0 )�0 ��0 `�0 �0 ��0 g�0 ��0 .�0 N�0 �0 ��0 ��0 ��0 ��0 0�0 u�0 ��0 ��0 Z�0 �0 Q�0 ;�0 y�0 V�0 ��0 I�0 \1 .I1 �I1 �L1 +N1 _N1 ,O1 �P1 �Q1 R1 %T1 :V1 �Y1 Z1 �[1 �]1 =_1 �c1 &e1 if1 �f1 �n1 �o1 �o1 �p1 &q1 �r1 z1 �{1 �1 h�1 �1 ��1 +�1 D�1 /�1 y�1 ��1 �1 ��1 ��1 ��1 ,�1 '�1 ��1 '�1 ��1 [�1 ��1 9�1 n�1 c�1 �1 ܫ1 ��1 �1 ӯ1 o�1 �1 ��1 ��1 ��1 |�1 i�1 ƺ1 ��1 1 ��1 5�1 {�1 U�1 6�1 d�1 �1 ��1 P�1 �1 ��1 X�1 G�1 ��1 ��1 ��1 ��1 Z�1 ��1 �1 72 e2 �2 T2 �2 2 2 �2 A2 �2 �2 �2 �2 d2 �2 � 2 I"2 �"2 �#2 7'2 ,2 j,2 .2 v.2 �.2 �O2 ]T2 V2 �V2 Y2 &[2 �[2 �p2 �x2 Oy2 {2 #{2 �|2 �}2 b�2 {�2 ��2 Y�2 ��2 ��2 ތ2 J�2 ��2 �2 ֚2 #�2 ߝ2 �2 �2 '�2 é2 �2 |�2 �2 1�2 H�2 ��2 ��2 �2 B�2 �2 �2 ˹2 \�2 �2 ��2 ��2 ��2 ]�2 ��2 ��2 E�2 ��2 )�2 ��2 y 3 E3 3 �3 Q3 3 �3 �3 �3 c"3 �"3 l&3 �&3 �&3 '3 �'3 �:3 <3 �R3 �U3 �U3 GZ3 [3 �\3 ^3 �_3 �`3 �`3 �`3 �k3 hl3 Hm3 Ax3 ux3 �{3 �|3 �}3 ~3 �~3 ��3 Ȁ3 /�3 ��3 ׃3 l�3 ��3 ߙ3 K�3 {�3 ��3 _�3 ݞ3 �3 6�3 ܦ3 ��3 3�3 �3 q�3 �3 ��3 [�3 W�3 ��3 ��3 ��3 h�3 ��3 W�3 >�3 ��3 w�3 ,�3 ��3 8�3 �3 F�3 N�3 k�3 ��3 ��3 ��3 #�3 ��3 ��3 �3 ��3 �3 ��3 x�3 | 4 �4 �4 +4 z4
4 �
4 4 c4 �4 P4 -p4 �q4 v4 w4 �w4 �z4 v}4 n�4 o�4 f�4 a�4 U�4 �4 H�4 j�4 ]�4 ��4 8�4 ȥ4 ��4 �4 �4 ��4 ��4 �4 ��4 4 v�4 4 ]�4 )�4 g�4 ��4 ��4 �4 �4 p�4 ��4 ��4 7�4 }�4 m�4 ��4 o�4 ��4 ��4 ��4 ��4 N�4 ��4 9�4 35 w5 �5 �5 �
5 �
5 (5 W5 �5 \5 �5 D5 X5 @(5 �(5 $:5 �?5 O5 R5 pU5 AY5 �Z5 �[5 �]5 2`5 �`5 zl5 �n5 �n5 @o5 `o5 5 ��5 ��5 W�5 ʇ5 e�5 ��5 �5 �5 3�5 �5 ��5 ޛ5 ��5 ��5 ��5 _�5 �5 ��5 >�5 v-6 �56 �66 �86 �:6 �:6 <6 �<6 �=6 �A6 &E6 F6 4G6 J6 �K6 �K6 iQ6 *X6 ^Y6 C_6 �`6 �a6 �f6 7g6 �h6 1i6 �j6 yl6 =p6 ,r6 �t6 �v6 �y6 �y6 }6 �}6 ��6 ܁6 щ6 :�6 �6 �6 @�6 ݙ6 ��6 u�6 ��6 0�6 �6 w�6 ��6 �6 ��6 ��6 �6 ��6 (�6 ��6 =�6 ��6 C�6 ��6 6�6 ��6 <�6 ��6 5�6 ��6 !�6 ��6 7 P7 ?7 m7 �7 7 �7 � 7 �%7 x*7 �*7 �,7 �,7 &-7 �.7 �/7 �27 �67 C87 �87 |97 �>7 �@7 aA7 <B7 zT7 �U7 ~V7 �Z7 h[7 �n7 Nr7 �r7 ct7 �u7 ex7 y{7 �|7 �}7 �}7 _7 �7 A�7 �7 �7 _�7 ��7 ��7 č7 Z�7 ��7 ��7 ��7 ��7 �7 X�7 ��7 ��7 I�7 D�7 %�7 �7 3�7 ?�7 ��7 E�8 o�8 Y�8 �8 d�8 ��8 1�8 3�8 O�8 ��8 ��8 @�8 "�8 �8 �8 ��8 �9 9 X9 i9 (9 9 �9 X9 9 E9 �9 �9 �9 �9 N9 � 9 ;$9 �%9 ;&9 A'9 �59 �69 j:9 �:9 [;9 m;9 %=9 c=9 {>9 �>9 �B9 &D9 |D9 H9 kH9 vI9 �I9 dN9 zR9 QS9 �T9 U9 gW9 �Y9 �j9 ,o9 �o9 Or9 �9 U�9 !�9 K�9 ��9 ��9 �9 ��9 �9 j�9 �9 O�9 ݬ9 A�9 �9 �9 �9 ��9 Q�9 Ҵ9 ˵9 m�9 �9 w�9 �9 c�9 `�9 ��9 ��9 ��9 �9 g�9 e�9 ��9 �9 ��9 (�9 �9 U�9 ��9 x�9 ��9 ^�9 ��9 ��9 O�9 ��9 ��9 ��9 �9 ��9 �: �: �:
: K: �: �;: �U: �W: QY: Z: c\: �\: e]: z`: �`: �h: �h: �i: �i: �k: �o: !s: �u: �u: �w: m�: ��: ��: ��: i�: )�: �: �: ܒ: �: �: X�: ��: �: Ġ: ��: I�: ��: Z�: p�: "�: <�: a�: ��: ��: ��: .�: �: �: `�: �: ]�: Q�: S�: �: ��: ��: O�: ��: ��: �: ��: ��: 8; �; 8; U; ; �; �; ; S; a(; ,; p,; �,; 4/; t/; �4; 5; �5; n>; �B; C; �H; �J; EQ; �T; �U; aV; �V; �W; \; {a; ~b; �i; m; n; |q; St; u; �w; �w; tz; �{; @}; B; v�; Y�; ��; Յ; ��; ��; ؈; ʊ; �; ��; -�; "�; ��; ��; �; 1�; �; ��; ��; :�; ӟ; Ϡ; 1�; ��; !�; ~�; ��; ܩ; �; _�; ��; ��; Ϳ; 4�; ��; I�; ��; ��; M�; ��; ��; ��; P�; ��; O�; �<