verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1� Y � � � M* t/ �2 3 _3 �4 �: ~; �@ RB ^D �D vF � ؛ o� Ɲ �� � d� f� η D� %� �� � 1� |� y� V� �� }� �� 1� �� _� �� �� �� ,� � � .� $) 0) 2+ �+ �, �. �. �1 j3 8 D H ZI 1Q @^ �b �f �h �j \k :m 7p /v �v �y P� �� c� � R� �� ֊ �� ے z� ϖ �� ~� \� �� Y� �� U� �� !� R� � e� � � x� p� ̷ �� �� ;� h� �� �� z� G� � � Y� f� O� � �� U� v� �� �� �� �� �� �� d� � � � + � ] � l ~ � M B � O :! " j" �" # �% �( $* ,, �0 1 w4 �5 �: �; A= �= `? �? ;A #B xD �H �I �J �J �K �T U �Y �Y ZZ �Z 7[ �[ �^ ?` Xc �f g �i �i �j �k �l p qr �u #y |y �y Gz | a �� x� �� � �� � H� �� �� � !� �� �� ˍ �� ǐ <� �� � �� �� � 3� �� p� � � �� n� �� � '� � e� B� E� �� �� �� �� � J� v� �� �� �� {� �� ^� �� P� $� �� O� � � � �2 � � X
�
�
: � � � � � E" �" �# =$ �$ e% t& ' �' ;( S* �* 6 �7
8 V8 �9 �9 �: �;
< #= ]= hg h �i j Oj Nk `o �{ ܁ X� |� �� � I� � �� �� ,� � ɣ �� �� ϯ � /� �� � c� ¼ E� 2R �R S �V �V X �_ za �h 6i �u x Sy Jz �{ 4| �} � p� ;� t� V� � *� K� � �� ס � � � |� �� Ҭ � s� F� � � .� ľ �� � �� �� � �� �� v� ;� �� �� �� ;� �� �� �� :� |� �� �� u� �� I� �� �� �� ;� �3 � N � � O! �) �) o* �- 9/ v3 5 d6 �6 9 �9 ;: ;@ #A iE F �G �H �I �J �J 2N KO �O P �T U T\ +c �f �g �i �i 1l �l n p �q |x 7� � F� �� A� �� �� }� t� � �� �� �� �� �� � �� �� �� d� � Q� y� �� � �� s� � � R� d � 4 � - � ( x t � P � % �% �& �' ) �) �) �) t, �.
< @ �D 1M �N �O ;e 5f �g <k .r �r ps �| �� �� R� � R � ` y � �# �# �2 [b d �d 0q �t N~ � �� �� �� �� ݃ !�
� k� �� D� v� �� �� $� ښ Ɲ � V� �� �� '� �� ˩ Ū ѫ )� <� � d� �� �� �� �� �� �� �� �� P� |� �� �� �� 6� �� �� �� �� ;� \� �� �� j� �� �� �� � [ � 9 � � ^! Q# u& �& �& �' �' -( S6 �6 a7 �8 : u: ; �@ �D �E �G �G 2H �R xT �U aV 5W �[ �b tf �f G� ܆ [� 4� � Ӎ z� J� B� %� l� ϓ
� e� r� �� � Y� � Ƅ �� �� w� �� � k� � }� � �� �� � �� О � �� {� � 4� W� � Q� �� Q� � �� �� �� �� ]� �� � H� �� �� A� �� �� �� �� �� � ��
�
�
7
�
0
W
a`
d
�d
Be
�e
�x
z
�z
H{
�}
�
�
0�
@�
��
-�
v�
��
��
�
�y �z �{ �{ %~ � � �� �� ]� F� � ڇ � &� {� �� �� Г �� �� �� �� t� �� Ģ �� � \X �Y �Z �_ yf g bj �p Er +u �u �| W} �} � �� �� � � g� `� �� � � ȧ ը ҩ v� �� � �� 6� � ߿ ?� y� �� �� �� )� k� v� \� �� �� :� �� �� l� �� i� 6� G� �� � J� �� � � � A � ] V � 8
� � � � % g F \! �- �. �/ "3 RV W QX �X Y �[ %\ ` ;` �c �e �g �k �q *t �x �y ^z �z �~ � I� � D� �� � )� �� �� ۪ 9� |� � '� �� � � :� �� �� �� �� �� ,� �� �� �� �� �� @� #� Y� � k� �# 2$ v$ ' p' �6 �8 h9 �9 `> �> n@ �A B c` B� � l� 9� O� �� ͧ � � j� {� � L� b� .� &� � �� B� �� X� �� �� �� �� 8� p� f� J� �� �� .� � � c � � � � V l) �) �+ �1 �c �j >n �u �x �| t~ � �� �� � Ɋ V� �� �� [� �� D� :� � �� u� � \� �� �� �� �� !� ?� �� }� �� [� �� �� R� �� � �� � �� � �� _� �� �� �� �� 0� [� }� H� � [& J] ^ � � z � � � � � � 1* �+ �+ =- �/ �4 V6 �6 L8 �8 9 : = �= > �> �@ 9D �F �H I DM gN [O <P �P !R �S �T Z �` a Ia �a (b �c �f �g :j m �n ko �q dr �� 7� g� �� �� "� a� 5� �� �� �� � u� � �� � �� R� �� o � � � � � j
�
! � � � � � 7 � r � ( �" �# �$ )+ . 3 �< +A OB �B C SE SG �H [ }[ �[ '\ �\ y] �_ )a ]p 9s Os *u �u �z \{ 8~ �~ � -� �� �� �� �� ͎ i� � �� �� � \� � ,� �� c� '� � � �� �� �� \� � � h� � -� |� �� 8� *� � F� /� �� �� �� �� �� J� �� I �
� � } � & �+ �- a. }. �1 o2 77 {; �> @ �@ :D �D �F �R �S �T MU �U �Z �Z �] '^ �^ �l �p �q �t 8w �x �x � � �� � 2� �� [� u� � � ՙ 5� � �� *� Ʈ .� p� �� �� 5� � Ͻ � G� �� � !� � � �� �� V� �� � � 5� �� x� 9� �� �� �� � �� � �� �� �� �� �� �� D� �� 3� >� �� a � p % Z � Z � � � ] v � � � x8 9 j9 �; < �= > �? �? �@ >D �E �] k l =l Nu ?} t} E~ �� �� ͓ �� �� ח � �� H� � B� )� �� =� �� � M� �� %� �� ʽ �� � r� �� 8� �� 8� � S� �� �� � 0� %� �� � /( �( �( �* �. / �/ �2 6 �8 fE �E �K �K ML N O �O �O jP �Q �T �W Sd f oh �h �i 3k �k �l "m Xm cn 2p �p Kr �r �u �w `x �x Dy �y [z g| 3� � {� /� ˈ � &� � ϐ +� �� ג � �� � z� � &� � 1� ՚ ?� ؝ � �� Ѫ "� . j1 �� ߩ �� � G� ,� � �� � A� � ^� � Q� ӹ 4� �� �� X� �� �� �� � :� �� � \� �� d� �� �� �� �� e� ~� �� �� �� �� �� L ~ � � � . � � � � H �) �0 �7 �< �= X? �G I �I �I �P hQ �S �U �U 9W l` �a �b �� �� [� �� �� � {� �� � R� �� �� �� � (� .� �� � �� �� I� � ]� �� �� � � M� �� � �� �� �� 1� �� � �� �� d� ^� L� �� �� �� N� �� | � & � 910 _ �
� � l �) �* �+ �- ~8 �9 �9 �: �: 8; �; �= 9> �> �> A �C �G MH �L �L �N R &R S ^S 5T �T (U �U �V �� �� �� �� �� -� � � (� �� � �� � ?� j� 11� x� �� �� 7� �� 4� �� p� {� ְ x� U� ʳ �� �� � �� J� �� �� � � �� �� � q � 12 J � @ � 8 � � M � ~ � � � �2 6 �7 �= �> G �J �N ՙ `� b� �� �� �� C� l� ٤ q� �� >� �� �� a� S� �� g� е � � ȹ �� �� μ �� �� � �� �� �� |� 7� �� s� M� �� �� �� �� �� � n� B� � � � S � � � � � � � h w �8 )9 ; �; pB 9C �D �D �c d �f lg bj Fl \n vn �n u u �| � I� ɀ � �� �� P� �� � �� t� � �� �� �� � �� �� v� ۠ � _� Ǧ � �� � <� �� �� v� �� /� �� j� `� �� W� �� �� �� W� A� B� � Y� �� K� (� �� W� �� �� �� �� 4� �� �� �� <� �� K13! ! ! ! �! �"! .'! \'! �'! �*! 4,! !3! h4! �4! 5! �5! q6! �9! F! �N! Eb! �c! �d! ue! Uk! �k! \p! �x! x�! �! {�! ��! �! /�! N�! ��! ��! =�! ��! (�! X�! ��! ��! ��! ��! ��! .�! ˰! �! P�! k�! u�! T�! ��! ��! ��! {�! s�! ��! X�! D�! ��! I�! ��! ��! �! ��! �! ��! t�! w�! q�! ��! �! ��! p�! � "