verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1�
� l . � � n E! �! �! =1 �2 �3 f6 4? ^? �? �A HB �C UE �E MF �G {I �I �J +L �N aO <P S 6S �U �V F\ ] �^ ` To vp <q �q 0r v ev {y { k| �| S� � M� �� � �� � � a� �� f� � � ۬ �� }� �� �� v� �� � � �� � C� �� -� �� � �� �� F� n� �� 7� O� �� j� �� e� � � i� U� �� �� k� - a � � z R$ �* , �, �, �- �. �0 83 �3 9 �9 ': �: ~B WC )D �D =H GJ dM 2N �O GV !i Gm �m *o Rq �q �r �t �u v cv �w �x y iy �{ H� �� Γ � [� V� U� �� �� %� Q� �� V� v� ?� �� �� �� �� �� F� � ;� T� �� m� }� �� �� � \� C� �� ?� h� � T� S 0 j �m �m n �o p x Cx uy �z { �{ U| ]} ڕ
� �� ݟ Z� �� �� �� � �� e� �� � T� 1� o� l� �� �� �� �� �� � � � 8, �/ �; �< = *> �@ �K fL �L �M �P 6T uV �W X %X �X %Z sZ
^ �^ a b 5c �d �e Sf �h �j nk �m =n Ap (q �v Aw _w x �{ '| �� � � \� m� � � �� Ϣ ?� �� �� W� �� �� � � ( m � � ` d> 5@ �B �O R �S U �V �V W <X �X ^Z �Z �] �^ b �b �c ?e g 1g 7i �j 1k l nl 1m n �n �n Mp �p �q �t ~u qw �w x bx �~ � �� j� *� K� Ƈ � �� `� � G� *� "� S� � �� � �� ��
� �� ѭ �� �� ]� �� O� h� � �� �� �� �� �� �� �� 8� *� �� �� � �� Z� �� �� �� (� � � a ~ � �2 �3 � L � a � �
Y ] < m � � � O A!
I YI �I YK �K �L \N xO �P 0R �S �S �T �X FZ yZ �] �y .z ;{ g{ +| ~ =� � �� 8� ފ �� M� � c� ߧ ި � �� �� �� �� � K� Ȯ ѳ 9� �� � ȵ �� r� N� �� ]� N� � k� �� �� o� � �� � !� a� �� �� �� \� � _� �� x� �� d� �� �� �� -� � d � � Z �# f5 �7 �: > �> �@ A xB @D ~I �J -U �U ]W �W �Y �\ �_ ` (b �j �l 7n �r �s �s �t Vw p� � �� � _� 7� Ǐ "� �� � � �� O� �� �� �� � �� &� �� N� .� �� � \� �� � �� d� � 1� �� D� ^� �� �� � � � ) � � � e E i � � � � � �! �1 |2 �2 �5 8 '9 �: �A �B FC �D �D �E %H HS \ he �� $� �� 1� �� � 7� �� �� �� � �� � �� I� $� o� }� �� �� =� 5� q� � �� � �� 8� @� �� L� ~� w� +� �� E� �� �� �� �� �� b� �� � � . { � G L � 4 5 86 & � � c r% �% �% �& 9( j* �* �. �? �t �t nv }v Dx z 7z q{ �{ *| Q� ��
� ŕ � c� Ƞ �� �� L� �� �� �� d� �� � B� �� �� � T � 7 o
� l $ �$ G% M& �9 8J VJ -L pM �Q S �T �{ ]~ �~ � 7� � p� �� �� O� � ̙ � � � �� � ~� ު (� �� z� �� �� 4� u� � $� B� (� s� � a� R� � �� �� #� U� �� �� �� m� 2� Q9