verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1� � � � � �K �L WN O R �S �S �S ^T �T U �W f[ �[ �^ �^ �^ xp &q t 4v �v Yx gy 2| � �� \� �� %� �� ^� �� �� s� ˜ � �� Σ (� ک
� *� �� �� � w� �� �� �� � �� U� i� � 4� �� r� M� �� T� t� \� �� �� �� �� � �� �� M� #� �� �� �� � � W � � b ! m # P � � W! �# C$ �+ �+ , . / n/ H1 3 }4 27 �:
N �N �Q !W �Y �Z I[ +^ _ �` qa se !j �k m �n �q s �s �t �u x y x} h~ � z� � V� � t� S� ˪ [� �� � � � �� � �� �� F� �� �� j� p� ]� �� �� �� �� 2� �� �� O� a� b� �� l� �� �� � <� O� � �� �� �� '� � � A2 �3 � �
� � � � � � z � x �# �% �) �) �* �+ K, 41 �1 �2 _3 �> )C DD N O �U Sj dr /u -w lx �{ | �| �~ N ц '� H� *� � �� �� �� � W� �� �� 4� �� �� �� F� J� �� p� �� s� � g� � �� ˾ �� � N� �� � �� F� �� �� +� �� � � �� �� 8� V� �� �� �� �� �� �� �� �� �� � � � � 9 T � � � n � D � $ y$ 2& �' �) + �. V/ R2 �2 3 4 '5 u5 �5 G6 �7 1:
; �> �D E �O bQ �Y \ 5\ &� � Ɋ 5� F� a� � ޒ � (� �� 7� �� �� /� M� �� �� Z� 5� �� �� �� �� � ^� h� �� �� �� �� �� �� �� 7� ]� �� �� �� � � � � �4 �
� Q � � � � N! �" ## 5& �& �( $+ + o1 )3 :4 d8 �= M> ? k@ �B dC �G DH ZJ �J �K �W �[ �[ /] ]_ �o 5s �t � j� &� � �� ̌ �� 8� Ҕ R� o� ֙ .� 9� ̞ � � u� ϧ � �� �� a� $� `� �� �� �� +� _� ,� �� 1� �� �� �� �� � s� �� n� /� � 8� � 5 �6 � ^ A0 �0 1 3 �3 �5 �6 �9 �; e< )@ _@ �E vG xI �I )K L M -V zV S] �] v_ Ua �d e � �� � �� �� +� �� �� Z� S� nQ NS }j �l �t v pw �| �} 9� V� �� O� |� ]� � (� � <� +� 6� � 7� O� �� �� �
� `� ;� \� �� *� �� I� �� �� )� � x� �� � K� �� �� - ] . � C! 31 �2 �3 �= ;K �K �L 8M N vN �N _O @Q fR kU TW �W ` �e �e �g �g ~h �i )k l Xm �m �n tw �x �y �z ]~ �~ j� �� i� �� R� �� �� ;� � �� e� x� �� �� r� _� �� י �� /� Z� ̦ &� H� �� ]� �� ~� +� � =� S� �� �� V� �� � U� �� l� �� �� �� |� �� f� �� � �� �� �� � �� S� �� 4� h � 1 j F p W � * 6" �" �8 y; = �= > �> �? '@ /D tD �F �F �F �G �K �O nQ �R �S �T �U �U aV �Y �e Kg �g =l n �n =o 0p �p �q r �r �s �t �u �x "y �y �7