verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1� � � b � � � 4 � � � � � � � �I �K �L �L QT �T ]U �W �Y �Z �Z �[ T\ �\ @] �j k zm �o ^p �p gu �w Mx �z �} ݂ �� �� � <� �� �� e� E� џ � �� �� �� � �� � ۵ J� O� � .� �
� � �� e� j� �� � �� �� .� Y� �� h� �� � B� �� "� L� � C� � � � < � � � | � �% & �& �( �* �* 9+ . W: �: �> �> F G �G TI 3K qM �M �N �P �Q kR S WT �U <W �[ U\ �\ e 'k `l �t �t �u �u x U| �} Ä � u� �� �� �� e� � �� ϥ �� �� �� K� �� �� -� � �� w� �� c� �� �� � c� �� �� -� w� �� �� ?� �� �� �� �� &� �� 0� � �� )� �� �� �� e M � 4 2( �( ) �) 8* �* �+ q� ԡ ԣ � 0� �� F� �� S� �� �� � �� �� � �� �� E� I� D� � � � � r � � < a ; � W m # 2 � �4 �6 �8 9 �; 7? �C f� �� �� �� ?� m� y� �� ݩ �� �� �� >� �� ~� A� 8� �� � x� �� P� �2 �3 M � �
� � � T# >$ - ^- �. �0 N1 84 �4 7 �7 D= �? �? T@ �A G 2G NJ } S} � ڇ � ^� �� L� � �� �� �� R� �� d� �� ]� �� � %� H� � �� ~� �� .� j� к � T� � Z� ?� ,� P� �� �� �� �� �� �� �� T� d� �� �� P� ) D
x � � � L Z � �$ S% B' �' �( �* ~, �1 53 s3 �3 �4 �6 �; = �= ? ? �A yB �B C R �R �S bW �[ �\ cs �s �~ � _� �� � 1� �� ҧ �� �� � Q� �� �� w� 9� �� �� W� й � �� �� h� !� �� A� � n� �� '� �� �� n� �� M� ;� �� �� �� �� �� �� �� �� � � � \ � 6 � 4 � � � ' T
� A6 �9 V: �: j; �> D �E �E �J �L �O JQ %U =Z \ �\ �] �^ jb c Cc �e �g �g Kl �l �o �o qp �w �w /y �y
� k� � �� Ō ؎ �� ܓ � #� ˢ �� � &� �� ǥ � �� � b� f� �� �� }� /� n� �� �� �� �� o� �� A� �� �� J� �� � � G� �� �� o� � � � ( � � �: �; p< = r= �= G> G? �? PF �F H �X \ �` �a $b >d �d \t �v { : � b� o� �� �
� �� d� �� .� �� �� �� � $� �� �� M� �� d� ܽ �� {� � �� �� )� �� �� �� �� �� 9� <� �� �� [� �� ?� J� � �� X� r � [ � -5 O 7V |V W �W �X �a �b �c �d 9f �f g �g �p w ,w 6{ | J� 6� �� �� Z� @� Փ >� � 3� � X� � � � � �� Ѽ �� � Z� � �� b� �� I� � �� � �� C� h� 6� �� �� �� �� ^� � �� 5� �� L� � f� ~� X� �� �� �� �� �� �� Q �
� �
� l ," �+ T, . �. Q/ �0 �0 z1 2 �4 H5 ;8 9 �9 �: ; Z ]\ �] �_ �b vc �d �f xh gw �w | } '~ � � ς �� � �� Ň 7� ;� � 1� �� $� ɑ M� x� � ۗ �� #� �� E� �� )� n� �� -� �� Y� �� \� �� �� �� �� P� �� �� E� �� � �� � {� �� �� Y� �� � �� �� �� :� �� �� �� U� �� (� � 8