verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1� }� �� L � o � � ^( �) /* �/ ?0 �0 �1 �1 �2 f3 �3 ? cB �C D �D |J 9K rL �M hN �N �O �P �P �R T sU �U �W V� ߇ n� �� �� �� �� &� �� F� %� Z� �� �� � �� �� z� � נ � � .� >� �� ݥ 2� �� (� � a� � j� J� '� f� � �� �
� � ݿ � �� �� �� �� 5� �� w� �� �� � �� �� �� ;� �� �� �� � �� �� �� �� H� �� j � r � . &2 �3 �4 � � + = � � D � � � 4 J$ �& �( �( g1 �5 �< 9= > �> *? A �\ }k �k �m �n �o �q �q �r �s t �w 5x hx ]z �z w} �~ 5 W� � �� ۇ @� � ͎ k� 6� R� Ɯ �� )� � �� � W� �� o� � �� P� m� �� .� i� V� b� �� (� �� �� `� �� �� +� � �� �� �� � � � � � � � 1 � � � �" �% [' ԅ 7� �� U� X� �� �� ^� � � Ҩ >� <� Y� �� P� �� �� 9� �� �� n� G� �� Y� �� P� �� �� � �� �� �� �� � � � # c � � � � : � � � � ]"
' L) * d- �1 $2 �2 3 �3 44 �4 �? �@ B �B (g Qk ]l �t u �w Gx $z �{ � 8� p� ń � �� Q� � #� ƞ �� }� � � r� �� Ѯ �� �� �� R� �� � � � X� � �� l� � � ?� �� +� �� � �� Y� K� �� �� �
�
o
c
�
K
`"
|*
�I
�T
�c
�e
f
5h
�i
�j
l
_n
�p
%r
Qr
� 5� � �� g� �� �� q� y� o�
� �� �� 6� �� a� G� � s� &� �� �� �� �� � � � � � � � � e �& !- v5 �7 "8 �8 �9 �: C< j< B `D J xP c d �d f ;g �g %k �k o �q ]r �s v w r� K� #� �� <� `� �� (� �� R� u� � ]� � ʽ �� �� �� �� b U# & l& �( d) e2 53 �3 M5 �@ �D �D E �I @J ?O �Y =Z �Z 7` tc Rj �j �k l 6l ir �s �v .z *~ �~ �� *� �� � �� �� /� � �� � � 2� g� � -� s� �� 6� i� �� �� ǝ � �� � 7� a�
� �� ҧ �� ~� U� 0� � �� � �� ?� $� >� �� r� �� ]� �� � �� �� 7� %� f� �� � ? W � \8 �9 U � � � �Q �T �V �V MW eX �b 7c �f i Ts :t �x �y vz �� �� �� @� Q� b� ԉ A� � n� �� #� �� ߝ _� &� z� M� J� � �� η �� ]� O� s� ^� �� �� a� �� �� � �� �� D � e& �( u5 7 �7 �8 �: �; �; �= @ �@ �H M NZ _Z �Z I[ �\ _ �o Ap Cs �s )u o{ �{ "| �| � � l� � 0� D� f� ҅ � �� ݉ � �� q� m� c� Ֆ ؛ �� �� �� ,� x� V� � e� �� �� �� � �� 1� �� �� �� �� ?� W� �� � .� �� W� %� �� �� �� w� �� � �� �� �� �� I , � � �% ) �+ �2 �= > �A �B DM �N ~Q �W �X \Z �[ �] F� k� G� v� -� � �� � �� � � � x� ڜ E� � E� �� � �� �� L� 3 � ]
�
& � �# 10I *I PJ �K WL VO <V uW �X }Y �[ \ �] J^ �_ �` Aa Te Lk � +� �� \� Ε �� V� � � �� {� �� � B� � �� .� [� �� �� �� @� ۿ 1� �� *� !� �� Y� �� D� �� �� �� 1� 3� {� �� ,� m� �� �� � �� � d
M � � i n � � � � � _ �� �� �� �� � 2� �� � �� p� �� �� 3� � �� g� %� � � � �11 � F
Y � � � ` � M! �! �# �* �1 �2 �r �s �s 12t �w -x rx uz V~ � ]� n� �� l� H� � P� V� �� ̦ �� Y� �� o� ϶ "� S� �� �� � �� ~� �� !� ��
� G � � ` M � & . �" p* W+ %N �O ?P �P *T h Nh j �j �v Nx �x �x �x �y �| �~ � �� g� �� �� ѓ k� �� � ݘ � -� �� � 4� � k� �� � ۲ 13� �� � �� � �� � G� �� ?� �� � g� q� �� #� q� Y� :� �� �� <� � w� �� �� �� z� �14 ` �
� , � � f E � x � � (! �# �% �& ' f) �+ �A *M �Q �R *T ET /U F[ �[ Cc �d ji j �l �w �y vz �� � �� f� e� (� K� �� *� �� 8� �� �� �� "� � �� (� �� U� O� �� >� l� (� ]� �� 2� �� >� � �� &� � �� �� e� �� >� � �$ �& ( �) �+ �, b. �4 �5 �8 �9 w: < �< HB �D �G �J pL M �Y �Z �l +p �p r Sv ow y �y { �~ �~ *� �� ԃ �� 5� [� �� �� �� ߏ � (� � n� %� E� 8� K� �� �� s� �� �� )� D� W� �� L� �� �� �� `� �� �� -� �� �� $� :� �� �� X� � X � � � 215 �
� � E ! [
M * � ; 16 ^ � D � �&