verify-ppt/smollm3-stack-v2-Java
Synthetic Pre-pretraining Datasets This repository contains the pre-processed datasets released with the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The datasets cover both pre-pretraining (PPT) tasks and pre-training (PT) mixtures used in the paper. PPT tasks include k-Shuffle Dyck, Set, MP-Struct Core, and NCA; PT mixtures include C4, SmolLM3, Olmo3, Marin, and FineWeb-Edu configurations. The pre-processed data are tokenized and organized by… See the full description on the dataset page: https://huggingface.co/datasets/verify-ppt/smollm3-stack-v2-Java.
0610
1m W � �( �* �+ �- �. �/ 61 �1 (x �x \z `{ ǀ �� �� � �� �� � �� �� ޡ � �� � s� �� �� >� l� �� e� r� ,� g� �� <� �� �� �� �� �� �� t� !� V� �� �� @ � G
� z � � � " �% I' �) �* �+ �+ l, �, �- �2 +7 �7 �< #? |? �? M@ ZE �E �H )J �J bK �K L <L �P �R \S �S �V X �X )b Jb d �d |n �o Vp �q nt �w �z �~ �� [� �� �� σ a� >� �� r� �� � w� �� � �� x� �� a� �� u� � ]� �� � � �� ص �� � � I� � |� �� �� �� s� �� � �� Z� �� �� T� �� �� %� �� �� �� ��
? | � z C �1 U2 �2 �2 h3 44 �4 �5 �6 �8 #; =? ~? �@ �C �E �X �Y �[ %_ �d �h �k dl Tm �n �n � t� �� � �� � :� � �� N� �� v� 2� 3� Q� ' 2 � ^4 �5 � � � 6 l# v& �' �' �( �) Z* �+ h- �- Q. @/
0 �0 J2 g3 �3 44 �� ڎ A� �� �� �� =� �� � $� �� �� >� x� � f� 0� ��
� �� �� z� �� -� V� �� �� �� �� 6� � %� �� $� n� � {� �� J y
�
j � T � d � �
� � P � � M! �' �( �: �; �; �< @ T@ �B �D �K �K ON hP �[ �] ^ S^ 9_ n` Aa �p �q �w Y| �~ � �� 9� �� \� s� �� ޤ '� �� Z� �� d� �� ϻ M� w� � �� _� ,� �� �� 8� �� �� t� �� �� G� �� �� �� �� ;� �� _ p7 D d � � � 9! " & �+ -- �- . h/ +0 11 �2 �7 �7 )9 z< �< > C> �> �? .@ �@ 8R �R VS �c Gd �d Ml �l o �p �s <� ς �� Ȇ � A� U� Ď �� <� �� :� Ӕ o� x� � � ,� .� � �� e� �� �� F� � ɶ � O� :� �� �� >� r� � �� �� 5� T� �� P� M� �� *� �� �� L� �� �� �� �� 1� $� �� � } - � �9 �
f 3 � �( �) k+ �+ �+ ]Q �Q �T �U mY �Y ;^ s` �` za �b �c �c �e dg <h �h ci �j Vk �k �m n -n oo �w � 10� �� c� �� � �� 3� �� �� �� �� S� 1� U� h� p� �� � �� �� �� ?� �� �� �� -� �� �� ]� �� �� � �� � 1 �
� L � � z s � ) � � Z! �% -) �) �* ]+ �- �- Z. K/ 2 �6 �7 8 �9 t> �@ �C �E �F �L DN �Q �e l m Ap �r Gs �u υ �� �� W� Љ S� R� x� B� Q� !� �� � ޓ � � Ǘ K� 5� � 7� l� ˥ �� 5� s� �� � A� 0� #� 4� �� 1� ~� �� � �� ]� �� g� �� �� �� �� �� � ,� � �� �� � {� 7+ 0 20 �0 �1 �2 <4 b4 �= > �B �D �E F MF �H �M Vd rf 7q �z �} �~ �~ y �� �� _� |� 3� � @� � ԉ 1� �� �
� K� �� �� m� |� �� 4� �� � �� r� � �� c� � �� �� �� `� t� �� q� �� �� :� �� �� �� � �� ,� �� A� L� �� �� L� � S� � � M � �% �% �% 6 V6 �6 �6
< �B �E �G N VN ~O zP �P �\ �\ l^ �h �n �o �r �u Bw ]y |� �� T� �� � � B� "� ~� ǒ \� � � �� ̚ � P� h� 4� ܿ �� R� � 3� �� s� �� G� �� J� �� ;� +� Z� �� J� X� � �� T� \� o� D� �� 8� �� /� � � 511