dzungpham/graphcodebert-code-classification
0
12026-04-15 10:25:16,900 - INFO - train_pipeline - Logging to ./taskA-codebert-base/training.log22026-04-15 10:25:16,911 - INFO - train_pipeline - Loading model & tokenizer for 'microsoft/codebert-base'32026-04-15 10:25:18,028 - INFO - train_pipeline - Model placed on cuda42026-04-15 10:25:18,039 - INFO - train_pipeline - ===== Model Architecture =====52026-04-15 10:25:18,050 - INFO - train_pipeline - 6RobertaForSequenceClassification(7 (roberta): RobertaModel(8 (embeddings): RobertaEmbeddings(9 (word_embeddings): Embedding(50265, 768, padding_idx=1)10 (position_embeddings): Embedding(514, 768, padding_idx=1)11 (token_type_embeddings): Embedding(1, 768)12 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)13 (dropout): Dropout(p=0.1, inplace=False)14 )15 (encoder): RobertaEncoder(16 (layer): ModuleList(17 (0-11): 12 x RobertaLayer(18 (attention): RobertaAttention(19 (self): RobertaSdpaSelfAttention(20 (query): Linear(in_features=768, out_features=768, bias=True)21 (key): Linear(in_features=768, out_features=768, bias=True)22 (value): Linear(in_features=768, out_features=768, bias=True)23 (dropout): Dropout(p=0.1, inplace=False)24 )25 (output): RobertaSelfOutput(26 (dense): Linear(in_features=768, out_features=768, bias=True)27 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)28 (dropout): Dropout(p=0.1, inplace=False)29 )30 )31 (intermediate): RobertaIntermediate(32 (dense): Linear(in_features=768, out_features=3072, bias=True)33 (intermediate_act_fn): GELUActivation()34 )35 (output): RobertaOutput(36 (dense): Linear(in_features=3072, out_features=768, bias=True)37 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)38 (dropout): Dropout(p=0.1, inplace=False)39 )40 )41 )42 )43 )44 (classifier): RobertaClassificationHead(45 (dense): Linear(in_features=768, out_features=768, bias=True)46 (dropout): Dropout(p=0.1, inplace=False)47 (out_proj): Linear(in_features=768, out_features=2, bias=True)48 )49)502026-04-15 10:25:18,063 - INFO - train_pipeline - ===== Tokenizer Summary =====512026-04-15 10:25:18,087 - INFO - train_pipeline - Vocab size: 50265 | Special tokens: ['<s>', '</s>', '<unk>', '<pad>', '<mask>']522026-04-15 10:25:18,097 - INFO - train_pipeline - ===== End of Architecture Log =====532026-04-15 10:25:18,108 - INFO - train_pipeline - Base model weights frozen – only classifier head will be trained.542026-04-15 10:25:19,118 - INFO - train_pipeline - === Starting training ===552026-04-15 10:25:16,900 - INFO - train_pipeline - Logging to ./taskA-codebert-base/training.log562026-04-15 10:25:16,911 - INFO - train_pipeline - Loading model & tokenizer for 'microsoft/codebert-base'572026-04-15 10:25:18,028 - INFO - train_pipeline - Model placed on cuda582026-04-15 10:25:18,039 - INFO - train_pipeline - ===== Model Architecture =====592026-04-15 10:25:18,050 - INFO - train_pipeline - 60RobertaForSequenceClassification(61 (roberta): RobertaModel(62 (embeddings): RobertaEmbeddings(63 (word_embeddings): Embedding(50265, 768, padding_idx=1)64 (position_embeddings): Embedding(514, 768, padding_idx=1)65 (token_type_embeddings): Embedding(1, 768)66 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)67 (dropout): Dropout(p=0.1, inplace=False)68 )69 (encoder): RobertaEncoder(70 (layer): ModuleList(71 (0-11): 12 x RobertaLayer(72 (attention): RobertaAttention(73 (self): RobertaSdpaSelfAttention(74 (query): Linear(in_features=768, out_features=768, bias=True)75 (key): Linear(in_features=768, out_features=768, bias=True)76 (value): Linear(in_features=768, out_features=768, bias=True)77 (dropout): Dropout(p=0.1, inplace=False)78 )79 (output): RobertaSelfOutput(80 (dense): Linear(in_features=768, out_features=768, bias=True)81 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)82 (dropout): Dropout(p=0.1, inplace=False)83 )84 )85 (intermediate): RobertaIntermediate(86 (dense): Linear(in_features=768, out_features=3072, bias=True)87 (intermediate_act_fn): GELUActivation()88 )89 (output): RobertaOutput(90 (dense): Linear(in_features=3072, out_f2026-04-15 10:25:18,121 - INFO - __main__ - Loading datasets from Hugging Face Hub...912026-04-15 10:25:18,988 - INFO - __main__ - Train samples: 500000, Val samples: 100000922026-04-15 10:25:18,992 - INFO - __main__ - Tokenizing datasets...93 (dense): Linear(in_features=768, out_features=768, bias=True)94 (dropout): Dropout(p=0.1, inplace=False)95 (out_proj): Linear(in_features=768, out_features=2, bias=True)96 )97)982026-04-15 10:25:18,063 - INFO - train_pipeline - ===== Tokenizer Summary =====992026-04-15 10:25:18,087 - INFO - train_pipeline - Vocab size: 50265 | Special tokens: ['<s>', '</s>', '<unk>', '<pad>', '<mask>']1002026-04-15 10:25:18,097 - INFO - train_pipeline - ===== End of Architecture Log =====1012026-04-15 10:25:18,108 - INFO - train_pipeline - Base model weights frozen – only classifier head will be trained.1022026-04-15 10:25:19,118 - INFO - train_pipeline - === Starting training ===1032026-04-15 10:25:16,900 - INFO - train_pipeline - Logging to ./taskA-codebert-base/training.log1042026-04-15 10:25:16,911 - INFO - train_pipeline - Loading model & tokenizer for 'microsoft/codebert-base'1052026-04-15 10:25:18,028 - INFO - train_pipeline - Model placed on cuda1062026-04-15 10:25:18,039 - INFO - train_pipeline - ===== Model Architecture =====1072026-04-15 10:25:18,050 - INFO - train_pipeline - 108RobertaForSequenceClassification(109 (roberta): RobertaModel(110 (embeddings): RobertaEmbeddings(111 (word_embeddings): Embedding(50265, 768, padding_idx=1)112 (position_embeddings): Embedding(514, 768, padding_idx=1)113 (token_type_embeddings): Embedding(1, 768)114 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)115 (dropout): Dropout(p=0.1, inplace=False)116 )117 (encoder): RobertaEncoder(118 (layer): ModuleList(119 (0-11): 12 x RobertaLayer(120 (attention): RobertaAttention(121 (self): RobertaSdpaSelfAttention(122 (query): Linear(in_features=768, out_features=768, bias=True)123 (key): Linear(in_features=768, out_features=768, bias=True)124 (value): Linear(in_features=768, out_features=768, bias=True)125 (dropout): Dropout(p=0.1, inplace=False)126 )127 (output): RobertaSelfOutput(128 (dense): Linear(in_features=768, out_features=768, bias=True)129 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)130 (dropout): Dropout(p=0.1, inplace=False)131 )132 )133 (intermediate): RobertaIntermediate(134 (dense): Linear(in_features=768, out_features=3072, bias=True)135 (intermediate_act_fn): GELUActivation()136 )137 (output): RobertaOutput(138 (dense): Linear(in_features=3072, out_features=768, bias=True)139 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)140 (dropout): Dropout(p=0.1, inplace=False)141 )142 )143 )144 )145 )146 (classifier): RobertaClassificationHead(147 (dense): Linear(in_features=768, out_features=768, bias=True)148 (dropout): Dropout(p=0.1, inplace=False)149 (out_proj): Linear(in_features=768, out_features=2, bias=True)150 )151)1522026-04-15 10:25:18,063 - INFO - train_pipeline - ===== Tokenizer Summary =====1532026-04-15 10:25:18,087 - INFO - train_pipeline - Vocab size: 50265 | Special tokens: ['<s>', '</s>', '<unk>', '<pad>', '<mask>']1542026-04-15 10:25:18,097 - INFO - train_pipeline - ===== End of Architecture Log =====1552026-04-15 10:25:18,108 - INFO - train_pipeline - Base model weights frozen – only classifier head will be trained.1562026-04-15 10:25:19,118 - INFO - train_pipeline - === Starting training ===1572026-04-15 10:25:16,900 - INFO - train_pipeline - Logging to ./taskA-codebert-base/training.log1582026-04-15 10:25:16,911 - INFO - train_pipeline - Loading model & tokenizer for 'microsoft/codebert-base'1592026-04-15 10:25:18,028 - INFO - train_pipeline - Model placed on cuda1602026-04-15 10:25:18,039 - INFO - train_pipeline - ===== Model Architecture =====1612026-04-15 10:25:18,050 - INFO - train_pipeline - 162RobertaForSequenceClassification(163 (roberta): RobertaModel(164 (embeddings): RobertaEmbeddings(165 (word_embeddings): Embedding(50265, 768, padding_idx=1)166 (position_embeddings): Embedding(514, 768, padding_idx=1)167 (token_type_embeddings): Embedding(1, 768)168 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)169 (dropout): Dropout(p=0.1, inplace=False)170 )171 (encoder): RobertaEncoder(172 (layer): ModuleList(173 (0-11): 12 x RobertaLayer(174 (attention): RobertaAttention(175 (self): RobertaSdpaSelfAttention(176 (query): Linear(in_features=768, out_features=768, bias=True)177 (key): Linear(in_features=768, out_features=768, bias=True)178 (value): Linear(in_features=768, out_features=768, bias=True)179 (dropout): Dropout(p=0.1, inplace=False)180 )181 (output): RobertaSelfOutput(182 (dense): Linear(in_features=768, out_features=768, bias=True)183 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)184 (dropout): Dropout(p=0.1, inplace=False)185 )186 )187 (intermediate): RobertaIntermediate(188 (dense): Linear(in_features=768, out_features=3072, bias=True)189 (intermediate_act_fn): GELUActivation()190 )191 (output): RobertaOutput(192 (dense): Linear(in_features=3072, out_features=768, bias=True)193 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)194 (dropout): Dropout(p=0.1, inplace=False)195 )196 )197 )198 )199 )200 (classifier): RobertaClassificationHead(201 (dense): Linear(in_features=768, out_features=768, bias=True)202 (dropout): Dropout(p=0.1, inplace=False)203 (out_proj): Linear(in_features=768, out_features=2, bias=True)204 )205)2062026-04-15 10:25:18,063 - INFO - train_pipeline - ===== Tokenizer Summary =====2072026-04-15 10:25:18,087 - INFO - train_pipeline - Vocab size: 50265 | Special tokens: ['<s>', '</s>', '<unk>', '<pad>', '<mask>']2082026-04-15 10:25:18,097 - INFO - train_pipeline - ===== End of Architecture Log =====2092026-04-15 10:25:18,108 - INFO - train_pipeline - Base model weights frozen – only classifier head will be trained.2102026-04-15 10:25:19,118 - INFO - train_pipeline - === Starting training ===2112026-04-15 10:25:16,900 - INFO - train_pipeline - Logging to ./taskA-codebert-base/training.log2122026-04-15 10:25:16,911 - INFO - train_pipeline - Loading model & tokenizer for 'microsoft/codebert-base'2132026-04-15 10:25:18,028 - INFO - train_pipeline - Model placed on cuda2142026-04-15 10:25:18,039 - INFO - train_pipeline - ===== Model Architecture =====2152026-04-15 10:25:18,050 - INFO - train_pipeline - 216RobertaForSequenceClassification(217 (roberta): RobertaModel(218 (embeddings): RobertaEmbeddings(219 (word_embeddings): Embedding(50265, 768, padding_idx=1)220 (position_embeddings): Embedding(514, 768, padding_idx=1)221 (token_type_embeddings): Embedding(1, 768)222 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)223 (dropout): Dropout(p=0.1, inplace=False)224 )225 (encoder): RobertaEncoder(226 (layer): ModuleList(227 (0-11): 12 x RobertaLayer(228 (attention): RobertaAttention(229 (self): RobertaSdpaSelfAttention(230 (query): Linear(in_features=768, out_features=768, bias=True)231 (key): Linear(in_features=768, out_features=768, bias=True)232 (value): Linear(in_features=768, out_features=768, bias=True)233 (dropout): Dropout(p=0.1, inplace=False)234 )235 (output): RobertaSelfOutput(236 (dense): Linear(in_features=768, out_features=768, bias=True)237 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)238 (dropout): Dropout(p=0.1, inplace=False)239 )240 )241 (intermediate): RobertaIntermediate(242 (dense): Linear(in_features=768, out_features=3072, bias=True)243 (intermediate_act_fn): GELUActivation()244 )245 (output): RobertaOutput(246 (dense): Linear(in_features=3072, out_features=768, bias=True)247 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)248 (dropout): Dropout(p=0.1, inplace=False)249 )250 )251 )252 )253 )254 (classifier): RobertaClassificationHead(255 (dense): Linear(in_features=768, out_features=768, bias=True)256 (dropout): Dropout(p=0.1, inplace=False)257 (out_proj): Linear(in_features=768, out_features=2, bias=True)258 )259)2602026-04-15 10:25:18,063 - INFO - train_pipeline - ===== Tokenizer Summary =====2612026-04-15 10:25:18,087 - INFO - train_pipeline - Vocab size: 50265 | Special tokens: ['<s>', '</s>', '<unk>', '<pad>', '<mask>']2622026-04-15 10:25:18,097 - INFO - train_pipeline - ===== End of Architecture Log =====2632026-04-15 10:25:18,108 - INFO - train_pipeline - Base model weights frozen – only classifier head will be trained.2642026-04-15 10:25:19,118 - INFO - train_pipeline - === Starting training ===265 2026-04-15 10:25:16,900 - INFO - train_pipeline - Logging to ./taskA-codebert-base/training.log2662026-04-15 10:25:16,911 - INFO - train_pipeline - Loading model & tokenizer for 'microsoft/codebert-base'2672026-04-15 10:25:18,028 - INFO - train_pipeline - Model placed on cuda2682026-04-15 10:25:18,039 - INFO - train_pipeline - ===== Model Architecture =====2692026-04-15 10:25:18,050 - INFO - train_pipeline - 270RobertaForSequenceClassification(271 (roberta): RobertaModel(272 (embeddings): RobertaEmbeddings(273 (word_embeddings): Embedding(50265, 768, padding_idx=1)274 (position_embeddings): Embedding(514, 768, padding_idx=1)275 (token_type_embeddings): Embedding(1, 768)276 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)277 (dropout): Dropout(p=0.1, inplace=False)278 )279 (encoder): RobertaEncoder(280 (layer): ModuleList(281 (0-11): 12 x RobertaLayer(282 (attention): RobertaAttention(283 (self): RobertaSdpaSelfAttention(284 (query): Linear(in_features=768, out_features=768, bias=True)285 (key): Linear(in_features=768, out_features=768, bias=True)286 (value): Linear(in_features=768, out_features=768, bias=True)287 (dropout): Dropout(p=0.1, inplace=False)288 )289 (output): RobertaSelfOutput(290 (dense): Linear(in_features=768, out_features=768, bias=True)291 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)292 (dropout): Dropout(p=0.1, inplace=False)293 )294 )295 (intermediate): RobertaIntermediate(296 (dense): Linear(in_features=768, out_features=3072, bias=True)297 (intermediate_act_fn): GELUActivation()298 )299 (output): RobertaOutput(300 (dense): Linear(in_features=3072, out_features=768, bias=True)301 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)302 (dropout): Dropout(p=0.1, inplace=False)303 )304 )305 )306 )307 )308 (classifier): RobertaClassificationHead(309 (dense): Linear(in_features=768, out_features=768, bias=True)310 (dropout): Dropout(p=0.1, inplace=False)311 (out_proj): Linear(in_features=768, out_features=2, bias=True)312 )313)3142026-04-15 10:25:18,063 - INFO - train_pipeline - ===== Tokenizer Summary =====3152026-04-15 10:25:18,087 - INFO - train_pipeline - Vocab size: 50265 | Special tokens: ['<s>', '</s>', '<unk>', '<pad>', '<mask>']3162026-04-15 10:25:18,097 - INFO - train_pipeline - ===== End of Architecture Log =====3172026-04-15 10:25:18,108 - INFO - train_pipeline - Base model weights frozen – only classifier head will be trained.3182026-04-15 10:25:19,118 - INFO - train_pipeline - === Starting training ===319 2026-04-15 10:25:16,900 - INFO - train_pipeline - Logging to ./taskA-codebert-base/training.log3202026-04-15 10:25:16,911 - INFO - train_pipeline - Loading model & tokenizer for 'microsoft/codebert-base'3212026-04-15 10:25:18,028 - INFO - train_pipeline - Model placed on cuda3222026-04-15 10:25:18,039 - INFO - train_pipeline - ===== Model Architecture =====3232026-04-15 10:25:18,050 - INFO - train_pipeline - 324RobertaForSequenceClassification(325 (roberta): RobertaModel(326 (embeddings): RobertaEmbeddings(327 (word_embeddings): Embedding(50265, 768, padding_idx=1)328 (position_embeddings): Embedding(514, 768, padding_idx=1)329 (token_type_embeddings): Embedding(1, 768)330 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)331 (dropout): Dropout(p=0.1, inplace=False)332 )333 (encoder): RobertaEncoder(334 (layer): ModuleList(335 (0-11): 12 x RobertaLayer(336 (attention): RobertaAttention(337 (self): RobertaSdpaSelfAttention(338 (query): Linear(in_features=768, out_features=768, bias=True)339 (key): Linear(in_features=768, out_features=768, bias=True)340 (value): Linear(in_features=768, out_features=768, bias=True)341 (dropout): Dropout(p=0.1, inplace=False)342 )343 (output): RobertaSelfOutput(344 (dense): Linear(in_features=768, out_features=768, bias=True)345 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)346 (dropout): Dropout(p=0.1, inplace=False)347 )348 )349 (intermediate): RobertaIntermediate(350 (dense): Linear(in_features=768, out_features=3072, bias=True)351 (intermediate_act_fn): GELUActivation()352 )353 (output): RobertaOutput(354 (dense): Linear(in_features=3072, out_features=768, bias=True)355 (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)356 (dropout): Dropout(p=0.1, inplace=False)357 )358 )359 )360 )361 )362 (classifier): RobertaClassificationHead(363 (dense): Linear(in_features=768, out_features=768, bias=True)364 (dropout): Dropout(p=0.1, inplace=False)365 (out_proj): Linear(in_features=768, out_features=2, bias=True)366 )367)3682026-04-15 10:25:18,063 - INFO - train_pipeline - ===== Tokenizer Summary =====3692026-04-15 10:25:18,087 - INFO - train_pipeline - Vocab size: 50265 | Special tokens: ['<s>', '</s>', '<unk>', '<pad>', '<mask>']3702026-04-15 10:25:18,097 - INFO - train_pipeline - ===== End of Architecture Log =====3712026-04-15 10:25:18,108 - INFO - train_pipeline - Base model weights frozen – only classifier head will be trained.3722026-04-15 10:25:19,118 - INFO - train_pipeline - === Starting training ===373