Modelpublic

buelfhood/SOCO-Java-CodeBERTa-Softmax-PairClass

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes27downloads

Model Card

SentenceTransformer based on huggingface/CodeBERTa-small-v1

This is a sentence-transformers model finetuned from huggingface/CodeBERTa-small-v1 on the soco_train_java dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

—Model Type: Sentence Transformer
—Base model: huggingface/CodeBERTa-small-v1
—Maximum Sequence Length: 512 tokens
—Output Dimensionality: 768 dimensions
—Similarity Function: Cosine Similarity
—Training Dataset:
—soco_train_java

Model Sources

—Documentation: Sentence Transformers Documentation
—Repository: Sentence Transformers on GitHub
—Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("buelfhood/SOCO-Java-CodeBERTa-Softmax-PairClass")
# Run inference
sentences = [
    '\n\n\n\n\n\n\nimport java.io.*;\nimport java.lang.Object;\n\npublic class WatchDog\n{\n\n   \n   public static void main(String args[])throws Exception\n   {\n        Process p1,p2,p3,p4,p5;\n        \n        for(;;)\n        {\n          \n\n          String s1[] = {"/usr/local//tcsh", "-c", "mailx -s \\"Part 2-Assignment2 \\"  < change.html"};\n          String s2[] = {"/usr/local//tcsh", "-c", "mv www.cs.rmit.edu./images/*.* predir"};\n          String s3[] = {"/usr/local//tcsh", "-c", "mv www.cs.rmit.edu./students/*.* predir"};\n          String s4[] = {"/usr/local//tcsh", "-c", "mv www.cs.rmit.edu./images/*.* postdir"};\n          String s5[] = {"/usr/local//tcsh", "-c", "mv www.cs.rmit.edu./students/*.* postdir"};\n          String s6[] = {"/usr/local//tcsh", "-c", "diff copy1 copy2 > diff.html"};\n\n\n          Process p = Runtime.getRuntime().exec("mkdir predir");\n          p.waitFor();\n          Process p1 = Runtime.getRuntime().exec("mkdir postdir");\n          p1.waitFor();\n\n          \n          p1 = Runtime.getRuntime().exec("wget -p --convert-links http://www.cs.rmit.edu./students/");\n          p1.waitFor();\n\n          Process q2 = Runtime.getRuntime().exec(s2);\n          q2.waitFor();\n          Process q3 = Runtime.getRuntime().exec(s3);\n          q2.waitFor();\n\n          \n          Thread.sleep(86400000);\n\n          p3 = Runtime.getRuntime().exec("wget -p --convert-links http://www.cs.rmit.edu./students/");\n          p3.waitFor();\n\n          Process q4 = Runtime.getRuntime().exec(s4);\n          q4.waitFor();\n          Process q5 = Runtime.getRuntime().exec(s5);\n          q5.waitFor();\n\n          try\n          {\n            String str;\n            p4 = Runtime.getRuntime().exec(s6);\n            DataInputStream inp1 = new DataInputStream(p4.getInputStream());\n            p4.waitFor();\n            \n            System.out.println("The WatchDog - Returns 0 if  change  else 1");\n            System.out.println("Value :" + p4.exitValue());\n            try\n            {\n                while ((str = inp1.readLine()) != null)\n                {\n                    System.out.println(str);\n                }\n            }\n            catch (IOException e)\n            {\n                System.exit(0);\n            }\n\n          }\n          catch(FileNotFoundException e )\n          {\n              e.printStackTrace();\n          }\n\n          BufferedReader in = new BufferedReader(new FileReader("change.html"));\n          \n          if (in.readLine() != null)\n          {\n\n          try\n          {\n            String str1;\n            p5 = Runtime.getRuntime().exec(s1);\n            DataInputStream inp2 = new DataInputStream(p5.getInputStream());\n            p5.waitFor();\n          try\n            {\n                while ((str1 = inp2.readLine()) != null)\n                {\n                    System.out.println(str1);\n                }\n            }\n            catch (IOException e1)\n            {\n                System.exit(0);\n            }\n\n          }\n          catch(FileNotFoundException exp)\n          {\n              exp.printStackTrace();\n          }\n\n        }\n    }\n  }\n}\n\n',
    'import java.io.*;\nimport java.util.*;\nimport java.net.*;\nimport java.net.Authenticator;\n\n\npublic class  BruteForce\n{\n\n\tprivate String result ="";\n\n\tpublic class customAuthenticator extends Authenticator {\n\t       public customAuthenticator(String passwd)\n           {\n            this.pass = passwd;\n           }\n\n\t       protected PasswordAuthentication getPasswordAuthentication()\n           {\n\t                 return new PasswordAuthentication("",pass.toCharArray());\n           }\n           public String pass;\n    }\n\n    public BruteForce() {\n            java.util.Date d = java.util.Calendar.getInstance().getTime();\n            System.out.println(d.toString());\n\t\tchar words[] = { \'a\',\'b\',\'c\',\'d\',\'e\', \'f\', \'g\', \'h\', \'i\',\'j\',\'k\',\'l\',\'m\',\'n\',\'o\',\'p\',\n\t\t\t\t\t\t\t  \'q\',\'r\',\'s\',\'t\',\'u\',\'v\',\'w\',\'x\',\'y\',\'z\', \'A\',\'B\',\'C\',\'D\',\'E\', \'F\', \'G\',\n\t\t\t\t\t\t\t   \'H\', \'I\',\'J\',\'K\',\'L\',\'M\',\'N\',\'O\',\'P\',\'Q\',\'R\',\'S\',\'T\',\'U\',\'V\',\'W\',\'X\',\'Y\',\'Z\'};\n\n\t\tString record = null;\n\n\n\n        String url = "http://sec-crack.cs.rmit.edu./SEC/2/";\n\n\t\tchar pass[] = {\'x\',\'x\',\'x\'};\n\t\tint count=1;\n\t\tString passwd=new String();\n        HttpURLConnection connection = null;\n        URL u = null;\n\n        try\n        {\n         u = new URL(url);\n\n        }\n        catch (MalformedURLException e)\n        {\n        }\n\n        for(int a=0;a<words.length;a++)\n        {\n                for(int b=0;b<words.length;b++)\n                {\n                        for(int c=0;c<words.length;c++)\n                        {\n                                 pass[0]=words[a];\n                                 pass[1]=words[b];\n                                 pass[2]=words[c];\n                                 passwd=passwd.copyValueOf(pass,0,3);\n                                 System.out.println(count+ " ) " + passwd);\n                                 count++;\n                                 try\n                                 {\n\n                                       connection = (HttpURLConnection) u.openConnection();\n                                       Authenticator.setDefault(new customAuthenticator(passwd));\n\n                                       if (connection.getResponseCode()!=401)\n                                       {\n                                            System.out.print("The password is : "+passwd);\n                                            System.out.println();\n                                            java.util.Date d1 = java.util.Calendar.getInstance().getTime();\n                                            System.out.println(d1.toString());\n                                            System.out.println("\\ntime taken in seconds:"+ (d1.getTime() - d.getTime())/1000+"\\n");\n\n                                            System.exit(0);\n                                       }\n                                       else\n                                       {\n                                       }\n                                       connection.disconnect();\n                                 }\n                                 catch (IOException e)\n                                 {\n                                       System.out.println(e);\n                                 }\n                        }\n                }\n        }\n    }\n\n\tpublic static void main(String[] args)\n\t{\n\n\n\t\tBruteForce  = new BruteForce();\n\t}\n}',
    'import java.Object;\nimport java.io.*;\nimport java.String;\nimport java.util.*;\n\nclass Dictionary{\n\n      public static void main(String [] args){\n              try\n                {\n                Date d = new Date();\n                String line1="";\n                String ps="";\n                String file1 = "words.txt";\n                String file2 = "index.html";\n                String endline="Authorization failed.";\n                String [] cmd = new String[4];\n                cmd[0] = "wget";\n                cmd[1] = "--http-user=";\n                cmd[3] = "http://sec-crack.cs.rmit.edu./SEC/2/";\n\n                FileReader fr1 = new FileReader(file1);\n                BufferedReader in1 = new BufferedReader(fr1);\n                while((line1 = in1.readLine())!=null)\n                {\n                  try{\n                    cmd[2] = connect(line1);\n                    Runtime.getRuntime().exec(cmd);\n                    if(line1.length()==3)\n                      ps = line1;\n                    System.out.println(cmd[2]);\n                    File f = new File(file2);\n                    if(f.exists())\n                    {\n                    System.out.println("password: " + ps);\n                    break;\n                    }\n                   }\n                    catch(IOException ex)\n                    {\n                        System.out.println("hello1");\n                    }\n\n                }\n                Date end = new Date();\n                System.out.println(d.toString());\n                System.out.println(end.toString());\n                System.out.println("Seconds: " + (end.getSeconds()-d.getSeconds()));\n               }\n\n            catch(IOException e)\n            {\n              System.out.println("hello,didnt find file.");\n            }\n      }\n    public static String connect(String str1)\n            {\n                 char data[] = {\'-\',\'-\',\'h\',\'t\',\'t\',\'p\',\'-\',\'p\',\'a\',\'s\',\'s\',\'w\',\'d\',\'=\'};\n                 String str = new String(data);\n                 return str + str1;\n            }\n\n}\n',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

socotrainjava

—Dataset: soco_train_java at 44ca4ff
—Size: 30,069 training samples
—Columns: <code>label</code>, <code>text1</code>, and <code>text2</code>
—Approximate statistics based on the first 1000 samples: | | label | text1 | text2 | |:--------|:-----------------------------------------------|:------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | type | int | string | string | | details | <ul><li>0: ~99.80%</li><li>1: ~0.20%</li></ul> | <ul><li>min: 51 tokens</li><li>mean: 448.9 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 54 tokens</li><li>mean: 464.58 tokens</li><li>max: 512 tokens</li></ul> |
—Samples: | label | text1 | text2 | |:---------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>0</code> | <code>import java.io.; import java.net.; import java.security.; import java.math.; import java.; import java.util.; public class WatchDog { public static FileWriter out = null, output = null; public static void main (String args[]) throws Exception { Socket socket = null; DataOutputStream = null; BufferedReader bf = null, fr = null; String retVal = null, StatusCode = "HTTP/1.1 200 OK"; int dirty = 0, count = 0; stime = System.currentTimeMillis(); System.out.println("Detecting the changes..."); try { URL yahoo = new URL("http://www.cs.rmit.edu./students/"); URLConnection yc = yahoo.openConnection(); BufferedReader in = new BufferedReader( new InputStreamReader( yc.getInputStream())); String inputLine; try { out = new FileWriter("newstudent"); while ((inp...</code> | <code> import java.util.; public class Cracker { private char[] letters = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'}; private Vector v; public Cracker() { v = new Vector( 52); } public void loadLetters() { int i; for( i = 0; i < letters.length; i++) { String s = new StringBuffer().append( letters[i]).toString(); v.add( s); } } public Vector getVictor() { return ; } public void loadPairs() { int i,j; for( i = 0; i < letters.length - 1; i++) { for( j = i + 1; j < letters.length; j++) { String s1 = new StringBuffer().append( letters[i]).append( letters[j]).toString(); String s2 = new StringBuffer().append( letters[j]).append( letters[i])....</code> | | <code>0</code> | <code> import java.net.; import java.; import java.io.; import java.util.GregorianCalendar; public class BruteForce { private char passwdArray []={'a','b','c','d','e','f','g','h','i','j','k', 'l','m','n','o','p','q','r','s','t','u','v','w','x','y','z','A','B', 'C','D','E','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V', 'W','X','Y','Z','0','1','2','3','4','5','6','7','8','9','\'','.','&'}; private int num=1; public void crackAddress() throws Exception { String line,username="",passwd; int flag=0; Runtime run = Runtime.getRuntime(); GregorianCalendar =new GregorianCalendar(); for(int k=0;k<passwdArray.length;k++) { for(int j=0;j<passwdArray.length;j++) { for(int i=0;i<passwdArray.length;i++) { passwd =(new StringBuffer().append(passwdArray[i]).append(passwdArray[j]).append(passwdArray[k])).toString(); System....</code> | <code> import java.io.InputStream; import java.util.Properties; import javax.naming.Context; import javax.naming.InitialContext; import javax.rmi.PortableRemoteObject; import javax.sql.DataSource; public class DictionaryPropertyHelper { private static Properties dictProps; public DictionaryPropertyHelper() { } public static String getProperty(String pKey){ try{ initProps(); } catch(Exception e){ System.err.println("Error init'ing the dictionary Props"); e.printStackTrace(); } return dictProps.getProperty(pKey); } private static void initProps() throws Exception{ if(dictProps == null){ dictProps = new Properties(); InputStream fis = DictionaryPropertyHelper.class.getResourceAsStream("/dictionary.properties"); dictProps.load(fis); } } } </code> | | <code>0</code> | <code> import java.net.*; import java.io.*; import java.util.Date; public class BruteForce { private URL url; private HttpURLConnection connection; private static String[] lowerCase = {"a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z"}; private static String[] upperCase = {"A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S", "T", "U", "V", "W", "X", "Y", "Z"}; private String userPassword, base64userPassword; private static char wrongPass; public BruteForce() { wrongPass = 'Y'; } public char determinePass(String inputURL, String userName, String passWord){ try{ url = new URL(inputURL); connection = (HttpURLConnection)url.openConnection(); this.getEncoded(userName,...</code> | <code>import java.io.*; class WatchDog { public static void main(String args[]) { if (args.length<1) { System.out.println("Correct Format Filename email address <username@cs.rmit.edu.> of the recordkeeper"); System.exit(1); } while (true) { FileInputStream stream=null; DataInputStream word=null; String input=" "; try { String lsstr; Process lsproc = Runtime.getRuntime().exec("wget http://www.cs.rmit.edu./students"); try { Thread.sleep(2000); }catch (Exception e) { System.err.println("Caught ThreadException: " +e.getMessage()); } String[] cmd = {"//sh","-c", "diff Index2.html index.html >report.txt "}; lsproc = Runtime.getRuntime().exec(cmd); try { Thread.sleep(2000); }catch (Exception e) { System.err.println("Caught ThreadException: " +e.getMessage()); } if (lsproc.exitValue()==2) { System.out.println("The file was checked for f...</code> |
—Loss: <code>SoftmaxLoss</code>

Evaluation Dataset

socotrainjava

—Dataset: soco_train_java at 44ca4ff
—Size: 3,342 evaluation samples
—Columns: <code>label</code>, <code>text1</code>, and <code>text2</code>
—Approximate statistics based on the first 1000 samples: | | label | text1 | text2 | |:--------|:-----------------------------------------------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | type | int | string | string | | details | <ul><li>0: ~99.50%</li><li>1: ~0.50%</li></ul> | <ul><li>min: 51 tokens</li><li>mean: 451.02 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 54 tokens</li><li>mean: 464.38 tokens</li><li>max: 512 tokens</li></ul> |
—Samples: | label | text1 | text2 | |:---------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>0</code> | <code>import java.io.; import java.net.; public class BruteForce { private String urlString = "http://sec-crack.cs.rmit.edu./SEC/2/index.php"; private static String password; private static int length; private static int tcounter; private static int fcounter; private static int cases; private static int respCode; public BruteForce() { Authenticator.setDefault(new BruteForceAuthenticator()); tcounter = 0; fcounter = 0; cases = 0; } public static void main (String[] args) { BruteForce bf = new BruteForce(); String file = " "; while(respCode != 200) { file = bf.fetchURL(); } System.out.println("Number of attempts: " + t_counter); System.out.println("Password: " + password); System.out.println(file); } private String fetchURL() { ...</code> | <code> import java.io.IOException; import java.net.; import java.io.; import java.util.; public class BruteForce { static String strLetter[]; static URL url = null; static URLConnection urlConnection; static InputStream urlStream; static String strExceptionPassword[]; static int intExceptionCount = -1; static int intNumberOfConnectionAttempts = 0; static String username = ""; static String strLastPasswordTested; public static void main (String args[]) { Calendar calStart; Calendar calFinish; Date dateStart; Date dateFinish; lngStart; lngFinish; calStart = new GregorianCalendar(); dateStart = calStart.getTime(); lngStart = dateStart.getTime(); System.out.println(); System.out.println(); populateArray(); boolean boolPasswordFound...</code> | | <code>0</code> | <code> import java.util.; import java.net.; import java.io.; import misc.BASE64Encoder; import javax.swing.; public class ConnectionThread extends Thread { private String url; private URL currURL; private URLConnection conn; private HoldSharedData sharedData; private int noOfThread; private int batch; public ConnectionThread( String pageURL, int wThread, int newBatch, HoldSharedData data ) { super(); url = pageURL; noOfThread = wThread; batch = newBatch; sharedData = data; } public void run() { try { currURL = new URL( url ); for( int i = noOfThreadbatch; (i < (noOfThread + 1)batch) && (i < sharedData.getPwdCount()); i ++ ) { String pwd = sharedData.getPasswordAt( i ); conn = currURL.openConnection(); ...</code> | <code>import java.net.; import java.io.; public class EmailClient { private String sender, recipient, hostName; public EmailClient(String nSender, String nRecipient, String nHost) { sender = nSender; recipient = nRecipient; hostName = nHost; } public void sendMail(String subject, String message) { try { Socket s1=null; InputStream is = null; OutputStream os = null; DataOutputStream = null; s1 = new Socket(hostName,25); is = s1.getInputStream(); os = s1.getOutputStream(); bd = new DataOutputStream(os); BufferedReader response = new BufferedReader(new InputStreamReader(is)); bd.writeBytes("HELO "+ InetAddress.getLocalHost().getHostName() + "\r\n"); waitForSuccessResponse(response); bd.writeBytes("MAIL FROM:"+sender+"\n"); waitForSuccessResponse(response); bd.writeBytes("RCPT :"+recipient+"\n"); waitForSuccessResponse(response); bd.writeBytes("data"+"\n"); bd.writeBytes("Subject:"+subject+"\n"); bd.writeBy...</code> | | <code>0</code> | <code>import java.io.; import java.util.; import java.text.; import java.net.*; public class LoginAttempt { private String urlName = "http://sec-crack.cs.rmit.edu./SEC/2/index.php"; private String userName = ""; private String connectString = ""; public LoginAttempt() { } public LoginAttemptResults tryPasswords(String [] casedPasswords, int passwordsTried) { boolean foundPassword = false; LoginAttemptResults results = new LoginAttemptResults(); for( i = 0; i < casedPasswords.length; i++) { passwordsTried++; try { URL targetURL; HttpURLConnection connection; targetURL = new URL(urlName); connection = (HttpURLConnection) targetURL.openConnection(); connectString = userName + ":" + casedPasswords[i].trim(); connectString = new targetURL.misc.BASE64Encoder().encode(connectString.getByt...</code> | <code> package java.httputils; import java.io.BufferedInputStream; import java.io.IOException; import java.net.HttpURLConnection; import java.net.MalformedURLException; import java.net.URL; import java.net.URLConnection; import java.sql.Timestamp; public class HttpRequestClient { protected URL serverURL; protected java.net.HttpURLConnection httpConnection; protected Timestamp start; protected Timestamp end; protected StringBuffer content = new StringBuffer(); protected int millis; protected int code; public HttpRequestClient(String url) throws MalformedURLException, IOException { setServerURL(new URL(url)); setStart(new Timestamp(System.currentTimeMillis())); setHttpConnection( (HttpURLConnection)this.getServerURL().openConnection()); doRequest(); } public HttpRequestClient() { super(); } public int doRequest() throws IOException {...</code> |
—Loss: <code>SoftmaxLoss</code>

Training Hyperparameters

Non-Default Hyperparameters

—eval_strategy: steps
—per_device_train_batch_size: 16
—per_device_eval_batch_size: 16
—num_train_epochs: 1
—warmup_ratio: 0.1
—fp16: True

All Hyperparameters

<details><summary>Click to expand</summary>

—overwrite_output_dir: False
—do_predict: False
—eval_strategy: steps
—prediction_loss_only: True
—per_device_train_batch_size: 16
—per_device_eval_batch_size: 16
—per_gpu_train_batch_size: None
—per_gpu_eval_batch_size: None
—gradient_accumulation_steps: 1
—eval_accumulation_steps: None
—torch_empty_cache_steps: None
—learning_rate: 5e-05
—weight_decay: 0.0
—adam_beta1: 0.9
—adam_beta2: 0.999
—adam_epsilon: 1e-08
—max_grad_norm: 1.0
—num_train_epochs: 1
—max_steps: -1
—lr_scheduler_type: linear
—lr_scheduler_kwargs: {}
—warmup_ratio: 0.1
—warmup_steps: 0
—log_level: passive
—log_level_replica: warning
—log_on_each_node: True
—logging_nan_inf_filter: True
—save_safetensors: True
—save_on_each_node: False
—save_only_model: False
—restore_callback_states_from_checkpoint: False
—no_cuda: False
—use_cpu: False
—use_mps_device: False
—seed: 42
—data_seed: None
—jit_mode_eval: False
—use_ipex: False
—bf16: False
—fp16: True
—fp16_opt_level: O1
—half_precision_backend: auto
—bf16_full_eval: False
—fp16_full_eval: False
—tf32: None
—local_rank: 0
—ddp_backend: None
—tpu_num_cores: None
—tpu_metrics_debug: False
—debug: []
—dataloader_drop_last: False
—dataloader_num_workers: 0
—dataloader_prefetch_factor: None
—past_index: -1
—disable_tqdm: False
—remove_unused_columns: True
—label_names: None
—load_best_model_at_end: False
—ignore_data_skip: False
—fsdp: []
—fsdp_min_num_params: 0
—fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
—fsdp_transformer_layer_cls_to_wrap: None
—accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
—deepspeed: None
—label_smoothing_factor: 0.0
—optim: adamw_torch
—optim_args: None
—adafactor: False
—group_by_length: False
—length_column_name: length
—ddp_find_unused_parameters: None
—ddp_bucket_cap_mb: None
—ddp_broadcast_buffers: False
—dataloader_pin_memory: True
—dataloader_persistent_workers: False
—skip_memory_metrics: True
—use_legacy_prediction_loop: False
—push_to_hub: False
—resume_from_checkpoint: None
—hub_model_id: None
—hub_strategy: every_save
—hub_private_repo: None
—hub_always_push: False
—gradient_checkpointing: False
—gradient_checkpointing_kwargs: None
—include_inputs_for_metrics: False
—include_for_metrics: []
—eval_do_concat_batches: True
—fp16_backend: auto
—push_to_hub_model_id: None
—push_to_hub_organization: None
—mp_parameters:
—auto_find_batch_size: False
—full_determinism: False
—torchdynamo: None
—ray_scope: last
—ddp_timeout: 1800
—torch_compile: False
—torch_compile_backend: None
—torch_compile_mode: None
—include_tokens_per_second: False
—include_num_input_tokens_seen: False
—neftune_noise_alpha: None
—optim_target_modules: None
—batch_eval_metrics: False
—eval_on_start: False
—use_liger_kernel: False
—eval_use_gather_object: False
—average_tokens_across_devices: False
—prompts: None
—batch_sampler: batch_sampler
—multi_dataset_batch_sampler: proportional

</details>

Training Logs

Epoch	Step	Training Loss	Validation Loss
0.0532	100	0.0842	0.0238
0.1064	200	0.0111	0.0203
0.1596	300	0.0061	0.0253
0.2128	400	0.0243	0.0203
0.2660	500	0.0311	0.0209
0.3191	600	0.0212	0.0203
0.3723	700	0.0137	0.0237
0.4255	800	0.03	0.0199
0.4787	900	0.0104	0.0195
0.5319	1000	0.0141	0.0201
0.5851	1100	0.0214	0.0186
0.6383	1200	0.0151	0.0192
0.6915	1300	0.0219	0.0185
0.7447	1400	0.0152	0.0184
0.7979	1500	0.0082	0.0192
0.8511	1600	0.0128	0.0182
0.9043	1700	0.0155	0.0172
0.9574	1800	0.0102	0.0174

Framework Versions

—Python: 3.11.13
—Sentence Transformers: 4.1.0
—Transformers: 4.52.4
—PyTorch: 2.6.0+cu124
—Accelerate: 1.7.0
—Datasets: 3.6.0
—Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers and SoftmaxLoss

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->