Modelpublic

buelfhood/SOCO-Java-CodeBERTa-ST-1

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes92downloads

Model Card

SentenceTransformer based on huggingface/CodeBERTa-small-v1

This is a sentence-transformers model finetuned from huggingface/CodeBERTa-small-v1. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

—Model Type: Sentence Transformer
—Base model: huggingface/CodeBERTa-small-v1
—Maximum Sequence Length: 512 tokens
—Output Dimensionality: 768 dimensions
—Similarity Function: Cosine Similarity

Model Sources

—Documentation: Sentence Transformers Documentation
—Repository: Sentence Transformers on GitHub
—Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("buelfhood/SOCO-Java-CodeBERTa-ST-1")
# Run inference
sentences = [
    '\npackage java.httputils;\n\nimport java.io.IOException;\nimport java.net.HttpURLConnection;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport java.sql.Timestamp;\n\n\npublic class BasicAuthHttpRequest extends HttpRequestClient\n{\n    String userName;\n    String password;\n    \n    protected BasicAuthHttpRequest(String url, String userName, String password)\n        throws MalformedURLException, IOException\n    {\n        setPassword(password);\n        setUserName(userName);\n        setServerURL(new URL(url));\n        \n        setStart(new Timestamp(System.currentTimeMillis()));\n\n        String userPassword = userName + ":" + password;\n\n        \n        String encoding = new url.misc.BASE64Encoder().encode (userPassword.getBytes());\n\n       \n\n       setHttpConnection(\n        (HttpURLConnection)this.getServerURL().openConnection());\n\n        \n        getHttpConnection().setRequestProperty ("Authorization", " " + encoding);\n        doRequest();\n    }\n\n    \n    protected BasicAuthHttpRequest(String url)\n        throws MalformedURLException, IOException\n    {\n        super(url);\n    }\n\n    \n    public BasicAuthHttpRequest()\n    {\n        super();\n    }\n\n\n    \n    public String getPassword()\n    {\n        return password;\n    }\n\n    \n    public String getUserName()\n    {\n        return userName;\n    }\n\n    \n    public void setPassword(String string)\n    {\n        password = string;\n    }\n\n    \n    public void setUserName(String string)\n    {\n        userName = string;\n    }\n\n    public static void main (String[] args)\n    {\n        BasicAuthHttpRequest client = null;\n        try\n        {\n            client = new BasicAuthHttpRequest(args[0], args[1], args[2]);\n        }\n        catch (MalformedURLException e)\n        {\n            e.printStackTrace();\n        }\n        catch (IOException e)\n        {\n            e.printStackTrace();\n        }\n        finally\n        {\n            if (client != null && client.getCode() != HttpURLConnection.HTTP_UNAUTHORIZED)\n            {\n                System.out.println(\n                    "Request response : \\n" + client.getCode());\n\n\n                System.out.println(\n                    "Request processing time (milliseconds): " +\n                    (client.getEnd().getTime() - client.getStart().getTime()));\n\n                System.out.println(\n                    "Request content: \\n" + client.getContent());\n            }\n            else\n            {\n                System.out.println(\n                    "Request response : \\n" + client.getCode());\n\n\n            }\n        }\n    }\n}\n',
    'import java.io.*;\nimport java.net.*;\nimport java.security.*;\nimport java.math.*;\nimport java.*;\nimport java.util.*;\n\n\npublic class WatchDog\n{\n    public static FileWriter out = null, output = null;\n\n    public static void main (String args[]) throws Exception {\n\tSocket socket = null;\n\tDataOutputStream  = null;\n\tBufferedReader bf = null, fr = null;\n\tString retVal = null, StatusCode = "HTTP/1.1 200 OK";\n    int dirty = 0, count = 0;\n\n         stime = System.currentTimeMillis();\n        System.out.println("Detecting the changes...");\n\n        try {\n\n\t        \n            URL yahoo = new URL("http://www.cs.rmit.edu./students/");\n            URLConnection yc = yahoo.openConnection();\n\n            \n            BufferedReader in = new BufferedReader(\n                                    new InputStreamReader(\n                                    yc.getInputStream()));\n\n            String inputLine;\n            try {\n                out = new FileWriter("newstudent");\n                while ((inputLine = in.readLine()) != null){\n                        out.write(inputLine + "\\n");\n                }\n            } catch (IOException ex) {\n                ex.printStackTrace();\n            }\n            in.print();\n            out.print();\n\n            dirty = diff();\n            if (dirty == 1){\n               sendMail();\n               System.out.println("Changes detected and email sent!");\n            }\n\n            if (diffimages() == 1){\n               sendMail();\n               System.out.println("Images modification detected and email sent!");\n            }\n\n            updatePage();\n            System.out.println("** End of WatchDog checking **");\n\n            } catch (Exception ex) {\n              ex.printStackTrace();\n            }\n    }\n\n    public static int diff()\n    {\n       int update = 0;\n\n       try{\n           Process process = Runtime.getRuntime().exec("diff -b RMITCSStudent newstudent");\n           BufferedReader pr = new BufferedReader(\n                                   new InputStreamReader(\n                                   process.getInputStream()));\n\n           output = new FileWriter("output");\n           String inputLine;\n           while ((inputLine = pr.readLine()) != null){\n                 output.write(inputLine + "\\n");\n                 update = 1;\n           }\n           output.promt();\n\n       }catch (Exception ex){\n              ex.printStackTrace();\n       }\n       return update;\n    }\n\n    public static int diffimages()\n    {\n       int update = 0;\n       String image;\n\n       try{\n           Process primages = Runtime.getRuntime().exec("./images.sh");\n           wait(1);\n           File imageFile = new File("imagesname");\n           BufferedReader fr = new BufferedReader(new FileReader(imageFile));\n\n           output = new FileWriter("output");\n           while ((image = fr.readLine()) != null) {\n                 primages = Runtime.getRuntime().exec("diff " + image + " o"+image);\n                 BufferedReader pr = new BufferedReader(\n                                       new InputStreamReader(\n                                       primages.getInputStream()));\n\n                 String inputLine;\n                 while ((inputLine = pr.readLine()) != null){\n                       output.write(inputLine + "\\n");\n                       update = 1;\n                 }\n           }\n           output.print();\n           fr.close();\n\n       }catch (Exception ex){\n              ex.printStackTrace();\n       }\n       return update;\n    }\n\n    public static void sendMail()\n    {\n       try{\n           Process mailprocess = Runtime.getRuntime().exec("./email.sh");\n       }catch (Exception ex){\n           ex.printStackTrace();\n       }\n    }\n\n    public static void updatePage()\n    {\n       String image;\n\n       try{\n           Process updateprocess = Runtime.getRuntime().exec("cp newstudent RMITCSStudent");\n           Process deleteprocess = Runtime.getRuntime().exec("rm newstudent");\n\n           File inputFile = new File("imagesname");\n           BufferedReader fr = new BufferedReader(new FileReader(inputFile));\n           while ((image = fr.readLine()) != null) {\n                 updateprocess = Runtime.getRuntime().exec("cp " + image + " o" + image);\n                 deleteprocess = Runtime.getRuntime().exec("rm " + image);\n           }\n           fr.close();\n       }catch (Exception ex){\n           ex.printStackTrace();\n       }\n    }\n\n    public static void wait(int time){\n\t   int timer, times;\n\t   timer = System.currentTimeMillis();\n\t   times = (time * 1000) + timer;\n\n\t   while(timer < times)\n\t\t\ttimer = System.currentTimeMillis();\n\t}\n}',
    'import java.net.*;\nimport java.io.*;\n\n\npublic class EmailClient\n{\n\tprivate String sender, recipient, hostName;\n\n\tpublic EmailClient(String nSender, String nRecipient, String nHost)\n\t{\n\t\tsender = nSender;\n\t\trecipient = nRecipient;\n\t\thostName = nHost;\n\t}\n\n\tpublic void sendMail(String subject, String message)\n\t{\n\t\ttry\n\t\t{\n\t\t\tSocket s1=null;\n\t\t\tInputStream\tis = null;\n\t\t\tOutputStream os = null;\n\n\t\t\tDataOutputStream  = null;\n\n\t\t\ts1 = new Socket(hostName,25);\n\t\t\tis = s1.getInputStream();\n\t\t\tos = s1.getOutputStream();\n\n\t\t\tbd = new DataOutputStream(os);\n\n\t\t\tBufferedReader response = new BufferedReader(new InputStreamReader(is));\n\n\t\t\tbd.writeBytes("HELO "+ InetAddress.getLocalHost().getHostName() + "\\r\\n");\n\n\t\t\twaitForSuccessResponse(response);\n\n\t\t\tbd.writeBytes("MAIL FROM:"+sender+"\\n");\n\n\t\t\twaitForSuccessResponse(response);\n\n\t\t\tbd.writeBytes("RCPT :"+recipient+"\\n");\n\n\t\t\twaitForSuccessResponse(response);\n\n\t\t\tbd.writeBytes("data"+"\\n");\n\n\t\t\tbd.writeBytes("Subject:"+subject+"\\n");\n\n\t\t\tbd.writeBytes(message+"\\n.\\n");\n\n\t\t\twaitForSuccessResponse(response);\n\t\t}\n\n\t\tcatch (UnknownHostException badUrl)\n\t\t{\n\t\t\tSystem.out.println("Host unknown.");\n\t\t}\n\n\t\tcatch (EOFException eof)\n\t\t{\n\t\t\tSystem.out.println("<EOF>");\n\t\t}\n\t\tcatch (Exception e)\n\t\t{\n\t\t\tSystem.out.println("got exception: "+e);\n\t\t}\n\t}\n\n\tprivate static void\twaitForSuccessResponse(BufferedReader response) throws IOException\n\t{\n\t\tString rsp;\n\t\tboolean r250 = false;\n\n\t\twhile( ! r250 )\n\t\t{\n\t\t\trsp = response.readLine().trim();\n\n\t\t\tif(rsp.startsWith("250"))\n\t\t\t\tr250 = true;\n\t\t}\n\n\t}\n}',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

—Size: 33,411 training samples
—Columns: <code>sentence0</code>, <code>sentence1</code>, and <code>label</code>
—Approximate statistics based on the first 1000 samples: | | sentence0 | sentence1 | label | |:--------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:-----------------------------------------------| | type | string | string | int | | details | <ul><li>min: 51 tokens</li><li>mean: 444.12 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 54 tokens</li><li>mean: 462.06 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>0: ~99.80%</li><li>1: ~0.20%</li></ul> |
—Samples: | sentence0 | sentence1 | label | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------| | <code> import java.net.; import java.io.; import java.Runtime; public class WatchDog{ public WatchDog(){} public void copyTo(){ } public static void main(String[] args) throws Exception { WatchDog wd= new WatchDog(); SendEMail t = new SendEMail(); PrintWriter pw=null; URL url = new URL("http://www.cs.rmit.edu./students"); URLConnection yc = url.openConnection(); System.out.println("Connection opened..."); BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream())); String inputLine; try{ pw=new PrintWriter(new FileOutputStream("newHtml")); while ((inputLine = in.readLine()) != null){ pw.println(inputLine); } pw.save(); }catch(IOException e){ System.out.println("Error saving the file"); } Process p = Runtime.getRuntime().exec("diff -b newHtml oldHtml"); ...</code> | <code> import java.io.; import java.net.; import java.; import java.util.; public class DictionaryAttack { public static void main ( String args[]) { String function,pass,temp1; int count =0; try{ FileReader fr = new FileReader("words.txt"); BufferedReader bfread = new BufferedReader(fr); Runtime rtime = Runtime.getRuntime(); Process prs = null; while(( bf = bfread.readLine()) != null) { if( f.length() < 4 ) { System.out.println(+ " The Attack Number =====>" + count++ ); pass = f; function ="wget --http-user= --http-passwd="+pass+" http://sec-crack.cs.rmit.edu./SEC/2/"; prs = rtime.exec(function); InputStreamReader stre = new InputStreamReader(prs.getErrorStream()); BufferedReader bread = new BufferedReader(stre); while( (temp1 = bread.readLine())!= null) { System.out.println(temp1); if(temp1.equals("HTTP request sent, awaiting resp...</code> | <code>0</code> | | <code> import java.net.; import java.io.; import java.util.; public class WatchDog { public WatchDog() { } public static void main(String[] args) { try { if( args.length != 2 ) { System.out.println("USAGE: java WatchDog <URL> <mailing UserName>"); System.exit(0); } Runtime.getRuntime().exec("rm LastWatch.html"); Runtime.getRuntime().exec("rm WatchDog.ini"); Thread.sleep(1000); while (true) { WatchDog myWatchDog = new WatchDog(); myWatchDog.readHTML(args[0], args[1]); Runtime.getRuntime().exec("rm Report.txt"); Runtime.getRuntime().exec("rm diffReport.txt"); Runtime.getRuntime().exec("rm NewWatch.txt"); System.out.println(" check after 2 ... press Ctrl-Z suspend WatchDog..."); Thread.sleep(2601000); } ...</code> | <code> import java.net.; import java.io.; class MyAuthenticator extends Authenticator { String password; public MyAuthenticator(String pwdin) { password = pwdin; } protected PasswordAuthentication getPasswordAuthentication(){ String pwd = password; return new PasswordAuthentication("",pwd.toCharArray()); } } </code> | <code>0</code> | | <code> import java.Runtime; import java.io.; public class differenceFile { StringWriter sw =null; PrintWriter pw = null; public differenceFile() { sw = new StringWriter(); pw = new PrintWriter(); } public String compareFile() { try { Process = Runtime.getRuntime().exec("diff History.txt Comparison.txt"); InputStream write = sw.getInputStream(); BufferedReader bf = new BufferedReader (new InputStreamReader(write)); String line; while((line = bf.readLine())!=null) pw.println(line); if((sw.toString().trim()).equals("")) { System.out.println(" difference"); return null; } System.out.println(sw.toString().trim()); }catch(Exception e){} return sw.toString().trim(); } }</code> | <code> public class HoldSharedData { private int numOfConnections = 0; private int startTime; private int totalTime = 0; private String[] password; private int pwdCount; public HoldSharedData( int time, String[] pwd, int count ) { startTime = time; password = pwd; pwdCount = count; } public int getPwdCount() { return pwdCount; } public void setNumOfConnections( ) { numOfConnections ++; } public int getNumOfConnections() { return numOfConnections; } public int getStartTime() { return startTime; } public void setTotalTime( int newTotalTime ) { totalTime = newTotalTime; } public int getTotalTime() { return totalTime; } public String getPasswordAt( int index ) { return password[index]; } } </code> | <code>0</code> |
—Loss: <code>BatchAllTripletLoss</code>

Training Hyperparameters

Non-Default Hyperparameters

—per_device_train_batch_size: 16
—per_device_eval_batch_size: 16
—num_train_epochs: 1
—fp16: True
—multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

—overwrite_output_dir: False
—do_predict: False
—eval_strategy: no
—prediction_loss_only: True
—per_device_train_batch_size: 16
—per_device_eval_batch_size: 16
—per_gpu_train_batch_size: None
—per_gpu_eval_batch_size: None
—gradient_accumulation_steps: 1
—eval_accumulation_steps: None
—torch_empty_cache_steps: None
—learning_rate: 5e-05
—weight_decay: 0.0
—adam_beta1: 0.9
—adam_beta2: 0.999
—adam_epsilon: 1e-08
—max_grad_norm: 1
—num_train_epochs: 1
—max_steps: -1
—lr_scheduler_type: linear
—lr_scheduler_kwargs: {}
—warmup_ratio: 0.0
—warmup_steps: 0
—log_level: passive
—log_level_replica: warning
—log_on_each_node: True
—logging_nan_inf_filter: True
—save_safetensors: True
—save_on_each_node: False
—save_only_model: False
—restore_callback_states_from_checkpoint: False
—no_cuda: False
—use_cpu: False
—use_mps_device: False
—seed: 42
—data_seed: None
—jit_mode_eval: False
—use_ipex: False
—bf16: False
—fp16: True
—fp16_opt_level: O1
—half_precision_backend: auto
—bf16_full_eval: False
—fp16_full_eval: False
—tf32: None
—local_rank: 0
—ddp_backend: None
—tpu_num_cores: None
—tpu_metrics_debug: False
—debug: []
—dataloader_drop_last: False
—dataloader_num_workers: 0
—dataloader_prefetch_factor: None
—past_index: -1
—disable_tqdm: False
—remove_unused_columns: True
—label_names: None
—load_best_model_at_end: False
—ignore_data_skip: False
—fsdp: []
—fsdp_min_num_params: 0
—fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
—fsdp_transformer_layer_cls_to_wrap: None
—accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
—deepspeed: None
—label_smoothing_factor: 0.0
—optim: adamw_torch
—optim_args: None
—adafactor: False
—group_by_length: False
—length_column_name: length
—ddp_find_unused_parameters: None
—ddp_bucket_cap_mb: None
—ddp_broadcast_buffers: False
—dataloader_pin_memory: True
—dataloader_persistent_workers: False
—skip_memory_metrics: True
—use_legacy_prediction_loop: False
—push_to_hub: False
—resume_from_checkpoint: None
—hub_model_id: None
—hub_strategy: every_save
—hub_private_repo: None
—hub_always_push: False
—gradient_checkpointing: False
—gradient_checkpointing_kwargs: None
—include_inputs_for_metrics: False
—include_for_metrics: []
—eval_do_concat_batches: True
—fp16_backend: auto
—push_to_hub_model_id: None
—push_to_hub_organization: None
—mp_parameters:
—auto_find_batch_size: False
—full_determinism: False
—torchdynamo: None
—ray_scope: last
—ddp_timeout: 1800
—torch_compile: False
—torch_compile_backend: None
—torch_compile_mode: None
—include_tokens_per_second: False
—include_num_input_tokens_seen: False
—neftune_noise_alpha: None
—optim_target_modules: None
—batch_eval_metrics: False
—eval_on_start: False
—use_liger_kernel: False
—eval_use_gather_object: False
—average_tokens_across_devices: False
—prompts: None
—batch_sampler: batch_sampler
—multi_dataset_batch_sampler: round_robin

</details>

Training Logs

Epoch	Step	Training Loss
0.2393	500	0.2031
0.4787	1000	0.1761
0.7180	1500	0.1914
0.9574	2000	0.2044

Framework Versions

—Python: 3.11.13
—Sentence Transformers: 4.1.0
—Transformers: 4.52.4
—PyTorch: 2.6.0+cu124
—Accelerate: 1.7.0
—Datasets: 3.6.0
—Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

BatchAllTripletLoss

bibtex

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->