google/code_x_glue_cc_code_completion_token

1---2annotations_creators:3- found4language_creators:5- found6language:7- code8license:9- c-uda10multilinguality:11- monolingual12size_categories:13- 10K<n<100K14source_datasets:15- original16task_categories:17- text-generation18- fill-mask19task_ids:20- language-modeling21- masked-language-modeling22pretty_name: CodeXGlueCcCodeCompletionToken23dataset_info:24- config_name: java25  features:26  - name: id27    dtype: int3228  - name: code29    sequence: string30  splits:31  - name: train32    num_bytes: 12831204533    num_examples: 1293434  - name: validation35    num_bytes: 3025916636    num_examples: 718937  - name: test38    num_bytes: 4302794839    num_examples: 826840  download_size: 3132033941  dataset_size: 20159915942- config_name: python43  features:44  - name: id45    dtype: int3246  - name: path47    dtype: string48  - name: code49    sequence: string50  splits:51  - name: train52    num_bytes: 68431945553    num_examples: 10000054  - name: test55    num_bytes: 33397802856    num_examples: 5000057  download_size: 21014352558  dataset_size: 101829748359configs:60- config_name: java61  data_files:62  - split: train63    path: java/train-*64  - split: validation65    path: java/validation-*66  - split: test67    path: java/test-*68- config_name: python69  data_files:70  - split: train71    path: python/train-*72  - split: test73    path: python/test-*74---75# Dataset Card for "code_x_glue_cc_code_completion_token"76 77## Table of Contents78- [Dataset Description](#dataset-description)79  - [Dataset Summary](#dataset-summary)80  - [Supported Tasks and Leaderboards](#supported-tasks)81  - [Languages](#languages)82- [Dataset Structure](#dataset-structure)83  - [Data Instances](#data-instances)84  - [Data Fields](#data-fields)85  - [Data Splits](#data-splits-sample-size)86- [Dataset Creation](#dataset-creation)87  - [Curation Rationale](#curation-rationale)88  - [Source Data](#source-data)89  - [Annotations](#annotations)90  - [Personal and Sensitive Information](#personal-and-sensitive-information)91- [Considerations for Using the Data](#considerations-for-using-the-data)92  - [Social Impact of Dataset](#social-impact-of-dataset)93  - [Discussion of Biases](#discussion-of-biases)94  - [Other Known Limitations](#other-known-limitations)95- [Additional Information](#additional-information)96  - [Dataset Curators](#dataset-curators)97  - [Licensing Information](#licensing-information)98  - [Citation Information](#citation-information)99  - [Contributions](#contributions)100 101## Dataset Description102 103- **Homepage:** https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/CodeCompletion-token104 105### Dataset Summary106 107CodeXGLUE CodeCompletion-token dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/CodeCompletion-token108 109Predict next code token given context of previous tokens. Models are evaluated by token level accuracy.110    Code completion is a one of the most widely used features in software development through IDEs. An effective code completion tool could improve software developers' productivity. We provide code completion evaluation tasks in two granularities -- token level and line level. Here we introduce token level code completion. Token level task is analogous to language modeling. Models should have be able to predict the next token in arbitary types.111 112### Supported Tasks and Leaderboards113 114- `language-modeling`: The dataset can be used to train a model for completing single code tokens.115 116### Languages117 118- Java **programming** language119- Python **programming** language120 121## Dataset Structure122 123### Data Instances124 125#### java126 127An example of 'test' looks as follows.128```129{130    "code": ["<s>", "package", "org", ".", "vaadin", ".", "teemu", ".", "clara", ".", "demo", ";", "import", "java", ".", "io", ".", "BufferedReader", ";", "import", "java", ".", "io", ".", "ByteArrayInputStream", ";", "import", "java", ".", "io", ".", "IOException", ";", "import", "java", ".", "io", ".", "InputStreamReader", ";", "import", "org", ".", "vaadin", ".", "teemu", ".", "clara", ".", "Clara", ";", "import", "org", ".", "vaadin", ".", "teemu", ".", "clara", ".", "inflater", ".", "LayoutInflaterException", ";", "import", "com", ".", "vaadin", ".", "Application", ";", "import", "com", ".", "vaadin", ".", "terminal", ".", "ThemeResource", ";", "import", "com", ".", "vaadin", ".", "ui", ".", "Button", ";", "import", "com", ".", "vaadin", ".", "ui", ".", "Button", ".", "ClickEvent", ";", "import", "com", ".", "vaadin", ".", "ui", ".", "Component", ";", "import", "com", ".", "vaadin", ".", "ui", ".", "Embedded", ";", "import", "com", ".", "vaadin", ".", "ui", ".", "HorizontalLayout", ";", "import", "com", ".", "vaadin", ".", "ui", ".", "HorizontalSplitPanel", ";", "import", "com", ".", "vaadin", ".", "ui", ".", "TextArea", ";", "import", "com", ".", "vaadin", ".", "ui", ".", "VerticalLayout", ";", "import", "com", ".", "vaadin", ".", "ui", ".", "Window", ";", "import", "com", ".", "vaadin", ".", "ui", ".", "Window", ".", "Notification", ";", "@", "SuppressWarnings", "(", "\"serial\"", ")", "public", "class", "DemoApplication", "extends", "Application", "{", "private", "DemoController", "controller", ";", "private", "TextArea", "xmlArea", ";", "private", "HorizontalSplitPanel", "split", "=", "new", "HorizontalSplitPanel", "(", ")", ";", "private", "Window", "mainWindow", ";", "@", "Override", "public", "void", "init", "(", ")", "{", "setTheme", "(", "\"clara\"", ")", ";", "setMainWindow", "(", "mainWindow", "=", "new", "Window", "(", ")", ")", ";", "controller", "=", "new", "DemoController", "(", "mainWindow", ")", ";", "mainWindow", ".", "setContent", "(", "split", ")", ";", "VerticalLayout", "editor", "=", "new", "VerticalLayout", "(", ")", ";", "editor", ".", "setSpacing", "(", "true", ")", ";", "editor", ".", "setMargin", "(", "false", ",", "false", ",", "false", ",", "true", ")", ";", "editor", ".", "setHeight", "(", "\"100%\"", ")", ";", "editor", ".", "addComponent", "(", "xmlArea", "=", "createXmlArea", "(", ")", ")", ";", "editor", ".", "setExpandRatio", "(", "xmlArea", ",", "1.0f", ")", ";", "editor", ".", "addComponent", "(", "createUpdateButton", "(", ")", ")", ";", "HorizontalLayout", "wrapper", "=", "new", "HorizontalLayout", "(", ")", ";", "wrapper", ".", "setMargin", "(", "true", ")", ";", "wrapper", ".", "setSizeFull", "(", ")", ";", "wrapper", ".", "addComponent", "(", "createLogo", "(", ")", ")", ";", "wrapper", ".", "addComponent", "(", "editor", ")", ";", "wrapper", ".", "setExpandRatio", "(", "editor", ",", "1.0f", ")", ";", "split", ".", "setFirstComponent", "(", "wrapper", ")", ";", "updateLayout", "(", ")", ";", "}", "private", "Component", "createLogo", "(", ")", "{", "Embedded", "logo", "=", "new", "Embedded", "(", "null", ",", "new", "ThemeResource", "(", "\"\"", ")", ")", ";", "logo", ".", "setHeight", "(", "\"90px\"", ")", ";", "logo", ".", "setWidth", "(", "\"90px\"", ")", ";", "return", "logo", ";", "}", "private", "TextArea", "createXmlArea", "(", ")", "{", "TextArea", "area", "=", "new", "TextArea", "(", ")", ";", "area", ".", "setStyleName", "(", "\"xml-area\"", ")", ";", "area", ".", "setSizeFull", "(", ")", ";", "area", ".", "setValue", "(", "readStartingPoint", "(", ")", ")", ";", "return", "area", ";", "}", "private", "Button", "createUpdateButton", "(", ")", "{", "return", "new", "Button", "(", "\"Update\"", ",", "new", "Button", ".", "ClickListener", "(", ")", "{", "public", "void", "buttonClick", "(", "ClickEvent", "event", ")", "{", "updateLayout", "(", ")", ";", "}", "}", ")", ";", "}", "private", "String", "readStartingPoint", "(", ")", "{", "BufferedReader", "reader", "=", "null", ";", "try", "{", "reader", "=", "new", "BufferedReader", "(", "new", "InputStreamReader", "(", "getClass", "(", ")", ".", "getClassLoader", "(", ")", ".", "getResourceAsStream", "(", "\"\"", ")", ")", ")", ";", "StringBuilder", "xml", "=", "new", "StringBuilder", "(", ")", ";", "String", "line", ";", "while", "(", "(", "line", "=", "reader", ".", "readLine", "(", ")", ")", "!=", "null", ")", "{", "xml", ".", "append", "(", "line", ")", ";", "xml", ".", "append", "(", "\"n\"", ")", ";", "}", "return", "xml", ".", "toString", "(", ")", ";", "}", "catch", "(", "IOException", "e", ")", "{", "e", ".", "printStackTrace", "(", ")", ";", "}", "finally", "{", "if", "(", "reader", "!=", "null", ")", "{", "try", "{", "reader", ".", "close", "(", ")", ";", "}", "catch", "(", "IOException", "e", ")", "{", "e", ".", "printStackTrace", "(", ")", ";", "}", "}", "}", "return", "null", ";", "}", "private", "void", "updateLayout", "(", ")", "{", "try", "{", "Component", "c", "=", "Clara", ".", "create", "(", "new", "ByteArrayInputStream", "(", "xmlArea", ".", "getValue", "(", ")", ".", "toString", "(", ")", ".", "getBytes", "(", ")", ")", ",", "controller", ")", ";", "split", ".", "replaceComponent", "(", "split", ".", "getSecondComponent", "(", ")", ",", "c", ")", ";", "}", "catch", "(", "LayoutInflaterException", "e", ")", "{", "mainWindow", ".", "showNotification", "(", "e", ".", "getMessage", "(", ")", ",", "Notification", ".", "TYPE_ERROR_MESSAGE", ")", ";", "}", "}", "}", "</s>"], 131    "id": 0132}133```134 135#### python136 137An example of 'train' looks as follows.138```139{140    "code": ["<s>", "from", "bootstrap", "import", "Bootstrap", "<EOL>", "from", "fund", "import", "InstantPaymentNotificationHandler", "<EOL>", "from", "fund", "import", "ThankYouHandler", "<EOL>", "from", "view", "import", "*", "<EOL>", "mapping", "=", "[", "(", "<EOL>", "r'/'", ",", "<EOL>", "Index", "<EOL>", ")", ",", "(", "<EOL>", "r'/ipn'", ",", "<EOL>", "InstantPaymentNotificationHandler", "<EOL>", ")", ",", "(", "<EOL>", "r'/thank-you'", ",", "<EOL>", "ThankYouHandler", "<EOL>", ")", ",", "(", "<EOL>", "r'/about\\/?'", ",", "<EOL>", "About", "<EOL>", ")", ",", "(", "<EOL>", "r'/guide\\/?'", ",", "<EOL>", "Guide", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Download", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Standards", "<EOL>", ")", ",", "(", "<EOL>", "r'/community\\/?'", ",", "<EOL>", "Community", "<EOL>", ")", ",", "(", "<EOL>", "r'/news\\/?'", ",", "<EOL>", "News", "<EOL>", ")", ",", "(", "<EOL>", "r'/support\\/?'", ",", "<EOL>", "Support", "<EOL>", ")", ",", "(", "<EOL>", "r'/contact\\/?'", ",", "<EOL>", "Contact", "<EOL>", ")", ",", "(", "<EOL>", "r'/press\\/?'", ",", "<EOL>", "Press", "<EOL>", ")", ",", "(", "<EOL>", "r'/legal/terms'", ",", "<EOL>", "Terms", "<EOL>", ")", ",", "(", "<EOL>", "r'/library\\/?'", ",", "<EOL>", "Library", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Library", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Library", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Users", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "User", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Design", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Design", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Design", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Design", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Design", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "RedirectSuccess", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "RedirectError", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "RedirectAfterDelete", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Moderate", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Bootstrap", "<EOL>", ")", ",", "(", "<EOL>", "r'/activity'", ",", "<EOL>", "ActivityScreen", "<EOL>", ")", ",", "(", "<EOL>", "r'/txns'", ",", "<EOL>", "TxnList", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Base64Blob", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "Base64Blob", "<EOL>", ")", ",", "(", "<EOL>", "r''", ",", "<EOL>", "MessageStrings", "<EOL>", ")", ",", "(", "<EOL>", "r'/.*'", ",", "<EOL>", "NotFound", "<EOL>", ")", "<EOL>", "]", "</s>"], 141    "id": 0, 142    "path": "00/wikihouse/urls.py\n"143}144```145 146### Data Fields147 148In the following each data field in go is explained for each config. The data fields are the same among all splits.149 150#### java151 152|field name|      type      |    description     |153|----------|----------------|--------------------|154|id        |int32           | Index of the sample|155|code      |Sequence[string]| Code Tokens        |156 157#### python158 159|field name|      type      |         description         |160|----------|----------------|-----------------------------|161|id        |int32           | Index of the sample         |162|path      |string          | Original path in the dataset|163|code      |Sequence[string]| Code Tokens                 |164 165### Data Splits166 167#### java168 169|    |train|validation|test|170|----|----:|---------:|---:|171|java|12934|      7189|8268|172 173#### python174 175|      |train |test |176|------|-----:|----:|177|python|100000|50000|178 179## Dataset Creation180 181### Curation Rationale182 183[More Information Needed]184 185### Source Data186 187#### Initial Data Collection and Normalization188 189[More Information Needed]190 191#### Who are the source language producers?192 193[More Information Needed]194 195### Annotations196 197#### Annotation process198 199[More Information Needed]200 201#### Who are the annotators?202 203[More Information Needed]204 205### Personal and Sensitive Information206 207[More Information Needed]208 209## Considerations for Using the Data210 211### Social Impact of Dataset212 213[More Information Needed]214 215### Discussion of Biases216 217[More Information Needed]218 219### Other Known Limitations220 221[More Information Needed]222 223## Additional Information224 225### Dataset Curators226 227https://github.com/microsoft, https://github.com/madlag228 229### Licensing Information230 231Computational Use of Data Agreement (C-UDA) License.232 233### Citation Information234 235```236@article{raychev2016probabilistic,237      title={Probabilistic Model for Code with Decision Trees},238      author={Raychev, Veselin and Bielik, Pavol and Vechev, Martin},239      journal={ACM SIGPLAN Notices},240      pages={731--747},241      year={2016},242      publisher={ACM New York, NY, USA}243    }244    @inproceedings{allamanis2013mining,245      title={Mining Source Code Repositories at Massive Scale using Language Modeling},246      author={Allamanis, Miltiadis and Sutton, Charles},247      booktitle={2013 10th Working Conference on Mining Software Repositories (MSR)},248      pages={207--216},249      year={2013},250      organization={IEEE}251    }252```253 254The data for "java" configuration comes from:255```256@dataset{rafael_michael_karampatsis_2020_3628665,257  author       = {Rafael - Michael Karampatsis and258                  Hlib Babii and259                  Romain Robbes and260                  Charles Sutton and261                  Andrea Janes},262  title        = {Preprocessed Java Code Corpus},263  month        = jan,264  year         = 2020,265  publisher    = {Zenodo},266  version      = {1.0},267  doi          = {10.5281/zenodo.3628665},268  url          = {https://doi.org/10.5281/zenodo.3628665}269}270```271 272### Contributions273 274Thanks to @madlag (and partly also @ncoop57) for adding this dataset.