leosltl/Android-Control

1---2license: apache-2.03task_categories:4  - robotics5  - image-to-text6tags:7  - android8  - gui-agent9  - device-control10  - mobile11  - ui-automation12  - google-research13pretty_name: Android Control14size_categories:15  - 10K<n<100K16---17 18# Android Control (Community Mirror)19 20> **This is a community mirror of the official [Android Control](https://github.com/google-research/google-research/tree/master/android_control) dataset by Google Research, hosted on Hugging Face for easier access.**21 22The original dataset is hosted on Google Cloud Storage at `gs://gresearch/android_control/`. This copy is provided to make it more accessible to the research community via Hugging Face.23 24## Dataset Description25 26Android Control is a dataset for training and evaluating Android device control agents. It contains **15,283 demonstrations** across **833 Android apps** covering **40 app categories**, with **14,548 unique tasks**.27 28Each demonstration includes:29- **Screenshots** (PNG-encoded)30- **Accessibility trees** (serialized protocol buffer format)31- **Natural language instructions** — both high-level goals and low-level step-by-step instructions32- **Actions** — `click`, `scroll`, `input_text`, `open_app`, `navigate_home`, `navigate_back`, `long_press`, `wait`33 34### Data Format35 36The data is stored as **TFRecord files** with GZIP compression. Each record contains:37- `episode_id`: unique identifier for the demonstration38- `goal`: high-level task description39- `step_instructions`: low-level action descriptions40- `screenshots`: PNG-encoded screen images41- `accessibility_trees`: serialized accessibility tree protos42- `actions`: JSON-encoded action sequences43 44### Loading the Data45 46```python47import tensorflow as tf48 49filenames = tf.io.gfile.glob('path/to/android_control*')50raw_dataset = tf.data.TFRecordDataset(filenames, compression_type='GZIP')51```52 53## Citation54 55If you use this dataset, please cite the original paper:56 57```bibtex58@article{li2024effects,59  title={On the Effects of Data Scale on UI Control Agents},60  author={Li, Wei and Bishop, William and Li, Alice and Rawles, Chris and Campbell-Ajala, Folawiyo and Tyamagundlu, Divya and Riva, Oriana},61  journal={arXiv preprint arXiv:2406.03679},62  year={2024}63}64```65 66## License67 68This dataset is released under the **Apache License 2.0** by Google Research.69 70## Disclaimer71 72This is an unofficial community mirror. All credit goes to the original authors at Google Research. If you are an author and would like this mirror removed, please open an issue.73