ibm-granite/granite-embedding-small-english-r2

1---2language:3- en4library_name: sentence-transformers5license: apache-2.06pipeline_tag: feature-extraction7tags:8- granite9- embeddings10- transformers11- mteb12- feature-extraction13---14 15# Granite-Embedding-Small-English-R216 17<!-- Provide a quick summary of what the model is/does. -->18 19**Model Summary:** Granite-embedding-small-english-r2 is a 47M parameter dense biencoder embedding model from the Granite Embeddings collection that can be used to generate high quality text embeddings. This model produces embedding vectors of size 384 based on context length of upto 8192 tokens. Compared to most other open-source models, this model was only trained using open-source relevance-pair datasets with permissive, enterprise-friendly license, plus IBM collected and generated datasets. 20 21The r2 models show strong performance across standard and IBM-built information retrieval benchmarks (BEIR, ClapNQ), 22code retrieval (COIR), long-document search benchmarks (MLDR, LongEmbed), conversational multi-turn (MTRAG), 23table retrieval (NQTables, OTT-QA, AIT-QA, MultiHierTT, OpenWikiTables), and on many enterprise use cases.24 25These models use a bi-encoder architecture to generate high-quality embeddings from text inputs such as queries, passages, and documents, enabling seamless comparison through cosine similarity. Built using retrieval oriented pretraining, contrastive finetuning, knowledge distillation, and model merging, granite-embedding-small-english-r2 is optimized to ensure strong alignment between query and passage embeddings.26 27The latest granite embedding r2 release introduces two English embedding models, both based on the ModernBERT architecture:28- _granite-embedding-english-r2_ (**149M** parameters): with an output embedding size of _768_, replacing _granite-embedding-125m-english_. 29- **_granite-embedding-small-english-r2_** (**47M** parameters): A _first-of-its-kind_ reduced-size model, with 8192 context length support, fewer layers and a smaller output embedding size (_384_), replacing _granite-embedding-30m-english_. 30 31## Model Details32 33- **Developed by:** Granite Embedding Team, IBM34- **Repository:** [ibm-granite/granite-embedding-models](https://github.com/ibm-granite/granite-embedding-models)35- **Project Page:** [IBM Granite](https://www.ibm.com/granite)36- **Paper:** [Granite Embedding R2 Models](https://arxiv.org/abs/2508.21085)37- **Language(s):** English38- **Release Date**: Aug 15, 202539- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)40 41## Usage42 43**Intended Use:** The model is designed to produce fixed length vector representations for a given text, which can be used for text similarity, retrieval, and search applications.44 45For efficient decoding, these models use Flash Attention 2. Installing it is optional, but can lead to faster inference.46 47```shell48pip install flash_attn==2.6.149```50 51<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->52 53**Usage with Sentence Transformers:** 54 55The model is compatible with SentenceTransformer library and is very easy to use:56 57First, install the sentence transformers library58```shell59pip install sentence_transformers60```61 62The model can then be used to encode pairs of text and find the similarity between their representations63 64```python65from sentence_transformers import SentenceTransformer, util66 67model_path = "ibm-granite/granite-embedding-small-english-r2"68# Load the Sentence Transformer model69model = SentenceTransformer(model_path)70 71input_queries = [72    ' Who made the song My achy breaky heart? ',73    'summit define'74    ]75 76input_passages = [77    "Achy Breaky Heart is a country song written by Don Von Tress. Originally titled Don't Tell My Heart and performed by The Marcy Brothers in 1991. ",78    "Definition of summit for English Language Learners. : 1 the highest point of a mountain : the top of a mountain. : 2 the highest level. : 3 a meeting or series of meetings between the leaders of two or more governments."79    ]80 81# encode queries and passages. The model produces unnormalized vectors. If your task requires normalized embeddings pass normalize_embeddings=True to encode as below.82query_embeddings = model.encode(input_queries)83passage_embeddings = model.encode(input_passages)84 85# calculate cosine similarity86print(util.cos_sim(query_embeddings, passage_embeddings))87```88 89**Usage with Huggingface Transformers:** 90 91This is a simple example of how to use the granite-embedding-small-english-r2 model with the Transformers library and PyTorch.92 93First, install the required libraries94```shell95pip install transformers torch96```97 98The model can then be used to encode pairs of text99 100```python101import torch102from transformers import AutoModel, AutoTokenizer103 104model_path = "ibm-granite/granite-embedding-small-english-r2"105 106# Load the model and tokenizer107model = AutoModel.from_pretrained(model_path)108tokenizer = AutoTokenizer.from_pretrained(model_path)109model.eval()110 111input_queries = [112    ' Who made the song My achy breaky heart? ',113    'summit define'114    ]115 116# tokenize inputs117tokenized_queries = tokenizer(input_queries, padding=True, truncation=True, return_tensors='pt')118 119# encode queries120with torch.no_grad():121    # Queries122    model_output = model(**tokenized_queries)123    # Perform pooling. granite-embedding-278m-multilingual uses CLS Pooling124    query_embeddings = model_output[0][:, 0]125 126# normalize the embeddings127query_embeddings = torch.nn.functional.normalize(query_embeddings, dim=1)128 129```130 131## Evaluation Results132Granite embedding r2 models show a strong performance across tasks diverse tasks. 133 134Performance of the granite models on MTEB Retrieval (i.e., BEIR), MTEB-v2, code retrieval (CoIR), long-document search benchmarks (MLDR, LongEmbed), conversational multi-turn (MTRAG), 135table retrieval (NQTables, OTT-QA, AIT-QA, MultiHierTT, OpenWikiTables),  benchmarks is reported in the below tables. 136 137The average speed to encode documents on a single H100 GPU using a sliding window with 512 context length chunks is also reported. 138Nearing encoding speed of 200 documents per second granite-embedding-small-english-r2 demonstrates speed and efficiency, while mainintaining competitive performance.139 140| Model                              | Parameters (M) | Embedding Size | BEIR Retrieval (15) | MTEB-v2 (41)| CoIR (10) | MLDR (En) | MTRAG (4) |  Encoding Speed (dosc/sec) |141|------------------------------------|:--------------:|:--------------:|:-------------------:|:-----------:|:---------:|:---------:|:---------:|:-------------------------------:|142| granite-embedding-125m-english     |      125       |      768       |        52.3         |     62.1   |   50.3    |   35.0    |   49.4   |               149             |143| granite-embedding-30m-english      |       30       |      384       |        49.1         |     60.2   |   47.0    |   32.6    |   48.6   |               198             |144| granite-embedding-english-r2       |      149       |      768       |        53.1         |     62.8   |   55.3    |   40.7    |   56.7   |               144             |145| granite-embedding-small-english-r2 |       47       |      384       |        50.9         |     61.1   |   53.8    |   39.8    |   48.1   |               199             |146 147 148 149|Model                              | Parameters (M)| Embedding Size|**AVERAGE**|MTEB-v2 Retrieval (10)| CoIR (10)| MLDR (En)| LongEmbed (6)| Table IR (5)| MTRAG (4) | Encoding Speed (docs/sec)|150|-----------------------------------|:-------------:|:-------------:|:---------:|:--------------------:|:--------:|:--------:|:------------:|:-----------:|:--------:|-----------:|151|e5-small-v2                        |33|384|45.39|48.5|47.1|29.9|40.7|72.31|33.8| 138|152|bge-small-en-v1.5                  |33|384|45.22|53.9|45.8|31.4|32.1|69.91|38.2| 138|153|||||||||||154|granite-embedding-english-r2       |149|768|59.5|56.4|54.8|41.6|67.8|78.53|57.6| 144|155|granite-embedding-small-english-r2 | 47|384|55.6|53.9|53.4|40.1|61.9|75.51|48.9| 199|156 157 158### Model Architecture and Key Features159 160The latest granite embedding r2 release introduces two English embedding models, both based on the ModernBERT architecture:161- _granite-embedding-english-r2_ (**149M** parameters): with an output embedding size of _768_, replacing _granite-embedding-125m-english_. 162- _granite-embedding-small-english-r2_ (**47M** parameters): A _first-of-its-kind_ reduced-size model, with fewer layers and a smaller output embedding size (_384_), replacing _granite-embedding-30m-english_. 163 164The following table shows the structure of the two models:165 166| Model                     | **granite-embedding-small-english-r2** | granite-embedding-english-r2   |167| :---------                | :-------:|:--------:| 168| Embedding size            | **384**      | 768      | 169| Number of layers          | **12**       | 22       | 170| Number of attention heads | **12**       | 12       | 171| Intermediate size         | **1536**     | 1152     | 172| Activation Function       | **GeGLU**    | GeGLU    | 173| Vocabulary Size           | **50368**    | 50368    | 174| Max. Sequence Length      | **8192**     | 8192     | 175| # Parameters              | **47M**      | 149M     | 176 177 178### Training and Optimization179 180The granite embedding r2 models incorporate key enhancements from the ModernBERT architecture, including: 181- Alternating attention lengths to accelerate processing 182- Rotary position embeddings for extended sequence length 183- A newly trained tokenizer optimized with code and text data 184- Flash Attention 2.0 for improved efficiency 185- Streamlined parameters, eliminating unnecessary bias terms186 187 188## Data Collection189Granite embedding r2 models are trained using data from four key sources: 1901. Unsupervised title-body paired data scraped from the web1912. Publicly available paired with permissive, enterprise-friendly license1923. IBM-internal paired data targetting specific technical domains1934. IBM-generated synthetic data194 195Notably, we _do not use_ the popular MS-MARCO retrieval dataset in our training corpus due to its non-commercial license (many open-source models use this dataset due to its high quality). 196 197The underlying encoder models using GneissWeb, an IBM-curated dataset composed exclusively of open, commercial-friendly sources.198 199For governance, all our data undergoes a data clearance process subject to technical, business, and governance review. This comprehensive process captures critical information about the data, including but not limited to their content description ownership, intended use, data classification, licensing information, usage restrictions, how the data will be acquired, as well as an assessment of sensitive information (i.e, personal information). 200 201## Infrastructure202We trained the granite embedding english r2 models using IBM's computing cluster, BlueVela Cluster, which is outfitted with NVIDIA H100 80GB GPUs. This cluster provides a scalable and efficient infrastructure for training our models over multiple GPUs.203 204## Ethical Considerations and Limitations205Granite-embedding-small-english-r2 leverages both permissively licensed open-source and select proprietary data for enhanced performance. The training data for the base language model was filtered to remove text containing hate, abuse, and profanity. Granite-embedding-small-english-r2 is trained only for English texts, and has a context length of 8192 tokens (longer texts will be truncated to this size).206 207- ⭐️ Learn about the latest updates with Granite: https://www.ibm.com/granite208- 📄 Get started with tutorials, best practices, and prompt engineering advice: https://www.ibm.com/granite/docs/209- 💡 Learn about the latest Granite learning resources: https://ibm.biz/granite-learning-resources210 211## Citation212```213@misc{awasthy2025graniteembeddingr2models,214      title={Granite Embedding R2 Models}, 215      author={Parul Awasthy and Aashka Trivedi and Yulong Li and Meet Doshi and Riyaz Bhat and Vignesh P and Vishwajeet Kumar and Yushu Yang and Bhavani Iyer and Abraham Daniels and Rudra Murthy and Ken Barker and Martin Franz and Madison Lee and Todd Ward and Salim Roukos and David Cox and Luis Lastras and Jaydeep Sen and Radu Florian},216      year={2025},217      eprint={2508.21085},218      archivePrefix={arXiv},219      primaryClass={cs.CL},220      url={https://arxiv.org/abs/2508.21085}, 221}222```