BGE Reranker V2 Gemma
Deploy BGE Reranker V2 Gemma on a dedicated GPU from $0.20 per hour, billed per minute, on 1× RTX A4000. A 2.51B multilingual LLM-based reranker that jointly reads each query-document pair and scores relevance through a Gemma-based Yes/No classification prompt.
HexGrid Cloud price
$0.20/hr
1× RTX A4000
Context
8K
architectural limit
Parameters
2.51B
dense
Languages
Multilingual
cross-language retrieval
Running BGE Reranker V2 Gemma on HexGrid Cloud
What you get when you deploy with us, beyond the hourly rate.
Single tenant by default
The model runs on a GPU that is yours for the life of the instance. No shared endpoint, no queue behind other tenants.
Per-minute billing
You pay for the GPU, not the token. Stop the instance and billing stops with it.
OpenAI-compatible endpoint
Point an existing SDK at your instance by changing the base URL. No rewrite required.
Your weights, your data
Prompts and outputs stay inside your instance. Nothing is logged, sampled or used for training.
BGE Reranker V2 Gemma GPU sizing and cost
What the model needs at each precision, and the cheapest HexGrid Cloud configuration that holds it.
| Precision | vRAM needed | Runs on | Price |
|---|---|---|---|
BF16 Recommended | 7 GB | 1× RTX A4000 | $0.20/hr |
FP8 | 4 GB | 1× RTX A4000 | $0.20/hr |
INT4 | 2 GB | 1× RTX A4000 | $0.20/hr |
The checkpoint configuration declares float32 weights. BAAI documents FP16 and BF16 inference options in FlagEmbedding, but does not publish standardized production throughput for this checkpoint. Benchmark pairs/s on the exact GPU, candidate count, pair length and batch size used by HexGrid.
About BGE Reranker V2 Gemma
What BGE Reranker V2 Gemma is built for, and where it falls short.
BGE Reranker V2 Gemma is BAAI's larger LLM-based reranker from the BGE v2 family. Instead of independently embedding the query and document, it processes them together so the model can directly model their relevance.
The checkpoint is initialized from Google's original Gemma 2B architecture. BAAI reformulates reranking as a language-model classification problem: the prompt contains query A and passage B, asks whether the passage answers the query, and the relevance signal is taken from the final-position logit for the token 'Yes'.
BAAI positions it as a higher-quality multilingual option in the BGE reranker family. The model configuration supports sequences up to 8192 positions, although BAAI's published reranking examples and benchmark charts commonly use 1024-token inference.
What people run it for
RAG reranking
Rescore the strongest chunks returned by dense, sparse or hybrid retrieval before passing context to a generation model.
Multilingual search
Improve relevance ordering in collections containing multiple languages or cross-language query-document pairs.
Enterprise search
Add a semantic second-stage ranking layer on top of an existing search engine without replacing the first-stage index.
High-precision retrieval
Apply a more computationally expensive relevance model only to a small top-K candidate set when ranking quality matters more than first-stage latency.
Strengths
- Strong multilingual reranking across English, Chinese and multilingual retrieval evaluations
- Joint query-document processing captures relevance interactions that independent embedding similarity cannot
- 60.71 average in BAAI's published BEIR evaluation when reranking candidates from BGE large EN v1.5
- 73.39 average in BAAI's published MIRACL evaluation when reranking candidates retrieved by BGE-M3
- Can be used directly through BAAI's FlagEmbedding library
- Raw relevance scores can be converted to a 0-1 range with a sigmoid when normalized scores are desirable
Limitations
- Every candidate document requires a full query-document forward pass, making it substantially more expensive than vector similarity
- The 2.51B LLM-style architecture is heavier than BGE Reranker V2 M3 and other encoder-only rerankers
- BAAI's normal examples use substantially shorter sequences than the underlying 8K architectural maximum
- It should be used on a bounded first-stage candidate set rather than as the primary retrieval mechanism for an entire corpus
- Licence metadata deserves extra review because BAAI labels the derivative checkpoint Apache 2.0 while the identified Google Gemma 2B base model is governed by Google's Gemma Terms
Quickstart
BAAI's verified interface is FlagEmbedding's LLM reranker. Use this until a model-specific HexGrid endpoint is documented.
from FlagEmbedding import FlagLLMReranker
reranker = FlagLLMReranker(
'BAAI/bge-reranker-v2-gemma',
use_bf16=True,
)
query = 'What is retrieval augmented generation?'
documents = [
'Retrieval-augmented generation retrieves external information before generating an answer.',
'Gradient descent is an optimization method for training neural networks.',
'Vector databases can provide first-stage semantic retrieval for RAG.'
]
pairs = [[query, document] for document in documents]
scores = reranker.compute_score(pairs)
ranked = sorted(zip(scores, documents), reverse=True)
for score, document in ranked:
print(score, document)BAAI's implementation also supports FP16. Its Transformers implementation derives relevance from the final-position 'Yes' token logit.
BGE Reranker V2 Gemma reported benchmarks
Scores published by the model authors. A starting point, not a guarantee on your own data.
- BEIR average — BAAI evaluation60.71
- CMTEB Retrieval average — BAAI evaluation71.74
- MIRACL average — BAAI evaluation73.39
BGE Reranker V2 Gemma specifications
Architecture and serving details for BGE Reranker V2 Gemma.
Architecture
- Type
- LLM-based pointwise cross-encoder reranker
- Base model
- Google Gemma 2B
- Parameters
- 2.51B
- Layers
- 18
- Hidden size
- 2048
- Intermediate size
- 16384
- Attention
- 8 query heads / 1 KV head
Context and scoring
- Max positions
- 8192
- Published evaluation length
- 1024 tokens
- Input
- Query + candidate passage
- Scoring formulation
- Final-position 'Yes' token logit
- Normalized score
- Optional sigmoid
- Language
- Multilingual
Serving
- FlagEmbedding
- FlagLLMReranker
- Transformers
- AutoModelForCausalLM
- FP16 inference
- Supported by BAAI
- BF16 inference
- Supported by BAAI
- HexGrid model-specific API
- Not independently verified
BGE Reranker V2 Gemma frequently asked questions
Common questions about deploying BGE Reranker V2 Gemma on HexGrid Cloud.
Is BGE Reranker V2 Gemma an embedding model?
No. It jointly processes each query and candidate passage and returns a relevance score. It is intended to rerank candidates produced by a first-stage retrieval system.
How does it calculate relevance?
BAAI formats the query and passage into a prompt asking for a Yes or No relevance decision. The reference Transformers implementation uses the model's final-position logit for the token 'Yes' as the relevance score.
What is its maximum context length?
The checkpoint configuration supports 8192 positions. BAAI's reference examples and published benchmark charts typically evaluate reranking at a 1024-token inference length, so the architectural maximum should not be confused with the recommended production pair length.
Is it multilingual?
Yes. BAAI explicitly positions BGE Reranker V2 Gemma as a multilingual reranker and publishes Chinese and MIRACL multilingual evaluations.
Is the licence simply Apache 2.0?
BAAI's model repository labels the checkpoint Apache 2.0. However, the checkpoint configuration identifies Google's original Gemma 2B as its base model, and Google applies Gemma Terms of Use to Gemma and model derivatives. Review both sets of terms before publishing a definitive licensing statement.
Deploy BGE Reranker V2 Gemma today
$0.20 per hour on 1× RTX A4000, billed per minute, never shared.