Qwen3 Reranker 8B
Deploy Qwen3 Reranker 8B on a dedicated GPU from $0.20 per hour, billed per minute, on 1× RTX A4000. An 8B cross-encoder reranker for multilingual, long-context and code retrieval, designed to rescore candidate documents with query-document interaction rather than independent embeddings.
HexGrid Cloud price
$0.20/hr
1× RTX A4000
Context
32K
tokens
Parameters
8B
dense
Languages
100+
multilingual
Running Qwen3 Reranker 8B on HexGrid Cloud
What you get when you deploy with us, beyond the hourly rate.
Single tenant by default
The model runs on a GPU that is yours for the life of the instance. No shared endpoint, no queue behind other tenants.
Per-minute billing
You pay for the GPU, not the token. Stop the instance and billing stops with it.
OpenAI-compatible endpoint
Point an existing SDK at your instance by changing the base URL. No rewrite required.
Your weights, your data
Prompts and outputs stay inside your instance. Nothing is logged, sampled or used for training.
Qwen3 Reranker 8B GPU sizing and cost
What the model needs at each precision, and the cheapest HexGrid Cloud configuration that holds it.
| Precision | vRAM needed | Runs on | Price |
|---|---|---|---|
BF16 Recommended | 20 GB | 1× RTX 4090 | $0.45/hr |
FP8 | 10 GB | 1× RTX A4000 | $0.20/hr |
INT4 | 5 GB | 1× RTX A4000 | $0.20/hr |
Qwen does not publish standardized throughput figures for this checkpoint. Native weights are BF16. Measure requests/s or pairs/s on the exact HexGrid GPU, sequence-length distribution and batch configuration before publishing performance claims.
About Qwen3 Reranker 8B
What Qwen3 Reranker 8B is built for, and where it falls short.
Qwen3-Reranker-8B is the largest text reranker in Qwen's original Qwen3 Embedding series. It is built on the dense Qwen3 foundation model and evaluates the query and candidate document together through a cross-encoder architecture.
Unlike an embedding model, which creates reusable vectors independently, the reranker runs once for every query-document pair. That makes it most useful as a second retrieval stage: retrieve a manageable candidate set with embeddings or lexical search, then use Qwen3-Reranker-8B to reorder those candidates.
The model is instruction-aware, supports more than 100 languages including programming languages, and accepts sequences up to 32K tokens.
What people run it for
RAG reranking
Rescore the top passages returned by a vector database before supplying context to a generation model.
Multilingual search
Improve ranking when queries and documents may use different languages or multilingual content.
Code search
Rerank retrieved functions, files or code snippets using joint query-code relevance scoring.
Enterprise document search
Add a high-accuracy second ranking stage to internal knowledge-base and document retrieval systems.
Strengths
- Cross-encoder scoring allows direct interaction between the query and document instead of comparing independently generated vectors
- Strong multilingual and cross-lingual retrieval results, including a 77.45 CMTEB-R score in Qwen's published evaluation
- 32K context supports substantially longer candidate passages than traditional short-context rerankers
- Instruction-aware scoring lets applications describe the retrieval objective instead of relying on a fixed relevance definition
- Apache 2.0 licensing permits commercial use, modification and redistribution subject to the licence terms
Limitations
- Much more computationally expensive than vector similarity because every candidate must be jointly processed with the query
- Best suited to reranking a bounded candidate set rather than searching an entire corpus directly
- The 8B model is not uniformly better than the 4B variant in Qwen's published benchmarks; for example, the 4B model reports higher MTEB-R and FollowIR scores
- Long query-document pairs increase latency and memory consumption even though the architecture supports 32K tokens
- Performance depends on the retrieval candidates supplied to it; a reranker cannot recover documents that the first-stage retriever never returned
Quickstart
Deploy the model through HexGrid and call the vLLM-compatible reranking endpoint with a query and candidate documents.
import requests
base_url = 'https://<your-endpoint>/v1'
api_key = '<your-key>'
response = requests.post(
f'{base_url}/rerank',
headers={'Authorization': f'Bearer {api_key}'},
json={
'model': 'Qwen/Qwen3-Reranker-8B',
'query': 'Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: What is retrieval augmented generation?',
'documents': [
'Retrieval-augmented generation combines document retrieval with language-model generation.',
'Gradient descent is an optimization algorithm used to train neural networks.',
'Vector databases store embeddings for semantic retrieval.'
],
'top_n': 2
}
)
print(response.json())HexGrid documents Qwen3 rerankers through vLLM's /v1/rerank API. Qwen recommends task-specific instructions for retrieval workloads.
Qwen3 Reranker 8B reported benchmarks
Scores published by the model authors. A starting point, not a guarantee on your own data.
- MTEB-R69.02
- CMTEB-R77.45
- MMTEB-R72.94
- MLDR70.19
- MTEB-Code81.22
- FollowIR8.05
Qwen3 Reranker 8B specifications
Architecture and serving details for Qwen3 Reranker 8B.
Architecture
- Type
- Dense cross-encoder reranker
- Backbone
- Qwen3
- Layers
- 36
- Attention
- GQA, 32 attention heads / 8 KV heads
- Native precision
- BF16
Context and scoring
- Sequence length
- 32K tokens
- Input
- Instruction + query + document
- Output
- Relevance score
- Instruction aware
- Yes
- Languages
- 100+
Serving
- HexGrid endpoint
- POST /v1/rerank
- Runtime
- vLLM
- Local libraries
- Transformers, Sentence Transformers
- Transformers
- >= 4.51.0
- vLLM
- >= 0.8.5 in Qwen examples
Qwen3 Reranker 8B frequently asked questions
Common questions about deploying Qwen3 Reranker 8B on HexGrid Cloud.
What is Qwen3-Reranker-8B used for?
It is normally used after first-stage retrieval. An embedding model, BM25 or another retriever finds candidate documents, then the reranker jointly evaluates each query-document pair and reorders the candidates.
Is Qwen3-Reranker-8B always better than the 4B model?
No. Qwen's published evaluation is mixed. The 8B model scores higher on CMTEB-R, MMTEB-R, MLDR and MTEB-Code, while the 4B model scores higher on MTEB-R and FollowIR. Benchmark the models on your own retrieval distribution before choosing solely by parameter count.
Does it support multilingual search?
Yes. Qwen lists support for more than 100 languages, including programming languages, and evaluates the series on multilingual and cross-lingual retrieval benchmarks.
Can I use it commercially?
Yes. Qwen released the series under Apache 2.0, which permits commercial use, modification and redistribution subject to the licence terms.
Which HexGrid API endpoint should I call?
HexGrid documents Qwen3 rerankers through POST /v1/rerank. The request contains a query, candidate documents and optionally top_n and other reranking controls.
Deploy Qwen3 Reranker 8B today
$0.20 per hour on 1× RTX A4000, billed per minute, never shared.