Back to all models

Qwen3 Reranker 4B

Deploy Qwen3 Reranker 4B on a dedicated GPU from $0.20 per hour, billed per minute, on 1× RTX A4000. A 4B instruction-aware cross-encoder for second-stage retrieval, combining 32K context and multilingual ranking with a substantially smaller footprint than the 8B variant.

Text4B paramsApache 2.0Deploys in ~5 minSecure Cloud
Qwen/Qwen3-Reranker-4B

HexGrid Cloud price

$0.20/hr

1× RTX A4000

Context

32K

tokens

Parameters

4B

dense

Languages

100+

multilingual

Running Qwen3 Reranker 4B on HexGrid Cloud

What you get when you deploy with us, beyond the hourly rate.

Single tenant by default

The model runs on a GPU that is yours for the life of the instance. No shared endpoint, no queue behind other tenants.

Per-minute billing

You pay for the GPU, not the token. Stop the instance and billing stops with it.

OpenAI-compatible endpoint

Point an existing SDK at your instance by changing the base URL. No rewrite required.

Your weights, your data

Prompts and outputs stay inside your instance. Nothing is logged, sampled or used for training.

Qwen3 Reranker 4B GPU sizing and cost

What the model needs at each precision, and the cheapest HexGrid Cloud configuration that holds it.

Qwen3 Reranker 4B vRAM requirement and hourly cost by precision
PrecisionvRAM neededRuns onPrice
BF16
Recommended		10 GB1× RTX A4000$0.20/hr
FP8
5 GB1× RTX A4000$0.20/hr
INT4
3 GB1× RTX A4000$0.20/hr

Official Qwen sources do not provide standardized production throughput numbers. The native checkpoint uses BF16. Populate these figures from HexGrid measurements using the exact GPU, sequence-length distribution, batching and runtime configuration shown to customers.

About Qwen3 Reranker 4B

What Qwen3 Reranker 4B is built for, and where it falls short.

Qwen3-Reranker-4B is a dense cross-encoder retrieval model built on Qwen3. It consumes a query and candidate document together and produces a relevance score, allowing attention across both sides of the pair.

The model is intended for second-stage retrieval: use a faster embedding or lexical system to obtain candidates, then rerank the strongest candidates with Qwen3-Reranker-4B.

It supports 32K-token inputs, more than 100 languages and user-defined retrieval instructions. Qwen's published evaluation shows that the 4B checkpoint remains highly competitive with the larger 8B variant.

What people run it for

Production RAG

Rerank vector-search results before sending the most relevant passages to the generation model.

Search result reranking

Reorder keyword, hybrid or dense-search candidates using joint query-document semantics.

Multilingual knowledge bases

Rerank documents across multilingual enterprise collections and cross-language retrieval workloads.

Code retrieval

Refine candidate code snippets or files returned by a faster code-search index.

Strengths

  • Strong relevance scoring without the serving footprint of the 8B reranker
  • Scores 69.76 on MTEB-R and 14.84 on FollowIR in Qwen's published evaluation
  • 32K context supports long documents and richer query-document pairs
  • Instruction-aware behavior lets applications specify what should count as relevant
  • Works with Sentence Transformers as a CrossEncoder and with vLLM-based serving

Limitations

  • Cross-encoder inference scales with the number of candidate query-document pairs
  • Not designed as the first-stage index for millions of documents; pair it with embeddings or another retrieval mechanism
  • A 32K theoretical sequence limit does not mean every production deployment should process 32K-token pairs because latency and memory rise with sequence length
  • Ranking quality remains dependent on the quality and recall of the first-stage candidate set
  • Published benchmark results should not be interpreted as universal performance on private corpora or domain-specific relevance judgments

Quickstart

Call the deployed model through HexGrid's vLLM-compatible reranking endpoint.

pythonPOST /v1/rerank
import requests

base_url = 'https://<your-endpoint>/v1'
api_key = '<your-key>'

response = requests.post(
    f'{base_url}/rerank',
    headers={'Authorization': f'Bearer {api_key}'},
    json={
        'model': 'Qwen/Qwen3-Reranker-4B',
        'query': 'Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: How does semantic search work?',
        'documents': [
            'Semantic search represents text by meaning and retrieves semantically similar content.',
            'A relational database organizes records into tables.',
            'Dense embeddings can be compared using cosine similarity.'
        ],
        'top_n': 2
    }
)

print(response.json())

For retrieval workloads, Qwen recommends a task-specific instruction. HexGrid's documented Qwen3 reranking endpoint is /v1/rerank.

Qwen3 Reranker 4B reported benchmarks

Scores published by the model authors. A starting point, not a guarantee on your own data.

  • MTEB-R69.76
  • CMTEB-R75.94
  • MMTEB-R72.74
  • MLDR69.97
  • MTEB-Code81.2
  • FollowIR14.84

Qwen3 Reranker 4B specifications

Architecture and serving details for Qwen3 Reranker 4B.

PublisherQwenParameters4BLicenceApache 2.0ReleasedJune 2025

Architecture

Type
Dense cross-encoder reranker
Backbone
Qwen3
Layers
36
Hidden size
2560
Attention
GQA, 32 attention heads / 8 KV heads
Native precision
BF16

Context and scoring

Sequence length
32K tokens
Input
Instruction + query + document
Output
Relevance score
Instruction aware
Yes
Languages
100+

Serving

HexGrid endpoint
POST /v1/rerank
Runtime
vLLM
Libraries
Transformers, Sentence Transformers
Transformers
>= 4.51.0
Sentence Transformers
CrossEncoder supported

Qwen3 Reranker 4B frequently asked questions

Common questions about deploying Qwen3 Reranker 4B on HexGrid Cloud.

How does Qwen3-Reranker-4B differ from an embedding model?

Embedding models encode queries and documents independently so their vectors can be indexed. A reranker evaluates each query-document pair jointly, which is slower but allows deeper relevance interaction.

Should I choose the 4B or 8B reranker?

The official benchmark table does not show a universal winner. The 4B model scores higher on MTEB-R and FollowIR, while the 8B model scores higher on several multilingual and long-document retrieval measurements. Serving cost and your own evaluation set should be considered alongside those results.

Can it return normalized relevance scores?

Sentence Transformers exposes raw score behavior by default and can apply a sigmoid activation for a 0-1 value. HexGrid's vLLM reranking endpoint returns a relevance_score for each result.

Can I use it commercially?

Yes. Qwen released Qwen3 Embedding and Reranker checkpoints under the Apache 2.0 licence.

Deploy Qwen3 Reranker 4B today

$0.20 per hour on 1× RTX A4000, billed per minute, never shared.

Deploy Qwen3 Reranker 4BSee all models