Skip to main content
Last Updated: July 13, 2026
Deploy from the SaladCloud Portal.

​
Overview

This recipe runs BAAI/bge-reranker-v2-m3 with Hugging Face Text Embeddings Inference on a Salad GPU. It accepts a query and candidate passages, scores each query-passage pair, and returns the candidates in descending order of relevance. A reranker complements embedding and vector search. Vector search efficiently retrieves a broad candidate set, while a reranker evaluates each candidate together with the original query to improve the final ordering. In a RAG pipeline, this helps select the most relevant passages before sending context to an LLM. BAAI/bge-reranker-v2-m3 is a multilingual reranker based on XLM-RoBERTa. It supports inputs up to 8192 tokens and is a good fit for multilingual search, document retrieval, and RAG result refinement.

​
Quick Start

  1. Open the SaladCloud Portal.
  2. Deploy the BGE Reranker API recipe.
  3. Enter a Container Group Name.
  4. Decide whether to enable Require Container Gateway Authentication:
    • Enabled: requests must include your SaladCloud API key.
    • Disabled: anyone with the URL can call the reranking service.
  5. Deploy and wait for the model to download and the replica to become ready.
The model is downloaded from Hugging Face at startup, so it can take several minutes before the deployment becomes ready.

​
Defaults

The recipe comes preconfigured with these defaults:
  • Server: Hugging Face Text Embeddings Inference
  • Model ID: BAAI/bge-reranker-v2-m3
  • Container image: ghcr.io/huggingface/text-embeddings-inference:89-1.9
  • Command equivalent: text-embeddings-router --model-id BAAI/bge-reranker-v2-m3 --port 3000
  • Container port: 3000
  • Host bind: ::
  • Data type: float16
  • Max batch tokens: 16384
  • Max client batch size: 32
  • Readiness probe: GET /health
  • Authentication: enabled by default

​
API Endpoints

Useful TEI endpoints include:
  • GET /health - readiness probe and health check
  • GET /info - model and server metadata
  • GET /docs - Swagger documentation
  • POST /rerank - score and rank one query against candidate texts
  • POST /predict - score query-text pairs using TEI’s prediction interface

​
Authentication

Require Container Gateway Authentication is available in the deployment form and is enabled by default.
  • Enabled: every request must include the Salad-Api-Key header.
  • Disabled: anyone with the deployment URL can call the API.
If you enable authentication, see Sending Requests for the header format.

​
Example Request

Send one query and multiple candidate passages to /rerank:
If you disabled authentication during deployment, omit the Salad-Api-Key header. The response is sorted by descending relevance. Scores depend on the input and model version, but the response shape is:
Each index refers to the candidate’s position in the original texts array. Set return_text to false if your application already tracks documents by index and does not need passage text repeated in the response. Set raw_scores to true only if you need raw model logits instead of normalized relevance scores.

​
Test A Deployment

Check health:
Check model metadata:
Confirm that the endpoint ranks the relevant passage first:
The first result should have index equal to 0. Add the Salad-Api-Key header if gateway authentication is enabled.

​
Tuning Notes

  • Keep MODEL_ID set to BAAI/bge-reranker-v2-m3 unless you intentionally want to repurpose the recipe.
  • The default MAX_CLIENT_BATCH_SIZE allows up to 32 candidate texts in one /rerank request. Increase it only after testing memory usage and latency with your expected passage lengths.
  • Long passages and large candidate lists consume more VRAM. Lower MAX_BATCH_TOKENS if replicas run out of memory.
  • For larger retrieval sets, retrieve broadly with vector search and rerank a smaller top-k candidate set rather than sending the full corpus to the reranker.

​
Source Code