BGE Embeddings: Complete Guide — Architecture, MTEB Benchmarks, Fine-Tuning, Local Deployment & RAG 2026

Model Overview

BGE (BAAI General Embedding) is a family of open-source text embedding models developed by the Beijing Academy of Artificial Intelligence (BAAI), first released in August 2023 with the v1.5 update in late 2023. The flagship bge-large-en-v1.5 model produces 1,024-dimensional vectors and achieves a 63.5 MTEB average score, making it the highest-performing open-source English embedding model. BGE belongs to the text embedding model category and solves the problem of measuring semantic similarity between text with a model that can be self-hosted, fine-tuned, and used completely free of charge. BGE is designed for developers building privacy-first RAG pipelines, on-premise semantic search systems, and domain-specific retrieval applications. Its key differentiators are full open-source availability under the Apache 2.0 license, local deployment without any API calls, and LoRA-based fine-tuning for domain adaptation. The BGE family includes large (1,024 dims), base (768 dims), and small (384 dims) variants, plus multilingual and Chinese-specific models. In 2026, BGE is the most widely deployed open-source embedding model in enterprise RAG, powering search for organizations that require data sovereignty and model customization.

Architecture & Technical Deep Dive

BGE uses a BERT-based encoder architecture built on XLM-RoBERTa-large, trained with a contrastive learning objective and a unique instruction-tuning approach. The models are fully open-source, enabling local deployment, fine-tuning, and inspection of model weights and architecture.

Core Architecture

BGE is built on the XLM-RoBERTa-large architecture — a transformer encoder with 24 layers, 1,024 hidden dimensions, 16 attention heads, and 560M parameters (large variant). The encoder processes text up to 512 tokens and produces a single dense vector per input via mean pooling over token representations. The large model outputs 1,024-dimensional vectors; the base model outputs 768-dimensional vectors; the small model outputs 384-dimensional vectors. All variants use the same architecture at different scales.

Instruction-Tuned Retrieval

A key innovation in BGE is instruction tuning for retrieval. When embedding queries, developers prepend an instruction such as "Represent this sentence for searching relevant passages: " to the query text. This produces query embeddings optimized for retrieval, similar to Cohere's input_type specification but achieved through prompt engineering rather than separate model heads. Documents are embedded without the instruction. This asymmetric approach improves retrieval accuracy by 3-5% over symmetric embeddings. The v1.5 update refined the instruction format and improved retrieval scores significantly over v1.0.

Training Methodology

BGE is trained in two stages. Stage 1: pre-training on a large web corpus using contrastive learning with a contrastive loss (InfoNCE). Stage 2: fine-tuning on labeled retrieval datasets including MS MARCO, NQ, and multi-domain retrieval data. BAAI uses hard-negative mining — training the model to distinguish relevant documents from superficially similar but irrelevant ones. The v1.5 update added more diverse training data and improved the contrastive learning curriculum. The model is trained on a single corpus of 100M+ text pairs across web text, academic papers, and domain-specific corpora.

Model Variants

The BGE family includes: bge-large-en-v1.5 (1,024 dims, 63.5 MTEB, 560M params) for maximum quality, bge-base-en-v1.5 (768 dims, 62.5 MTEB, 110M params) for balanced performance, bge-small-en-v1.5 (384 dims, 61.5 MTEB, 33M params) for fast inference, bge-m3 (multilingual, 100+ languages, 1,024 dims) for cross-lingual retrieval, and bge-large-zh-v1.5 for Chinese-specific applications. The small model runs on CPU at 100+ embeddings/second, making it ideal for edge deployment.

Inference Requirements

BGE can be self-hosted on modest hardware. bge-large-en-v1.5 requires 2.2GB VRAM (FP32) or 1.1GB (FP16) and runs on any GPU with 4GB+ VRAM, or on CPU with 8GB RAM (slower). bge-small-en-v1.5 runs on CPU with 4GB RAM at 100+ embeddings/second. Inference latency: 10-50ms per request on GPU, 50-200ms on CPU. Throughput: 1,000+ embeddings/second on a single A100. For production, use Hugging Face Text Embeddings Inference (TEI) server or FlagEmbedding library with batch processing.

MTEB Benchmark Performance & Scores

Scores based on publicly available data as of July 2026. Independent verification recommended.

MTEB Benchmark Comparison

Speed & Throughput

Latency: 10-50ms per request on GPU (bge-large), 50-200ms on CPU. bge-small runs at 100+ embeddings/second on CPU. Throughput: 1,000+ embeddings/second on a single A100 GPU. For production, use Hugging Face Text Embeddings Inference (TEI) server for optimized batch processing. Self-hosting eliminates API latency and rate limits, enabling unlimited throughput bounded only by hardware.

Speed & Latency

Latency: 10-50ms per request on GPU (bge-large), 50-200ms on CPU. bge-small runs at 100+ embeddings/second on CPU. Throughput: 1,000+ embeddings/second on a single A100 GPU. For production, use Hugging Face Text Embeddings Inference (TEI) server for optimized batch processing. Self-hosting eliminates API latency and rate limits, enabling unlimited throughput bounded only by hardware.

API Access, Pricing & Integration Guide

Looking for bge-large-en-v1.5 API pricing in 2026? Below is the complete pricing table, code examples, and integration guide.

API Pricing Table (as of July 2026)

Free Tier & Trial Access

BGE is completely free when self-hosted under the Apache 2.0 license — no API calls, no token limits, no rate limits. Hugging Face Inference API offers a free tier for testing. Together AI and other providers offer managed BGE inference at competitive rates. The only cost is compute hardware for self-hosting.

API Quick Start

# Install FlagEmbedding
pip install FlagEmbedding

from FlagEmbedding import FlagModel
import numpy as np

# Load model (downloads from Hugging Face)
model = FlagModel('BAAI/bge-large-en-v1.5',
                  query_instruction_for_retrieval="Represent this sentence for searching relevant passages: ",
                  use_fp16=True)

# Embed documents
documents = [
    "Machine learning is fascinating",
    "AI transforms industries",
    "I love pizza"
]
doc_embeddings = model.encode(documents)

# Embed a query (with retrieval instruction auto-prepended)
query = "What is AI?"
query_embedding = model.encode_query(query)

# Compute cosine similarity
sims = model.similarity(query_embedding, doc_embeddings)
print(f"Similarities: {sims}")

# Or using sentence-transformers
# from sentence_transformers import SentenceTransformer
# model = SentenceTransformer('BAAI/bge-large-en-v1.5')

Supported API Features

Compatible Platforms & Integrations

Fine-Tuning, RAG & Advanced Use

Fine-Tuning Availability

BGE supports full fine-tuning and LoRA-based parameter-efficient fine-tuning via the FlagEmbedding library. Developers can fine-tune on domain-specific query-document pairs using contrastive loss. LoRA fine-tuning requires as few as 1,000 query-document pairs and 8GB VRAM. Full fine-tuning requires 5,000+ pairs and 24GB+ VRAM. Fine-tuned BGE models often outperform commercial APIs on domain-specific retrieval by 5-15%.

Fine-Tuning Requirements

LoRA fine-tuning: 8GB+ VRAM (RTX 3060 or better), 1,000+ domain-specific query-document pairs, FlagEmbedding library. Full fine-tuning: 24GB+ VRAM (RTX 3090, A100), 5,000+ pairs, 2-4 hours training time. Recommended: use LoRA for most domain adaptation — 90% of the quality gain at 10% of the compute. Training data format: (query, positive_document, negative_documents) triples with hard-negative mining.

Fine-Tuning Use Cases

• Privacy-first RAG pipeline — self-host BGE for on-premise retrieval with zero data leaving your infrastructure
• Domain-specific retrieval — fine-tune BGE on legal, medical, or financial documents for 5-15% retrieval improvement over generic models
• Edge deployment — use bge-small-en-v1.5 for on-device embedding generation on laptops, phones, or IoT devices
• Unlimited-scale embedding — self-host for unlimited throughput without API rate limits or per-token costs
• Research and experimentation — full access to model weights enables custom architecture modifications and research

RAG Integration Guide

BGE is the leading open-source choice for privacy-first RAG. Recommended architecture: Documents → Chunker (256-512 tokens) → bge-large-en-v1.5 (self-hosted via TEI) → Vector DB (Qdrant, Milvus, Chroma, pgvector — all self-hosted) → Query bge-large-en-v1.5 (with retrieval instruction) → Retriever → LLM (Llama 3, Mistral, or any open-source LLM). For domain-specific RAG, fine-tune BGE on domain query-document pairs with LoRA before indexing. Use bge-small for cost-sensitive or edge deployments. The entire pipeline can run on-premise with zero external API calls.

Prompt Engineering Tips

• Prepend "Represent this sentence for searching relevant passages: " to queries for retrieval (auto-handled by FlagModel)
• Do NOT prepend the instruction to documents — only to queries
• Chunk documents at 256-512 tokens with 50-100 token overlap (within the 512-token limit)
• Use bge-small-en-v1.5 for CPU-only or edge deployment — 100+ embeddings/second on CPU
• Fine-tune with LoRA on 1,000+ domain query-document pairs for 5-15% retrieval improvement
• Use Hugging Face TEI server for production deployment — optimized for batch embedding inference
• Use FP16 inference to halve VRAM requirements with minimal quality loss

Use Cases, Strengths & Limitations

Top 10 Real-World Use Cases

Strengths

• ✓ Fully Open Source (Apache 2.0) — free to use, modify, and distribute with no restrictions
• ✓ Local Deployment — self-host with zero external API calls for complete data sovereignty
• ✓ Fine-Tunable — LoRA and full fine-tuning for domain adaptation with 5-15% retrieval improvement
• ✓ Top Open-Source MTEB Performance — 63.5 MTEB average, highest among open-source English models
• ✓ Multiple Model Sizes — large, base, and small variants for quality/speed/cost trade-offs
• ✓ Instruction-Tuned Retrieval — query instruction improves retrieval accuracy by 3-5%
• ✓ Unlimited Throughput — self-hosted with no rate limits or per-token costs
• ✓ Edge Deployment — bge-small runs on CPU at 100+ embeddings/second

Limitations & Weaknesses

• ✗ Lower MTEB Than Top Commercial — 63.5 vs 64.6 (OpenAI) and 64.8 (Jina v3); 1-2 points behind
• ✗ 512-Token Input Limit — shorter than OpenAI (8,191) and Jina (8,192), requiring more chunking
• ✗ Self-Hosting Overhead — requires GPU/CPU infrastructure and DevOps for production deployment
• ✗ No Managed API (Free) — Hugging Face API costs $0.19/1M; self-hosting requires hardware
• ✗ Smaller Ecosystem Than OpenAI — fewer pre-built integrations, though LangChain and LlamaIndex support it
• ✗ English-Focused (en models) — multilingual support requires bge-m3, which is larger and slower
• ✗ No Multimodal — text-only; for multimodal, use CLIP-based open-source models

Who Should Use This Model

Alternatives, Comparisons & Verdict

Top Alternatives

Detailed Comparison

BGE-large vs OpenAI text-embedding-3-large: BGE is free/open-source and runs locally (critical for privacy). OpenAI scores slightly higher (64.6 vs 63.5 MTEB) and has flexible dimensions and 8K context. BGE is fine-tunable; OpenAI is not. BGE is free; OpenAI costs $0.13/1M. Choose BGE for privacy, fine-tuning, and cost; OpenAI for convenience and maximum quality. BGE-large vs E5-large: Both are open-source and fine-tunable. BGE scores higher (63.5 vs 62.0 MTEB) and has instruction-tuned retrieval. E5 has a simpler architecture and is easier to fine-tune. BGE has better multilingual support via bge-m3. Choose BGE for higher quality, E5 for simpler fine-tuning.

Our Verdict

Internal Links

Frequently Asked Questions