Agentic Data Engineering Resources

Resource

What Are OpenAI Embeddings and What Can You Build With Them?

Build search, RAG, and agent memory with OpenAI embeddings. See how model choice, freshness, permissions, and re-indexing affect production.

Pedro Lopez

June 5, 2026

Last updated September 3, 2026

Summarize with AI:

OpenAI embeddings make semantic retrieval practical, but production success depends less on the API call than on the system around it. Model choice, chunking, freshness, permissions, metadata, and re-indexing determine whether search, RAG, and agent memory stay accurate as content changes. Embeddings convert text into numerical vectors where distance captures semantic relatedness, but turning that capability into a reliable system requires planning for storage, index behavior, access control, and update paths as source content changes.

TL;DR

OpenAI embeddings turn text into vectors that capture semantic relatedness and power search, RAG, classification, recommendations, and agent memory.
The text-embedding-3 model family improves on ada-002 with stronger multilingual retrieval and dimension shortening support.
Choosing between small and large depends on quality, cost, and storage needs, especially at production scale.
In production, results depend on more than the API call: freshness, permissions, and re-indexing strategy matter too.

How Do OpenAI Embeddings Work?

OpenAI embeddings turn text into fixed-size numerical vectors so semantically similar inputs sit near each other in a high-dimensional space. An embedding is a list of floating-point numbers produced by passing text through a neural network trained to place semantically similar inputs close together. "How do I reset my password?" and "I forgot my login credentials" produce vectors that are near each other, while unrelated text lands farther apart.

The text-embedding-3 models and ada-002 use the cl100k_base tokenizer and accept inputs up to a maximum token limit per request. Every input, whether it is a sentence or a full page of text, produces a fixed-size vector. These models output vectors normalized to length 1 under L2 normalization, so cosine similarity can be computed using a dot product.

Dimension Shortening With Matryoshka Behavior

The text-embedding-3 models support variable embedding dimensions through the dimensions parameter. In practice, shorter vectors can still preserve useful semantic information, and shortened v3 vectors can still beat ada-002 on standard benchmarks.

For teams balancing quality, storage footprint, and query speed, dimension choice matters because shorter vectors reduce storage and can change retrieval behavior. If a pipeline uses hardcoded similarity thresholds for top-N selection, those thresholds need recalibration after truncation.

How Do the text-embedding-3 Models Compare?

The main tradeoff is simple: text-embedding-3-small lowers cost and storage pressure, while text-embedding-3-large gives more retrieval headroom. Which model fits best depends on the workload and how much quality gain matters relative to storage and index size.

Feature	text-embedding-3-small	text-embedding-3-large	text-embedding-ada-002
Default dimensions	1,536	3,072	1,536
Massive Text Embedding Benchmark (MTEB) average score	Higher than ada-002	Highest of the three	Lowest of the three
MIRACL multilingual score	Improved over ada-002	Highest of the three	Lower multilingual performance than the v3 models
Dimension shortening	Yes	Yes	No
Storage per 1M vectors (full dim)	Lower than large	Higher than small	Similar to small at full size
Status	Current	Current	Older generation

Pricing, batch API rates, and max context length change over time. Confirm the latest figures on OpenAI pricing before finalizing a cost model.

Model Selection By Workload Type

For prototyping, early development, and cost-sensitive batch indexing of large corpora, text-embedding-3-small keeps costs lower while teams iterate on chunking, retrieval logic, and prompt design. For multilingual retrieval, text-embedding-3-large shows stronger benchmark results, so the decision comes down to whether that gain justifies the added dimensionality, cost, and storage pressure.

For latency-sensitive agent retrieval, benchmark both models on production data before committing. Aggregate benchmark scores are useful as a starting point, but conversational workloads should still be tested on the actual corpus and retrieval pipeline.

What Can You Build With OpenAI Embeddings?

OpenAI embeddings support semantic search, RAG pipelines, classification systems, recommendations, clustering workflows, agent memory, and cross-system enterprise search. The common thread is that both stored content and incoming queries are mapped into the same vector space, so similarity search becomes possible.

Use Case	Description	Production Example
Semantic search	Rank results by meaning, not keywords	Enterprise knowledge bases across business systems
RAG pipelines	Ground LLM responses in retrieved documents	Support agents answering from product docs and ticket history
Classification	Assign labels based on nearest embedding match	Classifying unstructured sources
Recommendations	Surface items with semantically related content	Matching content to learning objectives
Clustering	Group content by semantic similarity	Finding patterns in call transcripts
Agent memory	Store and retrieve knowledge for agent reasoning	Pre-embedded metadata retrieved alongside live queries
Cross-system enterprise search	Search across multiple tools with one indexed layer	Agents querying across systems with unified retrieval

Semantic Search And RAG

OpenAI embeddings support semantic search and Retrieval-Augmented Generation (RAG) by representing both documents and queries in the same vector space. In RAG, you embed the corpus once, embed each query at runtime, retrieve the most semantically similar chunks, and inject them into the large language model context window.

Dense retrieval captures semantic meaning, while keyword search still helps with exact entity names and terms. In production, that combination is often a practical baseline because each method covers the other method's blind spots.

Classification, Clustering, And Recommendations

OpenAI embeddings also support classification, clustering, and recommendations, and these use cases do not require a vector database. Classification assigns labels by finding the nearest embedding match. Clustering groups content by semantic similarity, and recommendations surface items with related content.

Agent Memory And Context Engineering

OpenAI embeddings can act as the storage layer for AI agents' semantic memory, where agents store facts and retrieve them through similarity search during reasoning. A common production pattern uses pre-embedded metadata for retrieval alongside fresh queries against live systems.

This fits directly into agent knowledge and context engineering. Pre-embedded context about data sources is separated from live queries against those data sources, which lets the agent reason over large catalogs without scanning raw metadata on every request.

How Do OpenAI Embeddings Fit Into AI Agent Architecture?

OpenAI embeddings fit into AI agent architecture mainly as the retrieval layer for semantic memory and parts of episodic memory. Semantic memory stores persistent facts and knowledge, so it uses pre-embedded knowledge bases with hybrid retrieval. Episodic memory captures what happened in specific interactions, tool calls, and outcomes, so it needs embeddings plus temporal metadata. Working memory uses no embeddings at all because it is the active context window.

Context engineering is the discipline of designing the complete system so agents receive the right information at the right time. In these systems, embeddings support pre-inference retrieval before the reasoning loop starts and just-in-time retrieval when sub-tasks surface new knowledge needs.

Permission-Aware Retrieval For Enterprise Agents

Enterprise agents need permission-aware embedding retrieval because retrieved content can be synthesized directly into a generated answer. Sensitive data from unauthorized documents can be woven into natural-language answers with no further permission check between retrieval and delivery.

The threat also extends to stored vectors, so permissions need to propagate into vector-store metadata at ingestion time and be enforced at retrieval time. Teams handling regulated data often need controls that support permission propagation and auditability.

Keeping Embeddings Fresh Across Enterprise Sources

Keeping embeddings fresh means updating vectors when source content changes so retrieval stays aligned with current documents and policies. Without that update path, a vector similarity score cannot tell that the underlying document changed months ago.

Manual re-embedding pipelines can create gaps, so teams often use incremental approaches. File-based sources use hash-based change detection, database sources use Change Data Capture (CDC), and SaaS applications often use webhook-based ingestion. Each vector record should carry metadata such as source document ID, content hash, embedding model version, and source permissions so targeted updates and delete propagation remain possible.

What Does It Take To Run OpenAI Embeddings In Production?

Running OpenAI embeddings in production means planning for API cost, vector storage, indexing behavior, and full re-indexing when models change. Embedding costs scale with both API calls and vector storage, so model selection affects infrastructure as much as retrieval quality.

Scale	text-embedding-3-small (1,536 dim)	text-embedding-3-large (3,072 dim)	text-embedding-3-large (shortened)
Vector size	Smaller footprint	Larger footprint	Much smaller footprint
1M vectors storage	Lower than large	Higher than small	Lower still after shortening
10M vectors storage	Lower than large	Higher than small	Lower still after shortening
API cost to embed large corpora	Lower than large	Higher than small	Same API cost as large; storage savings only
Batch API cost for same	Lower than large	Higher than small	Same API cost as large; storage savings only

Cost Modeling And The Batch API

The Batch API reduces embedding cost for workloads that can tolerate asynchronous processing. At small scale, the absolute cost differences are minimal, but the price ratio matters more when a team is embedding large document collections or re-embedding frequently.

Model Migration And Re-Indexing

Switching from text-embedding-ada-002 to text-embedding-3 generally requires a new index path and full re-vectorization because the vectors are not compatible. Truncating v3 vectors to 1,536 dimensions does not make them interchangeable. Mixing embeddings from different model versions can also change retrieval behavior even when the system does not show an explicit error.

What Should Teams Remember Before They Ship OpenAI Embeddings In Production?

The most reliable production pattern is to pair embedding generation with fresh sync, metadata management, and access controls across source systems. OpenAI embeddings are useful because they make semantic retrieval practical across search, RAG, classification, and agent memory, but production results depend less on the embedding call itself than on model choice, freshness, permissions, metadata, and re-indexing discipline.

Airbyte's agentic platform sits between agent frameworks and enterprise data sources. It includes embedding generation and metadata extraction across structured and unstructured data, incremental context sync to keep embeddings fresh without full re-processing, and access controls across sources. It also delivers to major vector databases and provides agent connectors so agents can reason across unified, permission-aware records from connected business systems.

Talk to sales to learn how Airbyte's agentic platform supports permission-aware retrieval across business systems, or try Airbyte's agentic platform today.

Frequently Asked Questions

What is the difference between text-embedding-3-small and text-embedding-3-large?

text-embedding-3-small produces smaller default vectors and lower cost, while text-embedding-3-large produces larger default vectors and stronger benchmark results. The larger model can also be shortened, which changes storage footprint more than API cost.

Can I use OpenAI embeddings for AI agent memory?

Yes. Embeddings power the semantic memory layer in AI agents by storing knowledge as vectors and retrieving relevant context during reasoning. A common pattern is to store pre-indexed metadata for retrieval alongside live system queries.

Do I need to re-embed my data when switching OpenAI embedding models?

Yes. Vectors from text-embedding-ada-002 are not compatible with text-embedding-3 vectors in the same index, even when v3 vectors are truncated to 1,536 dimensions. Switching models requires full corpus re-embedding.

How much does it cost to embed 10 million documents with OpenAI?

The total cost depends on token count, model choice, and whether you use standard or Batch API pricing. In general, text-embedding-3-small is cheaper than text-embedding-3-large, and shortening vectors affects storage costs rather than embedding API cost.

How do OpenAI embeddings compare to open-source alternatives?

The right choice depends on the domain, privacy requirements, and deployment constraints. OpenAI offers a managed API with published benchmark scores, while open-source alternatives require self-hosted infrastructure but can be fine-tuned for specific use cases.