LLM Embeddings and Vector Search Explained: How AI Finds Meaning, Not Just Keywords

November 12, 2026 · 15 min read

Type "tool to stop customers from leaving" into an old-fashioned site search and you may get nothing back, because no page on the site contains those exact words. Ask the same thing of a modern AI assistant and it will happily talk about churn prevention, retention analytics, and customer health scores, even though you never typed any of those terms. The difference is not magic. It is a technique called embeddings, and it quietly sits underneath most of the AI products businesses now use: semantic search, document chat, support bots, recommendation features, and the retrieval step that feeds web pages into AI answers.

This guide explains embeddings and vector search in plain English for U.S. B2B SaaS teams. It covers what an embedding actually is, how "similar meaning" is measured, how vector databases find matches among millions of passages, why hybrid search usually beats either method alone, where these systems fail, what they mean for privacy, and what they imply about how AI engines find and describe your brand. If you are new to the underlying technology, start with our practical guide to large language models.

Published November 12, 2026. This is an educational explainer. Embedding models, dimensions, and prices change frequently, and AI search providers rarely publish the full details of their retrieval systems. Where this guide describes how a specific product works, it relies on that vendor's public documentation; where it describes AI search in general, it describes common approaches rather than any one engine's private pipeline.

What an Embedding Is

An embedding is a list of numbers that represents the meaning of a piece of content. An embedding model reads a word, a sentence, a paragraph, an image, or a product description and outputs a fixed-length list, called a vector, such as 768, 1,536, or 3,072 numbers. On its own, any single number in that list means nothing a human could read. What matters is the relationship between lists: content with similar meaning produces vectors that sit close together, and unrelated content produces vectors that sit far apart.

A useful mental picture is a map. Imagine every sentence your company has ever published placed as a dot on a huge map, with sentences about pricing clustered in one region, sentences about security in another, and sentences about integrations in a third. "How much does it cost?" and "What are your plans and fees?" share almost no words, but they would land as neighbors on that map. A real embedding space has hundreds or thousands of dimensions instead of two, but the principle is the same: meaning becomes location.

This is the key difference from keyword matching. A keyword index asks, "Which documents contain these words?" An embedding index asks, "Which documents mean something close to this?" Both questions are useful, and as we will see, the best systems ask both.

A Short History: From Word Vectors to Sentence Embeddings

The idea of representing words as vectors is decades old, but it became practical at scale in 2013, when Google researchers published word2vec (Mikolov et al.). Word2vec learned vectors by predicting which words appear near each other in large amounts of text. The famous result was that simple arithmetic on the vectors captured relationships: the vector for "king" minus "man" plus "woman" landed near "queen."

Word vectors had a limitation. Each word got exactly one vector, so "bank" meant the same thing in "river bank" and "bank account." Transformer models, the architecture behind modern LLMs, fixed this by producing context-dependent representations, where a word's vector depends on the sentence around it. In 2019, Sentence-BERT (Reimers and Gurevych) showed how to train transformer models to produce one vector for a whole sentence that could be compared efficiently with other sentence vectors. That paper reported reducing the time to find the most similar pair among 10,000 sentences from about 65 hours with a standard BERT approach to about 5 seconds.

Today's commercial embedding models, offered by OpenAI, Google, Cohere, Voyage, and many open-source projects, are descendants of that line of work. They are trained on very large collections of related text pairs, such as questions and their answers, or titles and their articles, so that matching pairs land close together and unrelated pairs land far apart.

How Similarity Is Measured

Once content is turned into vectors, the system needs a way to say how close two vectors are. Three measures dominate.

• Cosine similarity measures the angle between two vectors, ignoring their length. A score near 1 means they point in nearly the same direction, which is read as very similar meaning. This is the most common choice for text.
• Dot product multiplies matching numbers and adds them up. When vectors are normalized to the same length, as many providers do, dot product and cosine similarity produce the same ranking.
• Euclidean distance measures the straight-line distance between two points. Smaller means more similar.

OpenAI's embeddings documentation recommends cosine similarity and notes that its embeddings are normalized to length 1, so a dot product gives the same result and is slightly faster to compute. The practical lesson for non-engineers is simpler: similarity scores are relative, not absolute. A cosine score of 0.82 does not mean "82 percent relevant." It only means that passage ranked closer than one that scored 0.71 under that particular model. Scores from different embedding models cannot be compared with each other at all.

Dimensions, Models, and Cost

Embedding models differ in how many numbers they produce, how well they capture meaning in specific domains and languages, and how much they cost to run.

OpenAI's documentation, for example, lists text-embedding-3-small with 1,536 dimensions by default and text-embedding-3-large with 3,072, and explains that developers can request shorter vectors through a dimensions parameter, trading a little accuracy for lower storage and faster search. That shortening works because of a training approach sometimes called Matryoshka representation learning (Kusupati et al.), named after Russian nesting dolls: the most important information is packed into the first numbers of the vector, so the vector can be truncated and still be useful.

Three cost lines matter in practice:

1. Creating embeddings. Providers usually charge per token of input text, and embedding is far cheaper per token than generating text with a chat model. Embedding a large help center once is typically a small line item; re-embedding everything every time you switch models is not.
2. Storing vectors. A 1,536-dimension vector stored as standard 32-bit numbers takes about 6 kilobytes. Ten million passages need roughly 60 gigabytes before any index overhead, which is why compression and shorter dimensions matter at scale.
3. Searching vectors. Every query must be embedded and then compared against the index, which costs compute and adds response time.

Benchmarks help narrow choices. The Massive Text Embedding Benchmark, or MTEB (Muennighoff et al.), launched in 2022 covering 8 task types across 58 datasets and 112 languages, and its public leaderboard is now the common starting point for comparing models. As with any benchmark, a high MTEB rank does not guarantee the best results on your documents, your jargon, or your customers' phrasing. Our guide to evaluating LLMs explains why your own small test set matters more than any public score.

How Vector Search Finds Matches Fast

Comparing a query against every stored vector, one by one, is called exact or brute-force search. It is perfectly accurate and perfectly fine for a few thousand passages. At millions or billions of passages, it becomes too slow, so production systems use approximate nearest neighbor, or ANN, search. ANN methods accept a small chance of missing the true closest match in exchange for answers that come back in milliseconds.

The most widely used ANN method is HNSW, short for Hierarchical Navigable Small World graphs (Malkov and Yashunin). It links each vector to its near neighbors in a layered graph. A search starts at a sparse top layer, takes long jumps toward the right region, then descends into denser layers to refine the result, a little like using an interstate to reach the right city before switching to local streets. Other approaches group vectors into clusters and search only the most promising clusters, or compress vectors so more fit in memory.

The software that stores vectors and runs these searches is often called a vector database. Options range from dedicated products to vector features added to systems teams already run, such as the open-source pgvector extension for PostgreSQL. For many B2B SaaS teams, adding vector search to an existing database is enough; a separate specialized system becomes worth it mainly at very large scale or with demanding latency requirements. The choice matters less than most vendor marketing suggests. As our RAG guide puts it, a well-organized small corpus with sensible search often beats a sophisticated index over messy documents.

Chunking: The Unglamorous Step That Decides Quality

Embedding models have input limits, and a single vector for a 40-page document blurs everything in it into one average meaning. So documents are split into chunks before embedding, typically a few hundred tokens each, often with some overlap between neighbors so a sentence is not cut off from the context it needs.

Chunking decisions have outsized effects:

• Too large, and a chunk about pricing, security, and onboarding at once will match many queries weakly and none of them precisely.
• Too small, and a chunk may lose the context that makes it meaningful, such as a sentence that says "it costs $49 per seat" without saying which plan "it" is.
• Split on structure where possible. Headings, sections, and question-and-answer pairs make natural chunk boundaries because each one tends to be about one thing.
• Carry context into each chunk. Many systems prepend the page title and section heading to every chunk so that a fragment remains understandable on its own.

This is why page structure matters for AI. A help article with a clear heading for each question and a self-contained answer under it chunks cleanly. A long, wandering page where the key fact appears only in the fourth paragraph of a section titled "Overview" does not. Our guide to tokens and context windows explains the related reason: AI systems usually work with selected fragments of your content, not the whole page.

Hybrid Search: Why Keywords Still Matter

Embeddings are excellent at paraphrase and intent. They are weaker at exact identifiers: product names, error codes, SKUs, version numbers, legal citations, and unusual brand names the embedding model rarely saw during training. A query for "error E-4012" may return passages about errors in general rather than the one page that mentions E-4012.

Traditional keyword ranking, most commonly an algorithm called BM25, has the opposite profile. It is precise about exact terms and blind to synonyms. That is why many production systems use hybrid search: run a keyword search and a vector search in parallel, then merge the results. A popular, simple merging method is reciprocal rank fusion, described by Cormack, Clarke, and Büttcher in 2009, which combines rankings by position rather than trying to reconcile incompatible scores.

Many systems then add a third stage, a reranker: a model that reads the query and each top candidate together and reorders them more precisely. Rerankers are slower than embedding comparison, so they are applied only to the top few dozen results. The overall pattern, broad and cheap retrieval followed by narrow and careful reranking, is how most serious search systems balance speed and quality.

For a business, the takeaway is practical. If your product has a distinctive name, write it consistently and in full, alongside a plain description of what it does. The keyword side of hybrid search will find the name; the embedding side will connect it to the problems buyers describe in their own words.

Where Embeddings Are Used Beyond Search

Semantic search is the best-known use, but the same vectors power a range of features:

• Retrieval-augmented generation. The retrieval step that finds passages for an LLM to answer from is very often an embedding search, sometimes combined with keywords.
• Recommendations. "Customers who read this also read" and "similar products" features can compare item vectors.
• Deduplication. Near-identical support tickets, duplicate leads, or repeated product listings sit very close together in embedding space.
• Clustering and topic discovery. Grouping thousands of customer comments by vector similarity reveals recurring themes without anyone writing categories in advance.
• Classification. Routing tickets or tagging documents by comparing them with labeled examples.
• Anomaly detection. Content far from every known cluster can flag something new or unusual.

Embedding models also extend beyond text. Multimodal embedding models place images and text in the same space, so a text query can retrieve a matching image, and some products apply the same idea to audio and video.

Common Failure Modes

Embeddings are powerful, and they fail in predictable ways. Knowing them prevents most surprises.

1. Similar is not the same as correct. Vector search returns passages that are about the same thing, not passages that answer the question. A query about canceling a plan may retrieve a page about upgrading a plan, because both are about plans and billing.
2. Negation and fine distinctions blur. "Supports single sign-on" and "does not support single sign-on" can embed very close together. Systems that need precision on such statements need rerankers, structured data, or explicit checks.
3. Stale indexes. If your pricing page changes but the stored vectors are not refreshed, search keeps returning the old text. Every embedding index needs a re-indexing plan.
4. Model changes break comparability. Vectors from one embedding model cannot be searched with queries embedded by another. Switching models means re-embedding the entire corpus.
5. Domain vocabulary gaps. General-purpose models may misplace specialized jargon, internal acronyms, or new product names. Test on your own terms before committing.
6. Bias inherited from training data. Research such as Bolukbasi et al. showed that word embeddings can encode social stereotypes present in their training text. Applications that rank people, such as candidate matching, deserve particular scrutiny.
7. Silent retrieval failure. When retrieval returns weak matches, the LLM on top may still write a confident answer from them. This is one of the main paths to the errors covered in our guide to LLM hallucinations.

The cure for most of these is the same discipline used for any AI system: a labeled set of real queries with the correct passages marked, measured before and after every change.

Privacy and Security: Vectors Are Not Anonymous

Teams sometimes assume that once text becomes a list of numbers, it is effectively anonymized. That assumption is wrong. Research titled "Text Embeddings Reveal (Almost) As Much As Text" (Morris et al., 2023) demonstrated a method that reconstructed 92 percent of 32-token text inputs exactly from their embeddings, and recovered personal information such as full names from clinical notes in the researchers' tests.

The practical rules follow directly:

• Treat vectors with the same sensitivity as the source text. If a document is confidential, its embeddings are confidential too.
• Apply access control at retrieval time. A search that ignores document permissions can surface restricted content to any user, and an LLM may then repeat it. Filtering must happen before passages are selected, not after an answer is written.
• Know where vectors go. Sending text to a third-party embedding service is a data transfer. Check the provider's retention and training terms for the specific plan you use.
• Plan for deletion. When a customer asks you to delete their data, their vectors and any cached chunks must be deleted too.

Our guide to LLM security and privacy covers the wider set of controls, including prompt injection through retrieved content.

What Embeddings Mean for AI Visibility

Here is where the mechanics meet marketing. When an AI search product answers a buyer's question with live web results, it must first find candidate passages. Google's documentation on AI features in Search explains that pages must be indexed and eligible to appear in Search to be shown as supporting links, and that the same foundational SEO practices apply. Providers rarely publish the full retrieval pipeline that follows, and approaches likely differ by engine and change over time. But the general pattern of modern retrieval, keyword and semantic matching over chunks of indexed content, followed by ranking and selection, has clear implications for how a B2B SaaS company should write.

1. Write the buyer's problem in the buyer's words. If customers search for "stop customers from leaving," a page that only says "proprietary lifecycle intelligence platform" is far from that query in meaning as well as in keywords. Plain problem statements put your content near the questions buyers actually ask.
2. Make each section about one thing. A section titled "Does it integrate with Salesforce?" with a direct answer underneath is easy to chunk and easy to match. A section that covers integrations, security, and pricing at once is a weak match for all three.
3. State key facts in self-contained sentences. Name the product, the plan, the number, and the date in the same sentence, so the fragment still makes sense when it is lifted out of the page.
4. Use your exact names consistently. Product names, plan names, and integration names should appear in full and the same way everywhere, which helps keyword retrieval and reduces confusion with similarly named products.
5. Keep facts current and dated. Retrieval only finds what is published and indexed. Outdated pages compete with current ones, and engines cannot tell which is right unless you make it obvious.
6. Be present where comparisons happen. For many buyer prompts, the passages retrieved come from third-party sources, such as roundups, documentation directories, and community discussions, not only your own site. Accurate, specific descriptions in those places give engines more correct material to work with.

None of this guarantees inclusion. Retrieval is only the first step, and each engine's ranking, model, and answer style vary. It does explain why vague positioning performs poorly in AI answers: content that does not clearly mean the thing a buyer is asking about is unlikely to be retrieved for that question. Our comparison of LLM SEO and traditional SEO covers how these practices fit with ordinary search work.

A 30-Day Plan for Teams Getting Started

Whether you are adding semantic search to your product or improving how your content performs in AI answers, the same first month works.

Week 1: Collect real queries. Pull 50 to 100 real questions from site search logs, support tickets, sales calls, and the prompts buyers use in AI assistants. Keep their original wording.

Week 2: Audit your content against them. For each question, find the page that should answer it. Mark gaps where no page answers it directly, and pages where the answer is buried, split across sections, or outdated.

Week 3: Restructure the top pages. Rewrite the ten most important pages so each question has a heading and a self-contained answer with names, numbers, and dates. If you run your own search, test chunk sizes and add hybrid search before switching embedding models.

Week 4: Measure and save the baseline. For internal search, measure how often the correct passage appears in the top three results. For AI visibility, run your buyer questions across the engines your buyers use, save the answers with dates, and repeat on a fixed schedule so changes are measured rather than guessed.

The Bottom Line

Embeddings turn meaning into numbers, which lets software find content that says the same thing in different words. Vector search makes that fast at scale, hybrid search adds back the precision of keywords, and rerankers sharpen the final selection. The technology is mature and widely available, but quality still comes from the basics: clean, current, well-structured content, sensible chunks, access control, and testing on real queries.

For brands, the lesson is direct. AI systems can only use what they can retrieve, and retrieval rewards content that plainly means what buyers are asking. If you want to see which buyer questions AI engines answer without your brand, and which passages they rely on instead, book a strategy call with LLM Recommend.

Frequently Asked Questions

What is an embedding in simple terms?

An embedding is a list of numbers that represents the meaning of a piece of content, such as a sentence or paragraph. Content with similar meaning produces lists that sit close together, so software can find related text even when it shares no exact words.

What is the difference between vector search and keyword search?

Keyword search finds documents containing the same words and is precise about names, codes, and identifiers. Vector search finds documents with similar meaning and handles paraphrase and intent. Many production systems combine both in hybrid search, then rerank the top results.

Do I need a vector database?

Not always. Small collections can be searched exactly without special software, and many teams add vector search to a database they already run, such as PostgreSQL with pgvector. Dedicated systems mainly pay off at very large scale or with strict response-time needs.

Which embedding model should I use?

Use public benchmarks such as MTEB to build a shortlist, then test candidates on 50 to 100 of your own real queries with the correct passages marked. Weigh accuracy against cost, vector size, language coverage, and the work of re-embedding everything if you switch later.

Are embeddings safe to store if they are just numbers?

No. Research has shown that much of the original text can be reconstructed from embeddings, including personal details. Treat vectors with the same sensitivity as the source text, enforce access control at retrieval time, and delete vectors when the source data is deleted.

How do embeddings affect my brand's visibility in AI answers?

AI systems that answer from the web must first retrieve passages, and modern retrieval commonly combines keyword and semantic matching over chunks of indexed content. Pages that state the buyer's problem plainly, keep one topic per section, and give self-contained, dated facts are easier to retrieve for the questions buyers actually ask. Retrieval does not guarantee inclusion.