Embeddings & Vector Search

Embeddings & Vector Search on Google Cloud is the practice of converting text, images, audio, video, logs, documents, or structured records into vector representations and retrieving semantically similar items with managed services such as Vertex AI Vector Search, Vector Search 2.0, BigQuery VECTOR_SEARCH, RAG Engine, Feature Store, AlloyDB, or supported vector stores. It underpins semantic search, hybrid search, RAG grounding, anomaly detection, deduplication, memory retrieval, and multimodal similarity workflows.

Synthesized across 41 notebooks. See the source pages under Notebooks below for line-level grounding.

This area starts with choosing an embedding model and task type that matches the workload, then preparing source data so each embedded item has enough context, clean text or media, stable IDs, and useful metadata. The same embedding model and compatible dimensionality must be used for indexed items and query vectors.

Google Cloud offers several retrieval patterns: Vertex AI Vector Search for managed low-latency nearest-neighbor serving, Vector Search 2.0 collections with schemas and optional auto-embeddings, BigQuery VECTOR_SEARCH for analytics-native workflows, RAG Engine for managed corpus retrieval, and database-backed options such as AlloyDB, Feature Store, Firestore, Weaviate, Chroma, or FAISS depending on scale and production needs.

High-quality systems require more than embedding and indexing: they need chunking, filtering, batching, quota handling, evaluation, threshold tuning, source citations, cleanup of billable resources, and careful handling of sensitive data before embeddings are generated.

Canonical workflow

1. Enable the required APIs, set project and region explicitly, authenticate, and grant service accounts the IAM roles needed for Vertex AI, BigQuery, Cloud Storage, Document AI, or the selected vector store.
2. Prepare and clean source data by filtering empty values, normalizing fields, chunking documents, extracting metadata, anonymizing sensitive data when needed, and storing assets such as PDFs, images, or videos in Cloud Storage when required.
3. Generate embeddings with the appropriate model, task type, dimensionality, batching, retry, throttling, and checkpointing strategy; persist embeddings and metadata for reuse.
4. Create or configure the vector backend, matching index dimensions, distance measure, update mode, schema, filters, and dense or sparse vector fields to the retrieval use case.
5. Import, upsert, or sync vectors in batches, then poll long-running operations and verify index readiness, coverage, deployment status, or Feature Store sync completion before querying.
6. Generate query embeddings with the same compatible model and task type, apply filters consistently, retrieve top-k neighbors, and order results by distance or service-specific ranking.
7. For RAG or agent workflows, pass retrieved chunks with source metadata into Gemini or the orchestration framework, require answers to stay grounded in retrieved context, and include citations where applicable.
8. Evaluate retrieval quality with metrics, plots, audits, thresholds, or source inspection, then tune chunking, top-k, distance thresholds, task types, hybrid weights, and cleanup unused billable resources.

Best practices

See Embeddings & Vector Search - Best Practices for the full distilled guide.

• Use the same embedding model, dimensionality, and compatible distance measure for stored vectors and query vectors.
• Choose task types deliberately, such as RETRIEVAL_DOCUMENT for indexed content, QUESTION_ANSWERING or RETRIEVAL_QUERY for questions, SEMANTIC_SIMILARITY for similarity baselines, and CLUSTERING for clustering-style anomaly detection.
• Match index dimensions to embedding output dimensionality; specify output_dimensionality when lower storage cost or faster search is acceptable.
• Batch embedding, import, upsert, update, delete, and search operations where supported, while respecting per-request limits such as maximum texts, tokens, or datapoints.
• Throttle embedding and generation traffic, use retries with exponential backoff, checkpoint long-running embedding jobs, and log per-item errors so large jobs can resume.
• Clean, normalize, and enrich source data before embedding, including removing null or blank text, combining relevant fields, converting HTML to Markdown, normalizing log fields, and preserving metadata such as source URI, page, chunk, and document name.

Key models & APIs

• gemini-embedding-001, text-embedding-005, text-embedding-004, gemini-embedding-2, gemini-embedding-2-preview, multimodalembedding@001, publishers/google/models/text-embedding-005, publishers/intfloat/models/e5@multilingual-e5-small, google/text-embedding-005, google/siglip2-base-patch16-512, gemini-2.0-flash, gemini-2.5-flash, gemini-3.5-flash, gemini-2.0-flash-live-preview-04-09, Vertex AI Embeddings API, Vertex AI Vector Search, Agent Platform Vector Search, Vector Search 2.0, BigQuery ML ML.GENERATE_EMBEDDING, BigQuery VECTOR_SEARCH, BigQuery VECTOR INDEX, Vertex AI RAG Engine, VertexRagStore, rag.retrieval_query, Vertex AI Feature Store, BigQueryVectorStore, VertexFSVectorStore, AlloyDB vector search, Cloud SQL or AlloyDB pgvector-compatible vector search, Document AI, Cloud DLP, Vertex AI Search, LlamaIndex, LangChain, Chroma, FAISS, Weaviate, Firestore

Notebooks (41)

Agents & ADK (2)

• Get started with Vertex AI Memory Bank — Builds a Vertex AI Memory Bank hotel concierge that stores and retrieves guest preferences across sessions.
• Intro to Skill Registry — Introduces Skill Registry for creating, ingesting, registering, and semantically retrieving agent skills.

Embeddings & Vector Search (15)

• 🛡️ AI Brand Safety: Three-Tier Agent Anomaly Detection — Builds ADK agent anomaly detection with Gemini baselines, Vector Search scoring, and tiered audits.
• Import from BigQuery into Vector Search — Imports BigQuery embedding rows into a Vertex AI Vector Search index using the import REST API.
• Use Gemini and OSS Text-Embedding Models Against Your BigQuery Data — Generate BigQuery embeddings with Gemini and a deployed OSS E5 model.
• Visualizing embedding similarity from text documents using t-SNE plots — Embeds 20 Newsgroups text with gemini-embedding-001 and visualizes similarity clusters using t-SNE.
• Combining Semantic & Keyword Search: A Hybrid Search Tutorial with Agent Platform Vector Search — Builds sparse and hybrid Vector Search indexes for Google Merch Shop product search.
• Getting Started with Text Embeddings + Agent Platform Vector Search — Builds text embeddings from Stack Overflow titles and serves semantic search with Vector Search.
• Get started with embeddings tuning on Agent Platform — Tunes text-embedding-005 for retrieval using synthetic Gemini queries and Document AI PDF chunks.
• Introduction to Gemini Multimodal Embeddings — Generates Gemini multimodal embeddings and applies them to search, similarity, PDFs, and RAG.
• Handling large-scale embedding generation for Agent Platform Vector Search — Generates large-scale text and image embeddings for Agent Platform Vector Search with throttling and checkpointing.
• Using “task type” embeddings for improving RAG search quality — Shows how task-type embeddings improve RAG Q&A retrieval quality and MRR.
• Log Anomaly Detection & Investigation with Text Embeddings + BigQuery Vector Search — Detects audit log anomalies using text-embedding-005 embeddings and BigQuery VECTOR_SEARCH.
• Anomaly Detection of Infrastructure Logs using Gemini and BigQuery Vector Search — Detects HDFS log anomalies with Gemini summaries, text embeddings, and BigQuery Vector Search.
• Introduction to Agent Platform Vector Search 2.0 — Builds an e-commerce product search demo with Agent Platform Vector Search 2.0 and auto-embeddings.
• Vector Search 2.0 Public Preview Quickstart — Creates a Vector Search 2.0 movie collection with CRUD, filters, semantic search, and ANN indexes.
• Agent Platform Vector Search Quickstart — Builds and queries a streaming-update Vector Search index from product embeddings.

Gemini (23)

• Interactive Loan Application Assistant (Financial Services) — Builds a Gemini 2.0 loan document assistant with RAG, large context, audio, Vertex AI Search, and Vector Search.
• LlamaIndex RAG Workflows using Gemini and Firestore — Builds a LlamaIndex RAG workflow with Gemini, Vertex embeddings, and Firestore storage.
• Intro to Building a Scalable and Modular RAG System with RAG Engine in Vertex AI — Builds a Vertex AI RAG Engine corpus, imports files, retrieves context, and grounds Gemini or Llama responses.
• Advanced RAG Techniques - Vertex RAG Engine Retrieval Quality Evaluation and Hyperparameters Tuning — Evaluates Vertex AI RAG Engine retrieval quality and tunes chunking, top-k, threshold, and embedding settings.
• Vertex AI RAG Engine with Vertex AI Feature Store — Builds a Vertex AI RAG Engine corpus backed by Vertex AI Feature Store and queries it with Gemini.
• Vertex AI RAG Engine with Vertex AI Vector Search — Builds a Vertex AI RAG Engine corpus backed by Vertex AI Vector Search and queries it with Gemini.
• Vertex AI RAG Engine with Weaviate — Creates a Vertex AI RAG Engine corpus backed by Weaviate and queries it with Gemini.
• Gen AI & LLM Security for developers — Shows prompt injection attacks against Gemini and layered mitigations with DLP, NL API, safety filters, and embeddings.
• AI-Assisted Data Science Workflows in BigQuery — Builds a BigQuery multimodal housing workflow with Gemini enrichment, BQML clustering, and vector search.
• Analyzing movie posters in BigQuery with Gemini — Analyzes movie poster images in BigQuery with Gemini, embeddings, and vector search.
• Text + multimodal embedding generation and vector search in BigQuery — Builds text and image embeddings in BigQuery for semantic product search.
• Performing Semantic Search in BigQuery — Builds semantic search over Stack Overflow questions in BigQuery with Vertex AI text embeddings.
• Use Retrieval Augmented Generation (RAG) with Gemini API — Builds a LangChain RAG pipeline over GitHub code notebooks using Gemini and Vertex AI embeddings.
• Comparing LlamaIndex and LlamaParse for Dense Document Questioning Answering on Vertex AI — Compares LlamaIndex and LlamaParse RAG parsing methods for dense 10-Q document QA on Vertex AI.
• Data Curation Pipeline: Splitting and Transcoding — Deduplicates video clips using video embeddings and BigQuery vector search.
• RAG Based on Sensitive Data Protection using Faker — Builds a RAG flow that anonymizes PII with Cloud DLP, Faker, Firestore, Chroma, and Gemini.
• Intra Knowledge QnA — Builds a Vertex AI and LangChain RAG Q&A app over an IRS PDF using Chroma embeddings.
• Leverage LlamaIndex with Vertex AI Vector Search to perform question answering RAG — Builds LlamaIndex RAG on Vertex AI Vector Search, compares prompts, and adds multi-document agents.
• Retrieval Augmented Generation(RAG) with AlloyDB — Builds a RAG workflow over patent abstracts using AlloyDB vector search and Gemini on Vertex AI.
• Building a Gen AI RAG application with Vertex AI Feature Store and BigQuery — Builds a LangChain RAG Q&A app using BigQuery Vector Search and Vertex AI Feature Store.
• Augment Gemini Output with Vector Embeddings from BigQuery — Builds BigQuery vector-search RAG over patent abstracts and uses Gemini to generate project ideas.
• Run RAG Pipelines in BigQuery with BQML and Vector Search — Builds a BigQuery RAG pipeline over a PDF using Document AI, embeddings, vector search, and Gemini.
• Production & Scalable RAG Pipeline Using BigFrames — Builds a scalable BigFrames RAG pipeline over Stack Overflow data with BigQuery, Vertex AI, and LangChain.

Vertex AI Search (1)

• Custom Embeddings with Vertex AI Search — Builds a Vertex AI Search app using custom text-embedding-005 embeddings from Stack Overflow data.

Related concepts

• RAG & Grounding · Applied Use Cases · Gemini Capabilities · Agents & ADK · Getting Started · Evaluation · Vision · Prompt Engineering

---

Part of the Generative AI Google Brain · Embeddings & Vector Search - Best Practices