RAG & Grounding

RAG & Grounding on Google Cloud is the pattern of retrieving relevant, attributable context from managed search, vector, graph, database, web, URL, or memory systems and using it to constrain Gemini or agent responses. The goal is to make responses more accurate, current, domain-specific, and inspectable through retrieved context, citations, grounding metadata, and evaluation.

Synthesized across 67 notebooks. See the source pages under Notebooks below for line-level grounding.

This area starts with content preparation: extract documents or records, split them into useful chunks, preserve metadata such as source URI, page number, chunk number, file ID, entity, or graph relationship, and generate embeddings with task-appropriate models. The retrieved context can come from Vertex AI RAG Engine, Vertex AI Search, Agent Platform Vector Search, Cloud SQL or AlloyDB with pgvector, BigQuery, Firestore, Feature Store, Pinecone, Weaviate, Spanner Graph, Neo4j, URL context, Google Search, Enterprise Web Search, Maps, or Memory Bank.

A typical grounded Gemini application retrieves context before generation, injects only the relevant facts into the prompt or passes a RAG retrieval tool to generate_content, and instructs the model to answer only from the provided sources. Grounding metadata, source documents, citations, logprobs, and retrieved context inspection are used to debug whether the answer is actually supported.

Production-quality RAG is less about simply adding a vector index and more about retrieval quality, operational control, and evaluation. The notebooks repeatedly emphasize chunking and overlap tuning, embedding task types, top_k and distance thresholds, quota-aware batching and retries, IAM setup, cleanup of billable resources, prompt-injection defenses, and groundedness or retrieval metrics such as recall, precision, nDCG, MRR, and custom EvalTask rubrics.

Canonical workflow

1. Enable the required Google Cloud APIs, configure authentication, project, region, billing, IAM roles, and any required storage or retrieval backend.
2. Ingest source material from Cloud Storage, PDFs, GitHub, BigQuery, Firestore, Cloud SQL, Vertex AI Search, graph databases, URLs, web search, or Memory Bank, while preserving source metadata.
3. Parse and chunk documents deliberately, including chunk_size, chunk_overlap, page numbers, file IDs, source URIs, and domain-specific extraction or parsing instructions where needed.
4. Generate embeddings with a consistent model and task type, then store vectors and metadata in a managed retrieval backend such as Vertex AI RAG Engine, Vector Search, Vertex AI Search, Cloud SQL pgvector, Feature Store, Pinecone, Weaviate, Firestore, Spanner, or Neo4j.
5. Retrieve candidate context using semantic, hybrid, graph, URL, web, memory, or search tools, tuning top_k, similarity_top_k, vector_distance_threshold, filters, and task-type embeddings for the use case.
6. Ground Gemini or an agent by injecting retrieved context into the prompt or passing an explicit retrieval or search tool, and instruct the model to answer only from retrieved sources with citations where appropriate.
7. Inspect retrieved contexts, grounding metadata, grounding chunks, grounding supports, source documents, logprobs, and citation fields to debug answer support and traceability.
8. Evaluate retrieval and answer quality with row-level and aggregate metrics, custom groundedness rubrics, recall, precision, nDCG, MRR, reference-free or referenced evaluation, then clean up billable resources.

Best practices

See RAG & Grounding - Best Practices for the full distilled guide.

• Use managed retrieval services such as Vertex AI RAG Engine, Vertex AI Search, Agent Platform Vector Search, or managed database vector stores when scalability, latency, IAM, and infrastructure management matter.
• Use the same embedding model and compatible dimensionality for document chunks, indexed vectors, and query embeddings; verify vector dimensions and distance measures match the backend.
• Choose embedding task types deliberately, such as RETRIEVAL_DOCUMENT for indexed documents and QUESTION_ANSWERING or RETRIEVAL_QUERY for user questions, instead of treating all embeddings as generic similarity embeddings.
• Configure chunk_size, chunk_overlap, top_k, similarity_top_k, vector_distance_threshold, and filters based on retrieval metrics rather than defaults.
• Preserve source metadata such as page number, chunk number, file path, URI, document ID, graph entity, or table key so generated answers can cite and debug their support.
• Inspect retrieval results directly before generation with tools such as rag.retrieval_query, search requests, source documents, grounding_chunks, grounding_supports, and url_context_metadata.

Key models & APIs

• Gemini 3.5 Flash, Gemini 2.5 Flash, Gemini 2.0 Flash, Gemini 3.1 Pro, Gemini 2.0 Flash Live Preview, gemini-embedding-001, gemini-embedding-2, text-embedding-005, Vertex AI RAG Engine, Vertex AI Gen AI SDK and Google Gen AI SDK, Vertex AI Agent Engine and ADK, Agent Engine Memory Bank, Agent Platform Vector Search and Vector Search 2.0, Vertex AI Search and Discovery Engine API, Enterprise Web Search grounding, Google Search grounding, Google Maps grounding, URL Context, Vertex AI Gen AI Evaluation Service, EvalTask, Rapid Evaluation API, and Dataflow ML, BigQuery AI.GENERATE, AI.GENERATE_TABLE, ObjectRefs, and remote models, Cloud SQL for PostgreSQL and pgvector, Vertex AI Feature Store, Firestore, Spanner Graph, Neo4j GraphRAG, Pinecone and Weaviate integrations, LangChain, LlamaIndex, LangGraph, Document AI, Cloud Storage, Secret Manager

Notebooks (67)

Agents & ADK (1)

• Get started with Memory Bank on ADK — Build ADK agents that generate, retrieve, preload, and customize Agent Engine Memory Bank memories.

Embeddings & Vector Search (5)

• Getting Started with Text Embeddings + Agent Platform Vector Search — Builds text embeddings from Stack Overflow titles and serves semantic search with Vector Search.
• Get started with embeddings tuning on Agent Platform — Tunes text-embedding-005 for retrieval using synthetic Gemini queries and Document AI PDF chunks.
• Introduction to Gemini Multimodal Embeddings — Generates Gemini multimodal embeddings and applies them to search, similarity, PDFs, and RAG.
• Using “task type” embeddings for improving RAG search quality — Shows how task-type embeddings improve RAG Q&A retrieval quality and MRR.
• Introduction to Agent Platform Vector Search 2.0 — Builds an e-commerce product search demo with Agent Platform Vector Search 2.0 and auto-embeddings.

Gemini (47)

• Get started with Vertex AI Memory Bank - LangGraph — Builds a LangGraph chatbot with Vertex AI Memory Bank for long-term personalized memory.
• Building a Multi-Agent RAG Application with LangGraph and Agent Engine — Builds and deploys a LangGraph multi-agent RAG app on Vertex AI Agent Engine with Cloud SQL vector stores.
• Building a Conversational Search Agent with Agent Engine and RAG on Vertex AI Search — Builds and deploys a Gemini movie-search RAG agent with LangChain, Agent Engine, and Vertex AI Search.
• Function Calling Agent — Builds a Gemini function-calling retail assistant over Cymbal Retail data in BigQuery.
• Task Planner Agent — Builds a LangGraph task-planner agent that plans, searches with Gemini, reflects, and responds.
• Introduction to Gemini Deep Research Agent — Shows how to run Gemini Deep Research Agent with streaming, multimodal input/output, and grounding tools.
• Evaluate groundedness with custom parsing — Evaluates Gemini response groundedness with Vertex AI EvalTask and custom JSON parsing.
• Evaluate generated answers from Retrieval-Augmented Generation (RAG) using Rapid Evaluation and Dataflow ML with Vertex AI pipelines — Builds a Vertex AI Pipeline to batch-evaluate RAG Q&A outputs with Rapid Eval API and Dataflow ML.
• Evaluate Generated Answers from Retrieval-Augmented Generation (RAG) for Question Answering with Gen AI Evaluation Service SDK — Evaluates BYO RAG QA answers with Vertex AI Gen AI Evaluation, custom metrics, and result visualizations.
• Gen AI Evaluation Service SDK Preview-to-GA Migration Guide — Migrates Vertex AI Gen AI Evaluation SDK preview patterns to GA EvalTask, pointwise, and pairwise metrics.
• Gemini 3.1 Flash Image (Nano Banana 2 🍌) Generation — Shows Gemini 3.1 Flash Image generation, grounding, video input, and image editing with Google Gen AI SDK.
• Gemini 3 Pro Image (Nano Banana Pro 🍌) Generation — Shows how to generate and edit images with Gemini 3 Pro Image using the Google GenAI SDK.
• Grounding with Vertex AI Search — Creates a Vertex AI Search engine and uses it to ground a Gemini response with retrieved context.
• Intro to Grounding with Gemini in Vertex AI — Shows how to ground Gemini 3.5 Flash responses with Search, Enterprise Web Search, Maps, and Vertex AI Search.
• Intro to Logprobs — Introduces Gemini logprobs on Vertex AI for confidence, autocomplete, and RAG grounding analysis.
• Interactive Loan Application Assistant (Financial Services) — Builds a Gemini 2.0 loan document assistant with RAG, large context, audio, Vertex AI Search, and Vector Search.
• Real-time Retrieval Augmented Generation (RAG) using the Multimodal Live API with Gemini 2.0 — Builds a retail RAG pipeline with Gemini Multimodal Live API for grounded text and audio answers.
• Getting Started with LangChain 🦜️🔗 + Gemini API in Vertex AI — Introduces LangChain with Gemini on Vertex AI for prompts, embeddings, retrieval, memory, and chains.
• LlamaIndex RAG Workflows using Gemini and Firestore — Builds a LlamaIndex RAG workflow with Gemini, Vertex embeddings, and Firestore storage.
• Intro to Building a Scalable and Modular RAG System with RAG Engine in Vertex AI — Builds a Vertex AI RAG Engine corpus, imports files, retrieves context, and grounds Gemini or Llama responses.
• Vertex AI Rag: Cross-Corpus Retrieval with AskContexts and AsyncRetrieveContexts Demo — Demonstrates cross-corpus Vertex AI RAG retrieval with ask_contexts and async_retrieve_contexts.
• Evaluating Vertex RAG Engine Generation with Vertex AI Python SDK for Gen AI Evaluation Service — Evaluates Vertex AI RAG Engine responses with a custom Gen AI Evaluation Service metric.
• Advanced RAG Techniques - Vertex RAG Engine Retrieval Quality Evaluation and Hyperparameters Tuning — Evaluates Vertex AI RAG Engine retrieval quality and tunes chunking, top-k, threshold, and embedding settings.
• Vertex AI RAG Engine with Vertex AI Feature Store — Builds a Vertex AI RAG Engine corpus backed by Vertex AI Feature Store and queries it with Gemini.
• Vertex AI RAG Engine with Pinecone — Shows how to use Vertex AI RAG Engine with Pinecone as the vector database for Gemini retrieval.
• Vertex AI RAG Engine with Vertex AI Vector Search — Builds a Vertex AI RAG Engine corpus backed by Vertex AI Vector Search and queries it with Gemini.
• Vertex AI RAG Engine with Vertex AI Search — Build a Vertex AI RAG Engine corpus backed by Vertex AI Search and query it with Gemini.
• Vertex AI RAG Engine with Weaviate — Creates a Vertex AI RAG Engine corpus backed by Weaviate and queries it with Gemini.
• Gen AI and LLM Security - ReAct and RAG attacks & mitigations — Demonstrates ReAct and RAG prompt-injection attacks with Gemini and simple mitigations.
• Intro to Url Context — Shows how to use Gemini URL context to summarize, compare, and analyze web pages with optional Google Search grounding.
• Analyze Multimodal Data in BigQuery — Shows BigQuery multimodal analysis over structured tables and GCS media using ObjectRefs and Gemini.
• Use Retrieval Augmented Generation (RAG) with Gemini API — Builds a LangChain RAG pipeline over GitHub code notebooks using Gemini and Vertex AI embeddings.
• Comparing LlamaIndex and LlamaParse for Dense Document Questioning Answering on Vertex AI — Compares LlamaIndex and LlamaParse RAG parsing methods for dense 10-Q document QA on Vertex AI.
• 🛡️ Agentic GraphRAG: Cybersecurity Threat Intelligence — Builds and deploys an ADK GraphRAG threat-intel agent using Neo4j and Vertex AI Agent Engine.
• GraphRAG on Google Cloud With Spanner and Vertex AI Agent Engine — Builds a GraphRAG Q&A agent with Spanner Graph, Gemini, ADK, and Vertex AI Agent Engine.
• 🌿 Eco-Nomad Swarm: 100% Real-Data Sustainable Travel Orchestration — Builds a Vertex AI ADK travel swarm using Neo4j GraphRAG and live APIs for sustainable trip briefs.
• Know Your Customer Use Case - Gemini Grounding with Google Search — Builds grounded Gemini KYC negative-news reports and evaluates response quality with custom Vertex AI metrics.
• Retail AI Location Strategy: Autonomous Site Selection & Market Analysis — Builds a Gemini 3 retail site-selection pipeline using search, Maps, code execution, reasoning, and JSON output.
• RAG Based on Sensitive Data Protection using Faker — Builds a RAG flow that anonymizes PII with Cloud DLP, Faker, Firestore, Chroma, and Gemini.
• Intra Knowledge QnA — Builds a Vertex AI and LangChain RAG Q&A app over an IRS PDF using Chroma embeddings.
• Leverage LlamaIndex with Vertex AI Vector Search to perform question answering RAG — Builds LlamaIndex RAG on Vertex AI Vector Search, compares prompts, and adds multi-document agents.
• Retrieval Augmented Generation(RAG) with AlloyDB — Builds a RAG workflow over patent abstracts using AlloyDB vector search and Gemini on Vertex AI.
• Building a Gen AI RAG application with Vertex AI Feature Store and BigQuery — Builds a LangChain RAG Q&A app using BigQuery Vector Search and Vertex AI Feature Store.
• Augment Gemini Output with Vector Embeddings from BigQuery — Builds BigQuery vector-search RAG over patent abstracts and uses Gemini to generate project ideas.
• Run RAG Pipelines in BigQuery with BQML and Vector Search — Builds a BigQuery RAG pipeline over a PDF using Document AI, embeddings, vector search, and Gemini.
• Building a Multimodal Chatbot for Warranty Claims using Gemini and Vector Search in Vertex AI — Builds a multimodal warranty-claims chatbot with Gemini, RAG, Vector Search, and function calling.
• Production & Scalable RAG Pipeline Using BigFrames — Builds a scalable BigFrames RAG pipeline over Stack Overflow data with BigQuery, Vertex AI, and LangChain.

Open Models (2)

• Running a Gemma 2-based agentic RAG with Ollama on Vertex AI and LangGraph — Deploys a Gemma 2 Ollama container on Vertex AI and uses it in a LangGraph SQL RAG agent.
• Cloud Run GPU Inference: Gemma 2 RAG Q&A with Ollama and LangChain — Deploys Gemma 2 on Cloud Run GPU with Ollama and builds a LangChain RAG Q&A chain.

Vertex AI Search (12)

• Create a Vertex AI Datastore and Search Engine — Creates a Vertex AI Search datastore, imports GCS PDFs, creates an enterprise search engine, and queries it.
• Custom Embeddings with Vertex AI Search — Builds a Vertex AI Search app using custom text-embedding-005 embeddings from Stack Overflow data.
• Intro to Gemini Enterprise — Shows how to call Gemini Enterprise search and answer APIs with the Discovery Engine Python SDK.
• Search tuning in Vertex AI Search — Tunes Vertex AI Search with JSONL/TSV Q&A data and tests a search app over Cloud Storage PDFs.
• Defining custom attributes based on URL patterns in Vertex AI Search Website Datastores — Shows how to add URL-pattern custom attributes to a Vertex AI Search website datastore.
• Ingestion of Unstructured Documents with Metadata in Vertex AI Search — Ingests PDFs with metadata into Vertex AI Search and queries results with metadata filters.
• Inline Ingestion of Documents into Vertex AI Search — Shows inline rawBytes document ingestion, document operations, search, filters, and cleanup in Vertex AI Search.
• Parsing and Chunking in Vertex AI Search: Featuring BYO Capabilities — Retrieves, reviews, exports, and reimports Vertex AI Search parsed and chunked documents with BYOC.
• Query-Level Boosting, Filtering, and Facets for Vertex AI Search Website Datastores — Configures Vertex AI Search website datastore schema, filters, facets, and boosting via REST.
• Recording Real-Time User Events in Vertex AI Search Datastores — Records real-time search and view-item user events for a Vertex AI Search website datastore.
• Q&A Chatbot with Vertex AI Search for summarized website results without advanced indexing — Builds a Q&A flow that searches a Vertex AI Search website data store, fetches the top page, and summarizes it with Gemini.
• Building Search Applications with Vertex AI Search — Builds Vertex AI Search workflows using Search API, Gemini grounding, and LangChain retrieval.

Related concepts

• Embeddings & Vector Search · Applied Use Cases · Vertex AI Search · Agents & ADK · Evaluation · Getting Started · Gemini Capabilities · Function Calling & Tools

---

Part of the Generative AI Google Brain · RAG & Grounding - Best Practices