Generative AI Google Brain

RAG & Grounding - Best Practices

25 min read

RAG & Grounding — Best Practices

Distilled from 67 notebooks tagged RAG & Grounding in the GoogleCloudPlatform/generative-ai repository. The From the notebooks section below cites the per-notebook source for grounding.

Do this

  • Use managed retrieval services such as Vertex AI RAG Engine, Vertex AI Search, Agent Platform Vector Search, or managed database vector stores when scalability, latency, IAM, and infrastructure management matter.
  • Use the same embedding model and compatible dimensionality for document chunks, indexed vectors, and query embeddings; verify vector dimensions and distance measures match the backend.
  • Choose embedding task types deliberately, such as RETRIEVAL_DOCUMENT for indexed documents and QUESTION_ANSWERING or RETRIEVAL_QUERY for user questions, instead of treating all embeddings as generic similarity embeddings.
  • Configure chunk_size, chunk_overlap, top_k, similarity_top_k, vector_distance_threshold, and filters based on retrieval metrics rather than defaults.
  • Preserve source metadata such as page number, chunk number, file path, URI, document ID, graph entity, or table key so generated answers can cite and debug their support.
  • Inspect retrieval results directly before generation with tools such as rag.retrieval_query, search requests, source documents, grounding_chunks, grounding_supports, and url_context_metadata.
  • Prompt the model to answer only from retrieved context when building grounded QA, and display citations or grounding metadata when the API returns them.
  • Test retrieval functions and agents locally before deploying to Agent Engine, and pin deployment requirements to the same versions used in development notebooks.
  • Use batch embedding and bulk import APIs, respect model request limits, and add exponential backoff or pacing for RESOURCE_EXHAUSTED and transient GenAI API errors.
  • Evaluate both retrieval and generation quality with metrics such as recall@k, precision@k, nDCG@k, MRR, groundedness, accuracy, completeness, reference-free evaluation, and referenced evaluation when golden answers exist.
  • Use structured schemas, Pydantic models, function declarations, and query parameters when grounding with tools or databases instead of allowing unconstrained SQL or free-form tool input.
  • For internal or regulated content, prefer Vertex AI Search, Enterprise Web Search, URL context, or private retrieval backends over public web grounding when privacy, logging, or residency requirements matter.
  • Scope long-term memory by user or application context, configure TTL, and store only meaningful extracted memories rather than indiscriminately persisting full conversations.
  • Apply defense in depth against RAG and ReAct prompt injection, including strict tool input and output validation, OCR for hidden text when needed, low temperature for reproducibility, and user consent for dangerous operations.
  • Clean up Agent Engine apps, RAG corpora, Vector Search indexes and endpoints, Cloud SQL instances, Feature Store resources, BigQuery datasets, Cloud Storage buckets, and search engines after tutorials or experiments to avoid unexpected costs.

Avoid this

  • Assuming retrieved context is available immediately after ingestion, indexing, memory generation, FeatureView sync, or Vector Search deployment; many notebooks note delays from seconds to tens of minutes.
  • Using mismatched embedding dimensions, distance measures, task types, or models between indexed documents and query embeddings.
  • Creating only a Vertex AI Search data store without the required search app or enabling required Enterprise and generative features for the grounding workflow.
  • Relying on pretrained model knowledge instead of passing retrieved context, or failing to instruct the model to answer only from provided sources.
  • Ignoring quotas and batching limits, which can cause embedding or generation calls to fail with rate-limit or RESOURCE_EXHAUSTED errors.
  • Leaving billable resources running, including deployed agents, search engines, indexes, index endpoints, Cloud SQL instances, buckets, corpora, Workbench instances, and pipeline jobs.
  • Changing Memory Bank scope keys or user_id scoping and then assuming previous memories should still be visible.
  • Treating demo parsers, regex ReAct logic, preview APIs, experimental live APIs, or simple logprob heuristics as production-ready implementations.

From the notebooks

Get started with Memory Bank on ADK

  • Use add_events_to_memory for production agents to stream recent events incrementally.
  • Use add_session_to_memory at the end of a session when processing the whole session is acceptable.
  • Provide both a memory tool on the Agent and a memory service on the Runner when using built-in ADK memory tools.
  • Use callbacks to automate memory generation after turns.
  • Use Agent Engine SDK generate with wait_for_completion when blocking memory generation is required.

Getting Started with Text Embeddings + Agent Platform Vector Search

  • Limit tutorial data size from the 23 million row Stack Overflow table before loading into memory.
  • Batch embedding requests and add delay to avoid quota errors.
  • Use dot product for similarity with the Google embedding model shown.
  • Store Vector Search input as JSONL with id and embedding fields.
  • Match index dimensions to the embedding output dimensionality.

Get started with embeddings tuning on Agent Platform

  • Prepare corpus, query, and labels files in the format required by embedding tuning.
  • Split labels into train and test sets before running the tuning pipeline.
  • Use a timestamped Cloud Storage path for tuning artifacts.
  • Evaluate tuned embeddings with pipeline-produced NDCG metrics.
  • Create cleanup flags for endpoint, model, job, bucket objects, and local tutorial files.

Introduction to Gemini Multimodal Embeddings

  • Embed multiple text prompts in one API call for efficiency.
  • Use output_dimensionality to reduce storage cost and improve search speed when lower dimensions are acceptable.
  • Choose input structure deliberately: one Content object aggregates parts into one embedding, while multiple contents return separate embeddings.
  • Create post-level multimodal representations by aggregating separate embeddings, for example by averaging.
  • Truncate PDFs before embedding when they exceed the six-page limit.

Using “task type” embeddings for improving RAG search quality

  • Use QUESTION_ANSWERING for question texts and RETRIEVAL_DOCUMENT for answer documents in Q&A RAG.
  • Use SEMANTIC_SIMILARITY as a baseline when comparing retrieval quality.
  • Evaluate search quality with ranking metrics such as Mean Reciprocal Rank.
  • Batch multiple texts per embedding API call to reduce repeated calls.
  • Consider tuning text embeddings when pre-trained task-type embeddings do not fit proprietary or specialized content.

Introduction to Agent Platform Vector Search 2.0

  • Use a schema-enforced collection with separate data_schema and vector_schema.
  • Use auto-embeddings by leaving vectors empty when vertex_embedding_config is configured.
  • Use random.seed(42) for reproducible sampling of the product dataset.
  • Use batch_create_data_objects instead of one create request per product for bulk imports.
  • Use task_type=“RETRIEVAL_DOCUMENT” for indexed product names and task_type=“QUESTION_ANSWERING” for semantic queries.

Get started with Vertex AI Memory Bank - LangGraph

  • Scope memories by user_id so retrieved facts are user-specific.
  • Retrieve memories before generation and inject them into the system prompt.
  • Use semantic search with top_k to limit retrieved memories to relevant facts.
  • Store both user and model messages after each turn to continuously build memory.
  • Use wait_for_completion when generating memories so persistence completes before continuing.

Building a Multi-Agent RAG Application with LangGraph and Agent Engine

  • Pin deployment requirements to the same package versions used in the notebook.
  • Test the LangGraph app locally before deploying it to Agent Engine.
  • Use separate vector store tables for movie and book data.
  • Use Cloud Storage as the source for reusable JSON document datasets.
  • Clean up Agent Engine apps and Cloud SQL instances after the tutorial to avoid billing.

Building a Conversational Search Agent with Agent Engine and RAG on Vertex AI Search

  • Test the Vertex AI Search function directly before wiring it into the agent.
  • Test the agent locally before deploying it to Agent Engine.
  • Use ChatMessageHistory and session_id to preserve conversational context across follow-up questions.
  • Set deployment requirements explicitly so Agent Engine has the needed LangChain and Discovery Engine packages.
  • Use temperature 0 for the demonstrated search agent behavior.

Function Calling Agent

  • Use function declarations to constrain database access instead of generating and executing arbitrary SQL.
  • Use BigQuery query parameters for filters such as price, store IDs, radius, product ID, and store ID.
  • Cap requested result counts with MAX_PRODUCT_RESULTS and MAX_STORE_RESULTS.
  • Use semantic search only when product_search_query is provided, otherwise use standard SQL filtering.
  • Execute multiple function calls asynchronously and feed function responses back to the model.

Task Planner Agent

  • Use Pydantic models for Task, Plan, Response, and PlanOrRespond to keep agent decisions structured.
  • Reset task results to None before executing newly generated plan tasks.
  • Separate planning, execution, reflection, and post-processing into explicit LangGraph nodes.
  • Use Google Search grounding in the executor for research tasks requiring live information.
  • Use a MemorySaver checkpointer and thread_id to preserve conversation state across turns.

Introduction to Gemini Deep Research Agent

  • Save the interaction_id immediately after initialization.
  • Use specific formatting instructions in the prompt to shape reports, sections, tables, and tone.
  • Prompt the agent to state when data is unavailable instead of estimating it.
  • Be cautious when combining sensitive internal data with public web browsing.
  • Verify citations returned by the agent.

Evaluate groundedness with custom parsing

  • Prompt the response model to answer using only the provided context.
  • Ask the autorater to be strict and avoid world knowledge unless trivial.
  • Return raw autorater output and parse it with CustomOutputConfig for detailed analysis.
  • Use structured labels for supported, unsupported, contradictory, and no_rad sentences.
  • Compute an overall groundedness score from parsed sentence verdicts.

Evaluate generated answers from Retrieval-Augmented Generation (RAG) using Rapid Evaluation and Dataflow ML with Vertex AI pipelines

  • Validate the evaluation dataset before starting remote evaluation.
  • Store pipeline data, source modules, requirements, outputs, and temporary files in Cloud Storage paths.
  • Package the Apache Beam module with requirements.txt and setup.py for Dataflow workers.
  • Use WaitGcpResourcesOp after DataflowPythonJobOp before reading output artifacts.
  • Retrieve both row-level metrics and aggregated summary metrics after pipeline completion.

Evaluate Generated Answers from Retrieval-Augmented Generation (RAG) for Question Answering with Gen AI Evaluation Service SDK

  • Use at least one evaluation example, with around 100 examples recommended for high-quality aggregated metrics and statistical significance.
  • Use reference-free evaluation when assessing generated answers against retrieved context without golden answers.
  • Use referenced evaluation when golden answers are available for comparison.
  • Inspect predefined metric templates before selecting metrics.
  • Use detailed explanations to understand why scores were assigned for individual instances.

Gen AI Evaluation Service SDK Preview-to-GA Migration Guide

  • Use MetricPromptTemplateExamples and adjust them for your use case instead of relying on removed black-box metrics.
  • Define custom PointwiseMetric or PairwiseMetric rubrics when discontinued metrics are still needed.
  • Use instruction_following instead of fulfillment and verbosity instead of summarization_verbosity.
  • Assemble instruction and context into a single prompt for GA metric templates.
  • Use PairwiseMetric in EvalTask for side-by-side evaluation of two models.

Gemini 3.1 Flash Image (Nano Banana 2 🍌) Generation

  • Use types.GenerateContentConfig to set response_modalities, image_config, thinking_config, and tools.
  • Check response.candidates[0].finish_reason before assuming an image was generated.
  • Skip parts marked thought when displaying generated images unless inspecting thoughts intentionally.
  • Use Part.from_bytes for local image or video bytes and Part.from_uri for YouTube or Cloud Storage inputs.
  • Use Google Search tools when prompts need current web or image grounding.

Gemini 3 Pro Image (Nano Banana Pro 🍌) Generation

  • Install or upgrade google-genai before running the notebook.
  • Fallback to the GOOGLE_CLOUD_PROJECT environment variable when PROJECT_ID is not set.
  • Use GenerateContentConfig with response_modalities and ImageConfig for image output control.
  • Check finish_reason before assuming an image was generated.
  • Skip thought parts when displaying final generated images.

Grounding with Vertex AI Search

  • Use environment variables for project and region defaults when parameters are not provided.
  • Create the Vertex AI Search engine with SEARCH_TIER_ENTERPRISE and SEARCH_ADD_ON_LLM for grounding quality.
  • Verify the search engine is ready by sending a SearchRequest before using it for grounded generation.
  • Inspect grounding_supports and grounding_chunks to understand which retrieved documents support the answer.
  • Delete the created engine and data store after the tutorial.

Intro to Grounding with Gemini in Vertex AI

  • Compare ungrounded and grounded responses to show the effect of grounding.
  • Use grounding metadata to display citations, grounding chunks, search queries, and retrieval queries.
  • Use Enterprise Web Search for web grounding when compliance requirements make Google Search logging unsuitable.
  • Use Vertex AI Search for internal documents that are not available on the public internet.
  • Delete projects, data stores, and disable APIs after the tutorial to avoid charges.

Intro to Logprobs

  • Use response_logprobs=True only when token confidence data is needed.
  • Use logprobs to inspect top alternative tokens and debug model behavior.
  • Constrain classification outputs with response_schema and text/x.enum before comparing label confidence.
  • Flag classifications for human review when top choices have close log probabilities.
  • Convert log probabilities with math.exp before applying probability thresholds.

Interactive Loan Application Assistant (Financial Services)

  • Ground generated answers in retrieved document chunks instead of asking the model without context.
  • Include page number and chunk number in citations when generating RAG answers.
  • Use retry with exponential backoff around embedding and generation calls for quota management.
  • Skip unreadable, empty, or blank PDF pages during extraction.
  • Use chunking and embeddings for faster targeted retrieval over large loan documents.

Real-time Retrieval Augmented Generation (RAG) using the Multimodal Live API with Gemini 2.0

  • Use genai.Client with vertexai=True, project, and location for Vertex AI endpoint calls.
  • Ground domain-specific answers with retrieved document context instead of relying on pretrained model knowledge.
  • Use the same embedding model for document chunks and user query embeddings.
  • Keep page number, chunk number, and document metadata with retrieved chunks.
  • Add exponential retry handling for quota-sensitive embedding and answer generation calls.

Getting Started with LangChain 🦜️🔗 + Gemini API in Vertex AI

  • Use PromptTemplate for prompts with runtime input variables.
  • Use StructuredOutputParser format instructions when structured model output is needed.
  • Split long documents with RecursiveCharacterTextSplitter before embedding or summarization.
  • Use retrievers over vector stores to combine documents with language models.
  • Use ConversationBufferMemory when a conversation chain needs prior message context.

LlamaIndex RAG Workflows using Gemini and Firestore

  • Pin LlamaIndex package versions used by the workflow.
  • Use google.auth.default with quota_project_id and refresh credentials before model setup.
  • Configure safety settings for dangerous content, harassment, and sexually explicit content.
  • Set Settings.embed_model and Settings.llm so LlamaIndex components share the same Vertex models.
  • Use custom Event classes to make workflow transitions explicit.

Intro to Building a Scalable and Modular RAG System with RAG Engine in Vertex AI

  • Use an explicit embedding model when creating the RAG corpus.
  • Configure chunk_size and chunk_overlap during bulk imports.
  • Use top_k and vector_distance_threshold to tune retrieval scope.
  • Run rag.retrieval_query to inspect retrieved context directly.
  • Pass a RAG retrieval tool into generate_content to ground model responses.

Vertex AI Rag: Cross-Corpus Retrieval with AskContexts and AsyncRetrieveContexts Demo

  • Initialize Vertex AI with an explicit project, location, and API endpoint.
  • Use RagResource objects to pass corpus paths into retrieval calls.
  • Configure retrieval behavior with RagRetrievalConfig, including top_k.
  • Use async_retrieve_contexts when integrating retrieval into an asynchronous application flow.

Evaluating Vertex RAG Engine Generation with Vertex AI Python SDK for Gen AI Evaluation Service

  • Use environment variables for GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION when project values are not provided.
  • Configure chunk_size and chunk_overlap when importing files into the RAG corpus.
  • Separate retrieved context collection from grounded response generation before evaluation.
  • Evaluate prompts, retrieved_context, and response together in a pandas DataFrame.
  • Use a custom metric rubric that checks accuracy, completeness, and groundedness.

Advanced RAG Techniques - Vertex RAG Engine Retrieval Quality Evaluation and Hyperparameters Tuning

  • Evaluate retrieval quality because poor retrieval can lead to irrelevant, incomplete, or hallucinated output.
  • Tune chunk size, chunk overlap, top-k, vector distance threshold, and embedding model based on recall, precision, and nDCG.
  • Use recall@k, precision@k, and nDCG@k to evaluate retrieval from different perspectives.
  • Reduce chunk size, increase chunk overlap, or increase top-k when recall is too low.
  • Reduce top-k, reduce chunk overlap, or increase chunk size when precision is too low.

Vertex AI RAG Engine with Vertex AI Feature Store

  • Use the required BigQuery schema fields for the RAG Feature Store source table.
  • Use optimized online serving for FeatureOnlineStore when using vector similarity search.
  • Use chunk_size and chunk_overlap when importing files into the RAG corpus.
  • List imported files after import because processing may take a few seconds.
  • Run FeatureView sync after uploading data to make it available for online serving.

Vertex AI RAG Engine with Pinecone

  • Use Secret Manager instead of embedding the Pinecone API key directly in the RAG corpus config.
  • Grant the RAG Engine service account only the secretAccessor role needed to read the Pinecone API key.
  • Set chunk_size and chunk_overlap explicitly when importing files.
  • Use similarity_top_k and vector_distance_threshold to control retrieved context quality.
  • Clean up the RAG corpus with rag.delete_corpus when the created resources are no longer needed.

Vertex AI RAG Engine with Vertex AI Vector Search

  • Use environment variables for project and region defaults when notebook parameters are not provided.
  • Create a public Vector Search index endpoint because RAG Engine supports public endpoints.
  • Tune chunk_size and chunk_overlap during file import.
  • Use similarity_top_k and vector_distance_threshold to control retrieved context.
  • List files after import to check ingestion progress.

Vertex AI RAG Engine with Vertex AI Search

  • Use environment variables for GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION when notebook parameters are not provided.
  • Use Vertex AI Search as the retrieval backend for large datasets, low-latency retrieval, and improved scalability.
  • Import documents from Cloud Storage using INCREMENTAL reconciliation mode.
  • Check the created corpus with rag.get_corpus before querying.
  • Clean up created RAG resources with rag.delete_corpus when delete_rag_corpus is enabled.

Vertex AI RAG Engine with Weaviate

  • Use GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION environment variables as fallbacks for project and region.
  • Configure a Google first-party embedding model for the corpus.
  • Set chunk_size and chunk_overlap when importing files.
  • Check created resources with rag.get_corpus and rag.list_files.
  • Use similarity_top_k and vector_distance_threshold to control retrieved context.

Gen AI and LLM Security - ReAct and RAG attacks & mitigations

  • Use ready Agents and RAG libraries such as Agent Builder, LangChain Agents, Vertex AI Search, and LangChain RAG.
  • Use strict schema validation of tool input and output.
  • Use out-of-band user consent for dangerous operations.
  • Apply defense in depth by layering multiple filters.
  • Use OCR for documents if concerned about invisible text.

Intro to Url Context

  • Install or upgrade google-genai before running the notebook.
  • Authenticate Colab users with auth.authenticate_user().
  • Use a Google Cloud Project for authentication in the tutorial workflow.
  • Inspect url_context_metadata to debug retrieved sources and verify information sources.
  • Inspect grounding_metadata when using Google Search grounding.

Analyze Multimodal Data in BigQuery

  • Use ObjectRefs as secure pointers instead of storing unstructured file bytes in BigQuery tables.
  • Use object tables when every file in a GCS path should automatically become an ObjectRef row.
  • Use OBJ.MAKE_REF and OBJ.FETCH_METADATA when URI columns already exist in structured tables.
  • Join structured records with ObjectRef columns to create a reusable multimodal table.
  • Define output_schema for AI.GENERATE and AI.GENERATE_TABLE to return typed, queryable results.

Use Retrieval Augmented Generation (RAG) with Gemini API

  • Use retrieved repository context rather than relying only on zero-shot prompting for code generation.
  • Split Python code with RecursiveCharacterTextSplitter.from_language using chunk overlap.
  • Persist crawled GitHub file URLs to avoid repeatedly downloading the file list.
  • Use low temperature for concise code generation.
  • Return source documents from RetrievalQA for traceability.

Comparing LlamaIndex and LlamaParse for Dense Document Questioning Answering on Vertex AI

  • Compare multiple parsing approaches against the same source document and query set.
  • Use similarity_top_k=2 consistently across query engines for apples-to-apples comparison.
  • Use domain-specific parsing instructions for LlamaParse on 10-Q financial documents.
  • Extract question and keyword metadata from parsed nodes before embedding for richer retrieval.
  • Print response source nodes, relevance scores, file names, page labels, and file paths for answer inspection.

🛡️ Agentic GraphRAG: Cybersecurity Threat Intelligence

  • Authenticate in Colab with google.colab.auth.authenticate_user when running in Colab.
  • Use vertexai.init with explicit project and location before Vertex AI operations.
  • Re-initialize Neo4j and imports inside the tool function for serialization contexts.
  • Give the ADK agent explicit instructions to use query_threat_graph first for threats, actors, and CVEs.
  • Test the ADK app locally before deploying to Vertex AI Agent Engine.

GraphRAG on Google Cloud With Spanner and Vertex AI Agent Engine

  • Constrain graph extraction with allowed_nodes and allowed_relationships.
  • Use temperature=0 for deterministic graph Q&A and entity rewriting.
  • Store embeddings alongside graph-related text in Spanner for semantic matching.
  • Combine vector search with graph search when exact graph entity names may not match user phrasing.
  • Configure ADK instructions to check the graph database before broader Google Search.

🌿 Eco-Nomad Swarm: 100% Real-Data Sustainable Travel Orchestration

  • Use live databases or real-world REST endpoints instead of mocked travel, weather, forex, or translation data.
  • Chain tool outputs deterministically, such as graph-derived country to RESTCountries currency to Frankfurter forex conversion.
  • Keep agent tools narrow and domain-specific: graph routing, demographics, treasury, climate, web intelligence, and translation.
  • Use exact 3-letter currency codes for live currency conversion.
  • Initialize Vertex AI with project, region, and a staging bucket before production deployment.

Know Your Customer Use Case - Gemini Grounding with Google Search

  • Use explicit system instructions to define role, scope, output format, and no-result behavior.
  • Ground sensitive claims with Google Search and display source metadata for verification.
  • Check for missing candidates, grounding metadata, grounding supports, and out-of-bounds chunks before extracting sources.
  • Separate reusable single-entity generation from batch entity processing.
  • Evaluate outputs with task-specific custom metrics and rubrics.

Retail AI Location Strategy: Autonomous Site Selection & Market Analysis

  • Use Search Grounding for fresh demographics, growth, rental, and market viability data.
  • Wrap Google Maps Places API as a tool for real competitor names, locations, and ratings.
  • Base gap analysis on prior market research and competitor findings rather than isolated prompts.
  • Use code execution for density, saturation, and scoring calculations.
  • Use Pydantic schemas and response_schema to make final recommendations machine-readable.

RAG Based on Sensitive Data Protection using Faker

  • Anonymize source documents before creating embeddings.
  • Anonymize the user query before retrieval against anonymized embeddings.
  • De-anonymize the generated response only after the RAG chain returns an answer.
  • Reuse existing mappings when the same original data is detected again.
  • Constrain the prompt to say it does not know when context is insufficient.

Intra Knowledge QnA

  • Initialize Vertex AI with an explicit project and location.
  • Use chunk overlap to preserve context across document splits.
  • Persist the vector database to disk for reuse.
  • Limit retrieval to the top 3 similar documents for answer generation.
  • Print unique source document paths from retrieved context.

Leverage LlamaIndex with Vertex AI Vector Search to perform question answering RAG

  • Check for existing buckets, indexes, and endpoints before creating resources.
  • Set LlamaIndex embed_model and llm settings before building indexes.
  • Use SimpleDirectoryReader and SentenceSplitter to parse documents into documents and nodes.
  • Display prompt templates before running RAG queries.
  • Inspect source text, relevance score, file name, page label, and file path with each response.

Retrieval Augmented Generation(RAG) with AlloyDB

  • Pin package versions for reproducible notebook setup.
  • Initialize Vertex AI with explicit project and location.
  • Use the AlloyDB connector and SQLAlchemy engine instead of raw unmanaged connections.
  • Create the google_ml_integration and vector extensions before generating and querying embeddings.
  • Concatenate title and abstract before embedding to improve retrieval context.

Building a Gen AI RAG application with Vertex AI Feature Store and BigQuery

  • Use BigQueryVectorStore for prototyping because it requires no infrastructure startup time.
  • Split documents so a few chunks can fit within the LLM context length.
  • Store document source, document name, and chunk number in metadata.
  • Use add_texts_with_embeddings when embeddings are already precomputed.
  • Use VertexFSVectorStore for production-ready user-facing Gen AI applications that need low-latency retrieval.

Augment Gemini Output with Vector Embeddings from BigQuery

  • Create BigQuery remote models through a Cloud resource connection for Vertex AI access.
  • Filter out rows with empty abstract and title before generating embeddings.
  • Store generated embeddings in a BigQuery table for reuse.
  • Check INFORMATION_SCHEMA.VECTOR_INDEXES for coverage_percentage and last_refresh_time before relying on the index.
  • Use the same embedding model for indexed embeddings and query embeddings.

Run RAG Pipelines in BigQuery with BQML and Vector Search

  • Use a Cloud resource connection so BigQuery can call Cloud Storage, Vertex AI, and Document AI.
  • Grant required IAM roles to the BigQuery connection service account before creating remote models.
  • Store source PDFs in Cloud Storage and expose them through a BigQuery object table.
  • Parse Document AI JSON into chunk content and metadata before embedding.
  • Use ML.GENERATE_EMBEDDING for both document chunks and user queries to keep embeddings consistent.

Building a Multimodal Chatbot for Warranty Claims using Gemini and Vector Search in Vertex AI

  • Use a unique lowercase RAG identifier without spaces for generated resources.
  • Keep chunk overlap when splitting retrieved text to preserve context across chunks.
  • Store Vector Search input as JSONL with id and embedding fields.
  • Use retrieved context in the prompt and instruct Gemini to answer only from provided text.
  • Return a fallback response when no matching page source is found.

Production & Scalable RAG Pipeline Using BigFrames

  • Use BigFrames to process BigQuery data with pandas-like syntax without moving terabyte-scale data out of BigQuery.
  • Parameterize scheduled runs with RUN_DATE, IS_INCREMENTAL, LOOK_BACK_DAYS, START_DATE, and END_DATE.
  • Sort by last_edit_date and drop duplicate question_id values before embedding.
  • Convert HTML questions and answers to Markdown before chunking and generation.
  • Use chunk overlap and consider preserving paragraphs, sections, markdown hierarchy, code blocks, lists, and question-answer grouping in production.

Running a Gemma 2-based agentic RAG with Ollama on Vertex AI and LangGraph

  • Use a Cloud Storage staging bucket when initializing Vertex AI SDK.
  • Use Artifact Registry and Cloud Build to build and store the custom serving image.
  • Expose Vertex AI-compatible health and predict routes in the custom container.
  • Validate prediction requests and return consistent error responses in the FastAPI proxy.
  • Test the serving container locally with Vertex AI LocalModel before deploying when debugging.

Cloud Run GPU Inference: Gemma 2 RAG Q&A with Ollama and LangChain

  • Store Gemma 2 9B and similarly sized model weights directly in the container image for startup time and scalability.
  • Consider storage requirements before placing larger model weights in the image.
  • Use e2-highcpu-32 for Cloud Build to speed up parallel downloads.
  • Set OLLAMA_KEEP_ALIVE=-1 to avoid unloading model weights from GPU memory.
  • Use a dummy startup request to load the model into GPU memory.

Create a Vertex AI Datastore and Search Engine

  • Use ClientOptions with a regional discoveryengine endpoint when location is not global.
  • Use the default_collection parent path for datastore and engine creation.
  • Import documents into the default_branch from Cloud Storage using data_schema=“content”.
  • Use INCREMENTAL reconciliation mode for document import.
  • Enable SEARCH_TIER_ENTERPRISE and SEARCH_ADD_ON_LLM when summaries or advanced LLM features are needed.

Custom Embeddings with Vertex AI Search

  • Use TextEmbeddingInput with task_type=“RETRIEVAL_DOCUMENT” for document retrieval embeddings.
  • Batch embedding calls instead of embedding all texts at once.
  • Convert BigQuery integer IDs to strings before using them as document IDs.
  • Store HTML content in Cloud Storage and reference it from JSONL content.uri.
  • Define the embedding vector in the Discovery Engine schema before importing documents.

Intro to Gemini Enterprise

  • Fallback to the GOOGLE_CLOUD_PROJECT environment variable when PROJECT_ID is not supplied.
  • Use ClientOptions to select a regional Discovery Engine endpoint for non-global locations.
  • Include citations in both search summaries and generated answers.
  • Use query expansion and spell correction in the search request.
  • Use query rephrasing and query classification in the answer request.

Search tuning in Vertex AI Search

  • Prepare datasets in JSONL format with identifier-text pairs.
  • Represent query-to-answer mappings in tab-separated TSV files.
  • Upload additional tuning documents and datasets to Cloud Storage before datastore refresh and tuning.
  • Use INCREMENTAL document import when adding files to the datastore.
  • Keep a baseline search result before tuning so it can be compared with post-tuning behavior.

Defining custom attributes based on URL patterns in Vertex AI Search Website Datastores

  • Use global datastore location unless there is a specific reason to use us or eu.
  • Verify schema and URL mapping with getUriPatternDocumentData after setting it.
  • Make custom attributes indexable, retrievable, and searchable when they should affect retrieval and appear in responses.
  • Poll datastore and app availability because creation can take a few minutes.
  • Delete the app and datastore when no longer needed.

Ingestion of Unstructured Documents with Metadata in Vertex AI Search

  • Use layout-based chunking with a 500 token chunk size and ancestor headings when accuracy on complex documents matters.
  • Provide an explicit metadata schema when fields must be retrievable, indexable, searchable, filterable, or specially mapped.
  • Represent each document as JSONL with id, structData, content.mimeType, and content.uri.
  • Poll long-running operations for schema updates and document imports before relying on the datastore.
  • Keep documents, metadata, and generated JSONL in the same temporary bucket when cleanup simplicity matters.

Inline Ingestion of Documents into Vertex AI Search

  • Use GCS buckets or BigQuery tables as a source of truth when importing structured or unstructured documents unless inline ingestion is required.
  • Use Incremental or Full reconciliation modes depending on the import strategy and conflict-resolution needs.
  • Set mimeType consistently with the source file format, such as application/pdf or text/plain.
  • Add structData metadata when it should influence search filtering.
  • Poll datastore availability after creation before continuing.

Parsing and Chunking in Vertex AI Search: Featuring BYO Capabilities

  • Create the datastore with layout parser for complex documents containing tables and lists.
  • Include ancestor headings with chunks to preserve heading context.
  • Use incremental reconciliation for document and chunk imports.
  • Poll datastore creation and import long-running operations before retrieving processed documents.
  • Export chunked JSON to Cloud Storage for offline review or editing before BYOC import.

Query-Level Boosting, Filtering, and Facets for Vertex AI Search Website Datastores

  • Evaluate Vertex AI Search results without extra rules before incrementally adding custom filters or boosts.
  • Apply boosting and filtering during retrieval rather than post-processing search results.
  • Use metadata fields as schema hooks for filters, facets, and boost conditions.
  • Use facets to translate user preferences into subsequent filtered search requests.
  • Start with smaller boost values and adjust as needed.

Recording Real-Time User Events in Vertex AI Search Datastores

  • Report both search events and view-item events with the same attribution token.
  • Include the impressions shown to the user in the search event.
  • Use userPseudoId to associate events with a user without requiring a named account.
  • Use JavaScript Pixel as the recommended alternative when the customer can control the page source.
  • Purge synthetic notebook events after testing.

Q&A Chatbot with Vertex AI Search for summarized website results without advanced indexing

  • Initialize Vertex AI with project and location before model use.
  • Use query expansion and spell correction with AUTO mode in SearchRequest.
  • Request snippets with ContentSearchSpec.SnippetSpec(return_snippet=True).
  • Handle empty search responses and exceptions by returning None or a fallback message.
  • Use low temperature for concise grounded answer generation.

Building Search Applications with Vertex AI Search

  • Use a custom prompt that tells the assistant to answer only from grounding snippets and not make up information.
  • Enable snippets, summaries, citations, query expansion, and spell correction in direct search requests.
  • Use safety settings when generating Gemini responses.
  • Ground Gemini answers in the Vertex AI Search data store instead of relying only on model knowledge.
  • Use extractive answers and document limits when configuring the LangChain VertexAISearchRetriever.

Back to RAG & Grounding · Best Practices Map

Graph View

Backlinks