Embeddings & Vector Search — Best Practices

Distilled from 41 notebooks tagged Embeddings & Vector Search in the GoogleCloudPlatform/generative-ai repository. The From the notebooks section below cites the per-notebook source for grounding.

Do this

• Use the same embedding model, dimensionality, and compatible distance measure for stored vectors and query vectors.
• Choose task types deliberately, such as RETRIEVAL_DOCUMENT for indexed content, QUESTION_ANSWERING or RETRIEVAL_QUERY for questions, SEMANTIC_SIMILARITY for similarity baselines, and CLUSTERING for clustering-style anomaly detection.
• Match index dimensions to embedding output dimensionality; specify output_dimensionality when lower storage cost or faster search is acceptable.
• Batch embedding, import, upsert, update, delete, and search operations where supported, while respecting per-request limits such as maximum texts, tokens, or datapoints.
• Throttle embedding and generation traffic, use retries with exponential backoff, checkpoint long-running embedding jobs, and log per-item errors so large jobs can resume.
• Clean, normalize, and enrich source data before embedding, including removing null or blank text, combining relevant fields, converting HTML to Markdown, normalizing log fields, and preserving metadata such as source URI, page, chunk, and document name.
• Use chunk_size and chunk_overlap for document corpora, then tune them with retrieval metrics instead of assuming one universal setting.
• Store embeddings and intermediate outputs in durable systems such as BigQuery, Cloud Storage, or a vector store so workflows are reproducible and do not require repeated model calls.
• Use metadata filters, restricts, numeric restricts, output_fields, or filter columns to constrain retrieval and control returned fields.
• Verify index readiness before relying on search results by polling long-running operations, checking VECTOR_INDEXES coverage, last_refresh_time, deployment state, sync status, or list_files output.
• Evaluate retrieval quality with recall@k, precision@k, nDCG@k, Mean Reciprocal Rank, distance distributions, source-node inspection, or answer keys with citations.
• For RAG, ground generation in retrieved context, instruct the model to answer only from provided sources, include source citations, and inspect retrieved context directly during debugging.
• Use hybrid dense and sparse search when semantic search alone may miss exact product names, SKUs, proprietary terms, or out-of-domain keywords; tune rrf_ranking_alpha for dense versus sparse weighting.
• Use managed services such as Vertex AI Vector Search, BigQuery VECTOR_SEARCH, Vertex AI Search, RAG Engine, Feature Store, or AlloyDB for scalable production retrieval instead of local FAISS or notebook-only stores.
• Delete or undeploy unused indexes, endpoints, collections, corpora, buckets, datasets, connections, Feature Store resources, endpoints, and model deployments to avoid ongoing charges.

Avoid this

• Creating an index with dimensions, task type assumptions, or distance settings that do not match the embeddings used for queries and stored datapoints.
• Assuming long-running index creation, deployment, import, sync, or vector index population is complete without polling or checking readiness fields.
• Ignoring quotas and batch limits, which can cause ResourceExhausted errors, timeouts, partial failures, or very long notebook runs.
• Leaving billable resources such as Vector Search endpoints, indexes, collections, Cloud Storage buckets, BigQuery datasets, Feature Stores, AlloyDB clusters, or deployed OSS endpoints running after tutorials.
• Using semantic search alone when keyword, SKU, codenames, sparse signals, filters, reranking, or hybrid search are needed for quality.
• Embedding sensitive data before anonymization, or failing to check both inputs and outputs when using embeddings in security-sensitive RAG flows.
• Running notebooks without required API enablement, billing, authentication, region setup, Cloud resource connections, or IAM grants for service accounts.
• Treating sample thresholds, top-k values, chunk settings, distance thresholds, or small synthetic datasets as production-ready without evaluation and tuning.

From the notebooks

Get started with Vertex AI Memory Bank

• Use a stable user_id scope to retrieve memories for a specific guest.
• Store the complete conversation in a session before generating memories.
• Use scope-based retrieval for complete profiles or small memory sets.
• Use similarity search for specific questions, many memories, fast targeted responses, or conversational context.
• Delete the Agent Engine and memories after the tutorial to avoid charges.

Intro to Skill Registry

• Use a mandatory SKILL.md file with YAML frontmatter and markdown instructions.
• Make the skill name a unique identifier matching the skill package name.
• Start the skill description in third person as a capability statement.
• Use local_path so the SDK packages, compresses, uploads, provisions, and indexes the skill.
• Use timestamped skill IDs to avoid collisions during registration.

🛡️ AI Brand Safety: Three-Tier Agent Anomaly Detection

• Compare prompts, tool calls, and responses against separate Vector Search indices to reduce structural false positives.
• Generate industry-specific golden baselines using the agent instruction and INDUSTRY_VERTICAL.
• Request JSON from Gemini with response_mime_type=“application/json” and parse it with json.loads.
• Log full execution traces to BigQuery as a flight recorder for later threshold tuning without new LLM calls.
• Route safety finish reasons and novelty anomalies to Tier 1 for 100% audit.

Import from BigQuery into Vector Search

• Initialize aiplatform with project and location before creating the index.
• Use BATCH_UPDATE when creating the index for BigQuery import.
• Map BigQuery columns explicitly to datapoint fields including id_column and embedding_column.
• Include restricts and numeric_restricts mappings when filterable fields are present.
• Check the REST response status code and print error text on failure.

Use Gemini and OSS Text-Embedding Models Against Your BigQuery Data

• Create a dedicated BigQuery dataset for demo tables and models.
• Filter out NULL text values before calling ML.GENERATE_EMBEDDING.
• Use the same ML.GENERATE_EMBEDDING interface for managed Gemini and deployed OSS remote models.
• Adjust min_replica_count, max_replica_count, and machine_type to balance scalability and cost.
• For batch workloads, deploy the OSS model, run inference, then immediately undeploy it.

Visualizing embedding similarity from text documents using t-SNE plots

• Use environment variables for GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION when notebook parameters are not provided.
• Subsample with stratification so labels are roughly evenly distributed.
• Clean emails, names, From headers, and Subject markers before embedding text.
• Use retry logic around embedding requests with a 300 second timeout.
• Set output_dimensionality to 768 before applying t-SNE for visualization.

Combining Semantic & Keyword Search: A Hybrid Search Tutorial with Agent Platform Vector Search

• Store both embedding and sparse_embedding on each item when building a hybrid index.
• Fit the sparse vectorizer on the corpus before generating item and query sparse embeddings.
• Use rrf_ranking_alpha to control the weight between dense and sparse search results.
• Consider subword tokenizers, BM25, or SPLADE instead of basic word-level TF-IDF for production requirements.
• Use fast Vector Search retrieval followed by reranking for higher-quality production retrieval or recommender systems.

Getting Started with Text Embeddings + Agent Platform Vector Search

• Limit tutorial data size from the 23 million row Stack Overflow table before loading into memory.
• Batch embedding requests and add delay to avoid quota errors.
• Use dot product for similarity with the Google embedding model shown.
• Store Vector Search input as JSONL with id and embedding fields.
• Match index dimensions to the embedding output dimensionality.

Get started with embeddings tuning on Agent Platform

• Prepare corpus, query, and labels files in the format required by embedding tuning.
• Split labels into train and test sets before running the tuning pipeline.
• Use a timestamped Cloud Storage path for tuning artifacts.
• Evaluate tuned embeddings with pipeline-produced NDCG metrics.
• Create cleanup flags for endpoint, model, job, bucket objects, and local tutorial files.

Introduction to Gemini Multimodal Embeddings

• Embed multiple text prompts in one API call for efficiency.
• Use output_dimensionality to reduce storage cost and improve search speed when lower dimensions are acceptable.
• Choose input structure deliberately: one Content object aggregates parts into one embedding, while multiple contents return separate embeddings.
• Create post-level multimodal representations by aggregating separate embeddings, for example by averaging.
• Truncate PDFs before embedding when they exceed the six-page limit.

Handling large-scale embedding generation for Agent Platform Vector Search

• Throttle API calls to stay within embedding quota limits.
• Use multithreading to reduce latency impact and improve quota utilization.
• Checkpoint generated embeddings periodically to Cloud Storage.
• Record errors to a log file during long-running jobs.
• Use moderate text batch sizes such as 20 instead of always using the maximum 250.

Using “task type” embeddings for improving RAG search quality

• Use QUESTION_ANSWERING for question texts and RETRIEVAL_DOCUMENT for answer documents in Q&A RAG.
• Use SEMANTIC_SIMILARITY as a baseline when comparing retrieval quality.
• Evaluate search quality with ranking metrics such as Mean Reciprocal Rank.
• Batch multiple texts per embedding API call to reduce repeated calls.
• Consider tuning text embeddings when pre-trained task-type embeddings do not fit proprietary or specialized content.

Log Anomaly Detection & Investigation with Text Embeddings + BigQuery Vector Search

• Aggregate daily user actions before embedding to make log analysis easier, faster, and more cost-effective.
• Include principal, action, resource, container, channel, IP, and count in the text used for embeddings.
• Normalize resource IDs and user-agent channels with BigQuery UDFs before embedding.
• Remove exact duplicate matches when comparing suspicious actions to historical actions.
• Use distance thresholds and visualizations to support anomaly investigation.

Anomaly Detection of Infrastructure Logs using Gemini and BigQuery Vector Search

• Group raw logs by a meaningful session identifier before summarization.
• Use a one-shot prompt and instruct Gemini not to repeat raw logs or speculate.
• Persist intermediate outputs in BigQuery tables so long-running jobs can resume after failures.
• Batch ML.GENERATE_TEXT calls to reduce loss from timeouts or quota exhaustion.
• Use CLUSTERING task_type for embeddings when the downstream task is clustering-based anomaly detection.

Introduction to Agent Platform Vector Search 2.0

• Use a schema-enforced collection with separate data_schema and vector_schema.
• Use auto-embeddings by leaving vectors empty when vertex_embedding_config is configured.
• Use random.seed(42) for reproducible sampling of the product dataset.
• Use batch_create_data_objects instead of one create request per product for bulk imports.
• Use task_type=“RETRIEVAL_DOCUMENT” for indexed product names and task_type=“QUESTION_ANSWERING” for semantic queries.

Vector Search 2.0 Public Preview Quickstart

• Define data_schema and vector_schema before creating data objects.
• Normalize generated dense vectors before storing or searching them.
• Sort sparse embedding indices when generating sparse vectors.
• Use batch create, batch update, batch delete, and batch search for multi-object operations.
• Use output_fields to control returned data, vector, and metadata fields.

Agent Platform Vector Search Quickstart

• Use STREAM_UPDATE when product catalog changes should appear in search results quickly.
• Use a public endpoint unless there is a specific VPC requirement; it is IAM-secured by default.
• Batch streaming upserts in chunks of 1000 datapoints.
• Reuse existing indexes and endpoints if a Colab runtime disconnects during long operations.
• Delete index endpoints and indexes after the tutorial to avoid unexpected costs.

Interactive Loan Application Assistant (Financial Services)

• Ground generated answers in retrieved document chunks instead of asking the model without context.
• Include page number and chunk number in citations when generating RAG answers.
• Use retry with exponential backoff around embedding and generation calls for quota management.
• Skip unreadable, empty, or blank PDF pages during extraction.
• Use chunking and embeddings for faster targeted retrieval over large loan documents.

LlamaIndex RAG Workflows using Gemini and Firestore

• Pin LlamaIndex package versions used by the workflow.
• Use google.auth.default with quota_project_id and refresh credentials before model setup.
• Configure safety settings for dangerous content, harassment, and sexually explicit content.
• Set Settings.embed_model and Settings.llm so LlamaIndex components share the same Vertex models.
• Use custom Event classes to make workflow transitions explicit.

Intro to Building a Scalable and Modular RAG System with RAG Engine in Vertex AI

• Use an explicit embedding model when creating the RAG corpus.
• Configure chunk_size and chunk_overlap during bulk imports.
• Use top_k and vector_distance_threshold to tune retrieval scope.
• Run rag.retrieval_query to inspect retrieved context directly.
• Pass a RAG retrieval tool into generate_content to ground model responses.

Advanced RAG Techniques - Vertex RAG Engine Retrieval Quality Evaluation and Hyperparameters Tuning

• Evaluate retrieval quality because poor retrieval can lead to irrelevant, incomplete, or hallucinated output.
• Tune chunk size, chunk overlap, top-k, vector distance threshold, and embedding model based on recall, precision, and nDCG.
• Use recall@k, precision@k, and nDCG@k to evaluate retrieval from different perspectives.
• Reduce chunk size, increase chunk overlap, or increase top-k when recall is too low.
• Reduce top-k, reduce chunk overlap, or increase chunk size when precision is too low.

Vertex AI RAG Engine with Vertex AI Feature Store

• Use the required BigQuery schema fields for the RAG Feature Store source table.
• Use optimized online serving for FeatureOnlineStore when using vector similarity search.
• Use chunk_size and chunk_overlap when importing files into the RAG corpus.
• List imported files after import because processing may take a few seconds.
• Run FeatureView sync after uploading data to make it available for online serving.

Vertex AI RAG Engine with Vertex AI Vector Search

• Use environment variables for project and region defaults when notebook parameters are not provided.
• Create a public Vector Search index endpoint because RAG Engine supports public endpoints.
• Tune chunk_size and chunk_overlap during file import.
• Use similarity_top_k and vector_distance_threshold to control retrieved context.
• List files after import to check ingestion progress.

Vertex AI RAG Engine with Weaviate

• Use GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION environment variables as fallbacks for project and region.
• Configure a Google first-party embedding model for the corpus.
• Set chunk_size and chunk_overlap when importing files.
• Check created resources with rag.get_corpus and rag.list_files.
• Use similarity_top_k and vector_distance_threshold to control retrieved context.

Gen AI & LLM Security for developers

• Do not store sensitive information in the prompt.
• Use low temperature for reproducible results in security demonstrations.
• Check both input and output with DLP before sending to or returning from Gemini.
• Treat links, binaries, and files from users as untrusted and validate them.
• Use Responsible AI safety filters and understand how to configure safety attributes.

AI-Assisted Data Science Workflows in BigQuery

• Create a dedicated BigQuery dataset to contain tutorial tables, views, and models.
• Use ML.DESCRIBE_DATA for quick exploratory statistics before modeling.
• Filter raw data and add engineered features before AI enrichment and clustering.
• Define an explicit AI.GENERATE_TABLE output schema for reliable typed columns.
• Inspect enriched rows and generated embeddings before downstream modeling or indexing.

Analyzing movie posters in BigQuery with Gemini

• Create a dedicated dataset for the demo resources.
• Use a Cloud resource connection for BigQuery access to Cloud Storage objects.
• Use output_schema with AI.GENERATE when extracting structured fields.
• Store intermediate Gemini analysis and embedding results in BigQuery tables.
• Use public datasets and joins to enrich generated analysis with structured metadata.

Text + multimodal embedding generation and vector search in BigQuery

• Create a dedicated BigQuery dataset before loading tutorial tables.
• Use BigQuery remote models instead of exporting data to call Vertex AI separately.
• Store embeddings in ARRAY columns for reuse with VECTOR_SEARCH.
• Combine product_name and description before generating text embeddings to improve semantic context.
• Order vector search results by distance ascending.

Performing Semantic Search in BigQuery

• Create a BigQuery Cloud resource connection before defining the remote model.
• Grant Vertex AI User to the connection service account used by BigQuery.
• Use SEMANTIC_SIMILARITY as the embedding task type for search embeddings.
• Concatenate question title and body before embedding to include both fields.
• Create an IVF cosine vector index on ml_generate_embedding_result before VECTOR_SEARCH.

Use Retrieval Augmented Generation (RAG) with Gemini API

• Use retrieved repository context rather than relying only on zero-shot prompting for code generation.
• Split Python code with RecursiveCharacterTextSplitter.from_language using chunk overlap.
• Persist crawled GitHub file URLs to avoid repeatedly downloading the file list.
• Use low temperature for concise code generation.
• Return source documents from RetrievalQA for traceability.

Comparing LlamaIndex and LlamaParse for Dense Document Questioning Answering on Vertex AI

• Compare multiple parsing approaches against the same source document and query set.
• Use similarity_top_k=2 consistently across query engines for apples-to-apples comparison.
• Use domain-specific parsing instructions for LlamaParse on 10-Q financial documents.
• Extract question and keyword metadata from parsed nodes before embedding for richer retrieval.
• Print response source nodes, relevance scores, file names, page labels, and file paths for answer inspection.

Data Curation Pipeline: Splitting and Transcoding

• Store videos in Cloud Storage before calling the Multimodal Embeddings API.
• Use a semaphore with ThreadPoolExecutor to control embedding API concurrency.
• Capture errors per video URI instead of failing the whole embedding run.
• Store embeddings in BigQuery for reproducible vector search workflows.
• Create a BigQuery VECTOR INDEX with COSINE distance before large vector search workloads.

RAG Based on Sensitive Data Protection using Faker

• Anonymize source documents before creating embeddings.
• Anonymize the user query before retrieval against anonymized embeddings.
• De-anonymize the generated response only after the RAG chain returns an answer.
• Reuse existing mappings when the same original data is detected again.
• Constrain the prompt to say it does not know when context is insufficient.

Intra Knowledge QnA

• Initialize Vertex AI with an explicit project and location.
• Use chunk overlap to preserve context across document splits.
• Persist the vector database to disk for reuse.
• Limit retrieval to the top 3 similar documents for answer generation.
• Print unique source document paths from retrieved context.

Leverage LlamaIndex with Vertex AI Vector Search to perform question answering RAG

• Check for existing buckets, indexes, and endpoints before creating resources.
• Set LlamaIndex embed_model and llm settings before building indexes.
• Use SimpleDirectoryReader and SentenceSplitter to parse documents into documents and nodes.
• Display prompt templates before running RAG queries.
• Inspect source text, relevance score, file name, page label, and file path with each response.

Retrieval Augmented Generation(RAG) with AlloyDB

• Pin package versions for reproducible notebook setup.
• Initialize Vertex AI with explicit project and location.
• Use the AlloyDB connector and SQLAlchemy engine instead of raw unmanaged connections.
• Create the google_ml_integration and vector extensions before generating and querying embeddings.
• Concatenate title and abstract before embedding to improve retrieval context.

Building a Gen AI RAG application with Vertex AI Feature Store and BigQuery

• Use BigQueryVectorStore for prototyping because it requires no infrastructure startup time.
• Split documents so a few chunks can fit within the LLM context length.
• Store document source, document name, and chunk number in metadata.
• Use add_texts_with_embeddings when embeddings are already precomputed.
• Use VertexFSVectorStore for production-ready user-facing Gen AI applications that need low-latency retrieval.

Augment Gemini Output with Vector Embeddings from BigQuery

• Create BigQuery remote models through a Cloud resource connection for Vertex AI access.
• Filter out rows with empty abstract and title before generating embeddings.
• Store generated embeddings in a BigQuery table for reuse.
• Check INFORMATION_SCHEMA.VECTOR_INDEXES for coverage_percentage and last_refresh_time before relying on the index.
• Use the same embedding model for indexed embeddings and query embeddings.

Run RAG Pipelines in BigQuery with BQML and Vector Search

• Use a Cloud resource connection so BigQuery can call Cloud Storage, Vertex AI, and Document AI.
• Grant required IAM roles to the BigQuery connection service account before creating remote models.
• Store source PDFs in Cloud Storage and expose them through a BigQuery object table.
• Parse Document AI JSON into chunk content and metadata before embedding.
• Use ML.GENERATE_EMBEDDING for both document chunks and user queries to keep embeddings consistent.

Production & Scalable RAG Pipeline Using BigFrames

• Use BigFrames to process BigQuery data with pandas-like syntax without moving terabyte-scale data out of BigQuery.
• Parameterize scheduled runs with RUN_DATE, IS_INCREMENTAL, LOOK_BACK_DAYS, START_DATE, and END_DATE.
• Sort by last_edit_date and drop duplicate question_id values before embedding.
• Convert HTML questions and answers to Markdown before chunking and generation.
• Use chunk overlap and consider preserving paragraphs, sections, markdown hierarchy, code blocks, lists, and question-answer grouping in production.

Custom Embeddings with Vertex AI Search

• Use TextEmbeddingInput with task_type=“RETRIEVAL_DOCUMENT” for document retrieval embeddings.
• Batch embedding calls instead of embedding all texts at once.
• Convert BigQuery integer IDs to strings before using them as document IDs.
• Store HTML content in Cloud Storage and reference it from JSONL content.uri.
• Define the embedding vector in the Discovery Engine schema before importing documents.

---

Back to Embeddings & Vector Search · Best Practices Map