Generative AI Google Brain

Evaluation - Best Practices

27 min read

Evaluation — Best Practices

Distilled from 77 notebooks tagged Evaluation in the GoogleCloudPlatform/generative-ai repository. The From the notebooks section below cites the per-notebook source for grounding.

Do this

  • Use a fixed dataset, metric set, prompt template, and experiment structure when comparing models, prompts, RAG outputs, or third-party systems.
  • Initialize Vertex AI with explicit project and location values, preferably with GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION fallbacks.
  • Keep evaluation datasets structured and task-specific, with columns such as prompt, response, reference, context, instruction, intermediate_events, predicted_trajectory, and reference_trajectory as required.
  • Run inference before evaluation when agent or model evaluation needs generated response columns, intermediate events, traces, or tool-call outputs.
  • Define clear metric criteria and rating rubrics before running model-based or custom evaluations, and inspect MetricPromptTemplateExamples before customizing templates.
  • Request structured JSON from autoraters and classifiers with response_mime_type, response_schema, or strict output formats, then parse and validate results explicitly.
  • Inspect row-level results, explanations, raw outputs, rubric verdicts, and mismatches instead of relying only on aggregate scores.
  • Use reference answers when available, and use reference-free groundedness or context-based evaluation when judging RAG answers without golden responses.
  • For agents, evaluate single tool selection before full trajectories, then evaluate trajectory order, precision, recall, final response quality, latency, and failure rate separately.
  • For RAG retrieval, use task-type embeddings appropriately and evaluate retrieval quality with ranking metrics such as Mean Reciprocal Rank.
  • For multimodal image and video evaluation, generate rubrics from prompts, inspect generated questions and validator reliability, and use counterexample prompts to validate score behavior.
  • Use batching, Cloud Storage JSONL, BigQuery joins, Dataflow pipelines, or asynchronous batch evaluation when evaluation volume is large.
  • Throttle evaluation calls with evaluation_service_qps or RateLimiter utilities when quota or autorater QPS is a concern.
  • Meta-evaluate autoraters against golden labels with agreement and correlation metrics before trusting them for important decisions.
  • Clean up Vertex AI experiments, remote agents, staging buckets, pipeline jobs, and evaluation resources when notebooks are complete.

Avoid this

  • Running notebooks without enabling Vertex AI API, setting PROJECT_ID and LOCATION, authenticating in Colab, or restarting the runtime after package installation.
  • Treating small demonstration datasets as rigorous benchmarks even though several notebooks recommend around 100 examples for stronger aggregate metrics.
  • Comparing models or prompts with different datasets, prompt templates, metrics, regions, or experiment settings, which weakens the comparison.
  • Passing incorrectly shaped datasets, mismatched column names, missing reference fields, or un-serialized JSON trajectories and tool calls into EvalTask.
  • Trusting autorater output without strict schemas, parsing functions, raw-output inspection, or meta-evaluation against golden labels.
  • Ignoring row-level failures, rubric verdicts, generated questions, validator reliability, or explanations and looking only at summary scores.
  • Forgetting that agent deployments can take around 10 minutes and often require Cloud Storage staging, session inputs, returned intermediate steps, or AgentInfo metadata.
  • Missing scale and environment constraints such as Cloud Storage destinations, service-account roles, online evaluation quotas, region-specific model availability, and GA versus preview SDK metric changes.

From the notebooks

🛡️ AI Brand Safety: Three-Tier Agent Anomaly Detection

  • Compare prompts, tool calls, and responses against separate Vector Search indices to reduce structural false positives.
  • Generate industry-specific golden baselines using the agent instruction and INDUSTRY_VERTICAL.
  • Request JSON from Gemini with response_mime_type=“application/json” and parse it with json.loads.
  • Log full execution traces to BigQuery as a flight recorder for later threshold tuning without new LLM calls.
  • Route safety finish reasons and novelty anomalies to Tier 1 for 100% audit.

Using “task type” embeddings for improving RAG search quality

  • Use QUESTION_ANSWERING for question texts and RETRIEVAL_DOCUMENT for answer documents in Q&A RAG.
  • Use SEMANTIC_SIMILARITY as a baseline when comparing retrieval quality.
  • Evaluate search quality with ranking metrics such as Mean Reciprocal Rank.
  • Batch multiple texts per embedding API call to reduce repeated calls.
  • Consider tuning text embeddings when pre-trained task-type embeddings do not fit proprietary or specialized content.

Evaluate a CrewAI agent on Vertex AI Agent Engine (Customized template)

  • Evaluate agents both online and offline, using subjective and objective evaluation signals.
  • Track single tool selection, trajectory order, response generation, latency, and failure rate when evaluating agents.
  • Use reference trajectories in the evaluation dataset when checking expected tool calls.
  • Wrap custom agent output so it includes response and predicted_trajectory for evaluation.
  • Clean up experiments and remote agents after the notebook run.

Evaluating a LangChain Agent on Vertex AI Agent Engine (Prebuilt template)

  • Evaluate agents with both objective metrics and subjective feedback to build trust in behavior.
  • Use agent_executor_kwargs={“return_intermediate_steps”: True} so tool calls can be evaluated.
  • Prepare evaluation data with prompts and reference trajectories for tool and trajectory metrics.
  • Evaluate single tool selection before broader trajectory and response quality metrics.
  • Use custom pointwise metrics when response quality depends on whether the response follows tool choices.

Evaluate a LangGraph agent on Vertex AI Agent Engine (Customized template)

  • Evaluate agents both online and offline using subjective and objective signals.
  • Use prompts, reference responses, and reference trajectories in evaluation datasets when available.
  • Start with single tool selection, then evaluate full trajectories and generated responses.
  • Use trajectory metrics such as exact match, in-order match, any-order match, precision, and recall.
  • Parse custom agent output into response and predicted_trajectory before passing it to EvalTask.

Guardrail Classifier Agent

  • Use a dedicated guardrails node before the chat node to decide whether to answer or block.
  • Use temperature=0 and seed=0 for guardrail classification consistency.
  • Return structured JSON with response_schema and validate it with Pydantic.
  • Provide a safe guardrail_response for blocked requests.
  • Store conversation history in state so the classifier and chat node can use prior turns.

Create & Deploy Agent and Run Gen AI Agent Evaluation

  • Create a small agent-specific evaluation dataset before running evaluation.
  • Run inference first so the dataset contains response and intermediate_events columns.
  • Use AgentInfo.load_from_agent with the agent definition and deployed resource name.
  • Persist managed evaluation results to Cloud Storage when results need to be retrieved later.
  • Poll evaluation runs until they reach SUCCEEDED, FAILED, or CANCELLED before displaying final results.

Create a Gen AI Agent Evaluation for a Deployed Agent

  • Run inference first so the dataset includes intermediate_events and response columns before evaluation.
  • Persist evaluation datasets and results to a Cloud Storage destination.
  • Provide AgentInfo with agent instruction and tool definitions for agent evaluation.
  • Poll the evaluation run until SUCCEEDED, FAILED, or CANCELLED before retrieving full results.
  • Retrieve with include_evaluation_items=True to inspect detailed evaluation items.

Bring-Your-Own-Autorater using CustomMetric

  • Use response_mime_type and response_schema to get structured autorater output.
  • Return both a numeric metric score and an explanation from the custom metric.
  • Keep the custom metric score field name aligned with the CustomMetric name.
  • Compare multiple prompt templates under the same EvalTask dataset and metrics.
  • Use notebook_utils to display results, explanations, radar plots, bar plots, and experiment runs.

Bring your own computation-based CustomMetric

  • Initialize vertexai with project and location before evaluation.
  • Use pandas DataFrame columns such as prompt, response, and reference for EvalTask datasets.
  • Return CustomMetric results as a dictionary from metric name to numeric score.
  • Use precomputed response and reference columns when evaluating without inference.
  • Display evaluation results with notebook_utils.display_eval_result.

Migrate from PaLM to Gemini model

  • Use an EvalTask with multiple metrics when comparing foundation models.
  • Include prompt and reference columns in the summarization evaluation dataset.
  • Generate unique experiment run names with notebook_utils.generate_uuid(8).
  • Use at least 100 examples for best evaluation results.
  • Visualize qualitative metrics separately from ROUGE and BLEU.

Compare Generative AI Models

  • Use the same dataset and prompt template across model runs for controlled comparison.
  • Define an EvalTask with explicit metrics before running evaluations.
  • Store runs under an experiment name for later display and comparison.
  • Inspect metric explanations, not only summary scores.
  • Visualize multiple metrics with a radar plot when comparing models.

Customize Model-based Metrics to Evaluate a Gen AI model

  • Initialize Vertex AI with an explicit project and location.
  • Use input_variables that match columns in the evaluation dataset.
  • Define clear criteria and rating rubrics for model-based metrics.
  • Use reference answers when evaluating summarization alignment.
  • Inspect reusable templates with MetricPromptTemplateExamples before customizing.

Enhancing quality and explainability with Vertex AI Evaluation

  • Generate multiple candidate responses before ranking for quality.
  • Use pairwise evaluation to select the best response among candidates.
  • Use pointwise evaluation to report quality and groundedness explanations for the selected response.
  • Use a prompt template that includes both instruction and context during evaluation.
  • Use unique experiment_run_name values with uuid.uuid4().

Evaluate agent final answer with custom parsing

  • Use a human reference response as the golden answer for judging agent output validity.
  • Request structured JSON from the autorater when downstream parsing is needed.
  • Use CustomOutputConfig with a parsing function to append parsed output to evaluation results.
  • Return raw autorater output alongside parsed results for inspection.
  • Separate helper functions, prompt template, metric definition, dataset preparation, and evaluation execution.

Evaluate your autorater with meta-evaluation

  • Compare automated evaluator outputs against golden labels before trusting the autorater.
  • Use strict output formats and parsing functions for LLM judge results.
  • Align ratings by shared sorted IDs before computing metrics.
  • Report multiple metrics: confusion matrix, Cohen’s kappa, Spearman correlation, and Kendall correlation.
  • Evaluate both a basic rater and a self-consistency rater.

Evaluate Generative Model Tool Use

  • Define explicit tool evaluation metrics before creating EvalTask.
  • Use a pandas DataFrame with response and reference columns for evaluation datasets.
  • Give FunctionDeclaration parameters clear types, descriptions, and required fields.
  • Use notebook_utils.generate_uuid to create distinct experiment run names.
  • Display evaluation results with notebook_utils.display_eval_result.

Evaluate groundedness with custom parsing

  • Prompt the response model to answer using only the provided context.
  • Ask the autorater to be strict and avoid world knowledge unless trivial.
  • Return raw autorater output and parse it with CustomOutputConfig for detailed analysis.
  • Use structured labels for supported, unsupported, contradictory, and no_rad sentences.
  • Compute an overall groundedness score from parsed sentence verdicts.

Evaluate images with Gecko

  • Generate rubrics from each prompt before validation so the metric adapts to prompt-specific challenges.
  • Use CustomOutputConfig with return_raw_output=True and parsing_fn for custom validator outputs.
  • Compare matching prompts with similar counterexample prompts to show high-quality and low-quality responses.
  • Inspect generated questions and validator reliability, and manually add questions when needed.
  • Aggregate validation results into final scores and per-question QA results.

Evaluate LangChain

  • Decompose multi-turn chats into per-turn examples with conversation_history before batch prediction.
  • Use temperature=0 for repeatable chatbot and evaluator outputs.
  • Map dataset columns explicitly with metric_column_mapping={“prompt”: “user”}.
  • Combine built-in metrics such as fluency, coherence, and safety with a task-specific custom metric.
  • Log evaluation results under an experiment name for run comparison.

Evaluating multimodal task

  • Use a structured response schema with score and explanation for autorater outputs.
  • Define a clear metric definition, criteria, rating rubric, and evaluation steps in the custom metric prompt.
  • Include both summary metrics and row-based metrics when reviewing evaluation results.
  • Inspect sampled explanations to validate the evaluation behavior.
  • Clean up the Vertex AI experiment when it is no longer needed.

Evaluate generated answers from Retrieval-Augmented Generation (RAG) using Rapid Evaluation and Dataflow ML with Vertex AI pipelines

  • Validate the evaluation dataset before starting remote evaluation.
  • Store pipeline data, source modules, requirements, outputs, and temporary files in Cloud Storage paths.
  • Package the Apache Beam module with requirements.txt and setup.py for Dataflow workers.
  • Use WaitGcpResourcesOp after DataflowPythonJobOp before reading output artifacts.
  • Retrieve both row-level metrics and aggregated summary metrics after pipeline completion.

Evaluate Generated Answers from Retrieval-Augmented Generation (RAG) for Question Answering with Gen AI Evaluation Service SDK

  • Use at least one evaluation example, with around 100 examples recommended for high-quality aggregated metrics and statistical significance.
  • Use reference-free evaluation when assessing generated answers against retrieved context without golden answers.
  • Use referenced evaluation when golden answers are available for comparison.
  • Inspect predefined metric templates before selecting metrics.
  • Use detailed explanations to understand why scores were assigned for individual instances.

Evaluate a Translation Model

  • Install google-cloud-aiplatform with the evaluation extra before importing evaluation APIs.
  • Initialize vertexai with an explicit project and location.
  • Use a structured pandas DataFrame with source, response, and reference columns.
  • Inspect both summary metrics and row-based metrics from EvalResult.
  • Delete the ExperimentRun created by evaluation when finished.

Evaluate videos with Gecko

  • Generate rubrics from the prompt so metrics reflect prompt-specific challenges.
  • Separate rubric generation from the validator step.
  • Use custom parsing functions for rubric generation and validation outputs.
  • Inspect generated questions and validator reliability before relying on the score.
  • Include matching and counterexample prompts to demonstrate score differences.

Evaluating Agents - Evaluate a CrewAI agent with Vertex AI Gen AI Evaluation Service

  • Evaluate agents with both monitoring-style task metrics and observability considerations such as latency and failure rate.
  • Use reference trajectories to evaluate expected tool choices and ordering.
  • Start with single-tool usage evaluation before broader trajectory evaluation.
  • Evaluate final responses separately from tool trajectory quality.
  • Use custom pointwise metrics when standard text metrics are insufficient for agent behavior.

Evaluating Agents - Evaluate a LangGraph agent with Vertex AI Gen AI Evaluation Service

  • Use an experiment name when initializing Vertex AI to organize evaluation runs.
  • Include prompts and reference trajectories in the evaluation dataset for agent trajectory metrics.
  • Evaluate single tool use before broader trajectory and response evaluations.
  • Use multiple trajectory metrics to compare exact order, in-order match, any-order match, precision, and recall.
  • Use custom pointwise metrics when response quality depends on the agent’s tool trajectory.

Evaluating prompts at scale with Gemini Batch Prediction API

  • Test the prompt on one image with generate_content before launching batch prediction.
  • Request JSON output in both the prompt and generation config.
  • Store evaluation_ts, evaluation_id, prompt_text, and gcs_uri with every request row.
  • Load ground truth into BigQuery and join predictions to compute correctness.
  • Use BigQuery views to parse raw responses and expose an evaluation table.

Intro to Batch Evaluations with the Gemini API

  • Store each evaluation item as one JSON object per JSONL line.
  • Include prompt, response, and optional reference fields in the evaluation dataset.
  • Use aggregation metrics such as AVERAGE and MEDIAN for dataset-level summaries.
  • Write evaluation outputs to Cloud Storage and read evaluation_results.jsonl separately from aggregation_results.jsonl.
  • Use helper functions to parse nested JSON results into readable tabular output.

Getting Started with Vertex AI Python SDK for Gen AI Evaluation Service

  • Define evaluation criteria and rating rubrics before running EvalTask.
  • Store responses in a pandas DataFrame with a response column for evaluation.
  • Inspect both summary metrics and row-based metrics in the EvalResult.
  • Delete the created ExperimentRun during cleanup.

Gen AI Evaluation Service SDK Preview-to-GA Migration Guide

  • Use MetricPromptTemplateExamples and adjust them for your use case instead of relying on removed black-box metrics.
  • Define custom PointwiseMetric or PairwiseMetric rubrics when discontinued metrics are still needed.
  • Use instruction_following instead of fulfillment and verbosity instead of summarization_verbosity.
  • Assemble instruction and context into a single prompt for GA metric templates.
  • Use PairwiseMetric in EvalTask for side-by-side evaluation of two models.

Rubric evaluation - Multimodal and Custom metric for text quality

  • Inspect, edit, or add generated rubrics before final evaluation.
  • Use predefined rubric metrics when they fit the multimodal task.
  • Use custom prompt templates when text-quality criteria must be controlled.
  • Return raw autorater output to debug reasoning and verdicts.
  • Write rubrics as granular binary yes/no constraints and avoid hallucinated or repeated criteria.

Evaluate and Optimize Prompt Template Design for Better Results

  • Use one shared EvalTask so prompt templates are compared against the same dataset, metrics, and experiment.
  • Use prompt template variables such as {instruction} and {context} to compile prompts from dataset fields.
  • Evaluate multiple prompt templates systematically before choosing one.
  • Use a consistent model, gemini-2.5-flash, across prompt variants for a fair comparison.
  • Inspect both evaluation results and summarization_quality explanations, not only aggregate scores.

Rubric-based instruction following evaluation using Gen AI Evaluation Service

  • Use predefined rubric metrics for supported use cases such as Instruction Following, Multimodal Understanding, and Text Quality.
  • Generate rubrics first when you want to review and revise them before scoring responses.
  • Display evaluation results to inspect rubrics, score, rubric_verdict_pairs, and raw_outputs.
  • Use environment variables for project and region defaults in notebook setup.

Evaluate Gemini Structured Output

  • Use structured output with response_mime_type application/json and response_schema for consistent JSON.
  • Keep reference ground truth alongside each model response in the evaluation dataset.
  • Evaluate both schema validity and extraction accuracy with custom metrics.
  • Use DeepDiff to inspect field-wise differences between reference and response.
  • Compare multiple Gemini model ids over the same prompt and input images.

Evaluate images with predefined Gecko

  • Generate rubrics from the user prompts before evaluating responses.
  • Use counterexample prompts with the same images to demonstrate high and low quality evaluations.
  • Inspect generated questions and validator reliability when analyzing quality.
  • Manually add questions when desired for an application.
  • Review Vertex AI pricing and estimate costs before running evaluation.

Use Gen AI Evaluation SDK to Evaluate Models in Vertex AI Studio, Model Garden, and Model Registry

  • Use the same EvalTask configuration with fixed dataset and metrics when comparing model architectures.
  • Use prompt_template variables that match columns in the evaluation dataset.
  • Use model-based pointwise, pairwise, and computation-based metrics for different evaluation needs.
  • Inspect evaluation results and explanations with notebook_utils display helpers.
  • Log prompt-template evaluations to an experiment and visualize comparisons with radar and bar plots.

Evaluate videos with predefined Gecko

  • Generate rubrics from the user prompts before evaluating responses.
  • Use similar counterexample prompts to demonstrate high-quality and low-quality response differences.
  • Inspect generated questions and validator reliability when analyzing quality.
  • Manually add questions when needed for an application.

Evaluate Generative Model Tool Use with Custom Code Execution

  • Define separate metrics for tool-call validity, tool-name match, parameter-key match, and parameter key-value match.
  • Handle negative examples where no tool calls are expected.
  • Return 0.0 for missing expected tool calls and 1.0 for true negatives.
  • Use remote custom functions to compute TP, FP, and FN before calculating precision, recall, and F1.

Evaluate your ADK agent using Vertex AI Gen AI Evaluation service

  • Use a small reference dataset with prompts and expected tool trajectories.
  • Evaluate single tool selection before broader trajectory metrics.
  • Track evaluation runs with unique experiment run names.
  • Separate trajectory metrics from response metrics such as safety and coherence.
  • Use custom pointwise criteria when response quality depends on tool choices.

Using Gen AI Evaluation SDK for Google Observability Gen AI multi-modal datasets

  • Use environment variables GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION as fallbacks for notebook parameters.
  • Keep prompt, response, and system instruction sources explicit when constructing ObservabilityEvalCase.
  • Call show() on the loaded dataset and evaluation result to inspect inputs and outputs.

Evaluating Third-Party LLMs with the Vertex AI Gen AI Evaluation SDK

  • Use environment variables or secure storage for provider API keys.
  • Enable Vertex AI API before running the workflow.
  • Set project and region from parameters or GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION.
  • Use the same dataset and metrics when comparing multiple models.
  • Generate rubrics from prompts before rubric-based comparison evaluation.

Migrating Foundation Models: A Practical Guide with Gen AI Evaluation Serivce

  • Use predefined rubric metrics such as GENERAL_QUALITY for structured model comparison.
  • Compare multiple candidates by passing a list of datasets to evaluate().
  • Use .show() on EvaluationDataset and EvaluationResult for in-notebook reports.
  • Use batch_evaluate() for large datasets or when immediate results are not required.
  • Use environment variables or secure storage for API keys instead of hardcoding them.

Overview

  • Load API keys from environment variables and prompt with getpass when missing.
  • Use explicit task context to enforce sequential dependencies between agents.
  • Store test inputs and expected reference trajectories in a Phoenix dataset.
  • Evaluate trajectories with multiple metrics: exact match, precision, in-order match, and any-order match.
  • Add a custom code evaluator for agent-name order matching.

Gen AI Eval - Multi-turn Agent Eval, User Simulation, Metric Registration, Auto-Loss Analysis

  • Load AgentInfo from the ADK agent before scenario generation.
  • Throttle evaluation calls with evaluation_service_qps.
  • Register custom metrics once and reference their metric_resource_name in evaluation runs.
  • Return a dictionary from custom LLM metric result_parsing_function.
  • Use predefined multi-turn metrics for tool use quality, trajectory quality, and task success.

Getting Started: Quick Gen AI Evaluation

  • Enable the Vertex AI API before using the SDK.
  • Use environment variables for PROJECT_ID and LOCATION defaults.
  • Generate responses with run_inference() before calling evaluate().
  • Use the SDK’s automatic handling of common data formats to avoid manual conversions.
  • Inspect intermediate and final results with show().

View Gen AI Agent Evaluation Run Results

  • Install google-cloud-aiplatform[evaluation] before using evaluation features.
  • Use include_evaluation_items=True when detailed case-level results are needed.
  • Use evaluation_run.show() to visualize failed run errors or embedded evaluation reports.

Intro to Logprobs

  • Use response_logprobs=True only when token confidence data is needed.
  • Use logprobs to inspect top alternative tokens and debug model behavior.
  • Constrain classification outputs with response_schema and text/x.enum before comparing label confidence.
  • Flag classifications for human review when top choices have close log probabilities.
  • Convert log probabilities with math.exp before applying probability thresholds.

Question Answering with Generative Models on Vertex AI

  • Use specific, context-rich, grammatically correct prompts for question answering.
  • Experiment with generation parameters such as temperature and max_output_tokens.
  • Use few-shot examples when the desired answer style is specific or short.
  • For closed-domain Q&A, provide the knowledge base as prompt context.
  • Tell the model to return “Information not available in provided context” when the context lacks the answer.

Text Classification with Generative Models on Vertex AI

  • Use explicit candidate labels or task instructions in classification prompts.
  • Use few-shot examples when the model should follow a specific class mapping or answer format.
  • Use low temperature and limited max output tokens for concise classification responses.
  • Evaluate classification outputs against ground truth labels when they are available.
  • Keep custom model training separate when prompt-based classification is sufficient for the notebook scope.

Text Summarization with Generative Models on Vertex AI

  • Initialize Vertex AI with an explicit project and location before model calls.
  • Use low temperature for concise summary generation when consistency is desired.
  • Adjust temperature, max_output_tokens, top_k, and top_p for different output styles.
  • Write prompts that specify the desired format, such as bullet points, TL;DR, to-dos, or title options.
  • Evaluate generated summaries against human-created summaries with ROUGE metrics.

Get Started with Vertex AI Prompt Optimizer

  • Use zero-shot optimization for rapid prompt refinement when no evaluation dataset is available.
  • Use data-driven optimization when sample inputs and expected outputs define what better performance means.
  • Provide examples where the current system instruction performs poorly for prompt optimization.
  • Use 50-100 distinct samples for reliable prompt optimization results.
  • Include a target field for computation-based metrics such as question_answering_correctness.

Get started with Vertex Prompt Optimizer - Custom metric

  • Use a structured Pydantic OptimizationConfig before submitting the optimizer job.
  • Validate the deployed custom metric endpoint with a test request before running optimization.
  • Use response_mime_type and response_schema to force the autorater to return JSON.
  • Store the optimizer configuration as config.json in Cloud Storage.
  • Use weighted metrics to balance question_answering_correctness and the custom engagement metric.

Get Started with Vertex AI Prompt Optimizer - Long prompt

  • Use placeholders to freeze static prompt sections while optimizing the rest of a long system instruction.
  • Provide examples where the current system instruction performs poorly.
  • Use 50-100 distinct samples for reliable prompt optimization results.
  • Use a target column for computation-based metrics such as question_answering_correctness.
  • Use response_schema and response_mime_type to constrain integer JSON output.

Get Started with Vertex AI Prompt Optimizer - Multimodality

  • Use labeled rows containing query, image GCS URI, and target for question_answering_correctness evaluation.
  • Use examples where the current system instruction performs poorly when building an optimization dataset.
  • Validate optimizer settings with the Pydantic OptimizationConfig before submitting the job.
  • Store both optimizer configuration and results in Cloud Storage.
  • Set has_multimodal_inputs to True when optimizing prompts with image inputs.

Get Started with Vertex AI Prompt Optimizer - Tool usage

  • Use a structured pydantic OptimizationConfig before submitting the job.
  • Validate tools and tool_config JSON before running optimization.
  • Pass FunctionDeclaration and ToolConfig as JSON structures to Prompt Optimizer.
  • Use weighted tool-call metrics for tool name, parameter key, and parameter key-value matching.
  • Retrieve the best prompt programmatically from GCS output files for application use.

Vertex Prompt Optimizer Notebook UI (Preview)

  • Use default optimization configurations as the initial setup.
  • Focus validation examples on the issues you want to address.
  • Use 50-100 distinct samples for reliable results.
  • Validate prompt and data before launching the optimizer job.
  • Use multi-metric settings only when more than one metric is needed.

Evaluating Vertex RAG Engine Generation with Vertex AI Python SDK for Gen AI Evaluation Service

  • Use environment variables for GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION when project values are not provided.
  • Configure chunk_size and chunk_overlap when importing files into the RAG corpus.
  • Separate retrieved context collection from grounded response generation before evaluation.
  • Evaluate prompts, retrieved_context, and response together in a pandas DataFrame.
  • Use a custom metric rubric that checks accuracy, completeness, and groundedness.

Advanced RAG Techniques - Vertex RAG Engine Retrieval Quality Evaluation and Hyperparameters Tuning

  • Evaluate retrieval quality because poor retrieval can lead to irrelevant, incomplete, or hallucinated output.
  • Tune chunk size, chunk overlap, top-k, vector distance threshold, and embedding model based on recall, precision, and nDCG.
  • Use recall@k, precision@k, and nDCG@k to evaluate retrieval from different perspectives.
  • Reduce chunk size, increase chunk overlap, or increase top-k when recall is too low.
  • Reduce top-k, reduce chunk overlap, or increase chunk size when precision is too low.

Prepare High-Quality Preference Data for Gemini 2.5

  • Filter examples with missing prompts, chosen content, or rejected content before evaluation.
  • Prepare separate prompt-response dataframes for preferred and dispreferred responses with columns named prompt and response.
  • Sort evaluation case results by eval_case_index before attaching scores back to the dataframe.
  • Visualize preferred and dispreferred score distributions before choosing filtering thresholds.
  • Use win-margin filtering to keep pairs with clear quality separation.

Supervised Fine-Tuning with integrated Gen AI Evaluation

  • Use separate training and validation JSONL datasets for supervised tuning with integrated evaluation.
  • Store detailed row-level evaluation results in Cloud Storage using output_config.
  • Use custom model-based metrics with clear prompt templates and optional judge model system instructions.
  • Monitor asynchronous tuning jobs by refreshing job state until terminal completion.
  • Inspect checkpoint evaluation runs in Vertex AI Experiments to compare model progress over time.

Integrate Custom Metrics into Gemini Supervised Fine-Tuning

  • Track task-specific quality criteria during tuning instead of relying only on training loss.
  • Use validation data so custom metrics can be evaluated during training.
  • Aggregate custom metric scores with aggregation_metrics such as AVERAGE.
  • Store detailed evaluation results in GCS for later inspection.
  • Replace sample training and validation paths with production datasets.

Supervised Fine Tuning with Gemini 2.5 Flash for Image Captioning

  • Use high-quality, well-labeled, task-relevant training data because low-quality data can hurt performance and introduce bias.
  • Use a separate validation set to evaluate model performance.
  • Choose evaluation metrics that reflect the task; this notebook uses ROUGE for image caption text generation.
  • Experiment with generation parameters and prompt structures to improve task performance.
  • Start with recommended default tuning hyperparameters, then customize epochs, learning rate multiplier, or adapter size for specific needs.

Supervised fine-tuning with Gemini 2.0 Flash for Q&A using the Google Gen AI SDK

  • Establish a baseline with the default model before fine-tuning.
  • Use high-quality, well-labeled, task-relevant training data.
  • Keep test data formatted like training data to prevent training and serving skew.
  • Use system instructions to define desired model behavior and response style.
  • Use a separate evaluation set to assess model performance.

Supervised Fine Tuning with Gemini 2.5 Flash for Article Summarization

  • Use high-quality, well-labeled, task-relevant training data.
  • Use a separate validation or evaluation dataset to measure tuned model performance.
  • Evaluate the base model before tuning and the tuned endpoint after tuning.
  • Choose task-appropriate metrics; the notebook uses ROUGE-L for summarization.
  • Experiment with generation parameters, prompt structure, epochs, and learning rate multiplier.

Evaluate

  • Use environment variables for project, location, and image path configuration.
  • Set RANDOM_STATE for reproducible sampling.
  • Use stratify=True for class-balanced evaluation samples.
  • Evaluate both all classes and a targeted subset of high-interest classes.
  • Inspect summary metrics together with confusion matrices and detailed rows.

Know Your Customer Use Case - Gemini Grounding with Google Search

  • Use explicit system instructions to define role, scope, output format, and no-result behavior.
  • Ground sensitive claims with Google Search and display source metadata for verification.
  • Check for missing candidates, grounding metadata, grounding supports, and out-of-bounds chunks before extracting sources.
  • Separate reusable single-entity generation from batch entity processing.
  • Evaluate outputs with task-specific custom metrics and rubrics.

Accelerate LLM Inference with EAGLE Speculative Decoding on Vertex AI

  • Pin package versions for reproducibility.
  • Deploy baseline and EAGLE endpoints on identical hardware for a fair comparison.
  • Run 100-prompt smoke tests before the full benchmark.
  • Use ShareGPT conversations as a realistic benchmark workload.
  • Use the model tokenizer locally so prompt and response token lengths are measured accurately.

Qwen 3 evaluation - Bring your own data eval

  • Use a custom dataset for domain-specific model assessment.
  • Compare a candidate response against a baseline_model_response row by row.
  • Use pairwise_summarization_quality for summary quality comparison.
  • Log evaluation runs to a Vertex AI Experiment for tracking and comparison.
  • Optionally clean up experiments after running evaluations.

Using open autorater for running evaluations with Vertex AI Gen AI Evaluation

  • Test the deployed judge endpoint with a sample prediction before running evaluation.
  • Use a structured evaluation dataset with user_input, ground_truth, assistant_response, and human rating columns.
  • Define the scoring rubric and required autorater output format explicitly in the metric prompt.
  • Use tokenizer.apply_chat_template for prompts sent to the Selene model.
  • Compare autorater scores against human ratings with evaluate_autorater before trusting judge alignment.

Hugging Face DLCs: Using Gemma for running evaluations with Vertex AI Gen AI Evaluation

  • Initialize Vertex AI with project and location before creating models, endpoints, and evaluations.
  • Use a tokenizer chat template before sending prompts to the Hugging Face TGI endpoint.
  • Wrap endpoint prediction in a model function so EvalTask can call it consistently.
  • Track evaluation runs with Vertex AI Experiments through experiment and experiment_run_name.
  • Combine reference-based metrics such as rouge_l_sum with model-based metrics such as summarization_quality and fluency.

MetaMath with Vertex AI Open Source Model Tuning

  • Use an 80/20 train-validation split with a fixed seed for reproducibility.
  • Limit validation rows to satisfy the Vertex AI validation dataset requirement.
  • Upload JSONL files to Cloud Storage because the Vertex AI tuning service cannot access local notebook files directly.
  • Use the same MetaMath prompt template for tuned-model testing and official-model comparison.
  • Use low temperature, top_p 1.0, and top_k 1 for factual math output.

Build and deploy a Hugging Face smolagent using DeepSeek-r1 on Vertex AI

  • Use environment variables GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION as fallbacks for project and location.
  • Initialize vertexai with project, location, and staging_bucket before creating Vertex AI resources.
  • Use Vertex AI Model Registry to manage the imported Hugging Face model lifecycle.
  • Use a dedicated endpoint display name derived from the model ID.
  • Set explicit serving container predict route, health route, port, and environment variables for the vLLM container.

Clearbox for Ranking Tuning

  • Pin random seeds for reproducible training.
  • Compare trained models against individual signal baselines.
  • Train on reciprocal ranks for better stability across signal distribution changes.
  • Use ClearBox feature utilities such as FillNaN so production serving uses the same logic as training.
  • Make rank-like signals monotonically increasing before reciprocal-rank computation.

Gemini Enterprise answer eval using BLEU, ROUGE, BERT, Similarity Score

  • Use a golden dataset with explicit query and expected-answer columns.
  • Evaluate each answer with multiple metrics instead of a single score.
  • Add timestamps to evaluation outputs for run tracking.
  • Convert tuple ratings and timestamps to strings before writing to Sheets or BigQuery.
  • Create BigQuery datasets and tables if they do not already exist.

Setup

  • Processes records in batches of 200 before calling the Rank API.
  • Uses ignore_record_details_in_response=True when only scores are needed.
  • Persists computed scores to Cloud Storage for reuse and inspection.
  • Filters evaluation to labeled datapoints to avoid underestimating model performance from unlabeled relevant examples.
  • Removes query-document pairs missing from either ground truth or predictions before metric calculation.

Tutorial for Running Prompt Management and Evaluation

  • Version prompts each time changes are saved so iterations can be compared.
  • Test a prompt with sample input before saving and evaluating it.
  • Create datasets in Cloud Storage and upload CSV, JSON, or JSONL files for evaluation.
  • Run an initial evaluation before prompt changes to establish a baseline.
  • Use human-in-the-loop rating and automated metrics for evaluation.

Back to Evaluation · Best Practices Map

Graph View

Backlinks