Generative AI Google Brain

gemini-prompts-prompt-optimizer-vertex-ai-prompt-optimizer-ui

3 min read

Vertex Prompt Optimizer Notebook UI (Preview)

Source notebook

Repo path: gemini/prompts/prompt_optimizer/vertex_ai_prompt_optimizer_ui.ipynb · Open on GitHub · intermediate

Shows how to configure, run, and inspect Vertex AI Prompt Optimizer jobs from a notebook UI.

Summary

This notebook teaches how to use Vertex AI Prompt Optimizer to refine or translate a system instruction for a target Gemini model. It walks through authentication, loading helper code, defining a system instruction and prompt template, configuring project paths, metrics, model, optimization settings, running the optimizer job, monitoring progress, and viewing generated prompt results in a notebook UI.

Key code patterns

Prompt template setup

SYSTEM_INSTRUCTION = "Answer the following question. Let's think step by step.\n"
PROMPT_TEMPLATE = "Question: {question}\n\nAnswer: {target}"

Separates fixed system instruction from dynamic prompt fields used for evaluation.

Prompt and data validation

label_enforced = vapo_lib.is_run_target_required(
    [EVAL_METRIC, EVAL_METRIC_1, EVAL_METRIC_2, EVAL_METRIC_3], ""
)
vapo_lib.validate_prompt_and_data(
    "\n".join([SYSTEM_INSTRUCTION, PROMPT_TEMPLATE]),
    input_data_path,
    PLACEHOLDER_TO_VALUE,
    label_enforced,
)

Checks whether labels are required for selected metrics and validates prompt inputs before starting optimization.

Optimizer parameter assembly

params = {
    "project": PROJECT_ID,
    "system_instruction": SYSTEM_INSTRUCTION,
    "prompt_template": PROMPT_TEMPLATE,
    "target_model": TARGET_MODEL,
    "target_model_location": LOCATION,
    "optimization_mode": OPTIMIZATION_MODE,
    "input_data_path": input_data_path,
    "output_path": output_path,
}

Collects the required job configuration for running Vertex AI Prompt Optimizer.

Run and monitor job

job = vapo_lib.run_apd(params, OUTPUT_PATH, display_name)
print(f"Job ID: {job.name}")
 
progress_form = vapo_lib.ProgressForm(params)
while progress_form.monitor_progress(job):
    time.sleep(5)

Starts the optimization job and polls progress in the notebook.

Inspect results UI

results_ui = vapo_lib.ResultsUI(RESULT_PATH)
display(HTML(results_df_html))
display(results_ui.get_container())

Displays generated templates and predictions from one or more optimizer runs.

Models & APIs used

When to use this

Use this pattern when optimizing or translating prompts for a target Gemini model with labeled validation data and metric-based evaluation.

Gotchas & caveats

  • Colab users must authenticate with google.colab.auth.authenticate_user().
  • PROJECT_ID, LOCATION, OUTPUT_PATH, and INPUT_DATA_PATH must be set before running the optimizer.
  • The input data must be CSV or JSONL validation samples, and some metrics require labeled targets.
  • The notebook recommends 50-100 distinct samples, though it states the tool can work with as few as 5.
  • The source model selection for prompt translation is limited to Google models.
  • All evaluation metrics are expected to be larger-is-better; MetricX is modified to a 0 to 25 scale.
  • Thinking budget defaults to -1, meaning no thinking for non-thinking models and auto thinking for thinking models.
  • QPS settings are explicitly configured for target model and evaluation calls.

Best practices

  • Use default optimization configurations as the initial setup.
  • Focus validation examples on the issues you want to address.
  • Use 50-100 distinct samples for reliable results.
  • Validate prompt and data before launching the optimizer job.
  • Use multi-metric settings only when more than one metric is needed.
  • Inspect generated responses from prompt templates after optimization.

Graph View

Backlinks