Generative AI Google Brain

gemini-prompts-prompt-optimizer-get-started-with-vertex-ai-prompt-optimizer

3 min read

Get Started with Vertex AI Prompt Optimizer

Source notebook

Repo path: gemini/prompts/prompt_optimizer/get_started_with_vertex_ai_prompt_optimizer.ipynb · Open on GitHub · intermediate

Shows zero-shot and data-driven prompt optimization with Vertex AI Prompt Optimizer.

Summary

This notebook teaches how to use Vertex AI Prompt Optimizer to refine prompts with and without an evaluation dataset. It sets up a Google Cloud project, Cloud Storage bucket, Vertex AI client, and service account permissions. The workflow runs zero-shot optimization, configures a VAPO data-driven job for a QA dataset, retrieves the best instruction from GCS, optionally explores results in Gradio, and cleans up resources.

Key code patterns

Initialize Vertex AI

PROJECT_ID = str(os.environ.get("GOOGLE_CLOUD_PROJECT"))
LOCATION = os.environ.get("GOOGLE_CLOUD_REGION", "us-central1")
BUCKET_URI = f"gs://{BUCKET_NAME}"
client = vertexai.Client(project=PROJECT_ID, location=LOCATION)

Sets the project, region, bucket URI, and Vertex AI client used by Prompt Optimizer.

Grant optimizer job roles

SERVICE_ACCOUNT = f"{PROJECT_NUMBER}-compute@developer.gserviceaccount.com"
for role in ["aiplatform.user", "storage.objectAdmin", "artifactregistry.reader"]:
    ! gcloud projects add-iam-policy-binding {PROJECT_ID} \
      --member=serviceAccount:{SERVICE_ACCOUNT} \
      --role=roles/{role} --condition=None

The backend job needs Vertex AI, Cloud Storage, and Artifact Registry permissions.

Zero-shot optimization

prompt = "You are a helpful assistant. Given a question with context, provide the correct answer to the question."
response = client.prompt_optimizer.optimize_prompt(prompt=prompt)
display(Markdown(response.suggested_prompt))

Refines a prompt without an evaluation dataset and returns a suggested prompt.

Validated VAPO config

vapo_data_settings = {
    "target_model": "gemini-2.5-flash",
    "optimization_mode": "instruction",
    "eval_metrics_types": ["question_answering_correctness", "fluency"],
    "eval_metrics_weights": [0.8, 0.2],
}
vapo_data_config = OptimizationConfig(**vapo_data_settings)
vapo_data_config_json = vapo_data_config.model_dump()

Uses a Pydantic model to structure data-driven optimization settings before submission.

Run VAPO job

config_path = f"{BUCKET_URI}/config.json"
with epath.Path(config_path).open("w") as config_file:
    json.dump(vapo_data_config_json, config_file)
result = client.prompt_optimizer.optimize(
    method=vertexai.types.PromptOptimizerMethod.VAPO,
    config=vapo_data_run_config,
)

Uploads the optimizer config to GCS and starts the Vertex AI backend optimization job.

Retrieve best prompt

best_instruction, _ = get_best_vapo_results(output_path)
print("The optimized instruction is:\n", best_instruction)

Reads optimizer output files in GCS and extracts the top-performing instruction.

Launch results viewer

interface = launch_app(
    share=True,
    server_port=7861,
    server_name="0.0.0.0",
    debug=False,
)

Starts a Gradio app for visually exploring VAPO results.

Models & APIs used

  • Models: gemini-2.5-flash
  • APIs / services: Vertex AI, Cloud Storage, Cloud IAM, Artifact Registry
  • SDKs / libraries: google-cloud-aiplatform, vertexai, google-cloud-storage, pydantic, etils, pandas, gradio

When to use this

Use this pattern when you need to improve Gemini prompts quickly or optimize them against task-specific examples and evaluation metrics.

Gotchas & caveats

  • The Vertex AI API must be enabled before running Prompt Optimizer jobs.
  • A Cloud Storage bucket is required for input data, config files, and optimization results.
  • The default Compute Engine service account needs Vertex AI User, Storage Object Admin, and Artifact Registry Reader roles.
  • Colab requires explicit user authentication with google.colab.auth.authenticate_user().
  • If the prompt dataset lacks a target ground-truth column, the config should set source_model instead.
  • The notebook installs google-cloud-aiplatform>=1.108.0 and pins protobuf==4.25.3.
  • Target, optimizer, and source model locations default to us-central1 in the config.
  • QPS settings should be based on quota, with target_model_qps and optimizer_model_qps defaulting to 1.

Best practices

  • Use zero-shot optimization for rapid prompt refinement when no evaluation dataset is available.
  • Use data-driven optimization when sample inputs and expected outputs define what better performance means.
  • Provide examples where the current system instruction performs poorly for prompt optimization.
  • Use 50-100 distinct samples for reliable prompt optimization results.
  • Include a target field for computation-based metrics such as question_answering_correctness.
  • Validate optimizer settings with a structured OptimizationConfig before starting the job.
  • Weight multiple evaluation metrics explicitly when using weighted_sum aggregation.
  • Retrieve the best optimized instruction programmatically from GCS for application use.

Graph View

Backlinks