Generative AI Google Brain

gemini-prompts-prompt-optimizer-get-started-with-vertex-ai-prompt-optimizer-long-prompt

3 min read

Get Started with Vertex AI Prompt Optimizer - Long prompt

Source notebook

Repo path: gemini/prompts/prompt_optimizer/get_started_with_vertex_ai_prompt_optimizer_long_prompt.ipynb · Open on GitHub · intermediate

Runs Vertex AI Prompt Optimizer to improve a long Gemini prompt using data-driven evaluation.

Summary

This notebook teaches how to configure and run the Vertex AI Prompt Optimizer Data-Driven Optimizer for a long prompt. It prepares a system instruction and prompt template with static and dynamic placeholders, loads a QA dataset from Cloud Storage, writes an optimization config to GCS, runs a Vertex AI prompt optimizer job, and retrieves the best optimized instruction from output files.

Key code patterns

Initialize Vertex AI client

LOCATION = os.environ.get("GOOGLE_CLOUD_REGION", "us-central1")
BUCKET_URI = f"gs://{BUCKET_NAME}"
client = vertexai.Client(project=PROJECT_ID, location=LOCATION)

Sets the project, region, bucket URI, and Vertex AI client used by the optimizer job.

Grant job service account roles

SERVICE_ACCOUNT = f"{PROJECT_NUMBER}-compute@developer.gserviceaccount.com"
for role in ["aiplatform.user", "storage.objectAdmin", "artifactregistry.reader"]:
    ! gcloud projects add-iam-policy-binding {PROJECT_ID} \
      --member=serviceAccount:{SERVICE_ACCOUNT} \
      --role=roles/{role} --condition=None

Gives the backend job permissions to call Vertex AI, access GCS objects, and download components.

Use static and dynamic placeholders

system_instruction = """
<INSTRUCTIONS>...</INSTRUCTIONS>
<EXAMPLES>
{{examples}}
</EXAMPLES>
"""
prompt_template = """
<QUESTION>
{{question}}
</QUESTION>
<ANSWER>
{{target}}
</ANSWER>
"""

Keeps examples fixed with a static placeholder while mapping question and target from dataset columns.

Build optimization config

vapo_data_settings = {
    "target_model": "gemini-2.5-flash",
    "optimization_mode": "instruction",
    "eval_metrics_types": ["question_answering_correctness"],
    "placeholder_to_content": json.loads(placeholder_to_content),
    "response_schema": response_schema,
    "input_data_path": input_data_path,
    "output_path": output_path,
}

Defines the model, metric, prompt inputs, placeholders, output format, and GCS paths for optimization.

Run Prompt Optimizer

vapo_data_run_config = {
    "config_path": config_path,
    "wait_for_completion": True,
    "service_account": SERVICE_ACCOUNT,
}
result = client.prompt_optimizer.optimize(method="vapo", config=vapo_data_run_config)

Submits the VAPO backend job and waits for completion.

Read best result from GCS

best_instruction, _ = get_best_vapo_results(output_path)
print("The optimized instruction is:\n", best_instruction)

Shows how an application can retrieve the top optimized instruction from optimizer outputs.

Models & APIs used

  • Models: gemini-2.5-flash
  • APIs / services: Vertex AI, Cloud Storage, Cloud IAM, Artifact Registry
  • SDKs / libraries: google-cloud-aiplatform, vertexai, google-cloud-storage, pydantic, etils, pandas

When to use this

Use this pattern when optimizing a long Gemini prompt against labeled examples and a task-specific evaluation metric.

Gotchas & caveats

  • Vertex AI API must be enabled for the project.
  • A Cloud Storage bucket is required for input data, config, and optimization results.
  • The backend job runs as the default Compute Engine service account and needs Vertex AI User, Storage Object Admin, and Artifact Registry Reader roles.
  • Dynamic placeholders must map to dataset column names such as question and target.
  • For reliable results, the notebook recommends 50-100 distinct samples where the current prompt performs poorly.
  • target_model_qps, optimizer_model_qps, and eval_qps should match available quota.
  • Default model and optimizer locations are us-central1.
  • Cleanup code can delete the latest custom job and remove the entire configured bucket.

Best practices

  • Use placeholders to freeze static prompt sections while optimizing the rest of a long system instruction.
  • Provide examples where the current system instruction performs poorly.
  • Use 50-100 distinct samples for reliable prompt optimization results.
  • Use a target column for computation-based metrics such as question_answering_correctness.
  • Use response_schema and response_mime_type to constrain integer JSON output.
  • Store the optimizer config in Cloud Storage and pass its GCS path to the optimizer job.
  • Retrieve the best instruction programmatically from Prompt Optimizer output files.

Graph View

Backlinks