Prompt Engineering — Best Practices

Distilled from 41 notebooks tagged Prompt Engineering in the GoogleCloudPlatform/generative-ai repository. The From the notebooks section below cites the per-notebook source for grounding.

Do this

• Be concise, specific, grammatically clear, and ask one task at a time.
• Use system instructions to set durable behavior, guardrails, persona boundaries, output restrictions, and source-of-truth rules.
• Use zero-shot, one-shot, or few-shot prompting intentionally; keep examples representative and avoid overloading the prompt with too many examples.
• For grounded Q&A, extraction, RAG, and agents, explicitly instruct the model to use provided context, retrieved documents, tools, or visible evidence instead of prior knowledge.
• Use low temperature or temperature 0 for deterministic classification, extraction, SQL generation, medical coding, security demonstrations, and repeatable evaluations.
• For structured outputs, set response_mime_type to application/json and use response_schema, response_json_schema, Pydantic schemas, enums, or closed vocabularies where appropriate.
• Mark required fields explicitly and use nullable fields when the input may not contain enough information.
• Test prompts on one or a few examples before running batch prediction, optimizer jobs, long document summarization, or large-scale evaluation.
• Compare prompt variants with the same EvalTask dataset, metrics, model, and experiment configuration for fair results.
• Inspect explanations, raw autorater output, intermediate steps, source documents, and per-example failures, not only aggregate scores.
• Use prompt templates with dataset columns or runtime variables so prompts can be compiled, evaluated, versioned, and reused consistently.
• Use 50-100 distinct validation samples when running Vertex AI Prompt Optimizer for more reliable results.
• Store evaluation IDs, timestamps, prompt text, input URIs, model version, usage metadata, and results for traceability.
• Use external tools, retrieval, BigQuery, Healthcare NLP, Tavily search, or other APIs when the model lacks current, private, or domain-specific information.
• Apply layered safety controls, including input and output checks, DLP where relevant, safety settings, untrusted-file validation, mission checks, and human review for sensitive domains.

Avoid this

• Assuming the model has real-time, private, or complete domain knowledge without retrieval, tools, or provided context.
• Relying on requested citations or confident wording as proof of factual grounding.
• Leaving fields optional by accident in structured output schemas or writing output instructions that contradict the schema.
• Parsing free-form markdown or regex output when JSON mode, schemas, enums, or closed vocabularies would be more robust.
• Using too many examples, poorly matched examples, or unbalanced examples that overfit or distort classification behavior.
• Launching batch prediction, optimization, tuning, or evaluation jobs before validating prompts and configs on small samples.
• Comparing prompts or models with different datasets, metrics, parameters, or model settings and treating the results as fair.
• Forgetting environment prerequisites such as project ID, location, authentication, API enablement, Cloud Storage buckets, IAM roles, quotas, runtime restarts, or billable resource cleanup.

From the notebooks

Gemini 3.1 Flash Text-to-Speech Generation

• Use VoiceConfig with PrebuiltVoiceConfig to choose documented voice options.
• Use response_modalities=[“AUDIO”] when requesting speech output.
• Use MultiSpeakerVoiceConfig with one SpeakerVoiceConfig per speaker for multi-speaker clips.
• Place audio tags immediately before the phrase or sentence they should influence.
• Avoid overusing audio tags and match them to natural changes in tone or pace.

Intro to Structured Output with the Gemini API

• Set response_mime_type to application/json for JSON outputs.
• Use response_schema with Pydantic or OpenAPI-style schemas for controlled JSON generation.
• Use response_json_schema when JSON Schema conditional logic is needed.
• Use response.parsed to consume structured outputs as objects or dictionaries.
• Use text/x.enum when the model must choose from predefined enum values.

Bring-Your-Own-Autorater using CustomMetric

• Use response_mime_type and response_schema to get structured autorater output.
• Return both a numeric metric score and an explanation from the custom metric.
• Keep the custom metric score field name aligned with the CustomMetric name.
• Compare multiple prompt templates under the same EvalTask dataset and metrics.
• Use notebook_utils to display results, explanations, radar plots, bar plots, and experiment runs.

Evaluate LangChain

• Decompose multi-turn chats into per-turn examples with conversation_history before batch prediction.
• Use temperature=0 for repeatable chatbot and evaluator outputs.
• Map dataset columns explicitly with metric_column_mapping={“prompt”: “user”}.
• Combine built-in metrics such as fluency, coherence, and safety with a task-specific custom metric.
• Log evaluation results under an experiment name for run comparison.

Evaluating prompts at scale with Gemini Batch Prediction API

• Test the prompt on one image with generate_content before launching batch prediction.
• Request JSON output in both the prompt and generation config.
• Store evaluation_ts, evaluation_id, prompt_text, and gcs_uri with every request row.
• Load ground truth into BigQuery and join predictions to compute correctness.
• Use BigQuery views to parse raw responses and expose an evaluation table.

Rubric evaluation - Multimodal and Custom metric for text quality

• Inspect, edit, or add generated rubrics before final evaluation.
• Use predefined rubric metrics when they fit the multimodal task.
• Use custom prompt templates when text-quality criteria must be controlled.
• Return raw autorater output to debug reasoning and verdicts.
• Write rubrics as granular binary yes/no constraints and avoid hallucinated or repeated criteria.

Evaluate and Optimize Prompt Template Design for Better Results

• Use one shared EvalTask so prompt templates are compared against the same dataset, metrics, and experiment.
• Use prompt template variables such as {instruction} and {context} to compile prompts from dataset fields.
• Evaluate multiple prompt templates systematically before choosing one.
• Use a consistent model, gemini-2.5-flash, across prompt variants for a fair comparison.
• Inspect both evaluation results and summarization_quality explanations, not only aggregate scores.

Use Gen AI Evaluation SDK to Evaluate Models in Vertex AI Studio, Model Garden, and Model Registry

• Use the same EvalTask configuration with fixed dataset and metrics when comparing model architectures.
• Use prompt_template variables that match columns in the evaluation dataset.
• Use model-based pointwise, pairwise, and computation-based metrics for different evaluation needs.
• Inspect evaluation results and explanations with notebook_utils display helpers.
• Log prompt-template evaluations to an experiment and visualize comparisons with radar and bar plots.

Getting Started with LangChain 🦜️🔗 + Gemini API in Vertex AI

• Use PromptTemplate for prompts with runtime input variables.
• Use StructuredOutputParser format instructions when structured model output is needed.
• Split long documents with RecursiveCharacterTextSplitter before embedding or summarization.
• Use retrievers over vector stores to combine documents with language models.
• Use ConversationBufferMemory when a conversation chain needs prior message context.

Chain of Thought & ReAct

• Use one-shot exemplars to show the model the desired reasoning format.
• Append “Let’s think step by step.” for zero-shot chain-of-thought reasoning.
• Use self-consistency by generating multiple candidate answers and selecting the most popular result.
• Avoid nested ReAct tool calls; parse work into separate actions.
• Use external tools when the model lacks current or external information.

Question Answering with Generative Models on Vertex AI

• Use specific, context-rich, grammatically correct prompts for question answering.
• Experiment with generation parameters such as temperature and max_output_tokens.
• Use few-shot examples when the desired answer style is specific or short.
• For closed-domain Q&A, provide the knowledge base as prompt context.
• Tell the model to return “Information not available in provided context” when the context lacks the answer.

Text Classification with Generative Models on Vertex AI

• Use explicit candidate labels or task instructions in classification prompts.
• Use few-shot examples when the model should follow a specific class mapping or answer format.
• Use low temperature and limited max output tokens for concise classification responses.
• Evaluate classification outputs against ground truth labels when they are available.
• Keep custom model training separate when prompt-based classification is sufficient for the notebook scope.

Text Extraction with Generative Models on Vertex AI

• Initialize Vertex AI with an explicit project and location.
• Use low temperature for extraction-oriented prompts.
• Ask for JSON format when downstream systems need structured output.
• Constrain answers to provided text for troubleshooting responses.
• Use few-shot prompting to guide output organization.

Text Summarization with Generative Models on Vertex AI

• Initialize Vertex AI with an explicit project and location before model calls.
• Use low temperature for concise summary generation when consistency is desired.
• Adjust temperature, max_output_tokens, top_k, and top_p for different output styles.
• Write prompts that specify the desired format, such as bullet points, TL;DR, to-dos, or title options.
• Evaluate generated summaries against human-created summaries with ROUGE metrics.

Prompt Design - Best Practices

• Be concise in prompts.
• Be specific and well-defined.
• Ask one task at a time.
• Use system instructions to guardrail the model from irrelevant responses.
• Turn generative tasks into classification tasks to reduce output variability.

Get Started with Vertex AI Prompt Optimizer

• Use zero-shot optimization for rapid prompt refinement when no evaluation dataset is available.
• Use data-driven optimization when sample inputs and expected outputs define what better performance means.
• Provide examples where the current system instruction performs poorly for prompt optimization.
• Use 50-100 distinct samples for reliable prompt optimization results.
• Include a target field for computation-based metrics such as question_answering_correctness.

Get started with Vertex Prompt Optimizer - Custom metric

• Use a structured Pydantic OptimizationConfig before submitting the optimizer job.
• Validate the deployed custom metric endpoint with a test request before running optimization.
• Use response_mime_type and response_schema to force the autorater to return JSON.
• Store the optimizer configuration as config.json in Cloud Storage.
• Use weighted metrics to balance question_answering_correctness and the custom engagement metric.

Get Started with Vertex AI Prompt Optimizer - Long prompt

• Use placeholders to freeze static prompt sections while optimizing the rest of a long system instruction.
• Provide examples where the current system instruction performs poorly.
• Use 50-100 distinct samples for reliable prompt optimization results.
• Use a target column for computation-based metrics such as question_answering_correctness.
• Use response_schema and response_mime_type to constrain integer JSON output.

Get Started with Vertex AI Prompt Optimizer - Multimodality

• Use labeled rows containing query, image GCS URI, and target for question_answering_correctness evaluation.
• Use examples where the current system instruction performs poorly when building an optimization dataset.
• Validate optimizer settings with the Pydantic OptimizationConfig before submitting the job.
• Store both optimizer configuration and results in Cloud Storage.
• Set has_multimodal_inputs to True when optimizing prompts with image inputs.

Get Started with Vertex AI Prompt Optimizer - Tool usage

• Use a structured pydantic OptimizationConfig before submitting the job.
• Validate tools and tool_config JSON before running optimization.
• Pass FunctionDeclaration and ToolConfig as JSON structures to Prompt Optimizer.
• Use weighted tool-call metrics for tool name, parameter key, and parameter key-value matching.
• Retrieve the best prompt programmatically from GCS output files for application use.

Vertex Prompt Optimizer Notebook UI (Preview)

• Use default optimization configurations as the initial setup.
• Focus validation examples on the issues you want to address.
• Use 50-100 distinct samples for reliable results.
• Validate prompt and data before launching the optimizer job.
• Use multi-metric settings only when more than one metric is needed.

Gen AI & LLM Security for developers

• Do not store sensitive information in the prompt.
• Use low temperature for reproducible results in security demonstrations.
• Check both input and output with DLP before sending to or returning from Gemini.
• Treat links, binaries, and files from users as untrusted and validate them.
• Use Responsible AI safety filters and understand how to configure safety attributes.

Supervised Fine Tuning with Gemini 2.5 Flash for Article Summarization

• Use high-quality, well-labeled, task-relevant training data.
• Use a separate validation or evaluation dataset to measure tuned model performance.
• Evaluate the base model before tuning and the tuned endpoint after tuning.
• Choose task-appropriate metrics; the notebook uses ROUGE-L for summarization.
• Experiment with generation parameters, prompt structure, epochs, and learning rate multiplier.

BigQuery DataFrames ML: Prescription Drug Name Generation

• Use PROJECT_ID from user input or GOOGLE_CLOUD_PROJECT environment variable.
• Create or reuse a BigQuery connection before invoking GeminiTextGenerator.
• Filter out missing, duplicate, short, spaced, and generic-matching brand names before using examples.
• Use random_state when sampling examples for reproducible few-shot prompts.
• Limit batch rows for demonstration purposes.

Code Vulnerability Scanning & Automated Remediation using Gemini API in Vertex AI (Gemini 2.0)

• Add each filename as a separator before its code so the model can identify findings per file.
• Use a clear prompt that specifies vulnerability name, description, recommendations, and recommended code.
• Set generation parameters including temperature, top_p, top_k, candidate_count, and max_output_tokens.
• Export findings to CSV and JSON for further analysis, benchmarking, and integration with security tools.
• Treat the workflow as experimental and continue validation before using it as a robust security tool.

Reduce Tech Debt with Gemini 3 Pro

• Generate passing baseline tests before refactoring legacy code.
• Verify generated tests against the original code before generating replacement code.
• Create a detailed specification from the legacy source before creating a design document.
• Generate new code from the specification and design document, not directly from the legacy code.
• Run the same tests against the generated code for final verification.

Text Summarization of Large Documents using LangChain 🦜🔗

• Use environment variables as fallbacks for Google Cloud project and region configuration.
• Keep summarization instructions in PromptTemplate objects for stuffing, map, combine, question, and refine prompts.
• Test stuffing on a small page subset and handle exceptions from oversized context.
• Use map_reduce or refine when document length exceeds what stuffing can handle.
• Set return_intermediate_steps=True to inspect chunk-level outputs.

ReAct (Reasoning + Acting) + Custom tool for Healthcare NL API + Gemini 2.0 + LangChain

• Use Healthcare NLP to preprocess unstructured medical records before asking the LLM for code suggestions.
• Limit extracted tool terms to clinically relevant entity categories before passing them to the agent.
• Use temperature 0 for more deterministic medical coding output.
• Hide ReAct reasoning steps with verbose=False when reasoning traces should not be shown.
• Use self-consistency by running repeated attempts and selecting the majority response.

Building Knowledge Graphs with Gemini

• Use only the provided input data to avoid relying on memorized general knowledge.
• Use explicit domain terminology such as entities, relationships, nodes, and edges.
• Include deterministic settings such as temperature=0.0, top_p=0.0, and seed=42 for extraction tasks.
• Separate data schema, extraction instructions, and output format in the prompt.
• Use TSV for table-shaped outputs when token efficiency matters.

Video Captioning with Gemini

• Use a detailed system prompt that specifies perspective, camera movement, subject motion, setting, lighting, body language, and visible text.
• Return only the caption by instructing the model not to send a preamble.
• Count video tokens separately when token accounting matters.
• Return usage metadata, model version, generated text, and video-only token count for traceability.
• Review the sample video before captioning to understand visual, textual, and audio elements.

Slide Generation with Gemini and Marp

• Use PROJECT_ID from GOOGLE_CLOUD_PROJECT when the placeholder is not replaced.
• Set a system instruction to restrict Gemini output to Markdown.
• Provide the Marp repository as context so generated slides can follow Marp style and formatting.
• Use max_output_tokens=8192 to allow enough room for a complete slide deck.
• Write generated Markdown to slide-deck.md before rendering it with Marp CLI.

Multimodal retail recommendation: using Gemini to recommend items based on images and image reasoning

• Combine text and image in a single prompt for visual understanding.
• Label each candidate image within the prompt so the model can reference options clearly.
• Provide the available item images when recommendations should be limited to store inventory.
• Use response_mime_type=“application/json” when recommendations need to plug into an application.

Product attributes extraction and detailed descriptions from images using Gemini 2.0

• Use temperature 0 for deterministic attribute extraction.
• Provide system instructions that constrain answers to visible product evidence.
• Use a closed vocabulary when attribute values must come from an approved set.
• Parse model JSON output before returning it to application code.
• Use debug mode to display the image and prompt during development.

Leverage LlamaIndex with Vertex AI Vector Search to perform question answering RAG

• Check for existing buckets, indexes, and endpoints before creating resources.
• Set LlamaIndex embed_model and llm settings before building indexes.
• Use SimpleDirectoryReader and SentenceSplitter to parse documents into documents and nodes.
• Display prompt templates before running RAG queries.
• Inspect source text, relevance score, file name, page label, and file path with each response.

Unlocking Multimodal Video Transcription with Gemini

• Use environment variables or Colab Secrets instead of hardcoding API configuration.
• Start with simple prompts to observe Gemini’s natural behavior before refining instructions.
• Craft prompts iteratively, precisely, and concisely.
• Decouple transcripts and speakers into linked tables with a consistent voice ID.
• Start transcript generation with audio-focused work before extracting speaker names from visual and audio cues.

Handling Reasoning with MaaS Models on Vertex AI using vLLM

• Use Google Cloud default credentials with the cloud-platform scope.
• Configure the Vertex AI endpoint with the model’s specific location.
• Use streaming for long reasoning chains.
• Cache credentials to avoid repeated authentication.
• Choose appropriate model locations for latency.

Guess who or what app using Hugging Face Deep Learning container model on Vertex AI

• Initialize aiplatform and vertexai with explicit project and location values.
• Retrieve the Hugging Face token with get_token instead of hardcoding it in the notebook.
• Register the Hugging Face model in Vertex AI Model Registry before deploying it to an endpoint.
• Use separate helper functions for Gemini content generation, subject extraction, prompt generation, and image generation.
• Use temperature 0 and candidate_count 1 for Gemini riddle solving and prompt generation.

Gemini Enterprise custom agent with prompt management

• Create the managed prompt once and reuse its prompt_id rather than hard-coding instructions in the deployed agent.
• Fetch the prompt in before_agent_callback so prompt text is centrally managed.
• Pass schema DDL as a text/plain file part instead of assuming schema context.
• Use low temperature for SQL generation.
• Enable tracing in AdkApp for local testing.

Tutorial for Running Prompt Management and Evaluation

• Version prompts each time changes are saved so iterations can be compared.
• Test a prompt with sample input before saving and evaluating it.
• Create datasets in Cloud Storage and upload CSV, JSON, or JSONL files for evaluation.
• Run an initial evaluation before prompt changes to establish a baseline.
• Use human-in-the-loop rating and automated metrics for evaluation.

Veo 3.1 Video Generation

• Use detailed prompts for better video quality.
• Include subject, action, scene, camera, style, temporal, audio, and dialogue details when relevant.
• Use Gemini to synthesize keyword choices into a cohesive Veo prompt.
• Use Veo 3.1 Fast when latency is a priority.
• Use Veo 3.1 Lite for rapid prototyping and iteration.

AI Agents for Engineers (Evolution of AI Agents)

• Use temperature=0 for deterministic LangChain and LangGraph essay workflows.
• Verify whether the GenAI client is using Gemini Developer API, Vertex AI project/location, or Vertex AI express mode.
• Use Tavily search when the prompt asks about recent events that the model may not know.
• Separate planning, research, writing, reflection, and critique research into explicit LangGraph nodes.
• Compile the graph with MemorySaver and run it with a thread_id for state management.

---

Back to Prompt Engineering · Best Practices Map