Generative AI Google Brain

Agent Engine - Best Practices

19 min read

Agent Engine — Best Practices

Distilled from 49 notebooks tagged Agent Engine in the GoogleCloudPlatform/generative-ai repository. The From the notebooks section below cites the per-notebook source for grounding.

Do this

  • Test tools directly before wiring them into an agent, and test the full agent locally before deploying to Agent Engine.
  • Keep deployable agent objects serializable: keep init lightweight and pickle-able, move heavy initialization to set_up, and use lazy initialization when executors or tools have non-pickleable state.
  • Wrap ADK agents with AdkApp for Agent Engine deployment and provide session_service, memory_service, and memory tools on the Runner or Agent when using managed state.
  • Use stable scopes such as user_id for Memory Bank so memories are isolated per user, client, or domain entity.
  • Generate memories from complete sessions or recent events, then retrieve them with scope-based lookup for small profiles or semantic search/top_k for targeted recall.
  • Use custom Memory Bank topics only after establishing a default baseline; keep topics focused, provide detailed descriptions with exclusions, and use realistic few-shot examples.
  • Persist complete message payloads and session events rather than plain text when frameworks need to reconstruct tool calls, intermediate steps, or conversation history.
  • Declare deployment dependencies explicitly through requirements, source packages, build_options, Dockerfiles, or Terraform artifacts so remote Agent Engine containers match the local environment.
  • Use Cloud Storage staging buckets and source-based or inline deployments for reproducible builds, CI/CD, and infrastructure-as-code workflows.
  • Pass project settings, model settings, credentials, and external API secrets through environment variables or secret-backed deployment mechanisms instead of hardcoding them.
  • For generated or untrusted code, use Agent Engine Sandbox or AgentEngineSandboxCodeExecutor instead of running code on the host system.
  • Enable tracing, inspect Cloud Trace or OpenInference spans, and convert traces to structured data when debugging tool use, latency, or execution paths.
  • Evaluate agents with datasets that include prompts, responses, intermediate events, predicted trajectories, and reference trajectories; start with tool selection before full trajectory and response metrics.
  • For A2A agents, define complete Agent Cards and AgentSkill metadata, use context_id/session_id for continuity, update task state with TaskUpdater, and poll until completed or failed before reading artifacts.
  • Clean up deployed agents, sandboxes, experiments, staging buckets, Agent Engine resources, GKE/Cloud Run/Cloud SQL resources, and Memory Bank data after tutorials or experiments.

Avoid this

  • Deploying before local validation often hides simple tool, prompt, packaging, or session bugs behind long-running Agent Engine builds.
  • Forgetting to enable required APIs, configure billing, set PROJECT_ID/LOCATION, authenticate, or grant service account IAM roles blocks deployment and tool access.
  • Using InMemorySessionService in production or scaled Cloud Run/GKE deployments loses state when processes restart or multiple instances serve requests.
  • Changing Memory Bank scope keys or user_id values makes prior memories invisible, and not every conversation produces a memory because Memory Bank stores extracted meaningful facts rather than raw dialogue.
  • Assuming wait_for_completion or add_events_to_memory behavior is always immediate is unsafe: blocking generation can slow notebooks, while non-blocking generation may not make memories available right away.
  • Putting non-pickleable MCP toolsets, local state, or heavy runtime objects into deployable agents causes serialization issues; root_agent.py, ModuleAgent, set_up, and lazy initialization are common fixes.
  • Omitting runtime requirements, source packages, custom installation scripts, or staging bucket permissions causes remote agents to fail even when local notebooks work.
  • Ignoring region, preview, and model-availability constraints leads to mismatches: many notebooks default to us-central1, some use global or europe-west1, and preview APIs or Live API endpoints have special requirements.

From the notebooks

Get started with Vertex AI Memory Bank

  • Use a stable user_id scope to retrieve memories for a specific guest.
  • Store the complete conversation in a session before generating memories.
  • Use scope-based retrieval for complete profiles or small memory sets.
  • Use similarity search for specific questions, many memories, fast targeted responses, or conversational context.
  • Delete the Agent Engine and memories after the tutorial to avoid charges.

Customizing Memory Topics

  • Create a baseline with default managed topics before adding custom topics.
  • Use the same conversation for default and custom engines to make the comparison apples-to-apples.
  • Keep custom topics focused on separate domain dimensions to reduce overlap and improve precision.
  • Write detailed topic descriptions with examples and exclusions for what belongs in other topics.
  • Use realistic few-shot conversations that show the desired memory extraction style.

Governance with Vertex AI Memory Bank

  • Use GenerateMemories with direct_memories_source or direct_contents_source when consolidation is needed instead of manual create.
  • Set granular TTL values for different memory creation and update paths.
  • Use scopes such as user_id to isolate customer memories.
  • Use managed topic labels to filter customer data by category.
  • Use revision labels such as data_source and verified for governance workflows.

Get started with Memory Bank on ADK

  • Use add_events_to_memory for production agents to stream recent events incrementally.
  • Use add_session_to_memory at the end of a session when processing the whole session is acceptable.
  • Provide both a memory tool on the Agent and a memory service on the Runner when using built-in ADK memory tools.
  • Use callbacks to automate memory generation after turns.
  • Use Agent Engine SDK generate with wait_for_completion when blocking memory generation is required.

Building a Multimodal Trip Planner with ADK on Vertex AI Agent Engine Memory Bank

  • Use managed topics for common memory categories and custom topics for domain-specific memory extraction.
  • Provide text context alongside file_data so multimodal memories are grounded in user intent.
  • Scope generated memories with user_id.
  • Use PreloadMemoryTool so relevant memories are fetched at the start of a conversation turn.
  • Instruct the agent not to make up facts when memories are unavailable.

Get started with A2A on Agent Engine

  • Test the A2A agent locally with set_up before deploying to Agent Engine.
  • Define an AgentSkill with id, name, description, tags, examples, input modes, and output modes.
  • Use context_id as the Vertex session_id to preserve continuity across A2A interactions.
  • Use VertexAiSessionService when GOOGLE_CLOUD_AGENT_ENGINE_ID is present and InMemorySessionService for local execution.
  • Return final answers as A2A artifacts and update task state through TaskUpdater.

Getting Started with Bidirectional Streaming v2 on Agent Runtime

  • Use bring-your-own-Dockerfile deployment with source_packages for custom Agent Runtime servers.
  • Set GOOGLE_GENAI_USE_VERTEXAI to 1 for the deployed ADK agent environment.
  • Use resource_limits and max_instances in Agent Runtime deployment config.
  • Refresh google.auth credentials before making HTTP or WebSocket calls.
  • Separate receive and send loops for bidirectional WebSocket handling.

Claude with ADK on Vertex AI Agent Engine

  • Test the ADK agent locally with InMemorySessionService and Runner before deploying.
  • Package the deployable agent in a root_agent.py module and expose an AdkApp entry point.
  • Use a VertexAiSessionService builder for production sessions on Agent Engine.
  • Pass runtime credentials and model settings through environment variables for the deployed app.
  • Use build_options installation steps to install external MCP runtime dependencies during deployment.

Deploy your containerized agent on Agent Runtime (prev. Agent Engine)

  • Stores project, model, model region, and location settings in config.json for the containerized agent.
  • Uses a Dockerfile and requirements.txt to make the agent runtime reproducible.
  • Wraps the ADK root_agent with agent_engines.AdkApp before exposing runtime endpoints.
  • Defines both regular and streaming FastAPI endpoints for agent invocation.
  • Uses agent_framework=“google-adk” so the deployed agent can be used through the Google Cloud console playground.

Deploy your first agent to Vertex AI Agent Engine

  • Start with Express Mode if new to Vertex AI Agent Engine.
  • Do not share or commit API keys publicly.
  • Use agent object deployment for interactive development in notebook environments like Colab.
  • Wrap ADK agents in AdkApp before deploying to Agent Engine.
  • Enable tracing on AdkApp for debugging.

Get started with Agent Engine Terraform Deployment

  • Use Terraform configuration files for version-controlled, repeatable Agent Engine deployments.
  • Define class_methods for the operations the deployed agent supports.
  • Package requirements.txt, agent.pkl, and dependencies.tar.gz as deployment artifacts.
  • Use set_up() for initialization logic and keep init() configuration pickle-able.
  • Use terraform output to retrieve deployed Reasoning Engine resource information.

Get started with Cloud API Registry on Vertex AI Agent Engine

  • Discover MCP servers and tools before enabling and using them.
  • Verify the BigQuery MCP server shows as ENABLED after enabling it.
  • Use x-goog-user-project in the ApiRegistry header provider for BigQuery MCP access.
  • Explore datasets and table schemas before writing SQL queries.
  • Use BigQuery tools to fetch real data rather than making assumptions.

Get started with Code Execution on Vertex AI Agent Engine

  • Run generated or untrusted code in an isolated Agent Engine Sandbox instead of the host system.
  • Initialize Vertex AI with explicit project and location before creating Agent Engine resources.
  • Parse sandbox outputs by mime_type and metadata, handling stdout, stderr, and generated files separately.
  • Use temperature=0 for the Gemini tool-calling example to make the function-call flow more deterministic.
  • Send function_response or tool_result messages back to the model before asking for the final answer.

Getting Started with Live API on Agent Engine

  • Keep init lightweight and pickle-able for Agent Engine serialization.
  • Put heavy initialization in set_up because Agent Engine calls it when the serverless container starts.
  • Make each stream_query yield a complete serializable response object.
  • Use bidi_stream_query with asyncio.Queue for continuous two-way sessions.
  • Declare agent dependencies in the Agent Engine requirements config.

MCP on Vertex AI Agent Engine with custom installation scripts

  • Test the agent locally with InMemorySessionService and Runner before deploying to Agent Engine.
  • Use environment variables for Google Cloud settings and Reddit credentials.
  • Use one chat_loop abstraction for local Runner and remote AgentEngine or AdkApp testing.
  • Close the aiofiles error log in a finally block after local testing.
  • Wrap the deployed ADK app in root_agent.py and reference it with ModuleAgent when MCPToolset is used.

Building multi-agent systems with Vertex AI and Claude

  • Test Bear and Bull agents locally before deployment.
  • Use environment variables for ADK and Vertex AI project configuration.
  • Define A2A agent cards with skills, descriptions, tags, and examples for discovery.
  • Package MCP tools as importable Python modules with a stdio server entry point.
  • Use lazy initialization for deployed agents to avoid pickling issues.

Building Multi-Agent Systems with Vertex AI and Llama model

  • Test the Bear and Bull agents locally before deployment.
  • Use separate role-specific prompts for risk-focused and opportunity-focused agents.
  • Expose agent capabilities through A2A Agent Cards with skills, tags, and examples.
  • Package MCP tools as importable modules with a separate stdio server entry point.
  • Use lazy initialization in deployed executors to avoid pickling issues.

Get started with Sessions and Memory Bank for ADK agents in Cloud Run

  • Store production ADK session data outside the agent runtime.
  • Use VertexAISessionService for scalable managed session storage.
  • Provide a memory tool on the Agent and a memory_service on the Runner.
  • Use a session_service on the Runner when using managed sessions.
  • Create new sessions during testing to prove long-term memory carries across sessions.

Get started with Sessions and Memory Bank for ADK agents in Google Kubernetes Engine

  • Store production ADK session data outside the agent runtime.
  • Use VertexAISessionService for scalable managed session storage.
  • Use VertexAiMemoryBankService for persistent long-term memory.
  • Provide both a memory tool on the Agent and a memory service on the Runner.
  • Provide a session service on the Runner when using managed sessions.

Intro to Managed Agents API on Agent Platform (cURL)

  • Validate authentication, API enablement, service agent role, and user IAM access before creating agents.
  • Use a targeted system_instruction to tailor custom agent behavior.
  • Delete custom agent configurations when no longer needed to keep the project clean.
  • Use update_mask when patching mutable agent fields.
  • Mount Cloud Storage sources into a remote environment instead of embedding skill content in requests.

Managed Agents API - Analyzing the 2026 World Cup

  • Validate project API enablement and IAM roles before creating the agent.
  • Mask access tokens in logs instead of printing full secrets.
  • Refresh OAuth tokens when missing or expired.
  • Use timezone-aware UTC expiry checks for credentials.
  • Keep agent skills in a configured Cloud Storage bucket or use the Google Cloud Skills Registry.

Vertex AI Agent Engine in Express Mode

  • Test the tool function directly before adding it to the agent.
  • Test the ADK agent locally before deploying it to Agent Engine.
  • Declare class_methods for the deployed async streaming and session methods.
  • Use source based deployment with source_packages, entrypoint_module, entrypoint_object, and requirements_file.
  • Inspect session events after remote conversations to view stored conversation history.

Evaluate a CrewAI agent on Vertex AI Agent Engine (Customized template)

  • Evaluate agents both online and offline, using subjective and objective evaluation signals.
  • Track single tool selection, trajectory order, response generation, latency, and failure rate when evaluating agents.
  • Use reference trajectories in the evaluation dataset when checking expected tool calls.
  • Wrap custom agent output so it includes response and predicted_trajectory for evaluation.
  • Clean up experiments and remote agents after the notebook run.

Evaluating a LangChain Agent on Vertex AI Agent Engine (Prebuilt template)

  • Evaluate agents with both objective metrics and subjective feedback to build trust in behavior.
  • Use agent_executor_kwargs={“return_intermediate_steps”: True} so tool calls can be evaluated.
  • Prepare evaluation data with prompts and reference trajectories for tool and trajectory metrics.
  • Evaluate single tool selection before broader trajectory and response quality metrics.
  • Use custom pointwise metrics when response quality depends on whether the response follows tool choices.

Evaluate a LangGraph agent on Vertex AI Agent Engine (Customized template)

  • Evaluate agents both online and offline using subjective and objective signals.
  • Use prompts, reference responses, and reference trajectories in evaluation datasets when available.
  • Start with single tool selection, then evaluate full trajectories and generated responses.
  • Use trajectory metrics such as exact match, in-order match, any-order match, precision, and recall.
  • Parse custom agent output into response and predicted_trajectory before passing it to EvalTask.

Intro to Building and Deploying an Agent with Agent Engine in Vertex AI

  • Test the Python tool directly before wiring it into the agent.
  • Test the agent locally with query before deployment.
  • Use stream_query to observe actions, messages, and output for debugging or real-time updates.
  • Re-define the agent before deployment to avoid stateful information from local testing.
  • Record remote_agent.resource_name so the deployed agent can be reused from another Python environment.

Persisting LangChain History with Vertex AI Session Service

  • Store the full LangChain message payload with message_to_dict instead of only message text.
  • Use messages_from_dict to reconstruct LangChain messages from stored raw_event payloads.
  • Wrap chains with RunnableWithMessageHistory to centralize history injection and persistence.
  • Use strict system instructions when the model must call a tool instead of using internal knowledge.
  • Delete the Agent Engine with force=True after the demo to clean up sessions.

Building and Deploying a Human-in-the-Loop LangGraph Application with Agent Engine on Vertex AI

  • Initialize Vertex AI with project, location, and staging bucket before using Agent Engine.
  • Set temperature to 0 for deterministic tool-review examples.
  • Use max_retries for model calls in the LanggraphAgent configuration.
  • Call agent.set_up() before local testing.
  • Use interrupt_before and interrupt_after around tools for human oversight.

Get started with Vertex AI Memory Bank - ADK

  • Use a unique USER_ID to scope memories to a particular user.
  • Use PreloadMemoryTool so the agent can retrieve relevant user context.
  • Prompt the agent to personalize responses and naturally reference past conversations when relevant.
  • Retrieve the completed session before adding it to Memory Bank.
  • Delete the Agent Engine after experimentation to avoid charges.

Get started with Vertex AI Memory Bank - CrewAI

  • Validate project, location, and Agent Engine name before using custom storage.
  • Scope memories by user_id to keep user memory separated.
  • Use wait_for_completion when generating memories before retrieval is expected.
  • Create a new Vertex AI client per storage operation for async event loop management.
  • Delete the Agent Engine resource after the tutorial to avoid ongoing charges.

Get started with Vertex AI Memory Bank - LangGraph

  • Scope memories by user_id so retrieved facts are user-specific.
  • Retrieve memories before generation and inject them into the system prompt.
  • Use semantic search with top_k to limit retrieved memories to relevant facts.
  • Store both user and model messages after each turn to continuously build memory.
  • Use wait_for_completion when generating memories so persistence completes before continuing.

Debugging and Optimizing Agents: A Guide to Tracing in Agent Engine

  • Enable tracing with enable_tracing=True when debugging agent execution.
  • Test the agent locally before deploying it to Agent Engine.
  • Use Cloud Trace filters such as openinference.span.kind:AGENT and root:AgentExecutor to narrow trace results.
  • Inspect traces in both the Cloud Console and the Cloud Trace Python SDK.
  • Convert spans to pandas DataFrames for programmatic trace analysis.

AG2 (formerly Autogen) Multi-Agents Example on Vertex AI Agent Engine

  • Authenticate in Colab with google.colab.auth or use Application Default Credentials outside Colab.
  • Test ResearchApp locally before deploying to Vertex AI Agent Engine.
  • Use Cache.disk() to reduce inference cost.
  • Use human_input_mode=“NEVER” for the deployed UserProxyAgent query flow.
  • Clean up deployed Agent Engines to avoid unnecessary costs.

Building and Deploying a Google Maps API Agent with Agent Engine

  • Test individual tool functions before wiring them into the agent.
  • Test the agent locally before deploying it to Agent Engine.
  • Use temperature 0 for deterministic agent behavior in this workflow.
  • Declare deployment requirements explicitly in the Agent Engine create config.
  • Grant only the needed Storage Object User role to the Agent Engine service account for bucket access.

Building and Deploying a LangGraph Agent with Agent Engine in Vertex AI

  • Test the LangGraph agent locally before deployment.
  • Provide deployment requirements in the Agent Engine config.
  • Use a staging bucket for Agent Engine deployment artifacts.
  • Delete the deployed agent after experimentation.
  • Optionally delete the staging bucket after cleanup.

Building a Multi-Agent RAG Application with LangGraph and Agent Engine

  • Pin deployment requirements to the same package versions used in the notebook.
  • Test the LangGraph app locally before deploying it to Agent Engine.
  • Use separate vector store tables for movie and book data.
  • Use Cloud Storage as the source for reusable JSON document datasets.
  • Clean up Agent Engine apps and Cloud SQL instances after the tutorial to avoid billing.

Deploying an Agent with Agent Engine and MCP Toolbox for Databases

  • Test the HotelBookingAgent locally before deploying it to Agent Engine.
  • Use Secret Manager to provide the Toolbox tools file to Cloud Run.
  • Grant service accounts explicit roles for Cloud SQL, Vertex AI, Secret Manager, and service usage.
  • Use parameterized SQL statements in Toolbox tool definitions.
  • Initialize Vertex AI with a Cloud Storage staging bucket before Agent Engine deployment.

Building a Conversational Search Agent with Agent Engine and RAG on Vertex AI Search

  • Test the Vertex AI Search function directly before wiring it into the agent.
  • Test the agent locally before deploying it to Agent Engine.
  • Use ChatMessageHistory and session_id to preserve conversational context across follow-up questions.
  • Set deployment requirements explicitly so Agent Engine has the needed LangChain and Discovery Engine packages.
  • Use temperature 0 for the demonstrated search agent behavior.

Create & Deploy Agent and Run Gen AI Agent Evaluation

  • Create a small agent-specific evaluation dataset before running evaluation.
  • Run inference first so the dataset contains response and intermediate_events columns.
  • Use AgentInfo.load_from_agent with the agent definition and deployed resource name.
  • Persist managed evaluation results to Cloud Storage when results need to be retrieved later.
  • Poll evaluation runs until they reach SUCCEEDED, FAILED, or CANCELLED before displaying final results.

Gen AI Eval - Multi-turn Agent Eval, User Simulation, Metric Registration, Auto-Loss Analysis

  • Load AgentInfo from the ADK agent before scenario generation.
  • Throttle evaluation calls with evaluation_service_qps.
  • Register custom metrics once and reference their metric_resource_name in evaluation runs.
  • Return a dictionary from custom LLM metric result_parsing_function.
  • Use predefined multi-turn metrics for tool use quality, trajectory quality, and task success.

Building a photo recognition agent: Agent Engine setup

  • Pin setup dependencies for the notebook environment.
  • Restart the runtime after installing packages.
  • Initialize Vertex AI with project, location, and staging bucket before creating Agent Engine resources.
  • Define tool functions with typed arguments, docstrings, and dictionary responses.
  • Test the agent locally with agent.query before deploying it remotely.

🛡️ Agentic GraphRAG: Cybersecurity Threat Intelligence

  • Authenticate in Colab with google.colab.auth.authenticate_user when running in Colab.
  • Use vertexai.init with explicit project and location before Vertex AI operations.
  • Re-initialize Neo4j and imports inside the tool function for serialization contexts.
  • Give the ADK agent explicit instructions to use query_threat_graph first for threats, actors, and CVEs.
  • Test the ADK app locally before deploying to Vertex AI Agent Engine.

GraphRAG on Google Cloud With Spanner and Vertex AI Agent Engine

  • Constrain graph extraction with allowed_nodes and allowed_relationships.
  • Use temperature=0 for deterministic graph Q&A and entity rewriting.
  • Store embeddings alongside graph-related text in Spanner for semantic matching.
  • Combine vector search with graph search when exact graph entity names may not match user phrasing.
  • Configure ADK instructions to check the graph database before broader Google Search.

🌿 Eco-Nomad Swarm: 100% Real-Data Sustainable Travel Orchestration

  • Use live databases or real-world REST endpoints instead of mocked travel, weather, forex, or translation data.
  • Chain tool outputs deterministically, such as graph-derived country to RESTCountries currency to Frankfurter forex conversion.
  • Keep agent tools narrow and domain-specific: graph routing, demographics, treasury, climate, web intelligence, and translation.
  • Use exact 3-letter currency codes for live currency conversion.
  • Initialize Vertex AI with project, region, and a staging bucket before production deployment.

Building an ADK agent using QWEN 3 on Vertex AI

  • Smoke test the deployed model endpoint with LiteLLM before building the full ADK agent.
  • Use InMemorySessionService and adk.Runner to test agent behavior locally before deployment.
  • Wrap Python functions with FunctionTool so ADK can call them as tools.
  • Use reasoning_engines.AdkApp before deploying the ADK agent to Agent Engine.
  • Delete the Vertex AI Endpoint and Agent Engine after use to avoid ongoing charges.

Gemini Enterprise custom agent with prompt management

  • Create the managed prompt once and reuse its prompt_id rather than hard-coding instructions in the deployed agent.
  • Fetch the prompt in before_agent_callback so prompt text is centrally managed.
  • Pass schema DDL as a text/plain file part instead of assuming schema context.
  • Use low temperature for SQL generation.
  • Enable tracing in AdkApp for local testing.

Gemini Enterprise custom agent with Vertex AI session

  • Use VertexAiSessionService when an agent needs persistent session state.
  • Split agent responsibilities into query completeness checking and itinerary generation.
  • Use a low temperature of 0.01 for the root agent configuration.
  • Use a session_service_builder when creating reasoning_engines.AdkApp.
  • Test the same multi-turn queries locally and through the remote Agent Engine app.

MCP Server with Gemini Enterprise

  • Keep the MCP server private on Cloud Run with —no-allow-unauthenticated.
  • Use environment variables for project, dataset, table, host, port, and region configuration.
  • Test the ADK agent locally with reasoning_engines.AdkApp before deploying to Agent Engine.
  • Declare Agent Engine runtime requirements explicitly when creating the remote app.
  • Use a structured agent instruction that asks for employee ID, start date, and end date before applying leave.

Open Source Models (Gemma) as a agent with Gemini Enterprise

  • Deploy the open-source model first, then integrate it through a tool function before agent deployment.
  • Keep the Cloud Run model endpoint private and authenticate with an ID token.
  • Test the ADK app locally with a session and stream_query before creating the remote Agent Engine app.
  • Pin runtime requirements when deploying the agent to Agent Engine.
  • Use a staging bucket when initializing Vertex AI for Agent Engine deployment.

Back to Agent Engine · Best Practices Map

Graph View

Backlinks