Generative AI Google Brain

Function Calling & Tools - Best Practices

27 min read

Function Calling & Tools — Best Practices

Distilled from 73 notebooks tagged Function Calling & Tools in the GoogleCloudPlatform/generative-ai repository. The From the notebooks section below cites the per-notebook source for grounding.

Do this

  • Use function declarations and schemas to constrain tool arguments instead of parsing freeform text or generating arbitrary SQL.
  • Give every tool a specific name, description, parameter type, property description, and required-field list where appropriate.
  • Set temperature to 0 for deterministic function-calling, guardrail, code execution, and tool-review examples when repeatability matters.
  • Always execute tools in application code or a managed executor, then append the tool results back to the model before requesting the final answer.
  • Handle multiple or parallel function calls by executing independent calls concurrently when safe and returning all function responses in bulk.
  • Validate tool-call names and arguments before execution, and validate returned rows or payloads with structured models such as Pydantic where used.
  • Test tool functions directly and test agents locally before deploying to Vertex AI Agent Engine or another remote runtime.
  • Use sessions, thread_id, context_id, ChatMessageHistory, RunnableWithMessageHistory, or VertexAiSessionService to preserve conversation and tool history when follow-up turns depend on state.
  • Use environment variables, Secret Manager, scoped ADC credentials, and IAM roles instead of hardcoding API keys or secrets.
  • Declare deployment requirements, source packages, entrypoint modules, class_methods, build options, and staging buckets explicitly for Agent Engine deployments.
  • Use root_agent.py, AdkApp, ModuleAgent, lazy initialization, or source-based deployment patterns when local objects contain non-pickleable tool state such as MCPToolset or ApiRegistry.
  • Run generated or untrusted code in Agent Engine Sandbox or another controlled environment, and parse stdout, stderr, files, executable_code, and execution results separately.
  • Enable tracing, streaming, state inspection, and evaluation datasets so tool behavior can be debugged and measured beyond the final answer.
  • Evaluate tool use separately from final response quality using single-tool metrics, trajectory metrics, custom pointwise metrics, and reference trajectories.
  • Delete deployed agents, sandboxes, Cloud Storage buckets, Cloud Run services, Cloud SQL instances, experiments, and other tutorial resources after use to avoid ongoing charges.

Avoid this

  • Assuming Gemini or OpenAI-compatible Chat Completions executes tools automatically; the application must run tools and return tool results.
  • Deploying before local testing, direct tool testing, or set_up initialization, which makes packaging, state, and IAM failures harder to debug.
  • Forgetting required Google Cloud setup: project, billing, Vertex AI or Agent Platform APIs, service agents, IAM roles, ADC or Colab authentication, and regional configuration.
  • Hardcoding secrets such as API keys, Reddit credentials, or Maps keys instead of using environment variables or Secret Manager.
  • Packaging non-pickleable local agent objects, MCP toolsets, or API registry state instead of using root_agent.py, ModuleAgent, source deployment, or lazy initialization.
  • Omitting deployment requirements, staging buckets, source packages, custom installation scripts, or runtime service account permissions needed by remote tools.
  • Ignoring model, region, endpoint, and preview limitations such as global versus regional endpoints, context caching availability, model-specific forced or parallel function-calling support, and preview API schema changes.
  • Reading outputs too early by failing to poll async deployments, tasks, evaluations, or indexing jobs until completed.

From the notebooks

Get started with A2A on Agent Engine

  • Test the A2A agent locally with set_up before deploying to Agent Engine.
  • Define an AgentSkill with id, name, description, tags, examples, input modes, and output modes.
  • Use context_id as the Vertex session_id to preserve continuity across A2A interactions.
  • Use VertexAiSessionService when GOOGLE_CLOUD_AGENT_ENGINE_ID is present and InMemorySessionService for local execution.
  • Return final answers as A2A artifacts and update task state through TaskUpdater.

Claude with ADK on Vertex AI Agent Engine

  • Test the ADK agent locally with InMemorySessionService and Runner before deploying.
  • Package the deployable agent in a root_agent.py module and expose an AdkApp entry point.
  • Use a VertexAiSessionService builder for production sessions on Agent Engine.
  • Pass runtime credentials and model settings through environment variables for the deployed app.
  • Use build_options installation steps to install external MCP runtime dependencies during deployment.

Deploy your first agent to Vertex AI Agent Engine

  • Start with Express Mode if new to Vertex AI Agent Engine.
  • Do not share or commit API keys publicly.
  • Use agent object deployment for interactive development in notebook environments like Colab.
  • Wrap ADK agents in AdkApp before deploying to Agent Engine.
  • Enable tracing on AdkApp for debugging.

Get started with Agent Engine Terraform Deployment

  • Use Terraform configuration files for version-controlled, repeatable Agent Engine deployments.
  • Define class_methods for the operations the deployed agent supports.
  • Package requirements.txt, agent.pkl, and dependencies.tar.gz as deployment artifacts.
  • Use set_up() for initialization logic and keep init() configuration pickle-able.
  • Use terraform output to retrieve deployed Reasoning Engine resource information.

Get started with Cloud API Registry on Vertex AI Agent Engine

  • Discover MCP servers and tools before enabling and using them.
  • Verify the BigQuery MCP server shows as ENABLED after enabling it.
  • Use x-goog-user-project in the ApiRegistry header provider for BigQuery MCP access.
  • Explore datasets and table schemas before writing SQL queries.
  • Use BigQuery tools to fetch real data rather than making assumptions.

Get started with Code Execution on Vertex AI Agent Engine

  • Run generated or untrusted code in an isolated Agent Engine Sandbox instead of the host system.
  • Initialize Vertex AI with explicit project and location before creating Agent Engine resources.
  • Parse sandbox outputs by mime_type and metadata, handling stdout, stderr, and generated files separately.
  • Use temperature=0 for the Gemini tool-calling example to make the function-call flow more deterministic.
  • Send function_response or tool_result messages back to the model before asking for the final answer.

MCP on Vertex AI Agent Engine with custom installation scripts

  • Test the agent locally with InMemorySessionService and Runner before deploying to Agent Engine.
  • Use environment variables for Google Cloud settings and Reddit credentials.
  • Use one chat_loop abstraction for local Runner and remote AgentEngine or AdkApp testing.
  • Close the aiofiles error log in a finally block after local testing.
  • Wrap the deployed ADK app in root_agent.py and reference it with ModuleAgent when MCPToolset is used.

Building multi-agent systems with Vertex AI and Claude

  • Test Bear and Bull agents locally before deployment.
  • Use environment variables for ADK and Vertex AI project configuration.
  • Define A2A agent cards with skills, descriptions, tags, and examples for discovery.
  • Package MCP tools as importable Python modules with a stdio server entry point.
  • Use lazy initialization for deployed agents to avoid pickling issues.

Building Multi-Agent Systems with Vertex AI and Llama model

  • Test the Bear and Bull agents locally before deployment.
  • Use separate role-specific prompts for risk-focused and opportunity-focused agents.
  • Expose agent capabilities through A2A Agent Cards with skills, tags, and examples.
  • Package MCP tools as importable modules with a separate stdio server entry point.
  • Use lazy initialization in deployed executors to avoid pickling issues.

Intro to Managed Agents API on Agent Platform (cURL)

  • Validate authentication, API enablement, service agent role, and user IAM access before creating agents.
  • Use a targeted system_instruction to tailor custom agent behavior.
  • Delete custom agent configurations when no longer needed to keep the project clean.
  • Use update_mask when patching mutable agent fields.
  • Mount Cloud Storage sources into a remote environment instead of embedding skill content in requests.

Intro to Managed Agents API on Agent Platform (Python)

  • Validate authentication, API enablement, service agent role, and user access before creating agents.
  • Use unique agent IDs with uuid to avoid naming collisions.
  • Poll agent creation status with client.agents.get because creation is asynchronous.
  • Delete test agents after the demo to keep the project clean.
  • Reuse environment IDs when filesystem or execution context must persist across turns.

Vertex AI Agent Engine in Express Mode

  • Test the tool function directly before adding it to the agent.
  • Test the ADK agent locally before deploying it to Agent Engine.
  • Declare class_methods for the deployed async streaming and session methods.
  • Use source based deployment with source_packages, entrypoint_module, entrypoint_object, and requirements_file.
  • Inspect session events after remote conversations to view stored conversation history.

Intro to Building and Deploying an Agent with Agent Engine in Vertex AI

  • Test the Python tool directly before wiring it into the agent.
  • Test the agent locally with query before deployment.
  • Use stream_query to observe actions, messages, and output for debugging or real-time updates.
  • Re-define the agent before deployment to avoid stateful information from local testing.
  • Record remote_agent.resource_name so the deployed agent can be reused from another Python environment.

Persisting LangChain History with Vertex AI Session Service

  • Store the full LangChain message payload with message_to_dict instead of only message text.
  • Use messages_from_dict to reconstruct LangChain messages from stored raw_event payloads.
  • Wrap chains with RunnableWithMessageHistory to centralize history injection and persistence.
  • Use strict system instructions when the model must call a tool instead of using internal knowledge.
  • Delete the Agent Engine with force=True after the demo to clean up sessions.

Building and Deploying a Human-in-the-Loop LangGraph Application with Agent Engine on Vertex AI

  • Initialize Vertex AI with project, location, and staging bucket before using Agent Engine.
  • Set temperature to 0 for deterministic tool-review examples.
  • Use max_retries for model calls in the LanggraphAgent configuration.
  • Call agent.set_up() before local testing.
  • Use interrupt_before and interrupt_after around tools for human oversight.

Debugging and Optimizing Agents: A Guide to Tracing in Agent Engine

  • Enable tracing with enable_tracing=True when debugging agent execution.
  • Test the agent locally before deploying it to Agent Engine.
  • Use Cloud Trace filters such as openinference.span.kind:AGENT and root:AgentExecutor to narrow trace results.
  • Inspect traces in both the Cloud Console and the Cloud Trace Python SDK.
  • Convert spans to pandas DataFrames for programmatic trace analysis.

Building and Deploying a Google Maps API Agent with Agent Engine

  • Test individual tool functions before wiring them into the agent.
  • Test the agent locally before deploying it to Agent Engine.
  • Use temperature 0 for deterministic agent behavior in this workflow.
  • Declare deployment requirements explicitly in the Agent Engine create config.
  • Grant only the needed Storage Object User role to the Agent Engine service account for bucket access.

Building and Deploying a LangGraph Agent with Agent Engine in Vertex AI

  • Test the LangGraph agent locally before deployment.
  • Provide deployment requirements in the Agent Engine config.
  • Use a staging bucket for Agent Engine deployment artifacts.
  • Delete the deployed agent after experimentation.
  • Optionally delete the staging bucket after cleanup.

Deploying an Agent with Agent Engine and MCP Toolbox for Databases

  • Test the HotelBookingAgent locally before deploying it to Agent Engine.
  • Use Secret Manager to provide the Toolbox tools file to Cloud Run.
  • Grant service accounts explicit roles for Cloud SQL, Vertex AI, Secret Manager, and service usage.
  • Use parameterized SQL statements in Toolbox tool definitions.
  • Initialize Vertex AI with a Cloud Storage staging bucket before Agent Engine deployment.

Building a Conversational Search Agent with Agent Engine and RAG on Vertex AI Search

  • Test the Vertex AI Search function directly before wiring it into the agent.
  • Test the agent locally before deploying it to Agent Engine.
  • Use ChatMessageHistory and session_id to preserve conversational context across follow-up questions.
  • Set deployment requirements explicitly so Agent Engine has the needed LangChain and Discovery Engine packages.
  • Use temperature 0 for the demonstrated search agent behavior.

Function Calling Agent

  • Use function declarations to constrain database access instead of generating and executing arbitrary SQL.
  • Use BigQuery query parameters for filters such as price, store IDs, radius, product ID, and store ID.
  • Cap requested result counts with MAX_PRODUCT_RESULTS and MAX_STORE_RESULTS.
  • Use semantic search only when product_search_query is provided, otherwise use standard SQL filtering.
  • Execute multiple function calls asynchronously and feed function responses back to the model.

Guardrail Classifier Agent

  • Use a dedicated guardrails node before the chat node to decide whether to answer or block.
  • Use temperature=0 and seed=0 for guardrail classification consistency.
  • Return structured JSON with response_schema and validate it with Pydantic.
  • Provide a safe guardrail_response for blocked requests.
  • Store conversation history in state so the classifier and chat node can use prior turns.

Task Planner Agent

  • Use Pydantic models for Task, Plan, Response, and PlanOrRespond to keep agent decisions structured.
  • Reset task results to None before executing newly generated plan tasks.
  • Separate planning, execution, reflection, and post-processing into explicit LangGraph nodes.
  • Use Google Search grounding in the executor for research tasks requiring live information.
  • Use a MemorySaver checkpointer and thread_id to preserve conversation state across turns.

Query a Remote LangGraph Agent Server

  • Use a unique thread_id to keep each test session isolated.
  • Pass an Authorization bearer header only when an id_token is available.
  • Handle multiple chunk shapes explicitly, including text, responses, guardrail classifications, router classifications, function calls, function responses, plans, executed tasks, errors, and unhandled keys.
  • Use get_state and get_state_history to inspect the remote agent session after streaming.

Using OpenAI libraries with Gemini on Vertex AI

  • Use google-auth cloud-platform scoped credentials instead of static API keys.
  • Use a regional Vertex AI endpoint when using context caching.
  • Use tool_choice to control whether the model may call tools.
  • Use response schemas with Pydantic when structured output is required.
  • Use context caching for repeated large inputs to reduce tokens sent and lower request cost.

Intro to Gemini Agentic Vision

  • Enable code execution when the model needs to crop, inspect, calculate, plot, or draw instead of guessing from a static image.
  • Parse response parts explicitly to inspect reasoning text, generated code, execution output, and resulting images.
  • Use PIL and IPython display to render images returned from code execution.
  • Use structured image parts from bytes or URI with the correct MIME type.

Intro to Generating and Executing Python Code with Gemini 3

  • Use the Google Gen AI SDK unified interface with Vertex AI for enterprise-ready projects.
  • Pass code_execution as a Tool instead of expecting code execution by default.
  • Inspect executable_code and code_execution_result response parts separately.
  • Save or display the output code, result, or outcome downstream in the application.
  • Use chat sessions when code needs to be rewritten iteratively with history.

Intro to Computer Use with Gemini

  • Run Computer Use agents in a secure controlled environment such as a sandboxed VM, container, or dedicated browser profile.
  • Implement client-side action handling and screenshot capture.
  • Append both model responses and function responses to conversation history.
  • Return a FunctionResponse for each executed action, including parallel actions.
  • Handle missing candidates because safety filters may return no candidates.

Create a Gen AI Agent Evaluation for a Deployed Agent

  • Run inference first so the dataset includes intermediate_events and response columns before evaluation.
  • Persist evaluation datasets and results to a Cloud Storage destination.
  • Provide AgentInfo with agent instruction and tool definitions for agent evaluation.
  • Poll the evaluation run until SUCCEEDED, FAILED, or CANCELLED before retrieving full results.
  • Retrieve with include_evaluation_items=True to inspect detailed evaluation items.

Evaluate Generative Model Tool Use

  • Define explicit tool evaluation metrics before creating EvalTask.
  • Use a pandas DataFrame with response and reference columns for evaluation datasets.
  • Give FunctionDeclaration parameters clear types, descriptions, and required fields.
  • Use notebook_utils.generate_uuid to create distinct experiment run names.
  • Display evaluation results with notebook_utils.display_eval_result.

Evaluating Agents - Evaluate a CrewAI agent with Vertex AI Gen AI Evaluation Service

  • Evaluate agents with both monitoring-style task metrics and observability considerations such as latency and failure rate.
  • Use reference trajectories to evaluate expected tool choices and ordering.
  • Start with single-tool usage evaluation before broader trajectory evaluation.
  • Evaluate final responses separately from tool trajectory quality.
  • Use custom pointwise metrics when standard text metrics are insufficient for agent behavior.

Evaluating Agents - Evaluate a LangGraph agent with Vertex AI Gen AI Evaluation Service

  • Use an experiment name when initializing Vertex AI to organize evaluation runs.
  • Include prompts and reference trajectories in the evaluation dataset for agent trajectory metrics.
  • Evaluate single tool use before broader trajectory and response evaluations.
  • Use multiple trajectory metrics to compare exact order, in-order match, any-order match, precision, and recall.
  • Use custom pointwise metrics when response quality depends on the agent’s tool trajectory.

Evaluate Generative Model Tool Use with Custom Code Execution

  • Define separate metrics for tool-call validity, tool-name match, parameter-key match, and parameter key-value match.
  • Handle negative examples where no tool calls are expected.
  • Return 0.0 for missing expected tool calls and 1.0 for true negatives.
  • Use remote custom functions to compute TP, FP, and FN before calculating precision, recall, and F1.

Evaluate your ADK agent using Vertex AI Gen AI Evaluation service

  • Use a small reference dataset with prompts and expected tool trajectories.
  • Evaluate single tool selection before broader trajectory metrics.
  • Track evaluation runs with unique experiment run names.
  • Separate trajectory metrics from response metrics such as safety and coherence.
  • Use custom pointwise criteria when response quality depends on tool choices.

Forced Function Calling with Tool Configurations in Gemini

  • Use typed FunctionDeclaration parameters with Schema and Type values.
  • Set temperature=0 for deterministic function-calling examples.
  • Use AUTO when the model should decide whether a tool is needed.
  • Use ANY with allowed_function_names to force a specific function or subset of functions.
  • Check response.function_calls[0].name before executing the external function.

Working with Data Structures and Schemas in Gemini Function Calling

  • Use explicit FunctionDeclaration names, descriptions, parameter types, and property descriptions.
  • Set temperature=0 for deterministic structured extraction examples.
  • Wrap function declarations in Tool objects before passing them to GenerateContentConfig.
  • Use required fields when every object in an array must include specific parameters.
  • Inspect response.function_calls to retrieve the structured function name and arguments.

Intro to Function Calling with the Gemini API & Python SDK

  • Define clear functions with specific parameters and data types instead of parsing freeform text.
  • Set temperature=0 for functions that require deterministic parameter values.
  • Attach tools when creating the chat session to avoid sending them with every request.
  • Send external system results back with Part.from_function_response before asking Gemini for the final user-facing answer.
  • Handle multiple function calls in one turn by returning a list of function responses.

Multimodal Function Calling with the Gemini API & Python SDK

  • Use FunctionDeclaration parameters to constrain predicted function arguments to a JSON schema.
  • Group function declarations into Tool objects before passing them to Gemini.
  • Set temperature=0 for deterministic function-call prediction examples.
  • Use Part.from_uri with the correct MIME type for image, video, audio, and PDF inputs.
  • Save the model response content when returning a later tool response with the thought signature.

Working with Parallel Function Calls and Multiple Function Responses in Gemini

  • Use FunctionDeclaration parameters to describe callable functions clearly.
  • Wrap function declarations in a Tool and pass it through GenerateContentConfig.
  • Use temperature=0 for these tool-calling examples.
  • Extract structured function calls from response.function_calls before executing application code.
  • Fan out independent external API calls in application code, then return all function responses to Gemini in bulk.

Intro to Gemini 2.5 Flash

  • Use a Google Cloud project with Vertex AI API enabled for most users.
  • Use streaming generation when responses should appear as they are generated.
  • Set thinking_budget based on task complexity to manage quality and speed.
  • Set thinking_budget to 0 for simpler examples where lower latency is desired.
  • Inspect usage_metadata, thoughts_token_count, and total_token_count when evaluating thinking cost.

Intro to Gemini 2.5 Flash-Lite

  • Use GenerateContentConfig to keep model parameters, system instruction, safety settings, tools, and schemas explicit per request.
  • Use system_instruction to provide task context, persona, formatting rules, and interaction guidelines.
  • Use safety_settings to adjust blocking behavior for specific harm categories.
  • Inspect usage_metadata to see prompt, candidate, thoughts, and total token counts.
  • Use response_schema with response_mime_type=“application/json” for controlled generation.

Intro to Gemini 2.5 Pro

  • Use a Google Cloud Project for authentication for most users.
  • Set thinking_budget to control quality and speed of response.
  • Inspect usage_metadata token counts when evaluating thinking budget behavior.
  • Use include_thoughts only when summarized thoughts are needed.
  • Use lower thinking budget for examples that do not need extra reasoning to reduce latency.

Intro to Gemini 3.1 Flash-Lite

  • Use thinking_level MINIMAL for low-complexity, lower-latency responses.
  • Use higher thinking levels for tasks requiring deeper reasoning.
  • Use Google Gen AI SDK chat history or append the full model response so thought signatures are handled automatically.
  • Append response.candidates[0].content when doing manual function calling across turns.
  • Set media_resolution based on detail needs and input length.

Intro to Gemini 3.1 Pro

  • Use Google Gen AI SDK chat history or append the full model response so thought signatures are handled automatically.
  • Set thinking_level to low for simple instruction following or chat when lower latency is desired.
  • Use high thinking level for non-trivial reasoning tasks such as concurrency-safe code generation.
  • Set media_resolution per media part when different images or videos need different detail levels.
  • Lower media resolution for very long inputs to manage token count.

Intro to Gemini 3.5 Flash

  • Use streaming for user-facing or chat-based interfaces to receive chunks immediately.
  • Use async generation to improve throughput in web apps and multi-agent systems.
  • Use per-part media resolution when different media inputs need different fidelity levels.
  • Use automatic function calling when possible because the SDK manages thought signatures behind the scenes.
  • Preserve response.candidates[0].content in manual tool histories to maintain reasoning context.

Intro to Gemini 3 Flash (Preview)

  • Use the Google Gen AI SDK chat history or append the full model response so thought signatures are handled automatically.
  • Set thinking_level to MINIMAL or LOW for lower-complexity tasks where lower latency is preferred.
  • Use HIGH thinking_level for non-trivial reasoning tasks such as the double-checked locking C++ example.
  • Lower media_resolution for long videos or extensive documents to fit token limits.
  • Keep Gemini 3 temperature at its default value of 1.0 as recommended in the notebook.

REST API

  • Set GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION before building the API endpoint.
  • Use the global aiplatform.googleapis.com host unless a regional location is configured.
  • Set generation parameters such as temperature, top_p, top_k, max_output_tokens, candidate_count, and stop_sequences deliberately.
  • Use safety_settings to configure harm blocking thresholds.
  • Specify role values for multi-turn chat content.

Getting started with Gemini using Vertex AI in Express Mode

  • Use streaming responses when reducing perceived latency matters.
  • Use system instructions to steer model behavior for a task.
  • Inspect finish_reason and safety_ratings when using safety settings.
  • Use response_schema and response_mime_type for structured JSON output.
  • Use count_tokens before sending a request to estimate input token usage.

Intro to Model Context Protocol (MCP) integration with Vertex AI

  • Use environment variables for project and region when explicit values are not provided.
  • Validate and normalize tool inputs such as two-letter US state codes.
  • Handle HTTP status errors, timeouts, request errors, JSON decode errors, and unexpected exceptions in external API calls.
  • Return structured tool responses with either result or error payloads.
  • Use a maximum tool-turn limit to avoid endless tool-calling loops.

Getting Started with Model Optimizer

  • Use environment variables for project and region fallbacks.
  • Set routing preferences explicitly with ModelSelectionConfig.
  • Keep system instruction and model selection config inside GenerateContentConfig.
  • Use streaming when incremental response display is desired.
  • Declare function parameters and response schemas before passing tools to the model.

Getting Started with Gemini Live API using WebSocket

  • Set PROJECT_ID and LOCATION before constructing the model path.
  • Use the regional aiplatform.googleapis.com WebSocket endpoint for non-global locations.
  • Send setup once, await the setup response, then start streaming.
  • Run send_loop and receive_loop concurrently with asyncio.gather for bidirectional interaction.
  • Use realtime_input for high-frequency audio and video chunks and client_content for discrete text turns.

Getting Started with Gemini Live API using Gen AI SDK

  • Use LiveConnectConfig to centralize response modalities, speech configuration, tools, transcription, and realtime input settings.
  • Set end_of_turn to True when generation should start after accumulated client content.
  • Declare tools when initiating the session rather than after the session starts.
  • Use turn_complete to know when to stop collecting an audio response in conversational loops.
  • Send audio_stream_end=True when a realtime audio stream is paused for more than a second.

From API to Report: Building a Currency Analysis Agent with LangGraph and Gemini

  • Separate the workflow into distinct nodes for API interaction, data validation, and report generation.
  • Use LangChain tools to expose external data sources to the agent.
  • Use conditional graph edges to route between tool execution and review.
  • Compile the graph with MemorySaver to support state management.
  • Use system and user prompts to define tool usage and the analysis task.

Build Your Own AI Podcasting Agent with LangGraph, Gemini, and Chirp 3

  • Defines AgentState with typed workflow fields for task, outline, queries, content, draft, critique, and tool calls.
  • Uses MemorySaver and thread_id to preserve unique workflow execution history.
  • Uses temperature=0 for deterministic agent node outputs.
  • Limits arXiv retrieval with load_max_docs=2 and get_full_documents=False.
  • Prompts the research agent to vary tools and avoid repeating prior sources and queries.

Chain of Thought & ReAct

  • Use one-shot exemplars to show the model the desired reasoning format.
  • Append “Let’s think step by step.” for zero-shot chain-of-thought reasoning.
  • Use self-consistency by generating multiple candidate answers and selecting the most popular result.
  • Avoid nested ReAct tool calls; parse work into separate actions.
  • Use external tools when the model lacks current or external information.

Get Started with Vertex AI Prompt Optimizer - Tool usage

  • Use a structured pydantic OptimizationConfig before submitting the job.
  • Validate tools and tool_config JSON before running optimization.
  • Pass FunctionDeclaration and ToolConfig as JSON structures to Prompt Optimizer.
  • Use weighted tool-call metrics for tool name, parameter key, and parameter key-value matching.
  • Retrieve the best prompt programmatically from GCS output files for application use.

Gen AI and LLM Security - ReAct and RAG attacks & mitigations

  • Use ready Agents and RAG libraries such as Agent Builder, LangChain Agents, Vertex AI Search, and LangChain RAG.
  • Use strict schema validation of tool input and output.
  • Use out-of-band user consent for dangerous operations.
  • Apply defense in depth by layering multiple filters.
  • Use OCR for documents if concerned about invisible text.

Building a photo recognition agent: Agent Engine setup

  • Pin setup dependencies for the notebook environment.
  • Restart the runtime after installing packages.
  • Initialize Vertex AI with project, location, and staging bucket before creating Agent Engine resources.
  • Define tool functions with typed arguments, docstrings, and dictionary responses.
  • Test the agent locally with agent.query before deploying it remotely.

Intro to thought signatures

  • Pass the model’s previous response content back into contents so the thought signature is preserved.
  • Send function execution results as role=“tool” with Part.from_function_response.
  • Group related function declarations in a Tool so the model can choose the needed function.
  • Use thought signatures for multi-turn interactions with external tools that require reasoning context.

Intro to thought signatures with REST API

  • Set GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION before constructing the endpoint.
  • Use function_declarations with explicit parameters and required fields.
  • Include thought signatures when sending function execution results back to the model.
  • Keep the full conversation history when requesting the final response.
  • Clean up generated response*.json files after the tutorial.

Intro to Url Context

  • Install or upgrade google-genai before running the notebook.
  • Authenticate Colab users with auth.authenticate_user().
  • Use a Google Cloud Project for authentication in the tutorial workflow.
  • Inspect url_context_metadata to debug retrieved sources and verify information sources.
  • Inspect grounding_metadata when using Google Search grounding.

Analyze a codebase with Gemini in Vertex AI

  • Use a shallow clone when loading the GitHub repository.
  • Create a context cache for the codebase to avoid resending the full context on every request.
  • Include a system instruction that frames the model as a coding expert for code-related questions.
  • Use streaming generation for longer developer guides, troubleshooting guides, and recommendations.
  • Separate tool-based GitHub issue retrieval from cached-content prompts because tools cannot be added at runtime with cached content.

Get hands-on with a customer support use case using Gemini and Gen AI SDK

  • Use system_instruction to give task role and response guidance across the interaction.
  • Use Part.from_uri with explicit MIME types for image inputs.
  • Use response_schema and response_mime_type=“application/json” for downstream structured processing.
  • Use temperature=0 when calling Google Search for store-location lookup.
  • Return function results with Part.from_function_response so the model can incorporate external data.

ReAct (Reasoning + Acting) + Custom tool for Healthcare NL API + Gemini 2.0 + LangChain

  • Use Healthcare NLP to preprocess unstructured medical records before asking the LLM for code suggestions.
  • Limit extracted tool terms to clinically relevant entity categories before passing them to the agent.
  • Use temperature 0 for more deterministic medical coding output.
  • Hide ReAct reasoning steps with verbose=False when reasoning traces should not be shown.
  • Use self-consistency by running repeated attempts and selecting the majority response.

Productivity Coaching with Gemini and Google Calendar

  • Use system instructions to define Gemini’s coaching role and analysis criteria.
  • Start with sample calendar screenshots before connecting live user data.
  • Request only readonly Calendar API access for event analysis.
  • Validate tool input bounds before calling the Calendar API.
  • Return a concise subset of event fields: summary, start, end, and status.

Retail AI Location Strategy: Autonomous Site Selection & Market Analysis

  • Use Search Grounding for fresh demographics, growth, rental, and market viability data.
  • Wrap Google Maps Places API as a tool for real competitor names, locations, and ratings.
  • Base gap analysis on prior market research and competitor findings rather than isolated prompts.
  • Use code execution for density, saturation, and scoring calculations.
  • Use Pydantic schemas and response_schema to make final recommendations machine-readable.

Building a Multimodal Chatbot for Warranty Claims using Gemini and Vector Search in Vertex AI

  • Use a unique lowercase RAG identifier without spaces for generated resources.
  • Keep chunk overlap when splitting retrieved text to preserve context across chunks.
  • Store Vector Search input as JSONL with id and embedding fields.
  • Use retrieved context in the prompt and instruct Gemini to answer only from provided text.
  • Return a fallback response when no matching page source is found.

Serving Open-Source LLMs on Vertex AI with LiteLLM and OpenAI-Compatible APIs

  • Check list_deploy_options() before deployment to verify supported configurations and resource needs.
  • Use environment variables for project and region defaults.
  • Validate the endpoint resource name before configuring LiteLLM.
  • Keep backend or external API execution separate from model tool-call argument generation.
  • Append assistant tool calls and tool responses to the message history before the second model call.

Multimodal Function Calling with Claude Models

  • Select a Claude model before creating the AnthropicVertex client because models have different location availability.
  • Use environment variables GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION when explicit project or region values are not provided.
  • Define tools with a name, description, and input_schema.
  • Append the assistant tool-use response and a user tool_result message before asking Claude for the final answer.
  • Pass tools again in the follow-up messages.create call.

Getting started with Google Generative AI using the Gen AI SDK

  • Read project and region from GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION when notebook parameters are unset.
  • Use GenerateContentConfig for system instructions, parameters, safety settings, tools, schemas, and cached content.
  • Use Part.from_uri for model-readable Cloud Storage or HTTPS file inputs.
  • Inspect response.candidates[0].safety_ratings after applying safety filters.
  • Use response_mime_type application/json with response_schema for controlled generation.

MCP Server with Gemini Enterprise

  • Keep the MCP server private on Cloud Run with —no-allow-unauthenticated.
  • Use environment variables for project, dataset, table, host, port, and region configuration.
  • Test the ADK agent locally with reasoning_engines.AdkApp before deploying to Agent Engine.
  • Declare Agent Engine runtime requirements explicitly when creating the remote app.
  • Use a structured agent instruction that asks for employee ID, start date, and end date before applying leave.

Building Search Applications with Vertex AI Search

  • Use a custom prompt that tells the assistant to answer only from grounding snippets and not make up information.
  • Enable snippets, summaries, citations, query expansion, and spell correction in direct search requests.
  • Use safety settings when generating Gemini responses.
  • Ground Gemini answers in the Vertex AI Search data store instead of relying only on model knowledge.
  • Use extractive answers and document limits when configuring the LangChain VertexAISearchRetriever.

AI Agents for Engineers (Evolution of AI Agents)

  • Use temperature=0 for deterministic LangChain and LangGraph essay workflows.
  • Verify whether the GenAI client is using Gemini Developer API, Vertex AI project/location, or Vertex AI express mode.
  • Use Tavily search when the prompt asks about recent events that the model may not know.
  • Separate planning, research, writing, reflection, and critique research into explicit LangGraph nodes.
  • Compile the graph with MemorySaver and run it with a thread_id for state management.

Back to Function Calling & Tools · Best Practices Map

Graph View

Backlinks