Generative AI Google Brain

gemini-agent-engine-tracing-agents-in-agent-engine

3 min read

Debugging and Optimizing Agents: A Guide to Tracing in Agent Engine

Source notebook

Repo path: gemini/agent-engine/tracing_agents_in_agent_engine.ipynb · Open on GitHub · intermediate

Builds, deploys, and traces a Gemini LangChain agent on Vertex AI Agent Engine.

Summary

This notebook teaches how to enable tracing for a LangchainAgent, run it locally, deploy it to Agent Engine, and inspect execution traces. It demonstrates a support-ticket routing agent with custom Python tools, then uses Cloud Trace, the Cloud Console, and pandas to filter and analyze spans from local and remote agent runs.

Key code patterns

Initialize Vertex AI

PROJECT_ID = "[your-project-id]"
LOCATION = "us-central1"
STAGING_BUCKET = f"gs://{PROJECT_ID}-agent-engine-staging"
 
import vertexai
vertexai.init(project=PROJECT_ID, location=LOCATION)

Sets the project and region before creating or deploying the agent.

Define agent tools

def classify_ticket(ticket_text: str) -> str:
    ...
 
def search_knowledge_base(category: str) -> list[dict]:
    ...
 
def escalate_to_human(ticket_text: str) -> str:
    ...

Uses plain Python functions as tools for ticket classification, knowledge lookup, and escalation.

Enable tracing

agent = LangchainAgent(
    model="gemini-2.5-flash",
    model_kwargs={"temperature": 0},
    tools=[classify_ticket, search_knowledge_base, escalate_to_human],
    enable_tracing=True,
)

The enable_tracing flag captures agent, LLM, and tool execution details.

Query locally

response = agent.query(
    input="""
    Classify the following ticket into a category and give me a relevant documentation link.
    Support ticket text:
    I need to update my billing information since my payment method has expired.
    """
)
print(response["output"])

Generates trace data before deploying the agent.

Fetch traces

trace_client = trace.TraceServiceClient()
result = [
    r for r in trace_client.list_traces(
        request=trace.types.ListTracesRequest(
            project_id=PROJECT_ID,
            filter="openinference.span.kind:AGENT",
        )
    )
]

Retrieves traces that contain Agent spans.

Deploy to Agent Engine

client = vertexai.Client(project=PROJECT_ID, location=LOCATION)
remote_agent = client.agent_engines.create(
    agent=agent,
    config={
        "staging_bucket": STAGING_BUCKET,
        "requirements": ["google-cloud-aiplatform[agent_engines,langchain]"],
    },
)

Packages the tracing-enabled agent with runtime requirements and a staging bucket.

Analyze spans with pandas

trace_data = trace_client.get_trace(project_id=PROJECT_ID, trace_id=result[0].trace_id)
spans = pd.DataFrame.from_records([_utils.to_dict(span) for span in trace_data.spans])
spans[spans["name"] == "ChatVertexAI"]
spans[spans["name"] == "ChatVertexAI"].labels.apply(pd.Series)

Turns Cloud Trace spans into tabular data for inspection.

Clean up resources

client.agent_engines.delete(name=remote_agent.api_resource.name)
 
# from google.cloud import storage
# storage.Client().bucket(STAGING_BUCKET.replace("gs://", "")).delete(force=True)

Deletes the deployed agent and optionally removes the staging bucket to avoid charges.

Models & APIs used

  • Models: gemini-2.5-flash
  • APIs / services: Vertex AI, Agent Engine, Cloud Trace, Cloud Storage
  • SDKs / libraries: google-cloud-aiplatform[agent_engines,langchain], google-cloud-trace, vertexai, pandas, langchain

When to use this

Use this pattern when you need to debug or optimize a tool-using Vertex AI Agent Engine agent with Cloud Trace data.

Gotchas & caveats

  • Requires an existing Google Cloud project with the Vertex AI API enabled.
  • Colab runs require google.colab auth.authenticate_user(project_id=PROJECT_ID).
  • The notebook uses LOCATION=“us-central1” and a Cloud Storage staging bucket for deployment.
  • Deployed agent requirements must include google-cloud-aiplatform[agent_engines,langchain].
  • Trace examples assume matching traces exist; result[0] will fail if no traces are returned.
  • Cleanup is recommended to avoid unexpected Google Cloud charges.

Best practices

  • Enable tracing with enable_tracing=True when debugging agent execution.
  • Test the agent locally before deploying it to Agent Engine.
  • Use Cloud Trace filters such as openinference.span.kind:AGENT and root:AgentExecutor to narrow trace results.
  • Inspect traces in both the Cloud Console and the Cloud Trace Python SDK.
  • Convert spans to pandas DataFrames for programmatic trace analysis.
  • Delete the deployed Agent Engine instance after experimentation.

Graph View

Backlinks