Generative AI Google Brain

agents-agent-engine-memory-bank-get-started-with-memory-bank

3 min read

Get started with Vertex AI Memory Bank

Source notebook

Repo path: agents/agent_engine/memory_bank/get_started_with_memory_bank.ipynb · Open on GitHub · intermediate

Builds a Vertex AI Memory Bank hotel concierge that stores and retrieves guest preferences across sessions.

Summary

This notebook teaches how to create a Vertex AI Agent Engine with Memory Bank configured for memory generation and similarity search. It walks through creating a guest session, appending conversation events, generating memories from that session, retrieving all memories by user scope, and using similarity search for targeted questions. It finishes by deleting the Agent Engine resource to avoid charges.

Key code patterns

Initialize Vertex AI client

import vertexai
 
PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT")
LOCATION = os.environ.get("GOOGLE_CLOUD_REGION", "us-central1")
client = vertexai.Client(project=PROJECT_ID, location=LOCATION)

Creates the client used for Agent Engine sessions and Memory Bank operations.

Configure Memory Bank

basic_memory_config = MemoryBankConfig(
    similarity_search_config=SimilaritySearchConfig(
        embedding_model=f"projects/{PROJECT_ID}/locations/{LOCATION}/publishers/google/models/text-embedding-005"
    ),
    generation_config=GenerationConfig(
        model=f"projects/{PROJECT_ID}/locations/{LOCATION}/publishers/google/models/gemini-2.5-flash"
    ),
)

Sets the embedding model for semantic retrieval and the Gemini model for memory extraction.

Create Agent Engine

agent_engine = client.agent_engines.create(
    config={"context_spec": {"memory_bank_config": basic_memory_config}}
)
agent_engine_name = agent_engine.api_resource.name

Provisions the Agent Engine container with Memory Bank capabilities.

Append session events

client.agent_engines.sessions.events.append(
    name=session_name,
    author=guest_id,
    invocation_id=str(invocation_id),
    timestamp=datetime.datetime.now(tz=datetime.timezone.utc),
    config={"content": {"role": turn["role"], "parts": [{"text": turn["message"]}]}}
)

Stores each conversation turn so Memory Bank can generate memories from the session.

Generate memories

operation = client.agent_engines.memories.generate(
    name=agent_engine_name,
    vertex_session_source={"session": session_name},
    config={"wait_for_completion": True},
)

Extracts and consolidates long-term memories from the stored conversation.

Retrieve by scope

results = client.agent_engines.memories.retrieve(
    name=agent_engine_name,
    scope={"user_id": guest_id}
)
memories = list(results)

Fetches all memories associated with a specific guest profile.

Similarity search memories

results = client.agent_engines.memories.retrieve(
    name=agent_engine_name,
    scope={"user_id": guest_id},
    similarity_search_params={
        "search_query": "What are Emma's dietary restrictions?",
        "top_k": 3,
    },
)

Retrieves only the most relevant memories for a specific natural-language question.

Models & APIs used

  • Models: text-embedding-005, gemini-2.5-flash
  • APIs / services: Vertex AI, Vertex AI Agent Engine, Vertex AI Memory Bank
  • SDKs / libraries: google-cloud-aiplatform, vertexai

When to use this

Use this pattern when building personalized agents that need persistent user preferences and targeted memory recall across sessions.

Gotchas & caveats

  • Requires an existing Google Cloud project with the Vertex AI API enabled.
  • Colab authentication is needed when running in Google Colab.
  • Colab may require a runtime restart after installing google-cloud-aiplatform.
  • The notebook defaults to us-central1 when GOOGLE_CLOUD_REGION is not set.
  • wait_for_completion=True is blocking and the notebook notes production should usually run generation in the background.
  • Agent Engine resources should be deleted after the tutorial to avoid charges.
  • Vertex AI Agent Engine Session is optional because source conversations can also be provided directly in JSON format.

Best practices

  • Use a stable user_id scope to retrieve memories for a specific guest.
  • Store the complete conversation in a session before generating memories.
  • Use scope-based retrieval for complete profiles or small memory sets.
  • Use similarity search for specific questions, many memories, fast targeted responses, or conversational context.
  • Delete the Agent Engine and memories after the tutorial to avoid charges.
  • Use typed Vertex AI SDK classes and aliases to keep configuration readable.

Graph View

Backlinks