Generative AI Google Brain

gemini-context-caching-intro-context-caching

3 min read

Intro to Context Caching with the Gemini API

Source notebook

Repo path: gemini/context-caching/intro_context_caching.ipynb · Open on GitHub · intermediate

Shows implicit and explicit context caching for Gemini in Vertex AI with the Google Gen AI SDK.

Summary

This notebook teaches how Gemini context caching can reduce repeated input tokens for subsequent requests. It demonstrates implicit caching with repeated image prompts, then explicit caching by creating a cache from two PDF papers in Cloud Storage. The workflow covers creating, retrieving, using, chatting with, updating, and deleting a context cache.

Key code patterns

Create Gen AI client

from google import genai
 
client = genai.Client(
    enterprise=True,
    project=PROJECT_ID,
    location=LOCATION,
)

Initializes the Google Gen AI SDK client for Vertex AI in a Google Cloud project and region.

Implicit cache attempt

response = client.models.generate_content(
    model=MODEL_ID,
    contents=[
        Part.from_uri(file_uri=image_uri, mime_type="image/png"),
        "Write a short and engaging blog post based on this image.",
    ],
)
cached = response.usage_metadata.cached_content_token_count or 0

Repeated requests with the same large prefix can produce cached token counts in usage metadata.

Create explicit cache

cached_content = client.caches.create(
    model=MODEL_ID,
    config=CreateCachedContentConfig(
        contents=[Content(role="user", parts=[pdf_1, pdf_2])],
        system_instruction=system_instruction,
        ttl="600s",
    ),
)

Stores large PDF inputs and a system instruction once so later requests can reference the cache.

Use cached content

response = client.models.generate_content(
    model=MODEL_ID,
    contents="What is the research goal shared by these research papers?",
    config=GenerateContentConfig(cached_content=cached_content.name),
)

Passes the cache resource name in generation config so cached content becomes the prompt prefix.

Chat with cache

chat = client.chats.create(
    model=MODEL_ID,
    config=GenerateContentConfig(cached_content=cached_content.name),
)
response = chat.send_message(prompt)

Reuses the cached content inside a multi-turn chat session.

Manage cache lifecycle

cached_content = client.caches.update(
    name=cached_content.name,
    config=CreateCachedContentConfig(ttl="3600s"),
)
client.caches.delete(name=cached_content.name)

Shows how to extend cache lifetime and remove cached content when it is no longer needed.

Models & APIs used

  • Models: gemini-3.5-flash
  • APIs / services: Vertex AI, Cloud Storage
  • SDKs / libraries: google-genai

When to use this

Use this pattern when repeated Gemini requests share a large common prefix such as files, images, or long documents.

Gotchas & caveats

  • Vertex AI API must be enabled for the Google Cloud project.
  • Colab requires notebook authentication with google.colab.auth.authenticate_user().
  • Implicit caching is enabled by default for Gemini 3 and 2.5 models, but cost savings only apply to Gemini 2.5 models.
  • Implicit caching minimum input token count is 2048 for 2.5 Flash and 2.5 Pro.
  • Explicit context caching minimum input token count is 2048 for all models.
  • Caches are model specific and cannot be reused with a different model.
  • Default context cache expiration time is 60 minutes unless ttl or expire_time is specified.
  • Re-enabling caching with cacheConfig requires the Vertex AI administrator role roles/aiplatform.admin.

Best practices

  • Put large and common contents at the beginning of the prompt to increase implicit cache hit chances.
  • Send requests with similar prefixes in a short amount of time for implicit caching.
  • Use usage_metadata.cached_content_token_count to verify cached token usage.
  • Use cached_content.name or resource_name to reference explicit cached content.
  • Set ttl or expire_time to control cache expiration.
  • Delete cached content when it is no longer needed.

Graph View

Backlinks