Generative AI Google Brain

gemini-long-context-intro-long-context

3 min read

Introduction to Long Context Window with Gemini on Vertex AI

Source notebook

Repo path: gemini/long-context/intro_long_context.ipynb · Open on GitHub · intro

Demonstrates Gemini long-context text, video, and audio prompts on Vertex AI with token counting.

Summary

This notebook introduces Gemini long context windows on Vertex AI and explains token accounting for text, images, video, and audio. It installs google-genai, authenticates in Colab when needed, creates a Vertex AI GenAI client, sets gemini-3.5-flash, then sends large Cloud Storage text, video, and audio inputs as prompt parts. Each workflow counts tokens, generates content, prints usage metadata, and displays the model response.

Key code patterns

Create Vertex AI client

from google import genai
 
PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT")
LOCATION = os.environ.get("GOOGLE_CLOUD_REGION", "us-central1")
client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)

Initializes the Google GenAI SDK for Vertex AI using project and region settings.

Attach GCS media parts

contents = [
    "Provide a detailed summary of the following novel.",
    Part.from_uri(
        file_uri="gs://github-repo/generative-ai/gemini/long-context/WarAndPeace.txt",
        mime_type="text/plain",
    ),
]

Uses Cloud Storage URIs and explicit MIME types to place large files into the model context.

Count tokens before generation

print(
    client.models.count_tokens(
        model=MODEL_ID,
        contents=contents,
    )
)

Checks prompt size before sending long-context requests to the model.

Generate and inspect usage

response = client.models.generate_content(
    model=MODEL_ID,
    contents=contents,
)
print(response.usage_metadata)
display(Markdown(response.text))

Runs the long-context prompt and inspects usage metadata alongside the generated answer.

Send multiple audio files

contents = [
    "According to the following podcasts, what can you tell me about AI/ML workloads on Kubernetes?",
    Part.from_uri(file_uri="gs://github-repo/generative-ai/gemini/long-context/20240417-kpod223.mp3", mime_type="audio/mpeg"),
    Part.from_uri(file_uri="gs://github-repo/generative-ai/gemini/long-context/20240430-kpod224.mp3", mime_type="audio/mpeg"),
]

Shows that multiple podcast episodes can be provided together as long-context audio inputs.

Models & APIs used

  • Models: gemini-3.5-flash
  • APIs / services: Vertex AI, Cloud Storage
  • SDKs / libraries: google-genai

When to use this

Use this pattern when Gemini on Vertex AI needs to summarize or answer questions over very large text, video, or audio inputs in one prompt.

Gotchas & caveats

  • An existing Google Cloud project and enabled Vertex AI API are required.
  • Colab users must run auth.authenticate_user().
  • PROJECT_ID falls back to GOOGLE_CLOUD_PROJECT, and LOCATION falls back to GOOGLE_CLOUD_REGION or us-central1.
  • Cloud Storage parts need valid gs:// URIs and correct MIME types.
  • The prose discusses Gemini 3 and Gemini 3 Flash, while the code sets MODEL_ID to gemini-3.5-flash.

Best practices

  • Start by putting all relevant tokens into the context window when the input fits, while recognizing RAG and summarization can still be relevant.
  • Use count_tokens() before generate_content() for long-context requests.
  • Use Part.from_uri with explicit MIME types for text, video, and audio files.
  • Inspect response.usage_metadata after generation.
  • Use context caching to reduce time and cost for repeated large-context requests.

Graph View

Backlinks