Generative AI Google Brain

gemini-getting-started-intro-gemini-3-1-pro

4 min read

Intro to Gemini 3.1 Pro

Source notebook

Repo path: gemini/getting-started/intro_gemini_3_1_pro.ipynb · Open on GitHub · intermediate

Quickstart for Gemini 3.1 Pro on Vertex AI with Google Gen AI SDK API features.

Summary

This notebook teaches how to authenticate to Vertex AI, create a Google Gen AI SDK client, and call the Gemini 3.1 Pro preview model. It demonstrates generation, streaming, thinking levels, media resolution, thought summaries, chat, safety settings, async requests, multimodal inputs, structured output, grounding, code execution, URL context, function calling, and token counting.

Key code patterns

Vertex AI client

PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT")
LOCATION = "global"
client = genai.Client(
    enterprise=True,
    project=PROJECT_ID,
    location=LOCATION,
)

Initializes Google Gen AI SDK access through a Google Cloud project on Vertex AI.

Thinking level

response = client.models.generate_content(
    model=MODEL_ID,
    contents="How does AI work?",
    config=types.GenerateContentConfig(
        thinking_config=types.ThinkingConfig(
            thinking_level=types.ThinkingLevel.LOW
        )
    ),
)

Uses low thinking level for faster, lower-latency responses when deep reasoning is not required.

Per-part media resolution

types.Part(
    file_data=types.FileData(
        file_uri="gs://cloud-samples-data/generative-ai/image/a-man-and-a-dog.png",
        mime_type="image/jpeg",
    ),
    media_resolution=types.PartMediaResolution(
        level=types.PartMediaResolutionLevel.MEDIA_RESOLUTION_HIGH
    ),
)

Controls token and latency trade-offs for individual image or video inputs.

Manual function calling history

history = [
    types.Content(role="user", parts=[types.Part(text=prompt)]),
    response.candidates[0].content,
    types.Content(
        role="tool",
        parts=[types.Part.from_function_response(
            name=function_call.name,
            response=function_response_data,
        )],
    ),
]

Preserves the model response content so thought signatures remain attached across turns.

Streaming function arguments

for chunk in client.models.generate_content_stream(
    model=MODEL_ID,
    contents="What's the weather in London and New York?",
    config=types.GenerateContentConfig(
        tools=[get_weather_tool],
        tool_config=types.ToolConfig(
            function_calling_config=types.FunctionCallingConfig(
                stream_function_call_arguments=True,
            )
        ),
    ),
):
    function_call = chunk.function_calls[0]

Enables partial function call argument streaming during tool use.

Structured output

response = client.models.generate_content(
    model=MODEL_ID,
    contents="Give me information for the United States.",
    config=types.GenerateContentConfig(
        response_mime_type="application/json",
        response_schema=CountryInfo,
    ),
)
print(response.parsed)

Constrains responses to JSON using a Pydantic schema and reads the parsed object.

Grounded search tool

response = client.models.generate_content(
    model=MODEL_ID,
    contents="Where will the next FIFA World Cup be held?",
    config=types.GenerateContentConfig(
        tools=[types.Tool(google_search=types.GoogleSearch())],
    ),
)

Uses Google Search as a tool so responses can include grounding metadata and citations.

Models & APIs used

  • Models: gemini-3.1-pro-preview
  • APIs / services: Vertex AI, Cloud Storage, Google Search
  • SDKs / libraries: google-genai, pydantic

When to use this

Use this pattern to evaluate Gemini 3.1 Pro features on Vertex AI before building chat, tool-use, multimodal, or grounded generation workflows.

Gotchas & caveats

  • Enable the Vertex AI API before using the Google Cloud project flow.
  • Colab users must run auth.authenticate_user().
  • The notebook uses LOCATION = “global”.
  • If thinking_level is omitted, Gemini 3.1 Pro defaults to high.
  • A thinking token budget of 0 returns an error.
  • Do not send thinking_level and legacy thinking_budget in the same request; it returns a 400 error.
  • Thinking cannot be turned off for Gemini 3.1 Pro.
  • Media resolution increases token usage and latency; long videos or documents may need low resolution.
  • Raw API clients must return thought_signature in the exact part where it was received or function calling can return a 400 error.
  • For Gemini 3, the notebook recommends keeping temperature at its default value of 1.0.
  • A blocked safety response can have response.text as None and finish_reason as SAFETY.

Best practices

  • Use Google Gen AI SDK chat history or append the full model response so thought signatures are handled automatically.
  • Set thinking_level to low for simple instruction following or chat when lower latency is desired.
  • Use high thinking level for non-trivial reasoning tasks such as concurrency-safe code generation.
  • Set media_resolution per media part when different images or videos need different detail levels.
  • Lower media resolution for very long inputs to manage token count.
  • Use safety_settings for harmful content categories and inspect finish_reason and safety_ratings.
  • Use response_mime_type=“application/json” with response_schema for structured output.
  • Use from_uri for Cloud Storage, web, YouTube, PDF, audio, and HTML inputs, and from_bytes for local files.
  • Use Google Search grounding metadata to print citations and sources.
  • Use count_tokens and compute_tokens to inspect token usage.

Graph View

Backlinks