Generative AI Google Brain

gemini-chat-completions-intro-chat-completions-api

3 min read

Using OpenAI libraries with Gemini on Vertex AI

Source notebook

Repo path: gemini/chat-completions/intro_chat_completions_api.ipynb · Open on GitHub · intermediate

Calls Gemini on Vertex AI through OpenAI Chat Completions with streaming, tools, schemas, caching, and safety.

Summary

This notebook teaches how to configure the OpenAI SDK to call Gemini through the Vertex AI Chat Completions API using Google credentials. It demonstrates chat, streaming, multimodal image input, function calling, tool_choice, structured output, reasoning_effort, safety settings, context caching with google-genai, and thought signatures for multi-turn tool use.

Key code patterns

OpenAI client for Vertex AI

credentials, _ = default(scopes=["https://www.googleapis.com/auth/cloud-platform"])
credentials.refresh(Request())
client = openai.OpenAI(
    base_url=f"https://{api_host}/v1/projects/{PROJECT_ID}/locations/{LOCATION}/endpoints/openapi",
    api_key=credentials.token,
)

Uses Google auth tokens with the OpenAI SDK against the Vertex AI OpenAPI endpoint.

Chat completion

response = client.chat.completions.create(
    model=MODEL_ID,
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
response.choices[0].message.content

Shows the basic message format and response extraction pattern.

Streaming response

for chunk in client.chat.completions.create(
    model=MODEL_ID,
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
    stream=True,
):
    output_text += chunk.choices[0].delta.content

Streams generated text chunks as they arrive instead of waiting for full completion.

Multimodal image input

response = client.chat.completions.create(
    model=MODEL_ID,
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "Write a short, engaging blog post based on this picture."},
        {"type": "image_url", "image_url": "gs://cloud-samples-data/generative-ai/image/meal.png"},
    ]}],
)

Passes text and a Cloud Storage image URI in one user message.

Function calling

response = client.chat.completions.create(
    model=MODEL_ID,
    messages=messages,
    tools=tools,
    tool_choice="auto",
)
print(response.choices[0].message.tool_calls)

Provides JSON function specs so Gemini can return tool calls and arguments.

Structured output

response = client.beta.chat.completions.parse(
    model=MODEL_ID,
    messages=[{"role": "user", "content": "List a few popular cookie recipes and their ingredients."}],
    response_format=RecipeList,
)
recipes = response.choices[0].message.parsed

Uses Pydantic models with response_format to parse schema-constrained output.

Context caching

cached_content = google_client.caches.create(
    model=MODEL_ID,
    config=CreateCachedContentConfig(contents=[...], ttl="3600s"),
)
response = openai_client.chat.completions.create(
    model=MODEL_ID,
    messages=[...],
    extra_body={"extra_body": {"google": {"cached_content": cached_content.name}}},
)

Creates a cache with google-genai and references it from OpenAI Chat Completions.

Thought signatures

response = client.chat.completions.create(
    model=MODEL_ID,
    messages=messages,
    tools=tools,
    extra_body={"extra_body": {"google": {"thinking_config": {"include_thoughts": True}}}},
)

Includes model thought context across turns for multi-step tool reasoning.

Models & APIs used

  • Models: google/gemini-2.5-flash
  • APIs / services: Vertex AI, Chat Completions API, Cloud Storage
  • SDKs / libraries: openai, google-auth, google-genai, requests, pydantic

When to use this

Use this pattern when migrating OpenAI SDK chat workflows to Gemini on Vertex AI while keeping OpenAI-style client code.

Gotchas & caveats

  • Vertex AI API must be enabled for the Google Cloud project.
  • Access tokens live for 1 hour by default and must be refreshed after expiration.
  • The notebook uses LOCATION default global, but context caching is not available on the global endpoint and switches to us-central1.
  • Gemini-specific parameters must be passed inside extra_content or extra_body or they will be ignored.
  • Function calling examples do not execute tools automatically; developers must run tools using model outputs.
  • Safety filtering can return finish_reason content_filter and a response message of None.

Best practices

  • Use google-auth cloud-platform scoped credentials instead of static API keys.
  • Use a regional Vertex AI endpoint when using context caching.
  • Use tool_choice to control whether the model may call tools.
  • Use response schemas with Pydantic when structured output is required.
  • Use context caching for repeated large inputs to reduce tokens sent and lower request cost.
  • Keep tool execution in application code and append tool results back into messages.

Graph View

Backlinks