Generative AI Google Brain

open-models-get-started-with-oss-maas-reasoning-open-ai-sdk

3 min read

Handling Reasoning with MaaS Models on Vertex AI using vLLM

Source notebook

Repo path: open-models/get_started_with_oss_maas_reasoning_open_ai_sdk.ipynb · Open on GitHub · intermediate

Shows how to call reasoning MaaS models on Vertex AI through the OpenAI SDK.

Summary

This notebook teaches how to use reasoning-capable MaaS open models on Vertex AI through the OpenAI-compatible endpoint. It authenticates with Google Cloud credentials, configures the OpenAI Python client against Vertex AI, calls DeepSeek and GPT-OSS models, extracts reasoning and final answers, streams reasoning chunks, requests guided JSON output, and compares model responses across regions.

Key code patterns

OpenAI client for Vertex AI

vertex_endpoint_url = f"https://{MODEL_LOCATION}-aiplatform.googleapis.com/v1beta1/projects/{PROJECT_ID}/locations/{MODEL_LOCATION}/endpoints/openapi"
client = openai.OpenAI(
    base_url=vertex_endpoint_url,
    api_key=credentials.token,
)

Uses the OpenAI SDK with a Vertex AI OpenAPI endpoint and Google Cloud access token.

DeepSeek reasoning extraction

response_text = response.choices[0].message.content
start_index = response_text.find("<think>")
end_index = response_text.find("</think>")
reasoning_content = response_text[start_index + len("<think>") : end_index].strip()
final_answer = response_text[end_index + len("</think>") :].strip()

DeepSeek R1 returns reasoning and final answer in one content string, so the notebook separates them by parsing think tags.

Enable DeepSeek thinking

response = client.chat.completions.create(
    model="deepseek-ai/deepseek-v3.1-maas",
    messages=messages,
    extra_body={"chat_template_kwargs": {"thinking": True}},
)

DeepSeek v3.1 uses a thinking parameter passed through extra_body.

GPT-OSS reasoning effort

response = client.chat.completions.create(
    model="openai/gpt-oss-20b-maas",
    messages=messages,
    reasoning_effort="high",
)

GPT-OSS exposes reasoning depth through reasoning_effort with low, medium, and high options.

Stream reasoning and answer

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta:
        if hasattr(chunk.choices[0].delta, "reasoning_content"):
            reasoning_content += chunk.choices[0].delta.reasoning_content or ""
        elif hasattr(chunk.choices[0].delta, "content"):
            final_content += chunk.choices[0].delta.content or ""

Streaming lets applications handle reasoning_content and final content as chunks arrive.

Guided JSON schema

class Person(BaseModel):
    name: str
    age: int
 
json_schema = Person.model_json_schema()
response = client.chat.completions.create(
    model="openai/gpt-oss-20b-maas",
    messages=[{"role": "user", "content": "Generate a JSON with a person's details"}],
    extra_body={"guided_json": json_schema},
)

guided_json constrains GPT-OSS output to a Pydantic-derived JSON schema while preserving reasoning_content.

Models & APIs used

  • Models: deepseek-ai/deepseek-r1-0528-maas, deepseek-ai/deepseek-v3.1-maas, openai/gpt-oss-20b-maas
  • APIs / services: Vertex AI
  • SDKs / libraries: openai, google-auth, pydantic

When to use this

Use this pattern when building Vertex AI applications that need OpenAI SDK compatibility and visible reasoning from MaaS open models.

Gotchas & caveats

  • Enable the Vertex AI API in an existing Google Cloud project.
  • Colab requires auth.authenticate_user() before using default credentials.
  • The OpenAI client api_key is a Google Cloud access token and may need refresh on expiration.
  • Model locations differ: deepseek-r1 and gpt-oss-20b use us-central1, while deepseek-v3.1 uses us-west2.
  • DeepSeek R1 returns reasoning inside content with tags, while GPT-OSS exposes reasoning_content separately.
  • Reasoning content adds to token usage; monitor response.usage and disable reasoning for simple queries.

Best practices

  • Use Google Cloud default credentials with the cloud-platform scope.
  • Configure the Vertex AI endpoint with the model’s specific location.
  • Use streaming for long reasoning chains.
  • Cache credentials to avoid repeated authentication.
  • Choose appropriate model locations for latency.
  • Check hasattr before reading reasoning_content.
  • Use guided_json with a Pydantic schema for structured JSON output.
  • Use reasoning for complex problem-solving, mathematical calculations, step-by-step analysis, and debugging assistance.
  • Skip reasoning for simple factual queries, quick responses, and high-volume requests.

Graph View

Backlinks