Generative AI Google Brain

gemini-model-optimizer-intro-model-optimizer

3 min read

Getting Started with Model Optimizer

Source notebook

Repo path: gemini/model-optimizer/intro_model_optimizer.ipynb · Open on GitHub · intro

Introduces Vertex AI Model Optimizer routing with Google Gen AI SDK prompts and function calls.

Summary

This notebook teaches how to use Vertex AI Model Optimizer through the Google Gen AI SDK. It configures a GenAI client, sets a Model Optimizer model ID, selects a routing preference, and sends single-turn, multi-turn, streaming, and function-call requests.

Key code patterns

Create Vertex AI GenAI client

from google import genai
 
client = genai.Client(
    vertexai=True,
    project=PROJECT_ID,
    location=LOCATION,
)

Initializes the Google Gen AI SDK against Vertex AI using project and region.

Configure routing preference

model_selection_config = ModelSelectionConfig(
    feature_selection_preference=FeatureSelectionPreference.BALANCED
)

Sets Model Optimizer routing preference, with options for quality, cost, or balanced behavior.

Generate content with Model Optimizer

response = client.models.generate_content(
    model=MODEL_ID,
    contents=prompt,
    config=GenerateContentConfig(
        system_instruction=system_instruction,
        model_selection_config=model_selection_config,
    ),
)

Sends a prompt through Model Optimizer using the configured routing settings.

Stream generated content

for chunk in client.models.generate_content_stream(
    model=MODEL_ID,
    contents=prompt,
    config=GenerateContentConfig(
        system_instruction=system_instruction,
        model_selection_config=model_selection_config,
    ),
):
    output_text += chunk.text

Shows how to consume streaming response chunks from the same Model Optimizer endpoint.

Declare and pass a function tool

get_current_weather_func = FunctionDeclaration(
    name="get_current_weather",
    description="Get the current weather in a given location",
    parameters=_REQUEST_FUNCTION_PARAMETER_SCHEMA_STRUCT,
    response=_REQUEST_FUNCTION_RESPONSE_SCHEMA_STRUCT,
)
 
weather_tool = Tool(function_declarations=[get_current_weather_func])

Defines a callable tool schema that the model can choose to invoke.

Execute returned function calls

function_map = {"get_current_weather": get_current_weather}
 
for function_call in response.function_calls:
    function = function_map[function_call.name]
    function_result = function(**function_call.args)
    function_response_parts.append(
        Part.from_function_response(name=function_call.name, response=function_result)
    )

Maps model-emitted function calls to local Python functions and formats function responses.

Models & APIs used

When to use this

Use this pattern when you want Vertex AI to route Gemini-family requests by quality or cost preference while keeping one SDK workflow.

Gotchas & caveats

  • Requires an existing Google Cloud project.
  • Vertex AI API must be enabled.
  • The notebook uses billable Vertex AI components.
  • Colab authentication is only run when google.colab is present.
  • LOCATION defaults to us-central1 when GOOGLE_CLOUD_REGION is unset.
  • PROJECT_ID falls back to GOOGLE_CLOUD_PROJECT when the placeholder is not replaced.

Best practices

  • Use environment variables for project and region fallbacks.
  • Set routing preferences explicitly with ModelSelectionConfig.
  • Keep system instruction and model selection config inside GenerateContentConfig.
  • Use streaming when incremental response display is desired.
  • Declare function parameters and response schemas before passing tools to the model.
  • Map returned function call names to trusted local functions before execution.

Graph View

Backlinks