Generative AI Google Brain

gemini-use-cases-customer-support-customer-support-gemini-genai-sdk

3 min read

Get hands-on with a customer support use case using Gemini and Gen AI SDK

Source notebook

Repo path: gemini/use-cases/customer-support/customer_support_gemini_genai_sdk.ipynb · Open on GitHub · intermediate

Builds a Gemini customer-support flow for retail product matching, room-fit reasoning, tools, search, and Live API audio.

Summary

This notebook teaches how to create a Google GenAI SDK client on Vertex AI and use Gemini for a retail customer-support workflow. It demonstrates multimodal product matching from catalog images, structured JSON output, room-fit reasoning across images, image generation, function calling with Google Search, and a text-to-audio Live API session.

Key code patterns

Create Vertex AI GenAI client

from google import genai
 
client = genai.Client(
    vertexai=True,
    project=PROJECT_ID,
    location=LOCATION,
)

Initializes Google GenAI SDK against Vertex AI using project and region.

Build multimodal catalog prompt

product_catalog_parts = []
for product in product_catalog:
    product_catalog_parts.append(f"Chair (id={product['id']}):")
    product_catalog_parts.append(
        Part.from_uri(file_uri=product["image_url"], mime_type="image/png")
    )

Combines text labels and image URI parts so Gemini can compare customer and catalog images.

Generate with system instruction

response = client.models.generate_content(
    model=MODEL_ID,
    contents=contents,
    config=GenerateContentConfig(
        system_instruction=system_instruction,
    ),
)

Shows the core GenAI SDK request pattern for guided multimodal generation.

Structured JSON output

response = client.models.generate_content(
    model=MODEL_ID,
    contents=response.text,
    config=GenerateContentConfig(
        response_mime_type="application/json",
        response_schema=response_schema,
    ),
)

Uses controlled generation so downstream code can consume parsed product matches.

Function calling tool

retail_tool = Tool(
    function_declarations=[
        get_product_info_function,
        get_store_location_function,
    ],
)
chat = client.chats.create(model=MODEL_ID, config=GenerateContentConfig(tools=[retail_tool]))

Lets the model choose declared business functions in a chat session.

Live API audio session

async with client.aio.live.connect(
    model="gemini-live-2.5-flash-native-audio",
    config=config,
) as session:
    await session.send_client_content(
        turns=Content(role="user", parts=[Part(text=text_input)])
    )

Demonstrates interactive text input with audio responses from the Live API.

Models & APIs used

When to use this

Use this pattern to prototype a retail support assistant that needs multimodal product reasoning, structured outputs, tool calls, search, image generation, and audio responses.

Gotchas & caveats

  • Requires google-genai installation and Colab authentication when running in Colab.
  • PROJECT_ID must be set directly or via GOOGLE_CLOUD_PROJECT.
  • LOCATION defaults to us-central1 from GOOGLE_CLOUD_REGION when not provided.
  • The product catalog is built from public PNG files in the cloud-samples-data Cloud Storage bucket.
  • Function calling example requires application code to execute returned function calls and send Part.from_function_response back to the chat.
  • Live API example is async and exits only when the user types q, quit, or exit.

Best practices

  • Use system_instruction to give task role and response guidance across the interaction.
  • Use Part.from_uri with explicit MIME types for image inputs.
  • Use response_schema and response_mime_type=“application/json” for downstream structured processing.
  • Use temperature=0 when calling Google Search for store-location lookup.
  • Return function results with Part.from_function_response so the model can incorporate external data.

Graph View

Backlinks