Generative AI Google Brain

sdk-intro-genai-sdk

4 min read

Getting started with Google Generative AI using the Gen AI SDK

Source notebook

Repo path: sdk/intro_genai_sdk.ipynb · Open on GitHub · intro

Introduces Google Gen AI SDK on Vertex AI for Gemini prompts, tools, caching, batches, and embeddings.

Summary

This notebook teaches how to use the Google Gen AI SDK with Vertex AI to call Gemini and embedding models. It walks from installation, authentication, client setup, and basic text generation through multimodal prompts, system instructions, parameters, safety filters, chat, structured output, streaming, async calls, token utilities, function calling, context caching, batch prediction, and text embeddings.

Key code patterns

Vertex AI client

from google import genai
 
PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT")
LOCATION = os.environ.get("GOOGLE_CLOUD_REGION", "global")
client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)

Creates a Gen AI SDK client targeting Vertex AI with project and region settings.

Text generation

response = client.models.generate_content(
    model=MODEL_ID,
    contents="What's the largest planet in our solar system?",
)
print(response.text)

Shows the basic synchronous Gemini request and text response pattern.

Multimodal prompt

response = client.models.generate_content(
    model=MODEL_ID,
    contents=[
        Part.from_uri(file_uri=image_uri, mime_type="image/png"),
        "Write a short and engaging blog post based on this picture.",
    ],
)

Demonstrates passing an image URI with text in one Gemini prompt.

Generation config

response = client.models.generate_content(
    model=MODEL_ID,
    contents=prompt,
    config=GenerateContentConfig(
        system_instruction=system_instruction,
        temperature=0.4,
        max_output_tokens=100,
    ),
)

Centralizes system instructions and model parameters in GenerateContentConfig.

Safety filters

safety_settings = [
    SafetySetting(
        category=HarmCategory.HARM_CATEGORY_HARASSMENT,
        threshold=HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
    )
]
response = client.models.generate_content(
    model=MODEL_ID,
    contents=prompt,
    config=GenerateContentConfig(safety_settings=safety_settings),
)

Applies Gemini safety thresholds and supports inspecting safety ratings.

Chat session

chat = client.chats.create(
    model=MODEL_ID,
    config=GenerateContentConfig(system_instruction=system_instruction, temperature=0.5),
)
response = chat.send_message("Write a function that checks if a year is a leap year.")

Creates a multi-turn chat with shared configuration and conversation state.

Structured JSON output

class Recipe(BaseModel):
    name: str
    description: str
    ingredients: list[str]
 
response = client.models.generate_content(
    model=MODEL_ID,
    contents="List a few popular cookie recipes and their ingredients.",
    config=GenerateContentConfig(response_mime_type="application/json", response_schema=Recipe),
)

Constrains model output to JSON using a Pydantic schema.

Function calling

get_destination = FunctionDeclaration(
    name="get_destination",
    description="Get the destination that the user wants to go to",
    parameters={"type": "OBJECT", "properties": {"destination": {"type": "STRING"}}},
)
tool = Tool(function_declarations=[get_destination])
response = client.models.generate_content(
    model=MODEL_ID, contents="I'd like to travel to Paris.", config=GenerateContentConfig(tools=[tool])
)

Defines a callable tool schema and lets Gemini return a matching function call.

Context cache

cached_content = client.caches.create(
    model="gemini-3.5-flash",
    config=CreateCachedContentConfig(
        system_instruction=system_instruction,
        contents=pdf_parts,
        ttl="3600s",
    ),
)

Stores repeated PDF context and references the cache in later generation calls.

Batch prediction

batch_job = client.batches.create(
    model=MODEL_ID,
    src=INPUT_DATA,
    config=CreateBatchJobConfig(dest=BUCKET_URI),
)
batch_job = client.batches.get(name=batch_job.name)

Runs large, non-latency-sensitive Gemini requests asynchronously with Cloud Storage I/O.

Text embeddings

response = client.models.embed_content(
    model="text-embedding-005",
    contents=["How do I renew my driver's license?"],
    config=EmbedContentConfig(output_dimensionality=128),
)
print(response.embeddings)

Gets text embeddings and customizes output dimensionality.

Models & APIs used

  • Models: gemini-3.5-flash, text-embedding-005
  • APIs / services: Vertex AI, Vertex AI API, Gemini API, Cloud Storage, BigQuery
  • SDKs / libraries: google-genai, pandas, fsspec, Pillow, requests, pydantic

When to use this

Use this pattern when starting a Python project that calls Gemini or text embedding models through Vertex AI with the Google Gen AI SDK.

Gotchas & caveats

  • Colab users must authenticate with google.colab.auth.authenticate_user().
  • Vertex AI use requires a Google Cloud project and the Vertex AI API enabled.
  • LOCATION defaults to global from GOOGLE_CLOUD_REGION when not explicitly set.
  • Context caching note says stable fixed-version models require a version postfix such as -001.
  • Batch Cloud Storage input must be JSONL, located in us-central1, and readable by the service account.
  • Batch output needs a Cloud Storage or BigQuery URI; the notebook creates a bucket with gsutil when BUCKET_URI is unset.
  • Batch jobs are asynchronous and require polling until JOB_STATE_SUCCEEDED or failure.

Best practices

  • Read project and region from GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION when notebook parameters are unset.
  • Use GenerateContentConfig for system instructions, parameters, safety settings, tools, schemas, and cached content.
  • Use Part.from_uri for model-readable Cloud Storage or HTTPS file inputs.
  • Inspect response.candidates[0].safety_ratings after applying safety filters.
  • Use response_mime_type application/json with response_schema for controlled generation.
  • Use count_tokens or compute_tokens before generation when token accounting matters.
  • Use batch prediction for many inputs that are not latency sensitive.
  • Delete cached content with client.caches.delete when it is no longer needed.

Graph View

Backlinks