Generative AI Google Brain

sdk-retries-configure-retries

3 min read

Configuring Retries in the Gen AI SDK

Source notebook

Repo path: sdk/retries/configure_retries.ipynb · Open on GitHub · intro

Configures Google Gen AI SDK HTTP retries for Gemini calls at client and request level.

Summary

This notebook teaches how to configure custom HTTP retry behavior in the Google Gen AI SDK for Gemini requests through Vertex AI. It installs and imports google-genai, authenticates in Colab when needed, sets a Google Cloud project and global location, then shows retry policies applied either to a genai.Client or to an individual generate_content request. The retry policy targets transient and rate-limit errors with backoff-related settings, status codes, and request timeout.

Key code patterns

Client-level retries

retry_options = types.HttpRetryOptions(
    initial_delay=1.0,
    attempts=5,
    exp_base=2,
    http_status_codes=[429, 500, 502, 503, 504],
)
http_options = types.HttpOptions(
    retry_options=retry_options,
    timeout=120 * 1000,
)
client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION, http_options=http_options)

Applies one retry policy to all API calls made with the configured client.

Inline client configuration

client = genai.Client(
    vertexai=True,
    project=PROJECT_ID,
    location=LOCATION,
    http_options=types.HttpOptions(
        retry_options=types.HttpRetryOptions(initial_delay=1.0, attempts=10, http_status_codes=[429, 500, 502, 503, 504]),
        timeout=120 * 1000,
    ),
)

Shows the complete setup pattern for retries and timeout in one client constructor.

Request-level retries

response = client.models.generate_content(
    model="gemini-3.5-flash",
    contents="Tell me a joke about a rabbit.",
    config=types.GenerateContentConfig(
        http_options=types.HttpOptions(
            retry_options=types.HttpRetryOptions(initial_delay=1.0, attempts=10, http_status_codes=[429, 500, 502, 503, 504]),
            timeout=120 * 1000,
        )
    ),
)

Overrides retry behavior for a single generate_content request using config.

Models & APIs used

  • Models: gemini-3.5-flash
  • APIs / services: Vertex AI, Gemini API
  • SDKs / libraries: google-genai

When to use this

Use this pattern when Gemini calls need custom resilience for transient network failures, 5xx service errors, or 429 rate limits.

Gotchas & caveats

  • The notebook requires google-genai to be installed or upgraded.
  • Colab users must authenticate with google.colab.auth.authenticate_user().
  • A Google Cloud project is required unless using a different API key path; this tutorial uses a project.
  • The Vertex AI API must be enabled for the project.
  • PROJECT_ID falls back to the GOOGLE_CLOUD_PROJECT environment variable when the placeholder is unchanged.
  • The notebook uses LOCATION = “global”.
  • Timeout is configured in milliseconds, shown as 120 * 1000.

Best practices

  • Configure retries for transient network errors, temporary server unavailability, and rate limits.
  • Use exponential backoff settings such as initial_delay, attempts, exp_base, max_delay, and jitter for fine-grained retry control.
  • Retry only selected HTTP status codes such as 429, 500, 502, 503, and 504.
  • Apply retries at the client level when all calls should share one policy.
  • Apply retries at the request level when only one call needs custom behavior.
  • Set an explicit request timeout alongside retry options.

Graph View

Backlinks