Generative AI Google Brain

audio-speech-getting-started-gemini-3-1-flash-tts

3 min read

Gemini 3.1 Flash Text-to-Speech Generation

Source notebook

Repo path: audio/speech/getting-started/gemini_3_1_flash_tts.ipynb · Open on GitHub · intro

Shows Gemini 3.1 Flash TTS generation with voices, languages, speakers, and audio tags.

Summary

This notebook teaches how to use the Google Gen AI SDK for Python with an Agent Platform client to generate speech audio from text. The workflow sets project and location, calls generate_content with AUDIO response modality and SpeechConfig, then plays returned inline audio. It demonstrates prebuilt voices, automatic language detection, two-speaker voice configuration, audio tags, and using Gemini to add tags to a podcast transcript before TTS generation.

Key code patterns

Agent Platform client

PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT")
LOCATION = os.environ.get("GOOGLE_CLOUD_REGION", "global")
client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)

Creates the enterprise Google Gen AI client scoped to a Google Cloud project and location.

Play returned audio

def play_audio(response):
    audio_bytes = response.candidates[0].content.parts[0].inline_data.data
    audio_array = np.frombuffer(audio_bytes, dtype="<i2")
    display(Audio(data=audio_array, rate=24000))

Extracts inline audio bytes from the model response and plays them as 24 kHz PCM audio.

Single-speaker TTS

response = client.models.generate_content(
    model=speech_model,
    contents=prompt,
    config=types.GenerateContentConfig(
        response_modalities=["AUDIO"],
        speech_config=types.SpeechConfig(
            voice_config=types.VoiceConfig(
                prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Fenrir")))))

Requests audio output from Gemini 3.1 Flash TTS with a named prebuilt voice.

Multi-speaker TTS

types.MultiSpeakerVoiceConfig(
    speaker_voice_configs=[
        types.SpeakerVoiceConfig(speaker="Ryan", voice_config=types.VoiceConfig(
            prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Umbriel"))),
        types.SpeakerVoiceConfig(speaker="Katie", voice_config=types.VoiceConfig(
            prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Leda")))])

Maps prompt speaker names to separate prebuilt voices for a two-speaker clip.

Gemini-assisted tags

response = client.models.generate_content(
    model=gemini_model,
    contents=f"Please take the {podcast} provided and insert audio tags from {example_audio_tags}.")
podcast_tags = response.text
response = client.models.generate_content(model=speech_model, contents=podcast_tags, config=tts_config)

Uses a Gemini text model to insert emotional audio tags before sending the transcript to TTS.

Models & APIs used

When to use this

Use this pattern when building expressive text-to-speech workflows with prebuilt voices, multilingual input, two-speaker clips, or audio-tagged narration on Agent Platform.

Gotchas & caveats

  • A Google Cloud project is required and the Agent Platform API must be enabled.
  • Colab users must run auth.authenticate_user() before calling the API.
  • PROJECT_ID is expected from the notebook parameter or GOOGLE_CLOUD_PROJECT.
  • LOCATION defaults to global unless GOOGLE_CLOUD_REGION is set.
  • MultiSpeakerVoiceConfig supports up to 2 speakers, and speaker names must match the prompt.
  • For best results, audio tags should be in English even when the transcript is in another language.
  • The helper assumes returned audio can be decoded as little-endian int16 and played at 24000 Hz.
  • By default, audio generated with Gemini 3.1 Flash TTS utilizes SynthID.

Best practices

  • Use VoiceConfig with PrebuiltVoiceConfig to choose documented voice options.
  • Use response_modalities=[“AUDIO”] when requesting speech output.
  • Use MultiSpeakerVoiceConfig with one SpeakerVoiceConfig per speaker for multi-speaker clips.
  • Place audio tags immediately before the phrase or sentence they should influence.
  • Avoid overusing audio tags and match them to natural changes in tone or pace.
  • Use Gemini to add audio tags to long transcripts before TTS generation.

Graph View

Backlinks