Generative AI Google Brain

audio-speech-getting-started-get-started-with-chirp3-instant-custom-voice

3 min read

Get started with Chirp 3: Instant custom voice

Source notebook

Repo path: audio/speech/getting-started/get_started_with_chirp3_instant_custom_voice.ipynb · Open on GitHub · intermediate

Creates and uses a Chirp 3 Instant Custom Voice with Cloud Text-to-Speech and a Gradio app.

Summary

This notebook teaches how to create a temporary Instant Custom Voice key from reference and consent audio using Google Cloud Text-to-Speech API. It then synthesizes text with the cloned voice through a synchronous REST request and builds a Gradio interface for uploading audio, creating the voice, generating speech, and resetting the UI.

Key code patterns

Authenticate and set quota project

! gcloud config set project {PROJECT_ID}
! gcloud auth application-default set-quota-project {PROJECT_ID}
! gcloud auth application-default login -q
credentials, _ = google.auth.default()
credentials.refresh(google.auth.transport.requests.Request())

Loads application default credentials and obtains an access token for REST calls.

Generate voice cloning key

url = f"https://{API_ENDPOINT}/v1beta1/voices:generateVoiceCloningKey"
request_body = {
  "reference_audio": {"audio_config": {"audio_encoding": "LINEAR16", "sample_rate_hertz": 24000}, "content": reference_audio_bytes},
  "voice_talent_consent": {"audio_config": {"audio_encoding": "LINEAR16", "sample_rate_hertz": 24000}, "content": consent_audio_bytes},
  "consent_script": "I am the owner of this voice and I consent to Google using this voice to create a synthetic voice model.",
  "language_code": "en-US",
}
response = requests.post(url, headers=headers, json=request_body)
voice_key = response.json().get("voiceCloningKey")

Creates the temporary voice_cloning_key from base64 reference and consent audio.

Synthesize cloned voice

url = f"https://{API_ENDPOINT}/v1beta1/text:synthesize"
request_body = {
  "input": {"text": text},
  "voice": {"language_code": "en-US", "voice_clone": {"voice_cloning_key": voice_key}},
  "audioConfig": {"audioEncoding": "LINEAR16", "sample_rate_hertz": 24000},
}
response = requests.post(url, headers=headers, json=request_body)
audio_content = response.json().get("audioContent")

Uses the generated voice key in the text:synthesize endpoint to produce LINEAR16 audio.

Gradio voice demo

with gr.Blocks(theme=gr.themes.Default(primary_hue="blue")) as app:
    full_voice_key = gr.State("")
    reference_audio = gr.Audio(label="Reference Voice", type="filepath")
    consent_audio = gr.Audio(label="Consent Audio", type="filepath")
    create_btn.click(create_voice_with_masking, inputs=[reference_audio, consent_audio], outputs=[full_voice_key, voice_key_output])
    generate_btn.click(generate_speech, inputs=[full_voice_key, text_input], outputs=[audio_output, status_output])
app.launch(share=True)

Wraps the API workflow in an upload-and-generate web UI.

Models & APIs used

  • Models: Chirp 3 Instant Custom Voice
  • APIs / services: Cloud Text-to-Speech API
  • SDKs / libraries: gradio, google.auth, requests, numpy

When to use this

Use this pattern when you need to prototype voice cloning and speech synthesis with Chirp 3 Instant Custom Voice from consented audio samples.

Gotchas & caveats

  • Voice cloning capability is restricted to allow-listed users.
  • The Text-to-Speech API must be enabled in an existing Google Cloud project.
  • Chirp 3 Instant Custom Voice has documented regional availability constraints.
  • The notebook uses TTS_LOCATION = “global” but points readers to regional availability documentation.
  • Reference and consent audio are expected as local WAV files, with LINEAR16 and 24000 Hz used in requests.
  • The consent audio must state the provided consent script.
  • Requests require an OAuth access token and x-goog-user-project header.
  • The Gradio launch uses share=True, which creates a shared app link.

Best practices

  • Use application default credentials and set the quota project before API calls.
  • Separate reference audio and voice talent consent audio in the generateVoiceCloningKey request.
  • Base64-encode WAV audio before sending it in the JSON payload.
  • Check for missing audioContent before trying to play generated audio.
  • Encapsulate REST calls and audio conversion in helper functions before wiring the UI.
  • Store the full voice key in Gradio state while showing a masked voice key in the interface.

Graph View

Backlinks