Generative AI Google Brain

audio-music-getting-started-lyria3-music-generation

2 min read

Lyria 3 Music Generation

Source notebook

Repo path: audio/music/getting-started/lyria3_music_generation.ipynb · Open on GitHub · intro

Generates Lyria 3 music tracks, clips, lyrics, and streams with the Google Gen AI SDK.

Summary

This notebook teaches how to call Lyria 3 from Agent Platform using the Google Gen AI SDK for Python. It demonstrates project setup, text-to-music generation, image-conditioned clips, custom lyrics, Interactions API calls, and streaming text/audio responses.

Key code patterns

Create GenAI client

client = genai.Client(
    enterprise=True,
    project=PROJECT_ID,
    location=LOCATION,
)

Initializes the SDK client for Agent Platform with project and region settings.

Generate text-to-music

response = client.models.generate_content(
    model=music_model,
    contents=prompt,
    config=types.GenerateContentConfig(
        response_modalities=["AUDIO", "TEXT"]
    ),
)

Requests both generated audio and text such as lyrics or song structure.

Use image prompt

contents=[
    types.Part.from_bytes(data=image, mime_type="image/png"),
    "Generate an instrumental clip based on this input image...",
]

Shows multimodal music clip generation from image bytes plus text guidance.

Stream interactions

stream = client.interactions.create(
    model=music_model,
    input="Generate a song about spending a day in Seoul in Korean.",
    stream=True,
)

Uses the Interactions API to process incremental text and audio deltas.

Models & APIs used

When to use this

Use this pattern when building applications that generate Lyria 3 music from text, images, lyrics, or streamed interactions.

Gotchas & caveats

  • Requires an existing Google Cloud project.
  • Requires enabling the Agent Platform API.
  • Colab requires authenticate_user().
  • LOCATION defaults to global from GOOGLE_CLOUD_REGION.
  • Image prompt requests can include up to 10 images.
  • Streaming audio data is base64 decoded before playback.
  • Supported music languages listed are English, German, Spanish, French, Hindi, Japanese, Korean, and Portuguese.

Best practices

  • Set response_modalities to include both AUDIO and TEXT when both outputs are needed.
  • Provide genre, vocal style, and instruments in prompts for text-to-music generation.
  • Use types.Part.from_bytes with the correct image MIME type for image-conditioned generation.
  • Read PROJECT_ID and LOCATION from environment variables when notebook parameters are unset.
  • Clean model text output before displaying Markdown and audio.

Graph View

Backlinks