Generative AI Google Brain

vision-getting-started-veo3-reference-to-video

3 min read

Veo 3.1 Reference to Video

Source notebook

Repo path: vision/getting-started/veo3_reference_to_video.ipynb · Open on GitHub · intro

Generates Veo 3.1 videos from reference images using the Google Gen AI SDK.

Summary

This notebook teaches how to use Veo 3.1 Reference-to-Video with asset images for subjects, settings, products, and combined references. It demonstrates project setup, Gen AI client initialization, local and Cloud Storage image inputs, video generation configuration, long-running operation polling, and displaying either video bytes or Cloud Storage output.

Key code patterns

Create enterprise Gen AI client

from google import genai
 
PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT")
LOCATION = os.environ.get("GOOGLE_CLOUD_REGION", "us-central1")
client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)

Initializes the SDK client against a Google Cloud project and region for Agent Platform usage.

Generate video from local reference images

operation = client.models.generate_videos(
    model=video_model_fast,
    prompt=prompt,
    config=types.GenerateVideosConfig(
        reference_images=[
            types.VideoGenerationReferenceImage(
                image=types.Image.from_file(location=first_image),
                reference_type="asset",
            )
        ],
        generate_audio=True,
    ),
)

Shows the core reference-to-video request using asset images and audio generation.

Poll long-running operation

while not operation.done:
    time.sleep(15)
    operation = client.operations.get(operation)
    print(operation)
 
if operation.response:
    show_video(operation.result.generated_videos[0].video.video_bytes)

Veo generation is asynchronous, so the notebook waits and retrieves the completed operation.

Use Cloud Storage image references and output

types.VideoGenerationReferenceImage(
    image=types.Image(gcs_uri=first_image_gcs, mime_type="image/png"),
    reference_type="asset",
)
 
config=types.GenerateVideosConfig(
    output_gcs_uri=output_gcs,
    resolution="1080p",
)

Demonstrates using gs:// assets directly and saving generated video output to Cloud Storage.

Models & APIs used

When to use this

Use this pattern when generating short videos that preserve subjects, products, or scenes from up to three reference images.

Gotchas & caveats

  • Requires an existing Google Cloud project and the Agent Platform API enabled.
  • Colab users must authenticate with google.colab.auth.authenticate_user().
  • PROJECT_ID falls back to GOOGLE_CLOUD_PROJECT when the placeholder is unchanged.
  • LOCATION defaults to us-central1 from GOOGLE_CLOUD_REGION when unset.
  • Reference-to-Video supports up to 3 asset images in one request.
  • 4k generation increases latency up to several minutes.
  • Cloud Storage output requires setting output_gcs to a bucket path.

Best practices

  • Use asset reference images for subjects, objects, or scenes that should appear in the final video.
  • Set aspect_ratio, number_of_videos, duration_seconds, resolution, person_generation, and generate_audio explicitly.
  • Poll the operation before reading generated_videos.
  • Use Cloud Storage URIs for multiple reference images when avoiding local downloads.
  • All Veo videos include SynthID digital watermarking.

Graph View

Backlinks