Generative AI Google Brain

Image & Video Generation

6 min read

Image & Video Generation

Image & Video Generation on Google Cloud covers generating, editing, upscaling, evaluating, and operationalizing visual media with Gemini image models, Imagen, Veo, Virtual Try-On, Vertex AI evaluation, and deployable open models. The area combines multimodal prompting, reference assets, Cloud Storage inputs and outputs, long-running generation operations, and quality evaluation workflows.

Synthesized across 18 notebooks. See the source pages under Notebooks below for line-level grounding.

The notebooks show two main production paths: managed Google models through the Google Gen AI SDK on Agent Platform or Vertex AI, and open-model deployment through Vertex AI Model Garden. Gemini image models are used for image generation, image editing, grounding, visual object detection, structured parsing, and reference-driven creative workflows; Imagen focuses on image generation, editing, and upscaling; Veo handles text-to-video, image-to-video, reference-to-video, frame interpolation, and video extension.

Most workflows depend on explicit project, location, authentication, and API setup. Image outputs usually require IMAGE in response_modalities, media inputs are passed as bytes or Cloud Storage URIs, and larger video outputs are commonly written to Cloud Storage. Many examples use global location for Gemini preview/image models, while Imagen, Veo, and Virtual Try-On commonly default to us-central1 when GOOGLE_CLOUD_REGION is unset.

Quality work is not limited to generation. Gecko-based evaluation notebooks generate prompt-specific rubrics, validate generated questions through VQA-style checks, compare positive examples with counterexamples, and aggregate per-question results into final scores. Production-grade use also requires quota awareness, cost review, cleanup of deployed endpoints, and inspection of generated media before reuse.

Canonical workflow

  1. Enable the required Google Cloud APIs, set PROJECT_ID and LOCATION or environment variables, and authenticate the notebook or runtime.
  2. Choose the appropriate model family: Gemini image models for multimodal image generation and editing, Imagen for image editing and upscaling, Veo for video generation and extension, Virtual Try-On for apparel try-on, or Model Garden for deployable open models.
  3. Prepare inputs as structured prompts plus local files, Part.from_bytes, Image.from_file, Cloud Storage URIs, reference images, masks, or structured schemas as required by the workflow.
  4. Configure generation explicitly, including response_modalities, ImageConfig or GenerateImagesConfig, aspect ratio, image size, number of images or videos, safety settings, person_generation, audio generation, output MIME type, and Cloud Storage output paths.
  5. Run generation or editing, then check candidates, finish_reason, thought parts, inline image data, or long-running operation status before reading or displaying results.
  6. Inspect outputs visually, compare originals and edits side by side when relevant, decode base64 endpoint responses when using deployed open models, and save intermediate assets when they will be reused.
  7. For evaluation, generate prompt-specific Gecko rubrics, inspect generated questions and validator reliability, add manual questions when needed, and aggregate validation results into final scores.
  8. Clean up billable resources such as deployed endpoints, uploaded models, Gradio apps, and generated output locations when the workflow is complete.

Best practices

See Image & Video Generation - Best Practices for the full distilled guide.

  • Set project, region, and credentials through environment variables, Colab Secrets, or platform defaults instead of hardcoding runtime configuration.
  • Enable the required API before running notebooks: Agent Platform or Vertex AI for managed model workflows, plus Artifact Registry when deploying Hugging Face container models.
  • Use explicit generation configuration rather than relying on defaults: response_modalities, image_config, aspect_ratio, image size, output_mime_type, number_of_images, number_of_videos, duration_seconds, resolution, safety_filter_level, person_generation, and generate_audio where applicable.
  • Always check response candidates, finish_reason, inline image data, thought parts, generated_videos, or operation status before assuming media was produced.
  • Use Part.from_bytes or Image.from_file for local media and Part.from_uri or gcs_uri Image objects for Cloud Storage or remote media inputs; specify mime_type for Cloud Storage image and video URIs.
  • Use Cloud Storage output paths for larger generated videos, video extension outputs, or persistent generated assets.

Key models & APIs

  • Google Gen AI SDK (google-genai), google-cloud-aiplatform and google-cloud-aiplatform[evaluation]>=1.122.0, Vertex AI API, Agent Platform API at aiplatform.googleapis.com, Vertex AI Gecko predefined image and video evaluation rubrics, Vertex AI Model Garden SDK, Gemini image models: gemini-2.5-flash-image, gemini-2.5-flash-image-preview, gemini-3.1-flash-image, gemini-3-pro-image, Gemini multimodal and prompt-helper models: gemini-2.5-flash, gemini-2.5-pro, gemini-2.0-flash, gemini-3.5-flash, gemini-3.1-pro-preview, Imagen models: imagen-3.0-generate-002, imagen-3.0-capability-001, imagen-4.0-generate-001, imagen-4.0-upscale-preview, Veo models: veo-3.1-generate-001, veo-3.1-generate-preview, veo-3.1-fast-generate-001, veo-3.1-fast-generate-preview, veo-3.1-lite-generate-001, Virtual Try-On model: virtual-try-on-001, Open models in Model Garden examples: black-forest-labs/FLUX.1-dev, stabilityai/stable-diffusion-xl-base-1.0, Google Search grounding tools for Gemini image generation workflows, Cloud Storage gs:// media inputs and output_gcs_uri or output_gcs paths

Notebooks (18)

Gemini (9)

Open Models (2)

Vision (7)

Part of the Generative AI Google Brain · Image & Video Generation - Best Practices

Graph View

Backlinks