Image & Video Generation — Best Practices

Distilled from 18 notebooks tagged Image & Video Generation in the GoogleCloudPlatform/generative-ai repository. The From the notebooks section below cites the per-notebook source for grounding.

Do this

• Set project, region, and credentials through environment variables, Colab Secrets, or platform defaults instead of hardcoding runtime configuration.
• Enable the required API before running notebooks: Agent Platform or Vertex AI for managed model workflows, plus Artifact Registry when deploying Hugging Face container models.
• Use explicit generation configuration rather than relying on defaults: response_modalities, image_config, aspect_ratio, image size, output_mime_type, number_of_images, number_of_videos, duration_seconds, resolution, safety_filter_level, person_generation, and generate_audio where applicable.
• Always check response candidates, finish_reason, inline image data, thought parts, generated_videos, or operation status before assuming media was produced.
• Use Part.from_bytes or Image.from_file for local media and Part.from_uri or gcs_uri Image objects for Cloud Storage or remote media inputs; specify mime_type for Cloud Storage image and video URIs.
• Use Cloud Storage output paths for larger generated videos, video extension outputs, or persistent generated assets.
• Poll Veo long-running operations with client.operations.get before reading generated results.
• Use detailed prompts for video generation, including subject, action, scene, camera, style, temporal details, audio, and dialogue when relevant.
• Use reference images, character sheets, previous scenes, and explicit image references such as Image 1 or Image 2 to preserve visual consistency across generations and edits.
• Keep edits precise: state what should be removed, preserved, or changed; use masks for mask-based Imagen edits; use empty prompts for simple inpainting removal when the object should simply disappear.
• Use structured outputs with response_mime_type and response_schema when object detection or downstream automation needs parseable results.
• For Gecko evaluation, generate rubrics from user prompts, compare matching prompts with similar counterexample prompts, inspect validator questions, and add manual questions when application quality requires it.
• For Model Garden deployments, list deployable models and deploy options first, accept EULAs when required, handle deployment exceptions, and delete endpoints with force=True during cleanup.
• For batch workflows, inspect inputs and outputs, track job progress, catch per-job exceptions, add short delays to reduce quota pressure, and package outputs for download.
• Save prompts, source IDs, and relevant generation metadata with assets so generated media can be restored or audited later.

Avoid this

• Running notebooks without enabling the required Agent Platform, Vertex AI, or Artifact Registry APIs causes setup failures before generation begins.
• Assuming Colab authentication, local Jupyter authentication, and managed runtime credentials behave the same leads to credential errors; Colab commonly needs google.colab.auth.authenticate_user().
• Omitting IMAGE from response_modalities prevents Gemini image generation, and omitting TEXT when text plus image is desired prevents multimodal output handling.
• Reading generated media before checking finish_reason, candidates, inline_data, thought parts, or long-running operation completion can produce missing or misleading outputs.
• Using the wrong location can break preview or model-specific workflows; several Gemini image workflows use global, while other notebooks default to us-central1 from GOOGLE_CLOUD_REGION.
• Leaving Cloud Storage placeholders such as gs://[your-bucket-path] unchanged prevents video output or extension workflows from saving results.
• Ignoring quota, regional accelerator availability, GPU requirements, or organization policy constraints can block Model Garden and Hugging Face deployments.
• For visual object detection, misreading Gemini bounding boxes causes incorrect crops because boxes are normalized to a 0-1000 coordinate space and ordered as [y1, x1, y2, x2].

From the notebooks

Evaluate images with Gecko

• Generate rubrics from each prompt before validation so the metric adapts to prompt-specific challenges.
• Use CustomOutputConfig with return_raw_output=True and parsing_fn for custom validator outputs.
• Compare matching prompts with similar counterexample prompts to show high-quality and low-quality responses.
• Inspect generated questions and validator reliability, and manually add questions when needed.
• Aggregate validation results into final scores and per-question QA results.

Evaluate images with predefined Gecko

• Generate rubrics from the user prompts before evaluating responses.
• Use counterexample prompts with the same images to demonstrate high and low quality evaluations.
• Inspect generated questions and validator reliability when analyzing quality.
• Manually add questions when desired for an application.
• Review Vertex AI pricing and estimate costs before running evaluation.

Evaluate videos with predefined Gecko

• Generate rubrics from the user prompts before evaluating responses.
• Use similar counterexample prompts to demonstrate high-quality and low-quality response differences.
• Inspect generated questions and validator reliability when analyzing quality.
• Manually add questions when needed for an application.

Gemini 2.5 Flash Image (Nano Banana 🍌) Generation

• Set response_modalities to match the desired output modalities.
• Use ImageConfig to specify aspect_ratio for generated images.
• Check finish_reason before assuming an image was generated.
• Iterate over response content parts and handle both text and inline_data.
• Use Part.from_bytes for local image inputs.

Gemini 3.1 Flash Image (Nano Banana 2 🍌) Generation

• Use types.GenerateContentConfig to set response_modalities, image_config, thinking_config, and tools.
• Check response.candidates[0].finish_reason before assuming an image was generated.
• Skip parts marked thought when displaying generated images unless inspecting thoughts intentionally.
• Use Part.from_bytes for local image or video bytes and Part.from_uri for YouTube or Cloud Storage inputs.
• Use Google Search tools when prompts need current web or image grounding.

Gemini 3 Pro Image (Nano Banana Pro 🍌) Generation

• Install or upgrade google-genai before running the notebook.
• Fallback to the GOOGLE_CLOUD_PROJECT environment variable when PROJECT_ID is not set.
• Use GenerateContentConfig with response_modalities and ImageConfig for image output control.
• Check finish_reason before assuming an image was generated.
• Skip thought parts when displaying final generated images.

Nano Banana 🍌: Gemini 2.5 Flash Image Recipes

• Defines shared model and generation configuration before running recipes.
• Uses helper functions to create canvases and display generated images consistently.
• Checks response candidates and inline image data before displaying output.
• Uses blank canvases to guide aspect ratio-sensitive outputs.
• Uses source and reference images with text prompts for grounded image edits.

Generating Consistent Imagery with Gemini 🍌

• Use environment variables or Colab Secrets instead of hardcoding API configuration.
• Use a character sheet as a reusable design reference for future image-generation tasks.
• Refer explicitly to input images, such as Image 1 and Image 2, to avoid ambiguity.
• Clarify removed or changed objects, such as no longer holding the map or removing ice axes.
• Spend time refining the first scene because it cascades into later generated scenes.

Detecting and Editing Visual Objects with Gemini

• Store environment configuration outside source code with environment variables, Colab Secrets, or platform defaults.
• Use structured outputs with response_mime_type and response_schema for automated parsing.
• Keep bounding box granularity controlled through prompt wording.
• Use precise prompts such as excluding captions from boxes or preserving original line breaks.
• Use low randomness, seed, and image response modalities for more deterministic restoration outputs.

Get started with Vertex AI Model Garden SDK

• Use list_deployable_models before selecting a model ID.
• Use list_deploy_options to verify supported deployment configurations before deploying.
• Use environment variables for project and region defaults.
• Accept model EULAs explicitly when required.
• Wrap deployment attempts in try/except to surface deployment errors.

Guess who or what app using Hugging Face Deep Learning container model on Vertex AI

• Initialize aiplatform and vertexai with explicit project and location values.
• Retrieve the Hugging Face token with get_token instead of hardcoding it in the notebook.
• Register the Hugging Face model in Vertex AI Model Registry before deploying it to an endpoint.
• Use separate helper functions for Gemini content generation, subject extraction, prompt generation, and image generation.
• Use temperature 0 and candidate_count 1 for Gemini riddle solving and prompt generation.

Imagen 3 Image Editing

• Set safety_filter_level and person_generation in generation and editing configs.
• Use RawReferenceImage for the source image and MaskReferenceImage for mask-based edits.
• Use MASK_MODE_USER_PROVIDED when supplying your own mask image.
• Use an empty prompt for inpainting removal requests where the object should simply be removed.
• Display original and edited images side by side for visual comparison.

Imagen 4 Image Upscale

• Install or upgrade google-genai before using the notebook.
• Authenticate only in Colab by checking for google.colab in sys.modules.
• Keep generation and upscaling model IDs in variables.
• Use types.GenerateImagesConfig for aspect ratio, image count, and image size.
• Use types.Image.from_file for local files and types.Image(gcs_uri=…) for Cloud Storage images.

Veo 3.1 Advanced Controls

• Use Cloud Storage URIs directly for remote images instead of downloading when appropriate.
• Poll long-running operations with client.operations.get before reading results.
• Set output_gcs_uri for larger generated videos and video extension outputs.
• Specify mime_type when passing Cloud Storage image or video URIs.
• Use SynthID-watermarked Veo outputs as noted by the notebook.

Veo 3.1 Reference to Video

• Use asset reference images for subjects, objects, or scenes that should appear in the final video.
• Set aspect_ratio, number_of_videos, duration_seconds, resolution, person_generation, and generate_audio explicitly.
• Poll the operation before reading generated_videos.
• Use Cloud Storage URIs for multiple reference images when avoiding local downloads.
• All Veo videos include SynthID digital watermarking.

Veo 3.1 Video Generation

• Use detailed prompts for better video quality.
• Include subject, action, scene, camera, style, temporal, audio, and dialogue details when relevant.
• Use Gemini to synthesize keyword choices into a cohesive Veo prompt.
• Use Veo 3.1 Fast when latency is a priority.
• Use Veo 3.1 Lite for rapid prototyping and iteration.

Virtual Try-On: Image Generation

• Use Image.from_file for local person and product images.
• Save an intermediate try-on output locally when it will be reused in a later request.
• Set output_mime_type and number_of_images explicitly in RecontextImageConfig or GenerateImagesConfig.
• Use safety_filter_level explicitly when generating or recontextualizing images.
• Combine multiple clothing items into one photo when trying on multiple items in a single request.

Virtual Try-On: Batch Generation Pipeline

• Reads project and region from environment variables when notebook parameters are not set.
• Displays input images before generation and displays generated images afterward for visual inspection.
• Uses response.raise_for_status() when fetching public image URLs.
• Tracks current job and total jobs while processing the Cartesian product of inputs.
• Catches per-job exceptions and skips failed jobs instead of stopping the whole batch.

---

Back to Image & Video Generation · Best Practices Map