Generative AI Google Brain

vision-getting-started-imagen3-editing

3 min read

Imagen 3 Image Editing

Source notebook

Repo path: vision/getting-started/imagen3_editing.ipynb · Open on GitHub · intermediate

Edits images with Imagen 3 using inpainting, background swap, outpainting, and mask-free prompts.

Summary

This notebook teaches how to use the Google Gen AI SDK for Python with Imagen 3 on Agent Platform. It initializes a Gen AI client for a Google Cloud project, generates or loads source images, builds raw and mask reference images, and calls image editing modes for insertion, removal, background swap, outpainting, and mask-free edits.

Key code patterns

Create Gen AI client

from google import genai
 
PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT")
LOCATION = os.environ.get("GOOGLE_CLOUD_REGION", "us-central1")
client = genai.Client(enterprise=True, project=PROJECT_ID, location=LOCATION)

Configures the SDK to call Imagen through the Google Cloud project and region.

Generate source image

generated_image = client.models.generate_images(
    model="imagen-3.0-generate-002",
    prompt=image_prompt,
    config=GenerateImagesConfig(
        number_of_images=1,
        aspect_ratio="1:1",
        safety_filter_level="BLOCK_MEDIUM_AND_ABOVE",
        person_generation="DONT_ALLOW",
    ),
)

Creates an initial image that can be passed into later edit operations.

Mask-based inpainting

raw_ref_image = RawReferenceImage(reference_image=image, reference_id=0)
mask_ref_image = MaskReferenceImage(
    reference_id=1,
    reference_image=None,
    config=MaskReferenceConfig(mask_mode="MASK_MODE_FOREGROUND", mask_dilation=0.1),
)
edited_image = client.models.edit_image(
    model="imagen-3.0-capability-001",
    prompt=edit_prompt,
    reference_images=[raw_ref_image, mask_ref_image],
    config=EditImageConfig(edit_mode="EDIT_MODE_INPAINT_INSERTION", number_of_images=1),
)

Shows the core Imagen editing pattern: raw image plus mask reference plus edit mode.

User-provided mask

initial_image = Image.from_file(location="image-dog.png")
initial_image_mask = Image.from_file(location="image-dog-mask.png")
mask_ref_image = MaskReferenceImage(
    reference_id=1,
    reference_image=initial_image_mask,
    config=MaskReferenceConfig(mask_mode="MASK_MODE_USER_PROVIDED", mask_dilation=0.1),
)

Uses a downloaded mask file when automatic or semantic masking is not enough.

Mask-free edit

raw_ref_image = RawReferenceImage(reference_image=original_image, reference_id=0)
edited_image = client.models.edit_image(
    model="imagen-3.0-capability-001",
    prompt=prompt,
    reference_images=[raw_ref_image],
    config=EditImageConfig(edit_mode="EDIT_MODE_DEFAULT", number_of_images=1),
)

Applies prompt-only changes with the original image as the sole reference image.

Models & APIs used

When to use this

Use this pattern when an application needs controlled image edits on existing assets with Imagen 3.

Gotchas & caveats

  • Requires an existing Google Cloud project and the Agent Platform API enabled.
  • Colab requires explicit user authentication with auth.authenticate_user().
  • PROJECT_ID must be set directly or via GOOGLE_CLOUD_PROJECT.
  • LOCATION defaults to us-central1 if GOOGLE_CLOUD_REGION is not set.
  • Outpainting requires padding the source image and creating a user-provided mask.
  • Semantic masks require the correct segmentation class ID from the notebook table.

Best practices

  • Set safety_filter_level and person_generation in generation and editing configs.
  • Use RawReferenceImage for the source image and MaskReferenceImage for mask-based edits.
  • Use MASK_MODE_USER_PROVIDED when supplying your own mask image.
  • Use an empty prompt for inpainting removal requests where the object should simply be removed.
  • Display original and edited images side by side for visual comparison.

Graph View

Backlinks