Generative AI Google Brain

gemini-use-cases-video-analysis-youtube-video-analysis

3 min read

YouTube Video Analysis with Gemini

Source notebook

Repo path: gemini/use-cases/video-analysis/youtube_video_analysis.ipynb · Open on GitHub · intermediate

Analyzes public YouTube videos with Gemini for summaries, structured JSON extraction, and cross-video insights.

Summary

This notebook teaches direct analysis of public YouTube videos with Gemini through the Google Gen AI SDK on Vertex AI. It demonstrates summarizing one video, extracting structured product announcements with controlled JSON output, and asynchronously analyzing multiple videos. The workflow ends by parsing Gemini responses into pandas DataFrames and aggregating athlete or team appearances across years.

Key code patterns

Create Vertex AI GenAI client

from google import genai
 
client = genai.Client(
    enterprise=True,
    project=PROJECT_ID,
    location=LOCATION,
)

Uses the Google Gen AI SDK with project and location settings for Vertex AI access.

Summarize YouTube video

response = client.models.generate_content(
    model=GEMINI_FLASH_MODEL_ID,
    contents=[
        Part.from_uri(file_uri=YOUTUBE_VIDEO_URL, mime_type="video/webm"),
        "Give a detailed summary of this video.",
    ],
)

Passes a public YouTube URL as a video Part and prompts Gemini for a detailed summary.

Controlled JSON output

config = GenerateContentConfig(
    max_output_tokens=8192,
    response_mime_type="application/json",
    response_schema=response_schema,
)
 
response = client.models.generate_content(
    model=GEMINI_PRO_MODEL_ID,
    contents=[prompt, Part.from_uri(file_uri=url, mime_type="video/webm")],
    config=config,
)

Constrains Gemini output to valid JSON matching a declared schema.

Async batch video analysis

@retry(wait=wait_random_exponential(multiplier=1, max=120), stop=stop_after_attempt(2))
async def async_generate(prompt, yt_link):
    response = await client.aio.models.generate_content(
        model=GEMINI_PRO_MODEL_ID,
        contents=[prompt, Part.from_uri(file_uri=yt_link, mime_type="video/webm")],
        config=multiple_video_extraction_json_generation_config,
    )
    return response.to_json_dict()

Runs multiple Gemini video extraction requests concurrently with retry handling.

Parse and aggregate results

extract_text = json.loads(gemini_response)["candidates"][0]["content"]["parts"][0]["text"]
extract_df = pd.DataFrame(json.loads(extract_text))
 
all_results = pd.concat(result_dfs, ignore_index=True)

Converts Gemini JSON text responses into tabular data for downstream analysis.

Models & APIs used

When to use this

Use this pattern when you need Gemini to summarize or extract structured insights from public YouTube videos, including batches of videos.

Gotchas & caveats

  • A Google Cloud project is required and the Vertex AI API must be enabled.
  • Colab users must authenticate with google.colab.auth.authenticate_user().
  • PROJECT_ID falls back to GOOGLE_CLOUD_PROJECT and LOCATION defaults to global.
  • The notebook notes long video analysis can take several minutes.
  • The YouTube video should be public and short enough to fit the model context window.
  • Async requests include retry handling with at most two attempts.

Best practices

  • Use Part.from_uri with mime_type=“video/webm” for public YouTube video inputs.
  • Use response_mime_type=“application/json” and response_schema for structured extraction.
  • Constrain enum fields in the response schema when only specific values are valid.
  • Prompt the model to use only information in the video itself for extraction.
  • Use asynchronous generation for analyzing multiple videos more efficiently.
  • Normalize text casing before grouping extracted entities.

Graph View

Backlinks