Generative AI Google Brain

Getting Started

10 min read

Getting Started

Getting Started on Google Cloud is the foundation for moving from a Google Cloud project to working generative AI calls on Vertex AI and Agent Platform: authentication, client setup, model selection, request configuration, response inspection, and cleanup. It spans Gemini text, multimodal, tool use, audio, embeddings, RAG, Vector Search, batch inference, Live API, and related SDK and REST entry points.

Synthesized across 53 notebooks. See the source pages under Notebooks below for line-level grounding.

Most getting-started notebooks follow the same operational spine: create or select a Google Cloud project, enable the required APIs, authenticate, set PROJECT_ID and LOCATION, install the right SDK versions, and initialize a client before making the smallest useful request.

From there, the notebooks branch by workload. Core Gemini examples use the Google Gen AI SDK, REST or cURL, OpenAI-compatible Chat Completions, LangChain, or WebSockets; specialized paths cover image generation, TTS, transcription, embeddings, Vector Search, RAG Engine, batch inference, data analytics agents, Live API audio, logging, evaluation, prompt optimization, and model routing.

The recurring engineering pattern is to make configuration explicit, inspect structured response metadata, handle long-running and streaming operations deliberately, preserve state correctly across turns, and clean up billable resources after tutorials.

Canonical workflow

  1. Create or select a Google Cloud project, confirm billing where required, and enable the specific APIs used by the notebook, such as Vertex AI, Agent Platform, Text-to-Speech, Speech-to-Text, BigQuery, Cloud Storage, Vector Search, Dataform, or Gemini Data Analytics.
  2. Authenticate for the runtime: use Colab authentication in notebooks, Application Default Credentials for local development, or project-based Vertex AI authentication; then set PROJECT_ID and LOCATION with environment-variable fallbacks.
  3. Install or upgrade the required SDKs, such as google-genai, google-cloud-aiplatform, google-cloud-texttospeech, Speech-to-Text clients, LangChain integrations, or OpenAI-compatible libraries, matching notebook version requirements.
  4. Initialize the appropriate client or endpoint, choosing global or regional locations intentionally based on the feature, model, endpoint, or data residency constraint.
  5. Start with a minimal text or chat request, then add system instructions, safety settings, generation parameters, structured output schemas, tools, response modalities, media parts, retrieval tools, or audio streaming configuration as needed.
  6. Inspect the response shape before consuming it: check finish_reason, safety_ratings, usage_metadata, response parts, parsed structured output, grounding metadata, artifacts, task state, or audio/image inline data.
  7. For stateful, tool, batch, indexing, or live workflows, preserve the required state and lifecycle: chat history, thought signatures, function responses, IAM policy bindings, operation polling, cancellation, backoff, and streaming turn completion.
  8. Store outputs only when needed and clean up billable resources such as batch jobs, indexes, endpoints, buckets, collections, logging configs, and tutorial data after validation.

Best practices

See Getting Started - Best Practices for the full distilled guide.

  • Use a real Google Cloud project with the required APIs enabled before running examples; set PROJECT_ID and LOCATION explicitly or through GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_REGION fallbacks.
  • Prefer the Google Gen AI SDK unified interface for Vertex AI projects, and use REST, cURL, OpenAI-compatible Chat Completions, LangChain, or WebSockets when the notebook’s integration path requires that level of control.
  • Keep GenerateContentConfig or equivalent request configuration explicit: system_instruction, safety_settings, generation parameters, thinking configuration, response_modalities, response_mime_type, response_schema, tools, and media_resolution should be deliberate.
  • Use concise, specific prompts that ask one task at a time; add representative zero-shot, one-shot, or few-shot examples when needed, and state when provided context is the only source of truth.
  • Use system instructions to steer behavior, formatting, persona, and task boundaries, but keep persona constraints controlled because they can override other instructions.
  • Use streaming for user-facing latency, async calls for higher-throughput applications, batch inference for large non-latency-sensitive workloads, and Live API or WebSockets for bidirectional audio, video, or native audio interaction.

Key models & APIs

  • Google Gen AI SDK for Python, google-genai, Vertex AI API and Agent Platform API, including GenerateContent, chat, streaming, count_tokens, compute_tokens, tools, safety settings, and structured output, Gemini API on Vertex AI through REST, cURL, OpenAI-compatible Chat Completions, LangChain, and WebSocket interfaces, gemini-3.5-flash, gemini-3.1-pro-preview, gemini-3.1-flash-lite, gemini-3-flash-preview, gemini-2.5-flash, gemini-2.5-flash-lite, gemini-2.5-pro, gemini-2.5-flash-image, gemini-3.1-flash-image, gemini-3-pro-image, gemini-live-2.5-flash-native-audio and Gemini Live API, gemini-3.1-flash-tts-preview, gemini-2.5-flash-tts, Chirp 3 HD voices, and Cloud Text-to-Speech API, chirp_3 and Speech-to-Text V2, gemini-embedding-001, text-embedding-004, text-embedding-005, Vertex AI Vector Search, Vector Search 2.0, and Vertex AI RAG Engine, Cloud Storage, BigQuery, Dataform, Gemini Data Analytics DataA2AService, and data agent APIs, Vertex AI EvalTask, Prompt Optimizer, Model Optimizer, request-response logging, and Skill Registry

Notebooks (53)

Agents & ADK (4)

Audio (5)

Embeddings & Vector Search (2)

Gemini (32)

Open Models (1)

SDK (2)

Vertex AI Search (2)

Translation (1)

Vision (4)

Part of the Generative AI Google Brain · Getting Started - Best Practices

Graph View

Backlinks