Open Models Notebooks

27 notebooks from the open-models/ area of GoogleCloudPlatform/generative-ai.

Agents (1)

Building an ADK agent using QWEN 3 on Vertex AI — Builds and deploys an ADK weather agent using Qwen3 on Vertex AI and Agent Engine.

Accelerate LLM Inference with EAGLE Speculative Decoding on Vertex AI — Benchmarks EAGLE speculative decoding for Llama 4 Scout on Vertex AI against a baseline endpoint.
Get started with Vertex AI Model Garden SDK — Deploy and test open models on Vertex AI with the Model Garden SDK.
Import, Deploy, and Serve custom open models on Vertex AI using Vertex AI Model Garden SDK. — Imports Hugging Face open-model weights to GCS, deploys them on Vertex AI, and serves predictions.
Get started with Model Garden Terraform Deployment — Deploys Model Garden and Hugging Face open models to Vertex AI endpoints with Terraform.
Handling Reasoning with MaaS Models on Vertex AI using vLLM — Shows how to call reasoning MaaS models on Vertex AI through the OpenAI SDK.

Qwen 3 evaluation - Bring your own data eval — Compares fine-tuned and base Qwen 3 medical summaries using Vertex AI pairwise evaluation.
Using open autorater for running evaluations with Vertex AI Gen AI Evaluation — Deploys Selene as an open judge on Vertex AI and uses Gen AI Evaluation to score LLM responses.
Hugging Face DLCs: Using Gemma for running evaluations with Vertex AI Gen AI Evaluation — Deploys Gemma 2 on Vertex AI TGI and evaluates summarization with Gen AI Evaluation.

MetaMath with Vertex AI Open Source Model Tuning — Fine-tunes a Llama 3.1 8B model on MetaMathQA using Vertex AI managed tuning.
Fine-tuning GPT-OSS 20B with Unsloth on Vertex AI Colab Enterprise and Nvidia A100 40GB GPU — Fine-tunes GPT-OSS 20B with Unsloth LoRA on Vertex AI Colab Enterprise using an A100 GPU.
Hugging Face DLCs: Fine-tuning Gemma with Transformer Reinforcement Learning (TRL) on Vertex AI — Fine-tunes google/gemma-2b with TRL SFT and LoRA in a Vertex AI custom container job.

Serving Gemma 3 with Ollama on Cloud Run — Deploys Gemma 3 with Ollama as a private GPU-backed Cloud Run API and tests it with curl and Python.
Running Qwen 3 with Ollama in Cloud Run for Agents — Deploys Qwen3:8b with Ollama on GPU Cloud Run and uses it as an ADK agent model.
Serving Gemma 3 with vLLM on Cloud Run — Deploys Gemma 3 on Cloud Run with vLLM as an OpenAI-compatible chat API.
Deploying Multiple LoRA Adapters on Vertex AI with vLLM — Deploys Gemma 2 with multiple LoRA adapters on Vertex AI using a custom vLLM container.
Running a Gemma 2-based agentic RAG with Ollama on Vertex AI and LangGraph — Deploys a Gemma 2 Ollama container on Vertex AI and uses it in a LangGraph SQL RAG agent.
Hugging Face DLCs: Serving PaliGemma using Pytorch Inference on Vertex AI with Custom Handler — Deploys gated PaliGemma from Hugging Face to Vertex AI using a PyTorch DLC and custom handler.
Hugging Face DLCs: Serving PLLuM using Pytorch Inference on Vertex AI with Custom Handler — Deploys CYFRAGOVPL/PLLuM-12B-chat to Vertex AI with a Hugging Face PyTorch custom handler.
Hugging Face DLCs: Serving Gemma with Text Generation Inference (TGI) on Vertex AI — Deploys gated Gemma 7B IT from Hugging Face to Vertex AI using a TGI deep learning container.
Hugging Face DLCs: Serving Gemma 2 with multiple LoRA adapters with Text Generation Inference (TGI) on Vertex AI — Deploys Gemma 2 with SQL and code LoRA adapters on Vertex AI using a Hugging Face TGI custom handler.

Use Any OSS Gen AI Model Against Your BigQuery Data — Deploys Llama 3.3 70B on Vertex AI and calls it from BigQuery ML for medical transcript analytics.
Cloud Run GPU Inference: Gemma 2 RAG Q&A with Ollama and LangChain — Deploys Gemma 2 on Cloud Run GPU with Ollama and builds a LangChain RAG Q&A chain.
Deploying Llama 3 on Google Kubernetes Engine with Cloud Functions and vLLM — Deploys Llama 3.2 on GKE with vLLM and exposes it through a Gen2 Cloud Function.
Guess who or what app using Hugging Face Deep Learning container model on Vertex AI — Builds a Gradio riddle game using Gemini and a Hugging Face FLUX model deployed on Vertex AI.
Serving Open-Source LLMs on Vertex AI with LiteLLM and OpenAI-Compatible APIs — Deploys a Llama 3.1 Model Garden model on Vertex AI and calls it through LiteLLM OpenAI-style APIs.
Build and deploy a Hugging Face smolagent using DeepSeek-r1 on Vertex AI — Deploys DeepSeek R1 Distill Qwen 7B on Vertex AI and wraps it in a smolagents math verifier agent.

Area index · Overview · Home