Generative AI Google Brain

search-gemini-enterprise-oss-model-with-gemini-enterprise

4 min read

Open Source Models (Gemma) as a agent with Gemini Enterprise

Source notebook

Repo path: search/gemini-enterprise/oss_model_with_gemini_enterprise.ipynb · Open on GitHub · advanced

Deploys Gemma on Cloud Run, wraps it with ADK, deploys to Agent Engine, and registers it in Gemini Enterprise.

Summary

This notebook demonstrates an end-to-end path for making an open-source Gemma model available as an agent in Gemini Enterprise. It deploys Gemma to Cloud Run, builds a Google ADK agent that calls the deployed model through an Ollama client, tests the agent with AdkApp, deploys it to Vertex AI Agent Engine, and registers it through the Discovery Engine API. It also shows an optional LiteLLM path for using Gemma as the root agent.

Key code patterns

Deploy Gemma on Cloud Run

!gcloud run deploy {SERVICE_NAME} \
  --image us-docker.pkg.dev/cloudrun/container/gemma/{GEMMA_PARAMETER} \
  --gpu 1 --gpu-type nvidia-l4 \
  --memory 32Gi --no-allow-unauthenticated \
  --region {REGION}

Hosts Gemma behind an authenticated Cloud Run endpoint for agent integration.

Call secured Gemma endpoint

auth_req = google.auth.transport.requests.Request()
id_token = google.oauth2.id_token.fetch_id_token(auth_req, api_base_url)
auth_headers = {"Authorization": f"Bearer {id_token}"}
client = Client(host=api_base_url, headers=auth_headers)
response = client.chat(model="gemma3:1b", messages=[{"role":"user","content":question}])

Uses an ID token and Ollama client to call the private Cloud Run model service.

Wrap Gemma as an ADK tool

gemma_agent = LlmAgent(
  model="gemini-2.5-pro",
  name="gemma_agent",
  instruction="Only answer the question based on provided context from tool.",
  tools=[get_answer],
)

Uses Gemini as the orchestrating root agent while delegating answers to the Gemma tool.

Test with AdkApp

app = reasoning_engines.AdkApp(agent=gemma_agent, enable_tracing=True)
session = app.create_session(user_id="u_1232")
contents = types.Content(role="user", parts=[types.Part.from_text(text=query)])
for event in app.stream_query(user_id="u_1232", session_id=session.id, message=contents.model_dump()):
    print(event)

Validates the agent locally before managed deployment.

Deploy and register agent

remote_app = agent_engines.create(
  display_name="Gemma Agent v7",
  agent_engine=app,
  requirements=["litellm (==1.74.7)", "google-adk (==1.7.0)"]
)

Packages the ADK app for Agent Engine, then the notebook registers it with Gemini Enterprise via Discovery Engine.

Models & APIs used

  • Models: gemma3-1b, gemma3:1b, gemini-2.5-pro
  • APIs / services: Cloud Run, IAM API, Vertex AI, Agent Engine, Cloud Storage, Discovery Engine API, Gemini Enterprise
  • SDKs / libraries: google-adk, litellm, ollama, google-genai, vertexai, google-auth, pydantic

When to use this

Use this pattern when you need to expose a self-hosted Gemma model as a governed conversational agent in Gemini Enterprise.

Gotchas & caveats

  • Project ID, project number, region, Cloud Run URL, GCS bucket, OAuth client ID, OAuth client secret, Gemini Enterprise engine ID, and reasoning engine ID are placeholders that must be updated.
  • The Cloud Run service is deployed with —no-allow-unauthenticated, so callers need a valid Google-signed ID token.
  • The deployment requests one nvidia-l4 GPU, 8 CPUs, 32Gi memory, max-instances 1, and us-central1 region.
  • The notebook pins google-adk1.7.0, litellm1.74.7, ollama0.5.1, google-genai1.24.0, and pydantic==2.11.7.
  • The LiteLLM Gemma-root-agent path is shown as commented code, while the active code uses gemini-2.5-pro as the LlmAgent model.
  • The final authorization IDs differ between create authorization and agent registration examples: customhr9893 versus customhr9885.

Best practices

  • Deploy the open-source model first, then integrate it through a tool function before agent deployment.
  • Keep the Cloud Run model endpoint private and authenticate with an ID token.
  • Test the ADK app locally with a session and stream_query before creating the remote Agent Engine app.
  • Pin runtime requirements when deploying the agent to Agent Engine.
  • Use a staging bucket when initializing Vertex AI for Agent Engine deployment.
  • Register the provisioned reasoning engine in Gemini Enterprise so authorized users can discover and use the agent.

Graph View

Backlinks