Generative AI Google Brain

gemini-orchestration-intro-langchain-gemini

3 min read

Getting Started with LangChain 🦜️🔗 + Gemini API in Vertex AI

Source notebook

Repo path: gemini/orchestration/intro_langchain_gemini.ipynb · Open on GitHub · intro

Introduces LangChain with Gemini on Vertex AI for prompts, embeddings, retrieval, memory, and chains.

Summary

This notebook teaches core LangChain components using Gemini API in Vertex AI for text generation, chat, and embeddings. It walks through prompt templates, few-shot example selection, structured output parsing, document loading, text splitting, vector retrieval, conversation memory, sequential chains, summarization, and retrieval-based PDF question answering.

Key code patterns

Initialize Gemini models

llm = GoogleGenerativeAI(model="gemini-3.5-flash", project=PROJECT_ID, location=LOCATION, enterprise=True)
chat = ChatGoogleGenerativeAI(model="gemini-3.5-flash", project=PROJECT_ID, location=LOCATION, enterprise=True)
embeddings = GoogleGenerativeAIEmbeddings(model="gemini-embedding-001", project=PROJECT_ID, location=LOCATION, enterprise=True)

Defines LangChain wrappers for text, chat, and embedding calls through Vertex AI.

Prompt template

prompt = PromptTemplate(input_variables=["location"], template=template)
final_prompt = prompt.format(location="Rome")
output = llm.invoke(final_prompt)

Builds reusable prompts with runtime variables before invoking the LLM.

Semantic example selection

example_selector = SemanticSimilarityExampleSelector.from_examples(
    examples, embeddings, FAISS, k=2
)
similar_prompt = FewShotPromptTemplate(
    example_selector=example_selector,
    example_prompt=example_prompt,
    prefix="Give the location an item is usually found in",
    suffix="Input: {noun}\nOutput:",
    input_variables=["noun"],
)

Uses embeddings and FAISS to choose relevant few-shot examples dynamically.

Structured output parser

response_schemas = [
    ResponseSchema(name="bad_string", description="This a poorly formatted user input string"),
    ResponseSchema(name="good_string", description="This is your response, a reformatted response"),
]
output_parser = StructuredOutputParser.from_response_schemas(response_schemas)
format_instructions = output_parser.get_format_instructions()
parsed = output_parser.parse(llm_output)

Adds format instructions to prompts and parses model text into a structured result.

Web retrieval with FAISS

documents = WebBaseLoader("http://www.paulgraham.com/worked.html").load()
texts = RecursiveCharacterTextSplitter(chunk_size=1500, chunk_overlap=50).split_documents(documents)
db = FAISS.from_documents(texts, embeddings)
retriever = db.as_retriever()
docs = retriever.invoke("what types of things did the author want to develop or build?")

Creates a local vector index from web text and retrieves semantically relevant chunks.

RetrievalQA over PDF

documents = PyPDFLoader(url).load()
docs = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=50).split_documents(documents)[:250]
db = Chroma.from_documents(docs, embeddings)
retriever = db.as_retriever(search_type="similarity", search_kwargs={"k": 2})
qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents=True)
result = qa.invoke({"query": query})

Combines PDF loading, chunking, embeddings, Chroma retrieval, and Gemini synthesis for document QA.

Map-reduce summarization

documents = WebBaseLoader(url).load()
texts = RecursiveCharacterTextSplitter(chunk_size=1500, chunk_overlap=50).split_documents(documents)
chain = load_summarize_chain(llm, chain_type="map_reduce", verbose=True)
chain.run(texts)

Summarizes long web content by splitting it and using a map-reduce chain.

Models & APIs used

  • Models: gemini-3.5-flash, gemini-embedding-001
  • APIs / services: Vertex AI
  • SDKs / libraries: langchain, langchain-core, langchain-text-splitters, langchain-google-genai, langchain-community, faiss-cpu, langchain-chroma, pypdf, langchain-classic, beautifulsoup4

When to use this

Use this pattern to prototype LangChain applications on Vertex AI that need prompting, embeddings, retrieval, memory, or chained LLM workflows.

Gotchas & caveats

  • Vertex AI is billable for this tutorial.
  • Colab users must authenticate with google.colab.auth.authenticate_user().
  • Local development requires Google Cloud SDK and application default credentials via gcloud auth application-default login.
  • PROJECT_ID must be set directly or read from GOOGLE_CLOUD_PROJECT.
  • LOCATION defaults to global from GOOGLE_CLOUD_REGION when unset.
  • Chroma document indexing may take a while because the API is rate limited.
  • The PDF QA example truncates split documents to docs[:250].

Best practices

  • Use PromptTemplate for prompts with runtime input variables.
  • Use StructuredOutputParser format instructions when structured model output is needed.
  • Split long documents with RecursiveCharacterTextSplitter before embedding or summarization.
  • Use retrievers over vector stores to combine documents with language models.
  • Use ConversationBufferMemory when a conversation chain needs prior message context.
  • Use SimpleSequentialChain to break tasks into focused deterministic LLM steps.
  • Return source documents from RetrievalQA when answering questions over retrieved content.

Graph View

Backlinks