Generative AI Google Brain

gemini-use-cases-retrieval-augmented-generation-intra-knowledge-qna

3 min read

Intra Knowledge QnA

Source notebook

Repo path: gemini/use-cases/retrieval-augmented-generation/intra_knowledge_qna.ipynb · Open on GitHub · intermediate

Builds a Vertex AI and LangChain RAG Q&A app over an IRS PDF using Chroma embeddings.

Summary

This notebook demonstrates retrieval-augmented question answering over intra-document knowledge. It downloads an IRS PDF, loads and chunks the document, embeds the chunks with Vertex AI text embeddings, persists them in Chroma, and uses Gemini through LangChain retrieval chains to answer questions with source references. It also adds an ipywidgets UI for asking concise or more detailed questions.

Key code patterns

Initialize Vertex AI

PROJECT_ID = "your-project-id"
LOCATION = "us-central1"
 
import vertexai
vertexai.init(project=PROJECT_ID, location=LOCATION)

Sets the Google Cloud project and region before using Vertex AI models.

Chunk PDFs and text files

def get_split_documents(index_path):
    split_docs = []
    for file_name in os.listdir(index_path):
        loader = UnstructuredPDFLoader(index_path + file_name) if file_name.endswith(".pdf") else TextLoader(index_path + file_name)
        splitter = CharacterTextSplitter(chunk_size=8192, chunk_overlap=128)
        split_docs.extend(splitter.split_documents(loader.load()))
    return split_docs

Loads local documents and creates overlapping chunks for retrieval.

Persist Chroma vector store

embeddings = VertexAIEmbeddings(
    model_name=EMBEDDING_MODEL,
    batch_size=5,
)
split_docs = get_split_documents(INDEX_PATH)
db = Chroma.from_documents(split_docs, embeddings, persist_directory=PERSIST_PATH)
db.persist()

Generates Vertex AI embeddings and stores them in a reusable Chroma database.

Create retrieval chain

llm = VertexAI(model=MULTIMODAL_MODEL, max_output_tokens=2048, temperature=0.2, top_p=0.8, top_k=40)
retriever = db.as_retriever(search_type="similarity", search_kwargs={"k": 3})
prompt = PromptTemplate.from_template(template)
combine_docs_chain = create_stuff_documents_chain(llm, prompt)
retrieval_chain = create_retrieval_chain(retriever, combine_docs_chain)

Connects Gemini, retrieved context, and a prompt that restricts answers to provided context.

Models & APIs used

  • Models: gemini-2.0-flash, text-embedding-005
  • APIs / services: Vertex AI
  • SDKs / libraries: google-cloud-aiplatform, vertexai, langchain, langchain_google_vertexai, chromadb, unstructured, ipywidgets

When to use this

Use this pattern to answer questions over a folder of PDFs or text files with persisted local vector search and Gemini-generated responses.

Gotchas & caveats

  • Vertex AI API must be enabled for the selected Google Cloud project.
  • Colab authentication and runtime restart are required after package installation in Colab.
  • The notebook pins specific package versions including google-cloud-aiplatform1.46.0 and langchain0.1.14.
  • PDF parsing depends on system packages poppler-utils and tesseract-ocr.
  • The prompt instructs the model to ask for rephrasing when the answer is not found in retrieved context.

Best practices

  • Initialize Vertex AI with an explicit project and location.
  • Use chunk overlap to preserve context across document splits.
  • Persist the vector database to disk for reuse.
  • Limit retrieval to the top 3 similar documents for answer generation.
  • Print unique source document paths from retrieved context.
  • Clean up downloaded files and the persisted database after the notebook run.

Graph View

Backlinks