Generative AI Google Brain

gemini-prompts-examples-question-answering

3 min read

Question Answering with Generative Models on Vertex AI

Source notebook

Repo path: gemini/prompts/examples/question_answering.ipynb · Open on GitHub · intro

Shows prompt patterns for Gemini question answering on Vertex AI, including simple fuzzy evaluation.

Summary

This notebook teaches open-domain and closed-domain question answering prompts with Gemini on Vertex AI. It demonstrates zero-shot prompting, few-shot prompting, adding private context, constraining answers when context is insufficient, extractive Q&A, and evaluating generated answers with fuzzy string matching.

Key code patterns

Initialize Vertex AI

PROJECT_ID = "your-project-id"
LOCATION = "us-central1"
 
import vertexai
vertexai.init(project=PROJECT_ID, location=LOCATION)

Sets the Google Cloud project and region before calling Gemini through Vertex AI.

Load Gemini model

from vertexai.generative_models import GenerationConfig, GenerativeModel
 
generation_model = GenerativeModel("gemini-2.0-flash")

Creates the Gemini model client used throughout the notebook.

Controlled zero-shot Q&A

prompt = """Q: What is the tallest mountain in the world?\n
A:
"""
generation_config = GenerationConfig(temperature=0.1, max_output_tokens=20)
response = generation_model.generate_content(
    contents=prompt,
    generation_config=generation_config,
).text

Uses low temperature and token limits to produce short factual answers.

Closed-domain context prompt

prompt = f"""Answer the question given in the context below:
Context: {context}\n
Question: {question}\n
Answer:
"""
response = generation_model.generate_content(contents=prompt).text

Passes private or internal knowledge in the prompt so the model answers from supplied context.

Fallback instruction

prompt = f"""Answer the question given the context below as {{Context:}}.
If the answer is not available in the {{Context:}} and you are not confident,
please say "Information not available in provided context".
 
Context: {context}
Question: {question}
Answer:
"""

Reduces unsupported answers by instructing the model what to return when context lacks the answer.

Fuzzy answer scoring

def get_fuzzy_match(df):
    return fuzz.partial_ratio(df["answer_groundtruth"], df["answer_prediction"])
 
qa_data_df["match_score"] = qa_data_df.apply(get_fuzzy_match, axis=1)
qa_data_df["match_score"].mean()

Evaluates short generated answers when exact string matching is too strict.

Models & APIs used

  • Models: gemini-2.0-flash
  • APIs / services: Vertex AI
  • SDKs / libraries: vertexai, google-cloud-aiplatform, pandas, fuzzywuzzy, python-Levenshtein

When to use this

Use this pattern when building prompt-only Gemini Q&A prototypes over public questions or small supplied context passages.

Gotchas & caveats

  • A Google Cloud project is required and the Vertex AI API must be enabled.
  • Colab requires explicit authentication with google.colab.auth.authenticate_user().
  • The notebook uses LOCATION = “us-central1” for Vertex AI initialization.
  • Models have knowledge cutoff dates, so recent open-domain questions can produce incomplete or incorrect answers.
  • Closed-domain answers depend on including the relevant private context in the prompt.
  • Exact string metrics can mark correct answers wrong when wording differs, so fuzzy matching is used.

Best practices

  • Use specific, context-rich, grammatically correct prompts for question answering.
  • Experiment with generation parameters such as temperature and max_output_tokens.
  • Use few-shot examples when the desired answer style is specific or short.
  • For closed-domain Q&A, provide the knowledge base as prompt context.
  • Tell the model to return “Information not available in provided context” when the context lacks the answer.
  • Use fuzzy matching for evaluating generated answers that may differ slightly from ground truth wording.

Graph View

Backlinks