Generative AI Google Brain

embeddings-hybrid-search

3 min read

Combining Semantic & Keyword Search: A Hybrid Search Tutorial with Agent Platform Vector Search

Source notebook

Repo path: embeddings/hybrid-search.ipynb · Open on GitHub · intermediate

Builds sparse and hybrid Vector Search indexes for Google Merch Shop product search.

Summary

This notebook teaches how hybrid search combines semantic search with token-based search to improve retrieval quality. It loads Google Merch Shop product titles, builds TF-IDF sparse embeddings, uploads index data to Cloud Storage, and creates a Vector Search index. It then adds dense embeddings from gemini-embedding-001 and queries a hybrid index using HybridQuery with RRF ranking.

Key code patterns

TF-IDF sparse embedding wrapper

vectorizer = TfidfVectorizer()
vectorizer.fit_transform(df.title.tolist())
 
def get_sparse_embedding(text):
    tfidf_vector = vectorizer.transform([text])
    values, dims = [], []
    for i, tfidf_value in enumerate(tfidf_vector.data):
        values.append(float(tfidf_value))
        dims.append(int(tfidf_vector.indices[i]))
    return {"values": values, "dimensions": dims}

Converts word-level TF-IDF output into the sparse_embedding values and dimensions format required by Vector Search.

Sparse index data file

items = []
for i in range(len(df)):
    title = df.title[i]
    sparse_embedding = get_sparse_embedding(title)
    items.append({"id": i, "title": title, "sparse_embedding": sparse_embedding})
 
with open("items.json", "w") as f:
    for item in items:
        f.write(f"{item}\n")
! gsutil cp items.json $BUCKET_URI

Builds per-item sparse embeddings and stages the index input file in Cloud Storage.

Create and deploy Vector Search index

aiplatform.init(project=PROJECT_ID, location=LOCATION)
my_sparse_index = aiplatform.MatchingEngineIndex.create_tree_ah_index(
    display_name=f"vs-tokensearch-index-{UID}",
    contents_delta_uri=BUCKET_URI,
    dimensions=768,
    approximate_neighbors_count=10,
)
my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint.create(
    display_name=f"vs-hybridsearch-index-endpoint-{UID}",
    public_endpoint_enabled=True,
)

Creates a Tree-AH Vector Search index from Cloud Storage and exposes it through a public index endpoint.

Dense Gemini embedding wrapper

embed_client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)
 
def get_dense_embedding(text):
    response = embed_client.models.embed_content(
        model="gemini-embedding-001",
        contents=text,
        config=EmbedContentConfig(output_dimensionality=768),
    )
    return response.embeddings[0].values

Generates 768-dimensional dense text embeddings for semantic search.

Hybrid query with RRF alpha

query_dense_emb = get_dense_embedding(query_text)
query_sparse_emb = get_sparse_embedding(query_text)
query = HybridQuery(
    dense_embedding=query_dense_emb,
    sparse_embedding_dimensions=query_sparse_emb["dimensions"],
    sparse_embedding_values=query_sparse_emb["values"],
    rrf_ranking_alpha=0.5,
)
response = my_index_endpoint.find_neighbors(
    deployed_index_id=DEPLOYED_HYBRID_INDEX_ID,
    queries=[query],
    num_neighbors=10,
)

Combines dense and sparse retrieval signals in one query and balances them with Reciprocal Rank Fusion.

Models & APIs used

  • Models: gemini-embedding-001
  • APIs / services: Agent Platform API, Agent Platform Vector Search, Cloud Storage
  • SDKs / libraries: google-cloud-aiplatform, google-cloud-storage, google-genai, pandas, scikit-learn

When to use this

Use this pattern when product, document, or catalog search needs both semantic matching and exact keyword matching for out-of-domain terms.

Gotchas & caveats

  • Vector Search deployments can take up to 30 minutes, and the tutorial estimates 1 to 1.5 hours because deployment waits occur twice.
  • An existing Google Cloud project and enabled Agent Platform API are required.
  • The Colab authentication cell applies only when running in Google Colab.
  • Colab Enterprise deployment may output a timeout error that the notebook says can be ignored.
  • Semantic search alone can miss out-of-domain SKUs, new product names, and proprietary codenames.
  • Indexes, Index Endpoints, and Cloud Storage buckets should be deleted after the tutorial to avoid unexpected costs.
  • rrf_ranking_alpha of 1 or omitted uses dense results only, 0 uses sparse results only, and values from 0 to 1 merge both.

Best practices

  • Store both embedding and sparse_embedding on each item when building a hybrid index.
  • Fit the sparse vectorizer on the corpus before generating item and query sparse embeddings.
  • Use rrf_ranking_alpha to control the weight between dense and sparse search results.
  • Consider subword tokenizers, BM25, or SPLADE instead of basic word-level TF-IDF for production requirements.
  • Use fast Vector Search retrieval followed by reranking for higher-quality production retrieval or recommender systems.
  • Clean up Indexes, Index Endpoints, and Cloud Storage buckets after finishing the tutorial.

Graph View

Backlinks