Stop Merging Search Results: A Better Architecture for Hybrid Search

Excellent search results require a combination of accuracy and intuition. Traditional (lexical) search engines such as Elasticsearch, OpenSearch, or Apache Solr are exceptional at highly-accurate search results served very quickly at almost any scale. Vector embedding search technologies offer great breadth and depth of intuitive results. Combining both is the ideal solution, and this has lately been approached by what we commonly refer to as Hybrid Search.

The typical approach to hybrid search architecture is to run lexical search and vector search in separate, parallel processes. The results are combined and some form of reranking occurs to attempt to blend the results into a relevant, intuitive, and hopefully accurate response.

In practice, though, the standard architectural pattern for hybrid search introduces an unexpected amount of friction. It gets you up and running quickly, but it breaks down the moment you hit real-world production requirements. Facets stop making sense, business rules lose their precision, deep pagination gets expensive, and your app server ends up doing heavy search engine work it was never built for.

We've adopted a different approach that combines the best of both approaches.

Use vector search to find candidates instead of ranking results

Instead of treating semantic search as a parallel search engine we treat it as a candidate generator. We fetch the most relevant document IDs from the vector embeddings along with their resulting KNN/ANN weights, inject those scores directly into your search engine’s query, and let Elasticsearch, OpenSearch, or Solr handle the entire ranking pass.

What Breaks with Merging?

The standard pattern usually looks like this:

The app issues two queries, gets two lists back, and stitches them together. This is elegant and on the surface it looks like it should work. But the problems encountered require a bit more attention as they manifest quickly once business requirements are realigned to the solution in production:

Relevancy calculations are wildly different

Vector search provides weights for how adjacent an embedding is to the requested text, typically with 1 being a perfect match and 0 being unrelated (1 to -1 being the most common scoring, but typicaly normalized for reliable calculation). Lexical search often uses BM25 similarity ranking which may have a very wide degree of score values with no explicit max. Combining these scores has many approaches, reranking being the most common technique, but they are prone to odd distributions and unpredictable blends. The calibration process is highly implementation-specific and typically depends on the search platform, embedding model, corpus characteristics, and desired business behavior.

Facets and aggregations lose context

Lexical engines calculate document attributes such as category counts and price facets across the entire matched document set. Once you split retrieval into two buckets and merge them, the search engine no longer knows what the user is actually looking at. Your facet counts either drift out of sync or require secondary aggregation queries to fix.

Business metrics get flattened

Search solutions get a great deal of relevancy heavy lifting from using data metrics. They don't just just match keywords they tune results using business data like inventory levels, profit margins, clickthrough rates, and location. Algorithms like RRF discard these raw scores and re-rank documents based purely on their relative position in a list. The important variations of document results are washed out in the reranking, and/or the reranking algorithms attempts to blend these metrics once again encounters the disparate mathematics.

Pagination and filters gets expensive fast

Paging down to result 50 isn't a simple offset request anymore. To render page five reliably, your app has to over-fetch hundreds of candidate items from both systems, ship them across the network, and sort them in memory just to display 10 items. Vector search also runs into scaling and performance issues when many filters are applied, as it often relies on overfetching results in order to satisfy the requested documents of the user's search.

Note
We do not feel that ranking approaches are inherently broken. They are powerful, suitable tools for the right circumstances. We feel they may be unsuitable when the application needs explicit, mathematically controllable interaction among lexical relevance, semantic similarity, and business signals.

A Better Hybrid Search Architecture: Semantic Candidate Injection

Instead of running two competing search engines, use vector search purely for candidate generation.

The vector system answers one simple question:

Which handful of documents match the conceptual intent of this query?

Your primary search engine handles the rest:

How should those documents be filtered, combined with exact matches, boosted by business metrics, and aggregated for the UI?

How It Works in 3 Steps

1. Extract candidates and scores

We turn the user’s query into an embedding and run a quick Approximate Nearest Neighbor (ANN) lookup. We don't fetch the full documents—just pull a lightweight array of document IDs and embedding weights:

[
  { "id": "doc_101", "score": 0.89 },
  { "id": "doc_202", "score": 0.74 },
  { "id": "doc_303", "score": 0.61 }
]

2. Calibrate the scores

Vector similarity scores usually sit between 0.0 and 1.0, while BM25 scores vary wildly depending on field lengths and term frequencies.

Before building the final search query, scale the vector scores so they align with your lexical scoring scale. Conceptually our scoring model looks like this:

This lets our search relevance engineers explicitly tune α, β, and γ instead of letting an opaque rank fusion algorithm decide for them.

We prefer using product for the calculations since we can predictably scale and influence each core operand to suit our business needs.

3. Run a single query pass

Inject those candidate IDs into your search query as a set of boosted conditional clauses. The search engine evaluates lexical text matches, applies your semantic ID boosts, filters out of stock items, calculates category facets, and sorts the final response in a single atomic pass.

Platform Queries

These are some sample patterns that work across all major search platforms without requiring special plugins. There are multiple ways to achieve this, modify as needed.

Note
The snippets below illustrate the structural mechanics of semantic candidate injection using standard platform parameters. Exact field names, scaling multipliers, and score modifiers (such as log1p vs. linear functions) should be adjusted to align with your specific index schema and scoring model.

Elasticsearch and OpenSearch

Use a bool query to combine standard lexical search with a set of boosted term matches for the semantic candidates, wrapped in a function_score to account for business data.

{
  "query": {
    "function_score": {
      "query": {
        "bool": {
          "should": [
            {
              "multi_match": {
                "query": "trail running shoes",
                "fields": ["title^2.0", "description"]
              }
            },
            {
              "bool": {
                "should": [
                  { "term": { "_id": { "value": "doc_101", "boost": 8.9 } } },
                  { "term": { "_id": { "value": "doc_202", "boost": 7.4 } } },
                  { "term": { "_id": { "value": "doc_303", "boost": 6.1 } } }
                ]
              }
            }
          ],
          "minimum_should_match": 1,
          "filter": [
            { "term": { "in_stock": true } }
          ]
        }
      },
      "field_value_factor": {
        "field": "margin",
        "factor": 1.2,
        "modifier": "log1p",
        "missing": 1.0
      },
      "boost_mode": "multiply"
    }
  },
  "aggs": {
    "categories": {
      "terms": {
        "field": "category.keyword"
      }
    }
  }
}

Apache Solr

Solr’s eDisMax query parser makes this extraordinarily clean. Candidate IDs are injected by a careful construction of the query parameter and business metrics via multiplicative boost (Boost Functions). We use parameter dereferencing to make it more redable.

q={!lucene v=$docids} {!edismax q.op='OR' v='trail%20running%20shoes'}"
&docids=id:"doc_101"^8.9 id:"doc_202"^7.4 id:"doc_303"^6.1
&defType=edismax
&qf=title^2.0 description^1.0
&fq=in_stock:true
&boost=product(log(sum(margin,1)),1.2)
&facet=true
&facet.field=category
&rows=20

Architectural Comparison

When to Use This Pattern (and How to Deploy It Fast)

Elasticsearch, OpenSearch and Apache Solr continue to build out native vector search features. However, injecting candidate IDs directly into standard queries remains the far better choice if:

1. You use an external vector database (like Qdrant, Milvus, or FAISS) for fast, specialized vector search.
2. You generate embeddings outside the search cluster and want to keep vector infrastructure decoupled from your search engine.
3. You need strict, deterministic control over how business metrics (margin, stock, recency) blend with semantic scores.
4. You want accurate facets and fast pagination without paying for excessive cluster hardware or writing middleware rank mergers.

The Fast Path: Where FindTuner Fits In

Designing the query building logic, managing score scaling, and tuning the weights between semantic intent and business rules takes real engineering cycles. If you’d rather not build and maintain that orchestration pipeline yourself, this is precisely what FindTuner was built to do.

FindTuner seamlessly integrates with your existing Elasticsearch, OpenSearch, or Solr cluster to handle candidate injection automatically. It translates semantic signals into native, highly optimized query structures while giving your relevance and merchandising teams a visual interface to finetune business boosts, rules, and metrics in real time.

Instead of spending weeks writing custom query builder glue code and rank fusion middleware, FindTuner lets you plug this architecture directly into your current search infrastructure giving you the full power of semantic candidate injection with zero custom boilerplate.

The Bottom Line

You don't need to merge two separate search responses and you don't need to rebuild your search engine from scratch to get semantic intelligence. By using vector search to hand your primary engine better candidates, you get the intent awareness of semantic search while keeping the speed, control, and native features of your core platform.

The architecture is straightforward in principle. Production implementation is considerably more challenging. Reliable score calibration, candidate management, query optimization, latency control, business rule interaction, and continuous tuning are where most implementations succeed or fail. FindTuner encapsulates these production concerns so organizations can adopt this architecture without building and maintaining it themselves.

Whether you build the query pipeline inhouse or accelerate it with a platform like FindTuner, candidate injection is simply a smarter way to do hybrid search.