The Challenge
A media and news organization relied on a shared search environment supporting approximately 50 properties and more than 250 million documents. High indexing throughput and heavy query traffic placed substantial demands on an already unstable system.
The existing architecture served multiple properties through a single sharded deployment hosted on premises. The broader delivery environment included Akamai and Varnish caching tiers, adding layers to the investigation of incidents affecting search availability.
The organization needed to stabilize operations while planning search modernization. That required addressing immediate failures alongside the architecture, capacity, and disaster recovery requirements necessary to support continued growth.
Innovent’s Approach
Innovent provided guidance to stabilize the existing environment and helped diagnose and triage several incidents in which external events brought search processing to a halt. The work required examining how the search platform interacted with the surrounding infrastructure and application environment.
Alongside production troubleshooting, Innovent assisted with a search platform upgrade and helped plan and carry out the transition from an on-premises deployment to a cloud-based, multi-tenant architecture.
As part of the ongoing work, Innovent diagnosed a request-routing bottleneck that concentrated query coordination on a small subset of search nodes, even though incoming traffic was distributed across the cluster. The team developed an approach that spread coordination across the existing nodes while preserving tenant isolation and the application’s existing search endpoint. This work is detailed in Solr Alias Request Routing: Diagnosing a Query Coordination Hotspot.
The solution also needed to accommodate changes as new tenant collections were created. Innovent addressed how to maintain the routing arrangement as the environment evolved, described in Maintaining a Gateway Anchor with Category Routed Aliases.
The team provided search architecture guidance that accounted for the organization’s substantial indexing and query workloads, the needs of its multiple properties, and its future capacity requirements.
Innovent also addressed disaster recovery and operational reliability as part of the architecture work. This connected the cloud migration with the requirements for maintaining a large, shared search service.
The Result
The engagement helped stabilize a demanding search environment and supported its transition to a cloud-based architecture serving approximately 50 properties and more than 250 million documents.
Targeted request-routing changes distributed query coordination across the cluster while preserving tenant isolation and avoiding changes to the upstream application. The organization also gained guidance for capacity planning, disaster recovery, and managing the architecture as its needs evolved.
The work demonstrated Innovent’s ability to combine production troubleshooting with search modernization, addressing urgent reliability concerns while helping evolve the architecture behind a large-scale, multi-property service.