The Challenge
A cloud-based subscription billing provider needed a way to search transaction logs through its customer-facing portal. The organization wanted to replace a paid search service while continuing to make operational information available for investigation and review.
The transaction log search environment needed to accommodate tens of billions of documents across dozens of large nodes, with an ingestion volume on the order of one million documents per day. Its scale made indexing, infrastructure sizing, and query performance important architectural considerations.
Users needed to find activity using identifiers such as names, IDs, and the last four digits of credit card numbers, as well as other transaction details. The logging and search design had to account for the appropriate handling of sensitive information.
Innovent’s Approach
Innovent designed and built the search architecture for large-scale transaction log retrieval and integration with the client-facing portal.
The architecture addressed a substantial existing collection alongside ongoing ingestion. It accounted for the relationship between document volume, distributed infrastructure, indexing, and the retrieval needs of portal users.
The team also considered the identifiers and transaction details users needed to search, connecting the data and infrastructure design with the requirements of the customer-facing application.
The Result
The engagement delivered a search architecture for a transaction log collection measured in tens of billions of documents and for integration with the provider’s customer-facing portal.
Innovent built the architecture to support the provider’s move away from its existing paid search service, addressing large-scale data storage, ongoing ingestion, and transaction retrieval within one solution.