Although the open source version of Hindsight is relatively new, its predecessor was developed last decade at Tellme/Microsoft to handle all the IVR (Interactive Voice Response) call logs of some of the top fortune 500 companies (American Airlines, UPS, FedEX, American Express, Fidelity to name a few). The system went into production in 2010 and was expanded that year to also process the global Bing voice search logs. At that point in time the system was handling billions of log messages a day accounting for ~1.5TB of data (in a proprietary binary log format). A single Hindsight box could accommodate all of the data for real time processing; that included ~60K concurrent analysis sandboxes (three for each toll free number we were servicing) monitoring things like voice recognition accuracy, call volumes/duration/latencies, and SLAs in addition to the actual service components and hardware utilization. However, the actual production setup consisted of two boxes per data center running in parallel; this setup provided a hot fail over and a mechanism to easily roll-out/test new releases in production. The purpose of the sandbox design was to allow each customer to run their own business logic against data relevant to their application and provide a self-serve monitoring portal in addition to what we already deliver. The system remained in this configuration until the Solaris based infrastructure was decommissioned a year or two after I left the company.

Hello world!