People talk about the challenges of big data, but the real problem is not size, it’s the growing number of variables within that data that make it complex, Ayasdi CEO Gurjeet Singh said.
Ayasdi provides machine-intelligence software, a combination of machine learning and artificial intelligence that solves very complex problems for the largest institutions. The company recently raised a $55 million round led by Kleiner Perkins Caufield & Byers for more than $100 million total raised to date.
Just as data is becoming increasingly complex, yet essential for operations, the number of possible insights is also growing and rendering the traditional approach to analytics ineffective. So, it’s no surprise that the number of data scientist or data analyst positions is on the rise. However, Ayasdi doesn’t think it’s a problem you can hire away: the fundamental approach needs to change.
The story of Ayasdi starts with the Defense Advanced Research Projects Agency (DARPA) and the National Science Foundation (NSF) awarding $10 million in grants to Stanford professor Gunnar Carlsson to apply Topological Data Analysis (TDA) to real-world problems. Ph.D. mathematics student Gurjeet Singh joined his then professor Gunnar Carlsson on the TDA research project, which continued to grow and ultimately became Ayasdi (Ayasdi means "to seek" in Cherokee).
Its customer base has grown significantly over the years as many pilot deployments are maturing, said Singh. Ayasdi deals exclusively with Fortune 1000-type companies in healthcare, financial services and others. These are sizable, seven-figure deals with customers showing even more substantial savings. Some publicly disclosed customers include Lockheed Martin and Citi Group. Ayasdi plays in the big leagues of analytics and is doing well as illustrated by what the company claims was a ninefold increase in annual recurring revenue last year.
A typical analytics approach is very manual, time-consuming, and inefficient.
“Traditional analytics have hit the wall,” said Ayasdi Chief Marketing Officer Patrick Rogers. “It starts with an analyst asking questions, and then applying them against data that may or may not find insight. You must then go back and reformulate until you find something impactful. There are a lot of tools, but it’s still fundamentally a human-driven process. That model is not going to scale—the number of possible questions grows exponentially with data sets.”
Rather than the hypothesis/test approach, Ayasdi takes a very machine-driven one to address complex data. At the heart of Ayasdi’s machine intelligence is topological math, which is building a more automated discovery process and eliminating manual processes. Ayasdi identifies what you should be looking for through patterns in complex data.
The old methodology is essentially guesswork,” said Singh. “There are many tools to validate or invalidate hypotheses. For large businesses, relying on luck is not the best strategy. We use this concept of topological math. We exploit TDA, we look at the shape of the data and accelerate the discovery process. It’s why we’re able to do it so quickly and accurately.”
Plugged Into Heart of Data Center
Automating this requires a lot of compute, according to Lawrence Spracklen, the company's vice president of engineering. “We’ve spun up a force around high-performance computing,” he said. “We’ve focused on looking at algorithms, so we can scale linearly on a single node and across a cluster.”
The company has relationships with Intel and others that help teach it tricks of the trade. It is continuously looking at ways it can leverage things like NVIDIA GPUs, super dense accelerator cards, in a way that is space- and power-efficient, said Spracklen.
It can horizontally scale to datasets that take up billions of rows and do so in a way that minimizes bottlenecks. It doesn’t require a lot of memory, so the company can deploy on public clouds. However, given the very sensitive nature of the data it works on, most deployments are on premises.
The company does spin up dedicated clusters in its own private cloud from time to time used mostly by the healthcare industry because of HIPAA certification.
A lot of customers want Ayasdi to deploy on their Hadoop data lake, said Spracklen. It natively runs on YARN and has a library of connectors to things like Teradata machines. “We’re plugged into the heart of the data center,” he said.
Mercy, the fifth largest Catholic health care system in the U.S., used Ayasdi to build a decision system from their patient data. It projects a savings of well over $100 million over the next three years and recently selected Ayasdi for another difficult problem in claims denial.
Lockheed uses it to better manage program outcomes. Citi Group, which previously failed Federal Reserve stress tests, used Ayasdi to take down thousands of variables and distill them to a handful, passing just two weeks ago.
The majority of funding is going toward marketing and sales expansion. Going after multiple verticals, the company will focus on product enhancement and engineering.
In addition to KPCB, the latest round (Series C) was supplied by existing investors Institutional Venture Partners, Khosla Ventures, Floodgate, and Citi Ventures as well as new investors Centerview Capital Technology and Draper Nexus.