NoSQL database provider Couchbase announced new product integrations with the Apache Hadoop ecosystem. Couchbase has integrated Hortonworks Data Platform (HDP) 2.2 into its Couchbase Server. Apache Kafka brings in distributed messaging, and Apache Storm integration brings in stream processing in the setup.
The goal of the integration, driven by Couchbase, is to provide operational and analytical requirements with a single solution.The Kafka connector strengthens the streaming data between Couchbase and Hadoop. Couchbase is the operational store and Hadoop is the deep analytics.
The big data market can be segmented into two halves: operational data management, where Couchbase and NoSQL vendors play, and analytical data processing, which has been led by Hadoop vendors.
Anil Madan, senior director of engineering at Paypal, a Couchbase customer, spoke about the power of the integration in a release. “As the operational data store, Couchbase is easily capable of processing tens of millions of updates a day, or more,” he said. “Streaming through Kafka into Hadoop, these key events are turned into business insight and that’s why we are very excited about this partnership.”
Kafka uses Couchbase Server’s Database Change Protocol (DCP), new to 3.0, to stream data from Couchbase Server to the message queue in real time. Apache Storm then consumes the messages for real-time analysis. The data is then written to Hadoop for further processing, with analysis written to Couchbase Server for access by real-time reporting and visualization dashboards.
“As more data streams flow into enterprises from a larger variety of sources, IT leaders must extend data processing to include more use of push (event-driven) models, “said Nick Heudecker, Gartner, in a release. “The opportunities and the risks of not harnessing high-velocity data streams are too significant to ignore. This shift in data models exposes new business and operational opportunities in information security, the Internet of Things customer engagement, fraud detection and telematics, among others.”
For offline analytics, Hortonworks certified the Couchbase Server plugin for Sqoop to support bi-directional data transfer bettwen Couchbase Server 3.0.2 and HDP 2.2. This lets enterprises export operational big data stored in Couchbase to HDP for offline analysis and refinement, then import back into Couchbase Server.
“Today’s enterprises need to build web, mobile and IoT applications that deliver contextual insight in real time,” said Rod Hamlin, vice president of business development at Couchbase, in a release. “The work we have done makes it faster and easier for IT organizations to deploy real-time big data architectures that enable them to analyze data continuously and support real-time decision making and personalization.”
Couchbase is the result of a 2010 merger between CouchOne and Membase. In 2012, it shifted focus away from Apache CouchDB into its own NoSQL database technology, which is also open source. Another CouchDB player is Cloudant, which was acquired by IBM. Cloudant had its own version of CouchDB called BigCouch and still contributes to Apache CouchDB.
Couchbase raised $60 million last June, noting sales were up 400 percent. In August, it partnered with CumuLogic, ElastiBox and Cloudsoft to help businesses deploy Couchbase Server to a cloud of choice.
