IBM and Hortonworks have announced the Hortonworks Data Platform (HDP) on IBM Elastic Storage (ESS) and IBM Spectrum Scale. The announcement comes a week after a series of other updates to the IBM Spectrum and Cloud Object Storage product lines. It sees HDP being certified on IBM Spectrum Scale on both Power8 and x86. This builds on the HDP Power8 announcement from last year.
Ed Walsh, general manager for IBM Storage and Software Defined Infrastructure said: “Every organization is becoming a digital organization. With this announcement IBM is delivering a powerful platform to extend the use of data and for cognitive applications. This announcement shows our partner community IBM’s commitment to help client’s grow, develop, and transform the use of their own data with less complexity.”
Hortonworks gets its first Software Defined Storage certification
This is the first enterprise SDS certification for Hortonworks. It gives it greater access to IBM’s customer base and delivers new capability to HDP. IBM has been changing the way large scale analytics are carried out. Rather than move the data to the analytics tools it has been focused on moving the tools to the data. This significantly reduces the time, compute power and network bandwidth lost to ETL and data preparation. It also reduces the duplication of data held by the enterprise and therefore its storage demands.
The press release highlights three features that HDP will gain from with IBM ESS and Spectrum Scale. They are:
- Better storage efficiency – unlike HDFS which creates three copies of every piece of data, IBM ESS uses erasure coding, eliminating the need for multiple copies and increasing storage efficiency.
- Hybrid Storage – extends on premise storage to the cloud and therefore realizing a myriad of economic, security, accessibility and other benefits.
- High Performance – as an optimized, highly parallel storage solution ESS is capable of high-speed data throughput able to exceed the performance of distributed HDFS.
There are other benefits that were announced to analysts a few weeks ago. According to UK Analyst company Creative Intellect Consulting (CIC): “Hortonworks will be able to take advantage of IBM’s underlying updates to its storage portfolio. IBM Spectrum Accelerate is adding non-disruptive data-at-rest encryption with Self Encryption Drives (SED). This removes the need for software to worry about data security on the drive. It is also supporting on-premises and cloud-based deployment. For HDP, this will allow it to query data wherever IBM customers store it.”
This is a big win for Hortonworks. It gets certification on the IBM SDS platform and Power8. Its ability to replace HDFS with IBM ESS for storage efficiency will open up a wider customer base. Many customers struggle to effectively take advantage of complex analytics due to storage, network and memory constraints.
IBM also gains from this announcement. While it has worked hard to move its own tools to the data and reduce ETL resource issues, getting HDP on its storage platform is a big step. It will be interesting to see how close these two vendors continue to get.
This is not the only ETL announcement this week. Oracle announced the Oracle Data Integrator Cloud Service, which helps customers to rapidly consolidate multiple big data repositories.