Hortonworks, Inc. (NASDAQ: HDP) today announced that American Water, the United States’ largest publicly traded water utility company, is leveraging global data management platforms from Hortonworks to modernize its data architecture. With Hortonworks Data Platform (HDP) and Hortonworks DataFlow (HDF), American Water is enhancing its ability to deliver critical data and insights to its field workers – in a matter of minutes.

HDP is an enterprise data management platform that helps organizations securely store, process and analyze any data asset. HDF is a scalable, real-time streaming data ingestion and analytics platform that collects, curates, and analyzes data so customers gain key insights for immediate actionable intelligence.

The connected data strategy that American Water has adopted by combining these two platforms has modernized its operations to build new applications powered by data lakes instead of legacy data warehouses.

The use of a data lake as the single source of truth to aggregate historical, traffic, real-time streaming and other types of data is unprecedented in the water utility industry. A data lake can combine historical insights with real-time insights from its consumer IoT devices to enable key decision-makers and field personnel with the right data at the right time. American Water was able to program its revolutionary data platform in a matter of days with a lean team and without going through rigorous new training – illustrating the ease of use and accelerated time-to-value that HDP and HDF bring.

Open Source Platforms Deliver Fast Results

American Water can now capture and process critical data previously only available in 24-hour increments in as little as 15 minutes. Open source data platforms from Hortonworks have powered this transformation and driven meaningful results to the business. To achieve even greater efficiencies, American Water transferred massive amounts of historical water plant data from its legacy systems, real-time Automated Metering Infrastructure (AMI) data from its IoT sensors and publicly available weather data into a data lake powered by HDP, using HDF as the backbone to collect and move data from the edge into the enterprise. The HDF platform incorporates real-time Supervisory Control and Data Acquisition (SCADA) sensor data to deliver IoT interactivity with American Water’s control modules.

“Our field employees are our front line with our customers and we knew they could serve customers more effectively if they had the data they need sooner,” said Radha Swaminathan, chief technology and innovation officer at American Water. “We are at the forefront of the water utilities industry, having modernized our data architecture with Hortonworks platforms to accelerate our operations and serve our customers more efficiently.”

HDF, and specifically Apache NiFi, were key to making American Water’s system work as it automates the movement of data between disparate data sources and systems. Together, they make data ingestion fast, easy and secure while allowing real-time data traceability.

“NiFi is a big tool for us because the extremely user-friendly interface allows us to ingest and move data from any legacy or streaming data source with little code to write,” said John Kuchmek, senior technologist at American Water. “The data provenance HDF provided – the traceability of the data – is exceptional. The data provenance within HDF can pinpoint the exact problems within any data source. We have better data quality in our data lake, thanks to HDF.

“HDP makes it so easy to visualize data in a readable table format that it gives our users instantaneous access to data originally hidden in ERP solutions or legacy data stores. Hive with LLAP enables us to execute some of the fastest queries due to its multi-threading and caching capabilities. We’ve completely changed the way to collect, store, search for, identify and use data to make actionable decisions. What previously took days to develop code, we can now do in minutes,” Kuchmek added.

“American Water is leading the water utilities industry by adopting HDP and HDF to modernize its data architecture and drive business transformation, including significant enhancements to its customer service and operations,” said Kenneth Smith, general manager of energy and utilities at Hortonworks. “Having an end-to-end open source data platform that can manage the entire lifecycle of data has been key to their success and we look forward to continuing to work with American Water to accelerate its business.”

American Water will share the success story of its production use case leveraging HDP and HDF at DataWorks Summit in San Jose, the industry’s premier Big Data community event. Kuchmek and Adam Michalsky, senior technologists at American Water, will speak in a session titled “Bridging the Gap: Achieving Fast Data Synchronization from SAP HANA by Leveraging Hadoop” on Wednesday, June 20, 2018 at 4 p.m. (PT). More information about the session can be found at https://dataworkssummit.com