GPU + In-Memory Data Management for Big Data Analytics

October 28th, 2014  |  Published in Research

Description

In this project, we develop a tool chain based on In-Memory Data Management and Parallel Data Processing in GPU using CUDA for large and intensive Smart Meter analytics. The global rollout of smart meters opens new business paradigm for utilities with data collection/transaction at such a high volume and velocity. For instance, a million meters collecting data at 15 minute intervals, if each meter reading is 1,000 bytes (1kB), then total transactional data collected from one million customer meters will reach about 30TB per year. Therefore, our aim is to utilize the processing power of GPU and the high-throughput low-latency feature of in-memory database to develop an adequate Big Data Analytics platform for an instant, in-depth analysis of massive volumes of smart meter data, towards advanced segmentation based on energy consumption patterns, and energy efficiency benchmarking. To achieve this goal, we research online learning methods to utilize new information to extend the existing knowledge bases.

The goal of our tool chain is to create a cognitive engine upon an application server with a web front-end interface.

 

GPU-in-Memory-Principle

Figure 1: Basic principle of the tool chain

 

The underlining GPU implementation focuses on a MapReduce schema with a vectorized interface. SAP HANA is our in-memory computing platform, which provides not only an in-memory data persistence, but also calculation logic directly in the database for extreme low-latency pre-processing.

 

GPU-in-Memory-Stack

Figure 2: Software stack

 

Although we begin this project with focus on the use case of smart meter data, the tool chain is also applicable for the Big Data analytics in other domains, such as Industry 4.0, Smart Cities, Personalized Medicine, etc.

Start

01/October/2014

Core partners

SAP

Topics

SAP HANA, GPU, In-Memory Data Management, Parallel Computing, Energy, Industry 4.0, Smart Cities, Personalized Medicine

 

Project team

 

External links

  • Similar GPU project as Big Data platform: mapD
Tags:

Comments are closed.