Taming IoT Data: Making Sense of Sensors with SQL Streaming

by | Sep 25, 2019 | Voxxed Days Cluj-Napoca 2019

Hans-Peter Grahsl is working as a technical trainer for Java Web Development and is responsible for the internal education department at NETCONOMY Software & Consulting GmbH in Graz, Austria. As an independent consultant he is supporting customers during the conception and implementation of on-premise or cloud-based data architectures using modern technology stacks and NoSQL data stores. Hans-Peter is also an associate lecturer for Software Engineering at CAMPUS 02 University of Applied Sciences.

We are living in a data streaming era, yet until recently it has been particularly hard to leverage existing stream processing technologies. On the one hand, because dealing with data in motion has its inherent challenges. On the other hand, most frameworks and APIs which are allowing for stream processing are typically very hard to employ and/or operate – NOT so for KSQL, the newest kid in Apache Kafka’s ecosystem.

Based on a simplified version of an IoT use case this session gives a gentle introduction into KSQL – Kafka’s SQL streaming engine for the masses. Join this fast-paced tour during which we are discussing a streaming IoT architecture. Concretely, we are going to: (1) ingest smart home energy data into Apache Kafka, (2) use KSQL for flexible, powerful and scalable SQL-only stream processing, (3) send raw data as well as pre-processed results to an operational NoSQL data store, (4) reactively serve data to clients in near real-time and (5) finally build informative live charts.