Data Engineering • IoT

MQTT Data Generator

A simple self-contained MQTT solution designed to simulate randomized sensor data and publish it to an MQTT broker.

Technologies: Python, Docker Compose, Mosquitto MQTT

View Project →

Data Engineering • Data Streaming • IoT

Structured Streaming Sensors

This demo simulates GPS tracking in a real-time streaming data pipeline. The streaming data pipeline collects location updates in an MQTT broker. Next, messages are published to Kafka through an MQTT Source Connector. Finally, Spark reads the streaming data from Kafka, processes it, and writes it to Elasticsearch where it is visualized on a Kibana map in real-time.

Technologies: Python, MQTT, Kafka, Spark, Elasticsearch, Kibana

View Project →