Data Engineering • IoT
MQTT Data Generator
A simple self-contained MQTT solution designed to simulate randomized sensor data and publish it to an MQTT broker.
Technologies: Python, Docker Compose, Mosquitto MQTT
Data Engineering • Data Streaming • IoT
Structured Streaming Sensors
This demo simulates GPS tracking in a real-time streaming data pipeline. The streaming data pipeline collects location updates in an MQTT broker. Next, messages are published to Kafka through an MQTT Source Connector. Finally, Spark reads the streaming data from Kafka, processes it, and writes it to Elasticsearch where it is visualized on a Kibana map in real-time.
Technologies: Python, MQTT, Kafka, Spark, Elasticsearch, Kibana