Data Engineering🔨 Personal Project
Smart City Vehicle Streaming Pipeline
Real-time IoT data pipeline simulating vehicle trip data from Malang-Surabaya route with Apache Kafka streaming to AWS S3.
Project Overview
🎯
Objective
Build a real-time data pipeline simulating smart city vehicle telemetry with GPS, speed, weather, and traffic event data streaming
💼
My Role
Data Engineer
⏱️
Timeline
2 weeks
🛠️
Tech Stack
Python, Apache Kafka, AWS S3
📈
Key Results
- ✓Developed real-time vehicle movement simulation between Malang and Surabaya
- ✓Implemented multi-topic Kafka streaming (vehicle_data, gps_data, weather_data, traffic_events)
- ✓Integrated AWS S3 persistence using Kafka Connect S3 Sink Connector
- ✓Built scalable architecture for IoT use cases and fleet management analytics
Impact
Real-time vehicle tracking simulation
Value
Scalable IoT data streaming architecture
🔗Project Links
Tools & Technologies
PythonApache KafkaAWS S3DockerFakerConfluent Kafka