Case Study – Confluent Kafka to Apache Kafka Migration

Confluent Kafka to Apache Kafka Migration

For one of the major IOT/IIOT and M2M manufacturing companies, that develops and manufactures end-to-end Remote Monitoring/Energy Management solutions


This solution is for the existing platform of the customer which had confluent Kafka, needed to be migrated & integrated with Apache Kafka, in order to collect device data from controller, sensors, meters installed on telecom tower sites and eventually pushes to cloud / on premises infrastructure.


Installation and integration of multi-node Kafka cluster as a middleware data streaming platform to consume the real time device data from various telecom towers and dump to RDBMS, so that monitoring team can take the appropriate action by visualizing the web based dash board


  • Design and architecture of the proposed Kafka cluster with optimizations
  • Implementation and configuration of scalable Kafka Cluster
  • Installation & configuration of Kafka Connect to ingest device data from Kafka cluster to MySQL Database
  • Accumulating and ingesting real time monitoring data from IOT devices by leveraging Apache Kafka framework and subsequent processing and rendering into the dashboard.
  • Tuning & Scale-up
  • Testing & Support


  • Target Kafka Configuration
  • Cluster Sizing
  • Desired Throughput