Tag - Apache Kafka

11Jan

Data Absorption (Ingestion)

By , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Comments Off
Read more...
08Jun

Why Lambda Architecture in Big Data Processing

Due to the exponential growth of digitization, the entire globe is creating minimum 2.5 Quintilian 2500000000000 Million) bytes of data every day and that we can denote as Big Data. Data generation is happening from everywhere starting from social media sites, various sensors, satellite, purchase transaction, Mobile, GPS signals and much more. With the advancement of technology, there is no sign of slowing down of data generation, instead it will grow in massive volume. All the major organizations, retailers,...

By Architecture, Data Engineering, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Comments Off
Read more...
29May

Apache Kafka, The next Generation Distributed Messaging System

In Big Data project, the main challenge is to collect an enormous volume of data. We need distributed high throughput messaging systems to overcome it. Apache Kafka is designed to address the challenge. It was originally developed at LinkedIn Corporation and later on became a part of Apache project. A Messaging System is typically responsible for transferring data from one application to another. A message is nothing but the bunch of data/information. To ingest huge volume of data into Hadoop...

By Data Ingestion, , , , , , , , , , , , , , , , , Comments Off
Read more...