Data Engineering

18Jan

The Internet Of Things (IOT)

The Internet Of Things has started vibrating now with a strong and bold promises to make a revolutionary, fully connected "Smart" world where the relationship between objects, their environment and people become more tightly twisted together. As a basic definition of IOT, we can refer to an environment where network connectivity and computing capability extends to objects , sensors and daily used articles normally bypassing computers, allowing those devices to generate, exchange and consume data with or without minimal...

By Data EngineeringComments Off
Read more...
18Jan

Why data becoming new fuel/oil and primary driver for all types of business strategy

From the practical reality, we can understand how companies are using data analysis to boost their revenue growth. Now a day's most of us are subscriber for Dish TV from well known DTH Service providers and we adopt our desired packages specific to our types of interest. However in the chosen package, all the channels of specific choice will not be provided. Instead , observing the pattern of the viewers frequency of watching channels, service providers gather the data...

By Data EngineeringComments Off
Read more...
17Jan

Analysis of CCTV footage

All of us know CCTV is commonly used for a variety of purposes like maintaining perimeter security in medium- to high-secure areas and defense installations, observing behavior of incarcerated inmates and potentially dangerous patients in medical facilities, traffic monitoring etc. The amount of digital data generated by a single camera after compression is about 20 gigabytes per day. The number of camera increases with the increase of premises and its surroundings. If we are planning to store those data...

By Data EngineeringComments Off
Read more...
17Jan

Ingesting Big Data into HDFS

we are always talking about Big data processing using Hadoop. And know the basic definition of Big Data which is huge volume of data those can not be stored in existing traditional database or data repository. Interestingly, how can we import such a huge volume of data to the cluster of computers where Hadoop is installed? Yes, using Flume we can continuously collect the stream of data. For example Twitter data can be collected for analysis of comments. Sqoop...

By Data Engineering, Data IngestionComments Off
Read more...
17Jan

Big Data generation statistics

Facebook is collecting our data more than 500 terabytes (1000 X 500 Gigabytes) a day which includes 2.9 billion Likes per day. More than 350 million photos uploaded per day. Data per day processed by Google 24 petabytes (24 X 1,000,000,000,000,000 bytes). 400 hours of video are uploaded to YouTube every minute. The Apache Cassandra database is accurate adoption when we need scalability and high availability without compromising performance. Facebook uses it for its Inbox search. Initially Cassandra was developed by...

By Data EngineeringComments Off
Read more...
17Jan

Leverage Big data for fraud detection

  Now a day's Financial industry as well as Banking system are facing most costly challenges in terms of fraud and other financial crimes. Big Data processing framework like Hadoop and its eco systems are started helping to detect, prevent and eliminate internal and external fraud as well as reduce the associated costs. Analyzing points of sales (POS) in shopping malls. hospitals, retail stores etc and other transactions places, Banking system are capable enough to identify and mitigate fraud immediately....

By Data Analysis, Data EngineeringComments Off
Read more...
17Jan

Japan started using Big data analysis

According to Yano research institute LTD, Tokyo Japan, Big data market could reach one trillion yen ($10 billion USD) by 2020 in Japan itself. By utilizing Hadoop on cloud computing, it is possible today to process huge volume of real-time earthquake warning system's data. Seismic data processing is slowly throwing light on accurate earthquake prediction. Besides, in Japan it is helping smart cities to minimize waste and improve efficiency. Oil and Natural gas exploration and production companies now a...

By Data EngineeringComments Off
Read more...
17Jan

Big Data in Hospital Network

Due to unprecedented scale of digitization, Big data is playing a major role in healthcare sector. Now a days in hospital network, data is continuously getting recorded from all the medical instruments in a pediatrics yard. The big data analytic team is now able to help the physicians to spot an infection trends 12 to 24 hours earlier than they may have spotted it. Hadoop framework is making it possible to process big data in terms of about 200...

By Data EngineeringComments Off
Read more...
17Jan

Big Data analysis not limited to IT graduates

Hundreds of Terabytes, Petabytes, Exabytes of data is possessed in fields viz.social media, medical science, space science, finance, Defense, Oil and Natural Gas industries etc. There is an urgent need of the hour to collect and preserve whatever data is being generated, for the betterment of analysis, research and possible new inventions. The main common characteristic of that massive amount of data is that they are unstructured. Hadoop is a cost and time effective option that helps in gaining...

By Data EngineeringComments Off
Read more...
17Jan

Future of Big Data analysis using Hadoop in Medical Science and Health Care Organizations.

Even though medical sciences are capable of diagnose the diseases like Cancer, Alzheimer’s etc, these diseases remain still incurable. Because to find the root cause of these diseases, the medical researchers need to analyze patient's medical records, various supportive information, climatic conditions in which they lived in, across different geographical locations. And these a need a platform where a huge volume of data can be stored and analyzed. Hadoop is a powerful platform that allows us to store huge...

By Data EngineeringComments Off
Read more...