Monthly Archives - January 2017

18Jan

Technology Platform behind Aadhaar card implementation

We are almost familiar with Aadhaar card which had been rolled out as a first initiative in 2003. It's a 12-digit unique identification number issued by the Indian government to every individual resident of India and that can be used to access a variety of services and benefits. Hadoop is an open-source big data processing framework, that has been customized excessively by the company named MapR to boost performance.The aadhaar card project has been developed using MapR's customized Hadoop and...

By Apache Hadoop, Data Engineering, Hadoop Eco SystemComments Off
Read more...
18Jan

Challenges in data analysis due to Demonetization

We are all aware of demonetization and its ongoing impact in the entire country. Government is trying hard to digitalize entire financial system. In nutshell, initially to create lower cash transaction before reaching to complete cashless transaction. Use of different types digital wallet, mobile banking, debit and credit cards are started booming among the people of the country and citizens slowly started adopting it mainly to get rid of long queues in ATM, Banks and due to their advantages...

By Data EngineeringComments Off
Read more...
18Jan

Introduction to Facebook graph Application Program Interface( API).

API is a consolidation of protocols, set of routines and tools for developing software application. Ideally Facebook recognizes the relationship among the entities as a "Social Graph". In Facebook, an entity can be a person, place, event or object that is relevant to a given system and an attribute is a property , characteristic of the entity. As shown in the diagram, Location is an entity and country, state, location id are the attributes. Based on the registered user...

By Data EngineeringComments Off
Read more...
18Jan

DYNAMICS OF ONLINE SOCIAL NETWORKS

Mathematical models for the analysis of spread of social influence have emerged as a major topic of interest among the researchers in diverse disciplines such as sociology, economics and computer science. Empirical studies of diffusion on social networks date back to the 1940s. Later on, theoretical propagation models were introduced in late 1970s. In the recent years, Online Social Networks such as Facebook, Twitter and Linkedin has experienced explosive growth and these have remarkably changed the way people communicate....

By Data EngineeringComments Off
Read more...
18Jan

Data storage mechanism in Facebook

  We are all almost familiar with social media mainly Facebook where photo uploads total 300 million per day. Daily generates 4.5 billion likes. Every 60 seconds, 510 comments are posted, 293,000 statuses are updated. It's really curious to know how Facebook stores such a huge volume of data that's totally impossible using traditional database management systems (RDBMS) . Facebook is using distributed database management system called Cassandra. Cassandra was initially developed by Facebook to enhance their inbox search feature. In...

By Data EngineeringComments Off
Read more...
18Jan

Mainframe Applications slowly migrating to Hadoop

The giant organizations across the globe are using legacy mainframe systems due to it's scalability, security and reliability of machine's processing capacity subjected to heavy and large workloads. Of course, these infrastructures desire huge hardware, software and processing capacity. As the technology advancing very rapidly, scarcity of mainframe technicians, developers etc are increasing and it has become a major challenge for those organizations to continue their operations. The maintenance/replacement of these hardware are also another threat due to low...

By Apache Hadoop, Data Engineering, Hadoop Eco SystemComments Off
Read more...
18Jan

How Social Media Making Irritating Login Process Comfortable !

  Nowadays social identities are becoming an integral part of a web presence that deals with different activities. With the advancement of technology, most of the traditional transactional activities have been lifted to the web like shopping in e-commerce portal, Travel booking for Bus, Train, Flight, Hotel reservation in holiday destination etc. Merchandisers involve in above businesses, host their own web portal to facilitate the services. When a visitor browses to a web portal, it is mandatory to create his new...

By EcommerceComments Off
Read more...
18Jan

Establishment of Data Lake specific to multi-channel e-commerce application to understand customer’s buying pattern

Post order fulfillment data is becoming a very important asset of e-commerce vendors to understand complete buying pattern of customers. Especially for the e-commerce vendors who sells multiple products starting from electronics to apparels. Extraction and transformation are time-consuming operations when partially structured data starts moving from the various sources and finally land into the relational data warehouse.  Data extracted from the social media are semi-structured (JSON or XML).  As an example, Facebook provides information in JSON format through Graph API and same...

By Apache Hadoop, Data Engineering, Data Ingestion, Hadoop Eco System, Processing Engine, Storage MechanismComments Off
Read more...
18Jan

How to understand Data Pipeline easily

  A data pipeline can be visualized as extraction, transformation and then loading of data into storage area referred as Database system or Data warehousing system. The data enters into one end of the multi-stage process in a particular shape / form and comes out of the other side in a different (desired) shape / form. The data pipeline has many stages that depends on the entered input data. There might be less number of stages if input data is...

By Data EngineeringComments Off
Read more...
18Jan

The Internet Of Things (IOT)

The Internet Of Things has started vibrating now with a strong and bold promises to make a revolutionary, fully connected "Smart" world where the relationship between objects, their environment and people become more tightly twisted together. As a basic definition of IOT, we can refer to an environment where network connectivity and computing capability extends to objects , sensors and daily used articles normally bypassing computers, allowing those devices to generate, exchange and consume data with or without minimal...

By Data EngineeringComments Off
Read more...