How Google news is able to group similar news together
Google news uses clustering machine learning techniques to group similar kind of news or articles together. Interestingly, they don’t have thousand news editors on trunk instead use the clustering techniques to forms groups of similar data based on the common characteristics. Mahout is a machine learning software from Apache community that applications leverage to analyse large sets of data. Before invention of Mahout, it was too complex to a analyse large sets of data. Mahout extensively utilize Apache Hadoop to get the power parallel and distributed computing. Three machine learning techniques that offers by Mahout.
- Recommendation
- Classification
- Clustering
Clustering is not useful to group data into an existing set of known categories. This is particularly useful when we are not sure how to organize the data in the first place. Google news uses this powerful technique to make change the ever-changing stream of news and articles from around the world to enabling us keep update with latest events around the globe.
Recommendation technique uses user information + community information to deliver what type of product/service etc. we would like or prefer mainly when we are browsing e-commerce sites.
Classification technique or engine generally utilize to segregate the emails (Spam and Non-spam emails) based on known data like,
- Previously marked spam
- Trigger words and phrases
- Blacklisted IP address
- Similar to known spam
- Different language
using above parameters on the new incoming email, classification engine determines where to go the email either spam or non-spam.
