Analyse Tweets using Flume, Hadoop and Hive
Note : Also don't forget to do check another entry on how to get some interesting facts from Twitter using R here . And also this entry on how to use Oozie for automating the below workflow. Here is a new blog on how to do the same analytics with Pig (using elephant-bird). It's not a hard rule, but almost 80% of the data is unstructured, while the remaining 20% is structured data. RDBMS helps to store/process the structured data (20%), while Hadoop solves the problem of storing/processing both types of data. The good thing about Hadoop, is that it scales incrementally with less CAPEX in terms of software and hardware. With the ever increasing usage of smart devices and the high speeds internet, unstructured data had been growing at a very fast rate. It's common to Tweet from a smart phone, take a picture and share it in Facebook. In this blog we will try to get Tweets using Flume and save them into HDFS for later analysis. Twitter exposes...