In our previous blog we learned that the platform that processes and organizes Big Data is Hadoop. Here we will learn more about Hadoop which is a core platform for structuring Big Data and solves the problems of utilizing it for analytic purposes. It is an Open Source software framework for distributed storage and distributed processing of Big Data on clusters of commodity hardware.
Main characteristics of Hadoop:
- Highly scalable (scaled out)
- Commodity hardware based
- Open Source, low acquisition and storage costs
Hadoop is basically divided into two parts namely : HDFS and Mapreduce framework. A Hadoop cluster is specially designed for storing and analyzing huge amounts of unstructured data. Workload is distributed across multiple cluster nodes that work to process data in parallel.
History of Hadoop
Doug Cutting is the brains behind Hadoop which has its origin in Apache and Nutch. Nutch was started in 2002 and it itself is an Open Source web search engine. Google published the paper that introduced the Mapreduce to the world. In early 2005 Nutch developers had a working Mapreduce implementation in Nutch. In February 2006 Hadoop was formed as an independent project by Nutch. In January 2008 Hadoop has made its own top level project at Apache and by this time major companies like Yahoo and Facebook started using Hadoop.
HDFS is the first aspect and Mapreduce is the secondary aspect of Hadoop. HDFS has an architecture which helps it in processing the data and organizing it. To get into details of HDFS, its architecture, functioning and several other concepts, keep an eye on the blogs that will be published in coming days.
Manasa Heggere
Senior Ruby on Rails Developer
Subscribe For Latest Updates
About Post Author
