What is NameNode and DataNode? How's it works in hadoop?

SingPrashant

Complete Programming Hub

Home Java Docker About

Hadoop-Tutorial !

HDFS

In the continuation of HDFS and MapReduce, let first understand about the HDFS.
HDFS(Hadoop Distributed file System): It's a storage part in Hadoop application. MapReduce consumes or take the data from HDFS. HDFS can create multiple replicas of data blocks and it can distribute them on compute nodes in cluster.This distribution enable the high rapid computation.
HDFS consists of two things:
1- NameNode which manages the filesystem,
2- DataNode that stores the actual data

Read Operation in HDFS:
1- HDFS client sent a read request by calling open() method of FileSystem object which is an object of type     DistributedFileSystem.
2- This object will try to connect to NameNode using RPC call and and get the metadata information
3- After the response this metadata request, now we have the addresses of DataNode(contains copy of that     block).
4- The object FSDataInputStream has came back to hdfs client , after receiving the addresses of DataNode.     FSDataInputStream contains the DFSInputStream object which is taking care of interactions with DataNode     and Namenode.
5- When the client will call read() method data has start coming/reading in the form of stream and the process of     read() will continues till it reaches end of block.
6- Once we reached at the end of block, DFSInput stream close the connection and then start point to next     DataNode for next block.
7- And at last once the client has done reading , it calls close() method to close all the connection.


Discover how to use Technologies and techniques to achieve an outstanding,knowledge Hadoop and Java website today!

@singprashant.com