且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

将新的名称节点数据目录添加到现有群集

更新时间:2023-11-24 21:53:40

我相信我可能只是回答了我自己的问题。我最终将其中一个现有namenode目录的全部内容复制到新的NFS namenode目录中,并且我能够启动namenode。 (请注意,我在复制之前停止了namenode以避免出现问题)

  cp -rp / data / 1 / dfs / nn / data / nfs / dfs / nn 

我想我的假设是namenode会自动将现有元数据复制到新目录不正确。


What procedure do i need to follow to properly add a new NameNode data directory (dfs.name.dir, dfs.namenode.name.dir) to an existing production cluster? I have added the new path to the comma-delimited list in the hdfs-site.xml file but when i try to start the namenode i get the following error:

Directory /data/nfs/dfs/nn is in an inconsistent state: storage directory does not exist or is not accessible.

In my case, i have two directories already in place and working. (/data/1/dfs/nn,/data/2/dfs/nn) When i add the new directory I can't start the namenode. When the new path is removed, it starts just fine. My fstab for the new directory looks like this:

backup-server:/hadoop_nn /data/nfs/dfs nfs tcp,soft,intr,timeo=10,retrans=10 1 2

In the above mount point I have created a folder called nn. That folder has identical ownership and permissions to the other two existing locations nn folder.

drwx------ 2 hdfs hadoop 64 Jan 22 16:30 nn

Do i need to manually replicate/copy all the files from one of the existing namenode directories or should the namenode service do that automatically when it's started?

I believe i may have just answered my own question. I ended up copying the entire contents of one of the existing namenode directories over to the new NFS namenode directory and i was able to start the namenode. (Note that i stopped the namenode before copying to avoid problems)

cp -rp /data/1/dfs/nn /data/nfs/dfs/nn

I guess my assumption that the namenode would automatically copy the existing metadata over to the new directory was incorrect.