---
layout: post
title: hadoop集群学习笔记(1)
category: 技术
tags: Hadoop
keywords:
description: hdfs faderation 配置
---
{:toc}
### 1.修改hdfs-site.xml
修改配置如下,并复制到集群的其他机器中
```xml
dfs.replication
3
replication num
dfs.namenode.name.dir
file:/root/hadoop/dfs/nn/name
name location
dfs.namenode.edits.dir
file:/root/hadoop/dfs/nn/edits
edit file location
dfs.namenode.checkpoint.dir
file:/root/hadoop/dfs/snn/name
seconde name file location
dfs.namenode.checkpoint.edits.dir
file:/root/hadoop/dfs/snn/edits
seconde edit file location
dfs.datanode.data.dir
file:/root/hadoop/dfs/dn/data
data location
dfs.webhdfs.enabled
true
dfs.permissions
false
dfs.nameservices
ns1,ns2
dfs.namenode.rpc-address.ns1
h2m1:8020
dfs.namenode.servicerpc-address.ns1
h2m1:8021
dfs.namenode.secondary.http-address.ns1
h2m1:9001
secondary namenode web config
dfs.namenode.secondary.https-address.ns1
h2m1:9002
secondary namenode web config
dfs.namenode.rpc-address.ns2
h2s1:8020
dfs.namenode.servicerpc-address.ns2
h2s1:8021
dfs.namenode.secondary.http-address.ns2
h2s1:9001
secondary namenode web config
dfs.namenode.secondary.https-address.ns2
h2s1:9002
secondary namenode web config
```
### 2. 修改core-site.xml
作为NameNode faderation的节点修改如下的配置将`h2m1`修改为自定主机名或ip
```xml
fs.defaultFS
hdfs://h2m1:9000
```
### 3. 删除logs、dfs目录
```
rm -rf /root/hadoop/dfs/ ### 自定义的dfs位置
rm -rf /usr/local/hadoop271/logs/
```
### 4. 格式化NameNode faderation节点,每个都需要执行命令
`hdfs namenode -format -clusterId clusterCustomId` clusterCustomId为指定的id,faderation节点必须相同,否则不会加入faderation集群
### 5.启动hdfs
`start-dfs.sh`