Zookeeper搭建集群

发表于 2014-11-14   |   分类于 中间件   |  

简介

ZooKeeper是一个为分布式应用所设计的分布的、开源的协调服务，它主要是用来解决分布式应用中经常遇到的一些数据管理问题，简化分布式应用协调及其管理的难度，提供高性能的分布式服务。ZooKeeper本身可以以Standalone模式安装运行，不过它的长处在于通过分布式ZooKeeper集群（一个Leader，多个Follower），基于一定的策略来保证ZooKeeper集群的稳定性和可用性，从而实现分布式应用的可靠性。

ZooKeeper是作为分布式协调服务，是不需要依赖于Hadoop的环境，也可以为其他的分布式环境提供服务。

单节点安装

下载,解压,配置

1
2
3
4
5
6
7
8
9
10
11
12
# 可能是受Windows影响,首次接触Mac时创建了这个目录用于存放开源框架
cd ~/ProgramFiles
# 个人感觉速度慢, 用迅雷下载该链接更快
curl -O http://apache.mirrors.tds.net/zookeeper/stable/zookeeper-3.4.6.tar.gz
# 解压
tar xzvf zookeeper-3.4.6.tar.gz
mv zookeeper-3.4.6 zookeeper346
cd zookeeper346/conf
cp zoo_sample.cfg zoo.cfg
vim zoo.cfg
# 修改内容如下, 需手动创建定义的数据目录
dataDir=/Users/arccode/data/zk/nodes/zk0

启动

1
2
3
# zookeeper 可执行文件目录
cd ../bin
./zkServer.sh start

至此单节点配置,启动完成.

集群安装

集群中节点树结构

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
└── nodes
    ├── zk0
    │   └── version-2
    │       └── log.1
    ├── zk1
    │   ├── data
    │   │   ├── myid
    │   │   └── version-2
    │   │       ├── acceptedEpoch
    │   │       ├── currentEpoch
    │   │       └── snapshot.0
    │   └── logs
    │       └── version-2
    │           └── log.100000001
    ├── zk2
    │   ├── data
    │   │   ├── myid
    │   │   └── version-2
    │   │       ├── acceptedEpoch
    │   │       └── currentEpoch
    │   └── logs
    │       └── version-2
    │           └── log.100000001
    └── zk3
        ├── data
        │   ├── myid
        │   └── version-2
        │       ├── acceptedEpoch
        │       ├── currentEpoch
        │       └── snapshot.100000000
        └── logs
            └── version-2
                └── log.100000001

配置节点一

不解释, 直接上配置文件(/Users/arccode/ProgramFiles/zookeeper346/conf/zk1.cfg)

1
2
3
4
5
6
7
dataDir=/Users/arccode/data/zk/nodes/zk1/data
dataLogDir=/Users/arccode/data/zk/nodes/zk1/logs
clientPort=2181
# 集群实例
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890

根据dataDir,dataLogDir创建相应的目录并在data目录执行如下命令

1
echo 1 > myid

配置节点二

不解释, 直接上配置文件(/Users/arccode/ProgramFiles/zookeeper346/conf/zk2.cfg)

1
2
3
4
5
6
7
dataDir=/Users/arccode/data/zk/nodes/zk2/data
dataLogDir=/Users/arccode/data/zk/nodes/zk2/logs
clientPort=2182
# 集群实例
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890

根据dataDir,dataLogDir创建相应的目录并在data目录执行如下命令

1
echo 2 > myid

配置节点三

不解释, 直接上配置文件(/Users/arccode/ProgramFiles/zookeeper346/conf/zk3.cfg)

1
2
3
4
5
6
7
dataDir=/Users/arccode/data/zk/nodes/zk3/data
dataLogDir=/Users/arccode/data/zk/nodes/zk3/logs
clientPort=2183
# 集群实例
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890

根据dataDir,dataLogDir创建相应的目录并在data目录执行如下命令

1
echo 3 > myid

启动Zookeeper集群

1
2
3
4
5
# zookeeper可执行文件目录
cd bin
./zkServer.sh start zk1.cfg
./zkServer.sh start zk2.cfg
./zkServer.sh start zk3.cfg

查看Zookeeper集群中各个节点的状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
./zkServer.sh status zk1.cfg

JMX enabled by default
Using config: /Users/arccode/ProgramFiles/zookeeper346/bin/../conf/zk1.cfg
Mode: follower

./zkServer.sh status zk2.cfg

JMX enabled by default
Using config: /Users/arccode/ProgramFiles/zookeeper346/bin/../conf/zk2.cfg
Mode: leader

./zkServer.sh status zk3.cfg

JMX enabled by default
Using config: /Users/arccode/ProgramFiles/zookeeper346/bin/../conf/zk3.cfg
Mode: follower

注: 可以考虑多种启动或停止zookeeper的情况进行状态查看, 可发现zookeeper的高可用(HA);

命令行操作

使用命令行client连接zookeeper并操作API

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
# 连接到任意节点
 ./zkCli.sh -server 127.0.0.1:2182
# help 查看指令
ZooKeeper -server host:port cmd args
	connect host:port
	get path [watch]
	ls path [watch]
	set path data [version]
	rmr path
	delquota [-n|-b] path
	quit 
	printwatches on|off
	create [-s] [-e] path data acl
	stat path [watch]
	close 
	ls2 path [watch]
	history 
	listquota path
	setAcl path acl
	getAcl path
	sync path
	redo cmdno
	addauth scheme auth
	delete path [version]
	setquota -n|-b val path
# 查看/目录节点
ls /
# 创建znode节点
create /config db.properties
# 查看/下节点, ls /
[config, zookeeper]
# 查看节点内容, get /config
db.properties
cZxid = 0x300000011
ctime = Fri Nov 14 23:05:05 CST 2014
mZxid = 0x300000011
mtime = Fri Nov 14 23:05:05 CST 2014
pZxid = 0x300000011
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 13
numChildren = 0
# 修改节点内容, set /config db2.properties
# 再次查看 get /config
db2.properties
cZxid = 0x300000011
ctime = Fri Nov 14 23:05:05 CST 2014
mZxid = 0x300000012
mtime = Fri Nov 14 23:08:10 CST 2014
pZxid = 0x300000011
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 14
numChildren = 0
# 删除节点, delete /config
# 查看, ls /
# 退出, quit

Java操作

源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
package com.ts.zookeeper;

import java.io.IOException;

import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooDefs.Ids;
import org.apache.zookeeper.ZooKeeper;

/**
 * Java 操作zookeeper API
 *
 */
public class Main
{
    public static void main( String[] args ) throws IOException, InterruptedException, KeeperException
    {
    	// 创建连接
    	ZooKeeper zk = new ZooKeeper("127.0.0.1:2182", 30000, new Watcher(){
    		// 监听的事件发生变化时, 调用该函数
			public void process(WatchedEvent event) {
				System.out.println("Event emit -> " + event.getType());
			}
    	});
    	
    	// 查看根目录, ls /
    	System.out.println("[line: 29]command: ls / -> " + zk.getChildren("/", true));
    	
    	// 创建数据节点config
    	if(zk.exists("/config", true) == null) {
    		zk.create("/config", "db.properties".getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
    		
    		System.out.println("创建数据节点config成功.");
    		
    		// 查看数据节点数据
    		System.out.println("[line: 38]command get /config: " + new String(zk.getData("/config", true, null)));
    		
    		// 查看根目录, ls /
    		System.out.println("[line: 41]command: ls / -> " + zk.getChildren("/", true));
    	}
    	
    	// 修改数据节点
    	if(zk.exists("/config", true) != null) {
    		zk.setData("/config", "db2.properteis".getBytes(), -1);
    		// 查看数据节点数据
    		System.out.println("[line: 48]command get /config: " + new String(zk.getData("/config", true, null)));
    		// 查看根目录, ls /
    		System.out.println("[line: 50]command: ls / -> " + zk.getChildren("/", true));
    	}
    	
    	// 删除数据节点
    	if(zk.exists("/config", true) != null) {
    		zk.delete("/config", -1);
    		// 查看根目录, ls /
    		System.out.println("[line: 57]command: ls / -> " + zk.getChildren("/", true));
    	}
    	
    	// 关闭连接
    	zk.close();
    }
}

结果

1
2
3
4
5
6
7
8
9
10
11
12
13
Event emit -> None
[line: 29]command: ls / -> [zookeeper]
Event emit -> NodeCreated
Event emit -> NodeChildrenChanged
创建数据节点config成功.
[line: 38]command get /config: db.properties
[line: 41]command: ls / -> [config, zookeeper]
Event emit -> NodeDataChanged
[line: 48]command get /config: db2.properteis
[line: 50]command: ls / -> [config, zookeeper]
Event emit -> NodeDeleted
Event emit -> NodeChildrenChanged
[line: 57]command: ls / -> [zookeeper]

参考资料

ZooKeeper伪分布式集群安装及使用

转载

本文出自<<arccode>>, 欢迎转载, 转载请注明出处.
#Java #Zookeeper
初识Gradle系列一
mongoose入门