In the mordern distrbuted applications, there are multiple coordination problems between nodes and nodes, including: leader election, group service, locking, configuration management, naming and synchronization. Apache Zookeeper, as its name implied, is a distributed, open-source coordination service framwork to address these demand.
In order to ensure the high performance, highly available and strictly ordered access, the performance aspects of ZooKeeper means it can be used in large, distributed systems and can also be deployed in cluster mode, which called ‘ZooKeeper ensemble’. In ZooKeeper ensemble, all write requests from clients are forwarded to a single server, called the leader, through the ZAB(Zookeeper Atomic Broadcast Protocol) to make sure the message in each nodes are same. Clients can access any one of the clusters to read and write data without worrying about inconsistencies in the data.
Image Credit : ebook -Zookeeper-Distributed Process Coordination from O’Reilly
The method to store the data in Zookeeper is similar as the standard UNIX file system, as a data model styled after the familiar directory tree structure of file systems. When we talking about ZooKeeper data nodes, we call it Znodes to clarify it.
Image Credit : ebook -Zookeeper-Distributed Process Coordination from O’Reilly
You could donwload and install Zookeeper directly1.
Or you could use Homebrew 2 brew install zookeeper
to install Zookeeper in Mac OS.
Considering the versatility, we run the Zookeeper by using docker in this blog. If you have not installed the docker yet, please prepare the docker environment first. 3
Execute the command to run zookeeper in a docker container
docker run --rm --name zookeeper -p 2181:2181 zookeeper
docker exec -it zookeeper bash
In the bin
directory, there is a command to start zookeeper zkServer
and the Management Console zkCli
bash-4.4# ls -l bin
total 36
-rwxr-xr-x 1 zookeepe zookeepe 232 Mar 27 04:32 README.txt
-rwxr-xr-x 1 zookeepe zookeepe 1937 Mar 27 04:32 zkCleanup.sh
-rwxr-xr-x 1 zookeepe zookeepe 1056 Mar 27 04:32 zkCli.cmd
-rwxr-xr-x 1 zookeepe zookeepe 1534 Mar 27 04:32 zkCli.sh
-rwxr-xr-x 1 zookeepe zookeepe 1759 Mar 27 04:32 zkEnv.cmd
-rwxr-xr-x 1 zookeepe zookeepe 2696 Mar 27 04:32 zkEnv.sh
-rwxr-xr-x 1 zookeepe zookeepe 1089 Mar 27 04:32 zkServer.cmd
-rwxr-xr-x 1 zookeepe zookeepe 6773 Mar 27 04:32 zkServer.sh
Since it was started through docker, the process of Zookeeper has been started and will provide the services to the public via port 2181.
bash-4.4# ps
PID USER TIME COMMAND
1 zookeepe 0:02 /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -Dzookeeper.log.dir=. -Dzookeeper.root
32 root 0:00 bash
42 root 0:00 ps
So, it allows you to access Zookeeper’s console directly through zkCli
for management.
bash-4.4# bin/zkCli.sh -server 127.0.0.1:2181
Connecting to 127.0.0.1:2181
...
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: 127.0.0.1:2181(CONNECTED) 0] help
ZooKeeper -server host:port cmd args
stat path [watch]
set path data [version]
ls path [watch]
delquota [-n|-b] path
ls2 path [watch]
setAcl path acl
setquota -n|-b val path
history
redo cmdno
printwatches on|off
delete path [version]
sync path
listquota path
rmr path
get path [watch]
create [-s] [-e] path data acl
addauth scheme auth
quit
getAcl path
close
connect host:port
Create /hello-zone
node:
[zk: 127.0.0.1:2181(CONNECTED) 19] create /hello-zone 'world'
Created /hello-zone
List the child nodes under /
and confirm that hello-zone
is created:
[zk: 127.0.0.1:2181(CONNECTED) 20] ls /
[zookeeper, hello-zone]
List the child nodes for /hello-zone
and verify that it is empty:
[zk: 127.0.0.1:2181(CONNECTED) 21] ls /hello-zone
[]
Get the data stored on the /hello-zone
node:
[zk: 127.0.0.1:2181(CONNECTED) 22] get /hello-zone
world
Zookeeper is used for service registration discovery and configuration management in Dubbo, and the structure of data in Zookeeper is shown in the following figure:
First, all data related to Dubbo is organized under the root node of /dubbo
.
The secondary directory is the service name like com.foo.BarService
.
The three-level directory has two child nodes, providers
and consumers
, representing the supplier and customers of the service.
The URL information for each application instance associated with the service will be recorded by the Level 4 directory. The providers
and consumer
will stored the providers information and the consumers information of the services seperately.
For example, the service provider of com.foo.BarService
will register its URL Information to /dubbo/com.foo.BarService/providers
; Similarly, service consumers will register their information under the corresponding consumer
node. At the same time, consumers will subscribe to the corresponding providers
node to be able to detect the changes of the service provider address list.
The code in this document can be found in https://github.com/dubbo/dubbo-samples/tree/master/dubbo-samples-zookeeper.
Define a simple greetingservice
interface with only one simple method named sayHello
to greet to the caller.
public interface GreetingService {
String sayHello(String name);
}
Implement the GreetingService
interface and mark it as a service for Dubbo via @Service
.
@Service
public class AnnotatedGreetingService implements GreetingService {
public String sayHello(String name) {
return "hello, " + name;
}
}
Define ProviderConfiguration to assemble Dubbo services.
@Configuration
@EnableDubbo(scanBasePackages = "com.alibaba.dubbo.samples.impl")
@PropertySource("classpath:/spring/dubbo-provider.properties")
static class ProviderConfiguration {}
Dubbo-provider.properties is an external configuration in a spring application, as follows:
dubbo.application.name=demo-provider
dubbo.registry.address=zookeeper://$DOCKER_HOST:2181
dubbo.protocol.name=dubbo
dubbo.protocol.port=20880
Since zookeeper runs in a docker container, please be noted that:
In the main
method, you could provide the Dubbo service by running a Spring Context.
public class ProviderBootstrap {
public static void main(String[] args) throws Exception {
AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(ProviderConfiguration.class);
context.start();
System.in.read();
}
}
Start the main
method of server, you will get the following output, which represents the success of the server’s startup, and the GreetingService
service is registered on the ZookeeperRegistry:
[03/08/18 10:50:33:033 CST] main INFO zookeeper.ZookeeperRegistry: [DUBBO] Register: dubbo://192.168.99.1:20880/com.alibaba.dubbo.samples.api.GreetingService?anyhost=true&application=demo-provider&dubbo=2.6.2&generic=false&interface=com.alibaba.dubbo.samples.api.GreetingService&methods=sayHello&pid=12938&side=provider×tamp=1533264631849, dubbo version: 2.6.2, current host: 192.168.99.1
You could find the registration information of the service provider through the Zookeeper management terminal:
$ docker exec -it zookeeper bash
bash-4.4# bin/zkCli.sh -server localhost:218
Connecting to localhost:2181
...
Welcome to ZooKeeper!
JLine support is enabled
...
[zk: localhost:2181(CONNECTED) 0] ls /dubbo/com.alibaba.dubbo.samples.api.GreetingService/providers
[dubbo%3A%2F%2F192.168.99.1%3A20880%2Fcom.alibaba.dubbo.samples.api.GreetingService%3Fanyhost%3Dtrue%26application%3Ddemo-provider%26dubbo%3D2.6.2%26generic%3Dfalse%26interface%3Dcom.alibaba.dubbo.samples.api.GreetingService%26methods%3DsayHello%26pid%3D12938%26side%3Dprovider%26timestamp%3D1533264631849]
You could find that the Dubbo services just registered its URL address at the providers
node as follows:
dubbo://192.168.99.1:20880/com.alibaba.dubbo.samples.api.GreetingService?anyhost=true&application=demo-provider&dubbo=2.6.2&generic=false&interface=com.alibaba.dubbo.samples.api.GreetingService&methods=sayHello&pid=12938&side=provider×tamp=1533264631849
You could declare the reference service by @Reference, while it will generate a full call. The target address of the service could be queried by the Zookeeper’s provider
node.
@Component("annotatedConsumer")
public class GreetingServiceConsumer {
@Reference
private GreetingService greetingService;
public String doSayHello(String name) {
return greetingService.sayHello(name);
}
}
Define the ConsumerConfiguration to assemble Dubbo service.
@Configuration
@EnableDubbo(scanBasePackages = "com.alibaba.dubbo.samples.action")
@PropertySource("classpath:/spring/dubbo-consumer.properties")
@ComponentScan(value = {"com.alibaba.dubbo.samples.action"})
static class ConsumerConfiguration {}
“dubbo-consumer.properties” is a method of external configuration in a Spring application, as follows:
dubbo.application.name=demo-consumer
dubbo.registry.address=zookeeper://$DOCKER_HOST:2181
dubbo.consumer.timeout=3000
Same as 3. Server: Assembling, You need to modify $DOCKER_HOST defined in dubbo.registry.address according to your own environment. You could find more instructions in step 3.
Run main
to initiate a remote call from a existed service provider. Dubbo first subscribes to the zookeeper service address and then selects one from the list of returned addresses to invoke the client:
public class ConsumerBootstrap {
public static void main(String[] args) {
public class ConsumerBootstrap {
public static void main(String[] args) throws IOException {
AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(ConsumerConfiguration.class);
context.start();
GreetingServiceConsumer greetingServiceConsumer = context.getBean(GreetingServiceConsumer.class);
String hello = greetingServiceConsumer.doSayHello("zookeeper");
System.out.println("result: " + hello);
System.in.read();
}
}
The output are as follows:
[03/08/18 01:42:31:031 CST] main INFO zookeeper.ZookeeperRegistry: [DUBBO] Register: consumer://192.168.99.1/com.alibaba.dubbo.samples.api.GreetingService?application=demo-consumer&category=consumers&check=false&default.timeout=3000&dubbo=2.6.2&interface=com.alibaba.dubbo.samples.api.GreetingService&methods=sayHello&pid=82406&side=consumer×tamp=1533274951195, dubbo version: 2.6.2, current host: 192.168.99.1 #1
[03/08/18 01:42:31:031 CST] main INFO zookeeper.ZookeeperRegistry: [DUBBO] Subscribe: consumer://192.168.99.1/com.alibaba.dubbo.samples.api.GreetingService?application=demo-consumer&category=providers,configurators,routers&default.timeout=3000&dubbo=2.6.2&interface=com.alibaba.dubbo.samples.api.GreetingService&methods=sayHello&pid=82406&side=consumer×tamp=1533274951195, dubbo version: 2.6.2, current host: 192.168.99.1 #2
...
result: hello, zookeeper
Description:
consumers
nodeproviders
, configurators
, routers
from Zookeepers. The configurations
is related to the Dubbo configuration, and routers
is related to routing rules. The providers node subscription should be noted. When a new service provider to join, due to the relationship between the subscription, the new address list will be pushed to the subscriber. So service consumers also dynamically perceive changes in address lists.You could find the registration information of the service provider through the Zookeeper management terminal:
$ docker exec -it zookeeper bash
bash-4.4# bin/zkCli.sh -server localhost:218
Connecting to localhost:2181
...
Welcome to ZooKeeper!
JLine support is enabled
...
[zk: localhost:2181(CONNECTED) 4] ls /dubbo/com.alibaba.dubbo.samples.api.GreetingService/consumers
[consumer%3A%2F%2F192.168.99.1%2Fcom.alibaba.dubbo.samples.api.GreetingService%3Fapplication%3Ddemo-consumer%26category%3Dconsumers%26check%3Dfalse%26default.timeout%3D3000%26dubbo%3D2.6.2%26interface%3Dcom.alibaba.dubbo.samples.api.GreetingService%26methods%3DsayHello%26pid%3D82406%26side%3Dconsumer%26timestamp%3D1533274951195]
You could see that consumers of Dubbo’s servicehas registered its URL address at the consumers
node:
consumer://192.168.99.1/com.alibaba.dubbo.samples.api.GreetingService?application=demo-consumer&category=providers,configurators,routers&default.timeout=3000&dubbo=2.6.2&interface=com.alibaba.dubbo.samples.api.GreetingService&methods=sayHello&pid=82406&side=consumer×tamp=1533274951195
This document focuses on how to use ZooKeeper as a registry in Dubbo. This document also mentioned that the Zookeeper could be a configuration center and a service management in Dubbo. Zookeeper is a single-node, standalone mode. However, developers always bulid a Zookeeper server cluster called * Zookeeper ensemble * in the real world.
Through this document, readers can learn: