这是用户在 2024-9-9 14:24 为 https://zookeeper.apache.org/doc/r3.4.14/zookeeperAdmin.html#sc_authOptions 保存的双语快照页面,由 沉浸式翻译 提供双语支持。了解如何保存?
Apache > ZooKeeper
Apache >动物园管理员
Last Published: 06/05/2022 15:18:10
最后发布: 06/05/2022 15:18:10
 

ZooKeeper Administrator's Guide
ZooKeeper 管理员指南

A Guide to Deployment and Administration
部署和管理指南

Deployment 部署

This section contains information about deploying Zookeeper and covers these topics:
本节包含有关部署 Zookeeper 的信息并涵盖以下主题:

The first two sections assume you are interested in installing ZooKeeper in a production environment such as a datacenter. The final section covers situations in which you are setting up ZooKeeper on a limited basis - for evaluation, testing, or development - but not in a production environment.
前两节假设您有兴趣在生产环境(例如数据中心)中安装 ZooKeeper。最后一部分介绍了在有限的基础上设置 ZooKeeper 的情况 - 用于评估、测试或开发 - 但不在生产环境中。

System Requirements 系统要求

Supported Platforms 支持的平台

ZooKeeper consists of multiple components. Some components are supported broadly, and other components are supported only on a smaller set of platforms.
ZooKeeper 由多个组件组成。某些组件得到广泛支持,而其他组件仅在较小的平台上受支持。

The following matrix describes the level of support committed for running each component on different operating system platforms.
以下矩阵描述了在不同操作系统平台上运行每个组件所承诺的支持级别。

Support Matrix 支持矩阵
Operating System  操作系统 Client  客户 Server  服务器 Native Client  本地客户端 Contrib  贡献
GNU/Linux Development and Production
开发生产
Development and Production
开发生产
Development and Production
开发生产
Development and Production
开发生产
Solaris  索拉里斯 Development and Production
开发生产
Development and Production
开发生产
Not Supported  不支持 Not Supported  不支持
FreeBSD  自由BSD Development and Production
开发生产
Development and Production
开发生产
Not Supported  不支持 Not Supported  不支持
Windows  视窗 Development and Production
开发生产
Development and Production
开发生产
Not Supported  不支持 Not Supported  不支持
Mac OS X Development Only  仅限开发 Development Only  仅限开发 Not Supported  不支持 Not Supported  不支持

For any operating system not explicitly mentioned as supported in the matrix, components may or may not work. The ZooKeeper community will fix obvious bugs that are reported for other platforms, but there is no full support.
对于矩阵中未明确提及支持的任何操作系统,组件可能会也可能不会工作。 ZooKeeper 社区将修复其他平台报告的明显错误,但没有完全支持。

Required Software 所需软件

ZooKeeper runs in Java, release 1.7 or greater (JDK 7 or greater, FreeBSD support requires openjdk7). It runs as an ensemble of ZooKeeper servers. Three ZooKeeper servers is the minimum recommended size for an ensemble, and we also recommend that they run on separate machines. At Yahoo!, ZooKeeper is usually deployed on dedicated RHEL boxes, with dual-core processors, 2GB of RAM, and 80GB IDE hard drives.
ZooKeeper 在 Java 版本 1.7 或更高版本中运行(JDK 7 或更高版本,FreeBSD 支持需要 openjdk7)。它作为 ZooKeeper 服务器的集合运行。三个 ZooKeeper 服务器是整体建议的最小规模,我们还建议它们在单独的计算机上运行。在 Yahoo!,ZooKeeper 通常部署在专用的 RHEL 机器上,配备双核处理器、2GB RAM 和 80GB IDE 硬盘驱动器。

Clustered (Multi-Server) Setup
集群(多服务器)设置

For reliable ZooKeeper service, you should deploy ZooKeeper in a cluster known as an ensemble. As long as a majority of the ensemble are up, the service will be available. Because Zookeeper requires a majority, it is best to use an odd number of machines. For example, with four machines ZooKeeper can only handle the failure of a single machine; if two machines fail, the remaining two machines do not constitute a majority. However, with five machines ZooKeeper can handle the failure of two machines.
为了获得可靠的 ZooKeeper 服务,您应该将 ZooKeeper 部署在称为ensemble的集群中。只要大多数集合都已启动,该服务就可用。因为Zookeeper需要多数,所以最好使用奇数台机器。例如,四台机器的ZooKeeper只能处理单台机器的故障;如果两台机器出现故障,剩下的两台机器不占多数。然而,如果有五台机器,ZooKeeper 就可以处理两台机器的故障。

Note 笔记

As mentioned in the ZooKeeper Getting Started Guide , a minimum of three servers are required for a fault tolerant clustered setup, and it is strongly recommended that you have an odd number of servers.
正如ZooKeeper 入门指南中提到的,容错集群设置至少需要三台服务器,并且强烈建议您拥有奇数数量的服务器。

Usually three servers is more than enough for a production install, but for maximum reliability during maintenance, you may wish to install five servers. With three servers, if you perform maintenance on one of them, you are vulnerable to a failure on one of the other two servers during that maintenance. If you have five of them running, you can take one down for maintenance, and know that you're still OK if one of the other four suddenly fails.
通常,三台服务器对于生产安装来说绰绰有余,但为了在维护期间获得最大可靠性,您可能希望安装五台服务器。对于三台服务器,如果您对其中一台进行维护,则在维护期间,其他两台服务器中的一台很容易出现故障。如果其中五个正在运行,您可以取下其中一个进行维护,并且知道即使其他四个之一突然发生故障,您仍然可以正常工作。

Your redundancy considerations should include all aspects of your environment. If you have three ZooKeeper servers, but their network cables are all plugged into the same network switch, then the failure of that switch will take down your entire ensemble.
您的冗余考虑因素应包括环境的所有方面。如果您有三台 ZooKeeper 服务器,但它们的网络电缆都插入同一个网络交换机,那么该交换机的故障将导致整个系统瘫痪。

Here are the steps to setting a server that will be part of an ensemble. These steps should be performed on every host in the ensemble:
以下是设置将成为集合一部分的服务器的步骤。应在整体中的每个主机上执行以下步骤:

  1. Install the Java JDK. You can use the native packaging system for your system, or download the JDK from: http://java.sun.com/javase/downloads/index.jsp
    安装 Java JDK。您可以使用适合您系统的本机打包系统,或者从以下位置下载 JDK: http://java.sun.com/javase/downloads/index.jsp

  2. Set the Java heap size. This is very important to avoid swapping, which will seriously degrade ZooKeeper performance. To determine the correct value, use load tests, and make sure you are well below the usage limit that would cause you to swap. Be conservative - use a maximum heap size of 3GB for a 4GB machine.
    设置 Java 堆大小。这对于避免交换非常重要,交换会严重降低 ZooKeeper 的性能。要确定正确的值,请使用负载测试,并确保远低于导致交换的使用限制。保守一点 - 对于 4GB 机器,使用最大堆大小 3GB。

  3. Install the ZooKeeper Server Package. It can be downloaded from: http://zookeeper.apache.org/releases.html
    安装 ZooKeeper 服务器包。可以从以下地址下载: http://zookeeper.apache.org/releases.html

  4. Create a configuration file. This file can be called anything. Use the following settings as a starting point:
    创建配置文件。该文件可以命名为任何名称。使用以下设置作为起点:

    tickTime=2000
    dataDir=/var/lib/zookeeper/
    clientPort=2181
    initLimit=5
    syncLimit=2
    server.1=zoo1:2888:3888
    server.2=zoo2:2888:3888
    server.3=zoo3:2888:3888
    

    You can find the meanings of these and other configuration settings in the section Configuration Parameters. A word though about a few here: Every machine that is part of the ZooKeeper ensemble should know about every other machine in the ensemble. You accomplish this with the series of lines of the form server.id=host:port:port. The parameters host and port are straightforward. You attribute the server id to each machine by creating a file named myid, one for each server, which resides in that server's data directory, as specified by the configuration file parameter dataDir.
    您可以在配置参数部分找到这些和其他配置设置的含义。这里简单说一下:ZooKeeper 整体中的每台机器都应该了解该整体中的所有其他机器。您可以通过server.id=host:port:port形式的一系列行来完成此操作。参数主机端口很简单。您可以通过创建一个名为myid的文件(每个服务器一个)来将服务器 ID 归属于每台计算机,该文件驻留在该服务器的数据目录中,如配置文件参数dataDir所指定。

  5. The myid file consists of a single line containing only the text of that machine's id. So myid of server 1 would contain the text "1" and nothing else. The id must be unique within the ensemble and should have a value between 1 and 255. IMPORTANT: if you enable extended features such as TTL Nodes (see below) the id must be between 1 and 254 due to internal limitations.
    myid 文件由一行组成,其中仅包含该计算机的 id 文本。因此服务器 1 的myid将包含文本“1”,仅此而已。 id 在整体中​​必须是唯一的,并且值应介于 1 和 255 之间。重要提示:如果启用 TTL 节点(见下文)等扩展功能,由于内部限制,id 必须介于 1 和 254 之间。

  6. If your configuration file is set up, you can start a ZooKeeper server:
    如果您的配置文件已设置,您可以启动 ZooKeeper 服务器:

    $ java -cp zookeeper.jar:lib/slf4j-api-1.7.5.jar:lib/slf4j-log4j12-1.7.5.jar:lib/log4j-1.2.17.jar:conf \\
    org.apache.zookeeper.server.quorum.QuorumPeerMain zoo.cfg
    

QuorumPeerMain starts a ZooKeeper server, JMX management beans are also registered which allows management through a JMX management console. The ZooKeeper JMX document contains details on managing ZooKeeper with JMX. See the script bin/zkServer.sh, which is included in the release, for an example of starting server instances.
QuorumPeerMain启动ZooKeeper服务器,还注册JMX管理bean,允许通过JMX管理控制台进行管理。 ZooKeeper JMX 文档包含有关使用 JMX 管理 ZooKeeper 的详细信息。有关启动服务器实例的示例,请参阅该版本中包含的脚本bin/zkServer.sh

  1. Test your deployment by connecting to the hosts: In Java, you can run the following command to execute simple operations:
    通过连接到主机来测试您的部署:在 Java 中,您可以运行以下命令来执行简单的操作:

    $ bin/zkCli.sh -server 127.0.0.1:2181
    

Single Server and Developer Setup
单一服务器和开发人员设置

If you want to setup ZooKeeper for development purposes, you will probably want to setup a single server instance of ZooKeeper, and then install either the Java or C client-side libraries and bindings on your development machine.
如果您想出于开发目的设置 ZooKeeper,您可能需要设置 ZooKeeper 的单个服务器实例,然后在您的开发计算机上安装 Java 或 C 客户端库和绑定。

The steps to setting up a single server instance are the similar to the above, except the configuration file is simpler. You can find the complete instructions in the Installing and Running ZooKeeper in Single Server Mode section of the ZooKeeper Getting Started Guide.
设置单个服务器实例的步骤与上面类似,只是配置文件更简单。您可以在ZooKeeper 入门指南以单服务器模式安装和运行 ZooKeeper部分找到完整的说明。

For information on installing the client side libraries, refer to the Bindings section of the ZooKeeper Programmer's Guide.
有关安装客户端库的信息,请参阅ZooKeeper 程序员指南绑定部分。

Administration 行政

This section contains information about running and maintaining ZooKeeper and covers these topics:
本节包含有关运行和维护 ZooKeeper 的信息,并涵盖以下主题:

Designing a ZooKeeper Deployment
设计 ZooKeeper 部署

The reliability of ZooKeeper rests on two basic assumptions.
ZooKeeper 的可靠性取决于两个基本假设。

  1. Only a minority of servers in a deployment will fail. Failure in this context means a machine crash, or some error in the network that partitions a server off from the majority.
    部署中只有少数服务器会发生故障。在这种情况下,故障意味着机器崩溃,或者网络中出现将服务器与大多数服务器隔离的错误。
  2. Deployed machines operate correctly. To operate correctly means to execute code correctly, to have clocks that work properly, and to have storage and network components that perform consistently.
    部署的机器运行正常。正确操作意味着正确执行代码、时钟正常工作以及存储和网络组件运行一致。

The sections below contain considerations for ZooKeeper administrators to maximize the probability for these assumptions to hold true. Some of these are cross-machines considerations, and others are things you should consider for each and every machine in your deployment.
以下部分包含 ZooKeeper 管理员的注意事项,以最大限度地提高这些假设成立的可能性。其中一些是跨机器的注意事项,另一些是您应该为部署中的每台机器考虑的事项。

Cross Machine Requirements
跨机器要求

For the ZooKeeper service to be active, there must be a majority of non-failing machines that can communicate with each other. To create a deployment that can tolerate the failure of F machines, you should count on deploying 2xF+1 machines. Thus, a deployment that consists of three machines can handle one failure, and a deployment of five machines can handle two failures. Note that a deployment of six machines can only handle two failures since three machines is not a majority. For this reason, ZooKeeper deployments are usually made up of an odd number of machines.
为了使 ZooKeeper 服务处于活动状态,必须有大多数无故障的计算机可以相互通信。要创建可以容忍 F 台机器故障的部署,您应该依靠部署 2xF+1 台机器。因此,由三台机器组成的部署可以处理一种故障,而由五台机器组成的部署可以处理两种故障。请注意,六台机器的部署只能处理两个故障,因为三台机器并不是大多数。因此,ZooKeeper 部署通常由奇数台机器组成。

To achieve the highest probability of tolerating a failure you should try to make machine failures independent. For example, if most of the machines share the same switch, failure of that switch could cause a correlated failure and bring down the service. The same holds true of shared power circuits, cooling systems, etc.
为了实现容忍故障的最高概率,您应该尝试使机器故障独立。例如,如果大多数计算机共享同一交换机,则该交换机的故障可能会导致相关故障并导致服务中断。共享电源电路、冷却系统等也是如此。

Single Machine Requirements
单机要求

If ZooKeeper has to contend with other applications for access to resources like storage media, CPU, network, or memory, its performance will suffer markedly. ZooKeeper has strong durability guarantees, which means it uses storage media to log changes before the operation responsible for the change is allowed to complete. You should be aware of this dependency then, and take great care if you want to ensure that ZooKeeper operations aren’t held up by your media. Here are some things you can do to minimize that sort of degradation:
如果 ZooKeeper 必须与其他应用程序竞争存储介质、CPU、网络或内存等资源的访问,其性能将受到显着影响。 ZooKeeper 具有强大的持久性保证,这意味着它在负责更改的操作被允许完成之前使用存储介质来记录更改。那么您应该意识到这种依赖性,并且如果您想确保 ZooKeeper 操作不会被您的媒体阻止,请务必小心。您可以采取以下措施来最大限度地减少这种退化:

Provisioning 配置

Things to Consider: ZooKeeper Strengths and Limitations
需要考虑的事项:ZooKeeper 的优势和局限性

Administering 管理

Maintenance 维护

Little long term maintenance is required for a ZooKeeper cluster however you must be aware of the following:
ZooKeeper 集群几乎不需要长期维护,但您必须注意以下事项:

Ongoing Data Directory Cleanup
正在进行的数据目录清理

The ZooKeeper Data Directory contains files which are a persistent copy of the znodes stored by a particular serving ensemble. These are the snapshot and transactional log files. As changes are made to the znodes these changes are appended to a transaction log. Occasionally, when a log grows large, a snapshot of the current state of all znodes will be written to the filesystem and a new transaction log file is created for future transactions. During snapshotting, ZooKeeper may continue appending incoming transactions to the old log file. Therefore, some transactions which are newer than a snapshot may be found in the last transaction log preceding the snapshot.
ZooKeeper数据目录包含的文件是由特定服务整体存储的 znode 的持久副本。这些是快照和事务日志文件。当对 znode 进行更改时,这些更改将附加到事务日志中。有时,当日志变大时,所有 znode 当前状态的快照将写入文件系统,并为将来的事务创建新的事务日志文件。在快照期间,ZooKeeper 可能会继续将传入事务附加到旧日志文件中。因此,可能会在快照之前的最后一个事务日志中找到一些比快照更新的事务。

A ZooKeeper server will not remove old snapshots and log files when using the default configuration (see autopurge below), this is the responsibility of the operator. Every serving environment is different and therefore the requirements of managing these files may differ from install to install (backup for example).
使用默认配置时,ZooKeeper 服务器不会删除旧的快照和日志文件(请参阅下面的自动清除),这是操作员的责任。每个服务环境都不同,因此管理这些文件的要求可能因安装而异(例如备份)。

The PurgeTxnLog utility implements a simple retention policy that administrators can use. The API docs contains details on calling conventions (arguments, etc...).
PurgeTxnLog 实用程序实现了管理员可以使用的简单保留策略。 API 文档包含有关调用约定(参数等)的详细信息。

In the following example the last count snapshots and their corresponding logs are retained and the others are deleted. The value of should typically be greater than 3 (although not required, this provides 3 backups in the unlikely event a recent log has become corrupted). This can be run as a cron job on the ZooKeeper server machines to clean up the logs daily.
在以下示例中,保留最后计数的快照及其相应的日志,并删除其他快照。的值通常应大于 3(尽管不是必需的,但这会在最近的日志损坏的情况下提供 3 个备份)。这可以作为 ZooKeeper 服务器计算机上的 cron 作业运行,以每天清理日志。

java -cp zookeeper.jar:lib/slf4j-api-1.7.5.jar:lib/slf4j-log4j12-1.7.5.jar:lib/log4j-1.2.17.jar:conf org.apache.zookeeper.server.PurgeTxnLog <dataDir> <snapDir> -n <count>

Automatic purging of the snapshots and corresponding transaction logs was introduced in version 3.4.0 and can be enabled via the following configuration parameters autopurge.snapRetainCount and autopurge.purgeInterval. For more on this, see Advanced Configuration below.
3.4.0 版本中引入了自动清除快照和相应事务日志的功能,可以通过以下配置参数autopurge.snapRetainCountautopurge.purgeInterval启用。有关详细信息,请参阅下面的高级配置

Debug Log Cleanup (log4j)
调试日志清理 (log4j)

See the section on logging in this document. It is expected that you will setup a rolling file appender using the in-built log4j feature. The sample configuration file in the release tar's conf/log4j.properties provides an example of this.
请参阅本文档中有关登录的部分。预计您将使用内置的 log4j 功能设置滚动文件附加程序。发行版 tar 的 conf/log4j.properties 中的示例配置文件提供了这样的示例。

Supervision 监督

You will want to have a supervisory process that manages each of your ZooKeeper server processes (JVM). The ZK server is designed to be "fail fast" meaning that it will shutdown (process exit) if an error occurs that it cannot recover from. As a ZooKeeper serving cluster is highly reliable, this means that while the server may go down the cluster as a whole is still active and serving requests. Additionally, as the cluster is "self healing" the failed server once restarted will automatically rejoin the ensemble w/o any manual interaction.
您将需要一个监督进程来管理每个 ZooKeeper 服务器进程 (JVM)。 ZK 服务器被设计为“快速失败”,这意味着如果发生无法恢复的错误,它将关闭(进程退出)。由于 ZooKeeper 服务集群高度可靠,这意味着虽然服务器可能出现故障,但整个集群仍然处于活动状态并为请求提供服务。此外,由于集群是“自我修复”的,故障服务器一旦重新启动将自动重新加入集群,无需任何手动交互。

Having a supervisory process such as daemontools or SMF (other options for supervisory process are also available, it's up to you which one you would like to use, these are just two examples) managing your ZooKeeper server ensures that if the process does exit abnormally it will automatically be restarted and will quickly rejoin the cluster.
拥有一个监控进程,例如daemontoolsSMF (监控进程的其他选项也可用,这取决于您想要使用哪一个,这只是两个示例)管理您的 ZooKeeper 服务器可确保如果进程确实异常退出将自动重启并快速重新加入集群。

Monitoring 监控

The ZooKeeper service can be monitored in one of two primary ways; 1) the command port through the use of 4 letter words and 2) JMX. See the appropriate section for your environment/requirements.
ZooKeeper 服务可以通过两种主要方式之一进行监控: 1) 通过使用4 个字母单词的命令端口和 2) JMX 。请参阅适合您的环境/要求的部分。

Logging 记录

ZooKeeper uses log4j version 1.2 as its logging infrastructure. The ZooKeeper default log4j.properties file resides in the conf directory. Log4j requires that log4j.properties either be in the working directory (the directory from which ZooKeeper is run) or be accessible from the classpath.
ZooKeeper 使用log4j版本 1.2 作为其日志记录基础设施。 ZooKeeper 默认的log4j.properties文件位于conf目录中。 Log4j 要求log4j.properties位于工作目录(ZooKeeper 运行的目录)中或者可以从类路径访问。

For more information, see Log4j Default Initialization Procedure of the log4j manual.
有关详细信息,请参阅 log4j 手册的Log4j 默认初始化过程

Troubleshooting 故障排除

Configuration Parameters 配置参数

ZooKeeper's behavior is governed by the ZooKeeper configuration file. This file is designed so that the exact same file can be used by all the servers that make up a ZooKeeper server assuming the disk layouts are the same. If servers use different configuration files, care must be taken to ensure that the list of servers in all of the different configuration files match.
ZooKeeper 的行为由 ZooKeeper 配置文件控制。该文件的设计使得组成 ZooKeeper 服务器的所有服务器都可以使用完全相同的文件(假设磁盘布局相同)。如果服务器使用不同的配置文件,则必须注意确保所有不同配置文件中的服务器列表匹配。

Minimum Configuration 最低配置

Here are the minimum configuration keywords that must be defined in the configuration file:
以下是配置文件中必须定义的最少配置关键字:

Advanced Configuration 高级配置

The configuration settings in the section are optional. You can use them to further fine tune the behaviour of your ZooKeeper servers. Some can also be set using Java system properties, generally of the form zookeeper.keyword. The exact system property, when available, is noted below.
该部分中的配置设置是可选的。您可以使用它们进一步微调 ZooKeeper 服务器的行为。有些还可以使用 Java 系统属性进行设置,通常采用Zookeeper.keyword形式。确切的系统属性(如果可用)如下所述。

Cluster Options 集群选项

The options in this section are designed for use with an ensemble of servers -- that is, when deploying clusters of servers.
本节中的选项设计用于服务器集合,即部署服务器集群时。

If you really need enable all four letter word commands by default, you can use the asterisk option so you don't have to include every command one by one in the list. As an example, this will enable all four letter word commands:
如果您确实需要默认启用所有四个字母的单词命令,则可以使用星号选项,这样您就不必将每个命令一一包含在列表中。例如,这将启用所有四个字母的单词命令:

4lw.commands.whitelist=*

Authentication and Authorization Options
身份验证和授权选项

The options in this section allow control over encryption/authentication/authorization performed by the service.
本节中的选项允许控制服务执行的加密/身份验证/授权。

$ perl -e "print 'stmk', pack('q>', 0b0011111010)" | nc localhost 2181
250

Experimental Options/Features
实验选项/功能

New features that are currently considered experimental.
目前被认为是实验性的新功能。

Unsafe Options 不安全选项

The following options can be useful, but be careful when you use them. The risk of each is explained along with the explanation of what the variable does.
以下选项可能很有用,但使用时要小心。每个风险的解释以及变量作用的解释。

Communication using the Netty framework
使用Netty框架进行通信

Netty is an NIO based client/server communication framework, it simplifies (over NIO being used directly) many of the complexities of network level communication for java applications. Additionally the Netty framework has built in support for encryption (SSL) and authentication (certificates). These are optional features and can be turned on or off individually.
Netty是一个基于 NIO 的客户端/服务器通信框架,它简化了(相对于直接使用 NIO 而言)java 应用程序网络级通信的许多复杂性。此外,Netty 框架内置了对加密 (SSL) 和身份验证(证书)的支持。这些是可选功能,可以单独打开或关闭。

In versions 3.5+, a ZooKeeper server can use Netty instead of NIO (default option) by setting the environment variable zookeeper.serverCnxnFactory to org.apache.zookeeper.server.NettyServerCnxnFactory; for the client, set zookeeper.clientCnxnSocket to org.apache.zookeeper.ClientCnxnSocketNetty.
在版本3.5+中,ZooKeeper服务器可以通过将环境变量zookeeper.serverCnxnFactory设置为org.apache.zookeeper.server.NettyServerCnxnFactory来使用Netty而不是NIO(默认选项);对于客户端,将Zookeeper.clientCnxnSocket设置为org.apache.zookeeper.ClientCnxnSocketNetty

TBD - tuning options for netty - currently there are none that are netty specific but we should add some. Esp around max bound on the number of reader worker threads netty creates.
TBD - netty 的调整选项 - 目前没有任何特定于 netty 的选项,但我们应该添加一些。特别是 netty 创建的读取器工作线程数量的最大限制。

TBD - how to manage encryption
TBD - 如何管理加密

TBD - how to manage certificates
TBD - 如何管理证书

ZooKeeper Commands: The Four Letter Words
ZooKeeper 命令:四个字母的单词

ZooKeeper responds to a small set of commands. Each command is composed of four letters. You issue the commands to ZooKeeper via telnet or nc, at the client port.
ZooKeeper 响应一小组命令。每个命令由四个字母组成。您可以在客户端端口通过 telnet 或 nc 向 ZooKeeper 发出命令。

Three of the more interesting commands: "stat" gives some general information about the server and connected clients, while "srvr" and "cons" give extended details on server and connections respectively.
三个更有趣的命令:“stat”提供有关服务器和连接的客户端的一些常规信息,而“srvr”和“cons”分别提供有关服务器和连接的扩展详细信息。

$ echo mntr | nc localhost 2185
              zk_version  3.4.0
              zk_avg_latency  0
              zk_max_latency  0
              zk_min_latency  0
              zk_packets_received 70
              zk_packets_sent 69
              zk_outstanding_requests 0
              zk_server_state leader
              zk_znode_count   4
              zk_watch_count  0
              zk_ephemerals_count 0
              zk_approximate_data_size    27
              zk_followers    4                   - only exposed by the Leader
              zk_synced_followers 4               - only exposed by the Leader
              zk_pending_syncs    0               - only exposed by the Leader
              zk_open_file_descriptor_count 23    - only available on Unix platforms
              zk_max_file_descriptor_count 1024   - only available on Unix platforms
              zk_fsync_threshold_exceed_count 0

The output is compatible with java properties format and the content may change over time (new keys added). Your scripts should expect changes. ATTENTION: Some of the keys are platform specific and some of the keys are only exported by the Leader. The output contains multiple lines with the following format:
输出与java属性格式兼容,内容可能会随着时间的推移而改变(添加了新的键)。您的脚本应该会发生变化。注意:有些密钥是特定于平台的,有些密钥仅由 Leader 导出。输出包含多行,格式如下:

key \t value

Here's an example of the ruok command:
以下是ruok命令的示例:

$ echo ruok | nc 127.0.0.1 5111
    imok

Data File Management 数据文件管理

ZooKeeper stores its data in a data directory and its transaction log in a transaction log directory. By default these two directories are the same. The server can (and should) be configured to store the transaction log files in a separate directory than the data files. Throughput increases and latency decreases when transaction logs reside on a dedicated log devices.
ZooKeeper 将其数据存储在数据目录中,将其事务日志存储在事务日志目录中。默认情况下这两个目录是相同的。服务器可以(并且应该)配置为将事务日志文件存储在与数据文件不同的目录中。当事务日志驻留在专用日志设备上时,吞吐量会增加,延迟会减少。

The Data Directory 数据目录

This directory has two or three files in it:
该目录中有两个或三个文件:

Each ZooKeeper server has a unique id. This id is used in two places: the myid file and the configuration file. The myid file identifies the server that corresponds to the given data directory. The configuration file lists the contact information for each server identified by its server id. When a ZooKeeper server instance starts, it reads its id from the myid file and then, using that id, reads from the configuration file, looking up the port on which it should listen.
每个 ZooKeeper 服务器都有一个唯一的 id。这个id用在两个地方: myid文件和​​配置文件。 myid文件标识与给定数据目录对应的服务器。配置文件列出了由服务器 ID 标识的每个服务器的联系信息。当 ZooKeeper 服务器实例启动时,它从myid文件中读取其 id,然后使用该 id 从配置文件中读取,查找它应该侦听的端口。

The snapshot files stored in the data directory are fuzzy snapshots in the sense that during the time the ZooKeeper server is taking the snapshot, updates are occurring to the data tree. The suffix of the snapshot file names is the zxid, the ZooKeeper transaction id, of the last committed transaction at the start of the snapshot. Thus, the snapshot includes a subset of the updates to the data tree that occurred while the snapshot was in process. The snapshot, then, may not correspond to any data tree that actually existed, and for this reason we refer to it as a fuzzy snapshot. Still, ZooKeeper can recover using this snapshot because it takes advantage of the idempotent nature of its updates. By replaying the transaction log against fuzzy snapshots ZooKeeper gets the state of the system at the end of the log.
存储在数据目录中的快照文件是模糊快照,因为在 ZooKeeper 服务器拍摄快照期间,数据树正在发生更新。快照文件名的后缀是zxid ,即快照开始时最后提交的事务的 ZooKeeper 事务 ID。因此,快照包括在快照处理期间发生的数据树更新的子集。那么,快照可能不对应于任何实际存在的数据树,因此我们将其称为模糊快照。尽管如此,ZooKeeper 仍可以使用此快照进行恢复,因为它利用了其更新的幂等性质。通过根据模糊快照重放事务日志,ZooKeeper 在日志末尾获取系统的状态。

The Log Directory 日志目录

The Log Directory contains the ZooKeeper transaction logs. Before any update takes place, ZooKeeper ensures that the transaction that represents the update is written to non-volatile storage. A new log file is started when the number of transactions written to the current log file reaches a (variable) threshold. The threshold is computed using the same parameter which influences the frequency of snapshotting (see snapCount above). The log file's suffix is the first zxid written to that log.
日志目录包含 ZooKeeper 事务日志。在进行任何更新之前,ZooKeeper 确保将代表更新的事务写入非易失性存储。当写入当前日志文件的事务数量达到(可变)阈值时,将启动新的日志文件。使用影响快照频率的相同参数计算阈值(请参见上面的 snapCount)。日志文件的后缀是写入该日志的第一个 zxid。

File Management 文件管理

The format of snapshot and log files does not change between standalone ZooKeeper servers and different configurations of replicated ZooKeeper servers. Therefore, you can pull these files from a running replicated ZooKeeper server to a development machine with a stand-alone ZooKeeper server for trouble shooting.
快照和日志文件的格式在独立 ZooKeeper 服务器和不同配置的复制 ZooKeeper 服务器之间不会改变。因此,您可以将这些文件从正在运行的复制 ZooKeeper 服务器拉取到具有独立 ZooKeeper 服务器的开发计算机,以进行故障排除。

Using older log and snapshot files, you can look at the previous state of ZooKeeper servers and even restore that state. The LogFormatter class allows an administrator to look at the transactions in a log.
使用较旧的日志和快照文件,您可以查看 ZooKeeper 服务器以前的状态,甚至可以恢复该状态。 LogFormatter 类允许管理员查看日志中的事务。

The ZooKeeper server creates snapshot and log files, but never deletes them. The retention policy of the data and log files is implemented outside of the ZooKeeper server. The server itself only needs the latest complete fuzzy snapshot, all log files following it, and the last log file preceding it. The latter requirement is necessary to include updates which happened after this snapshot was started but went into the existing log file at that time. This is possible because snapshotting and rolling over of logs proceed somewhat independently in ZooKeeper. See the maintenance section in this document for more details on setting a retention policy and maintenance of ZooKeeper storage.
ZooKeeper 服务器创建快照和日志文件,但从不删除它们。数据和日志文件的保留策略是在 ZooKeeper 服务器外部实现的。服务器本身只需要最新的完整模糊快照、其后的所有日志文件以及其前的最后一个日志文件。后一个要求必须包含在此快照启动后发生但当时进入现有日志文件的更新。这是可能的,因为日志的快照和滚动在 ZooKeeper 中某种程度上是独立进行的。有关设置保留策略和维护 ZooKeeper 存储的更多详细信息,请参阅本文档中的维护部分。

Note 笔记

The data stored in these files is not encrypted. In the case of storing sensitive data in ZooKeeper, necessary measures need to be taken to prevent unauthorized access. Such measures are external to ZooKeeper (e.g., control access to the files) and depend on the individual settings in which it is being deployed.
这些文件中存储的数据未加密。当ZooKeeper中存储敏感数据时,需要采取必要的措施来防止未经授权的访问。这些措施是 ZooKeeper 的外部措施(例如,控制对文件的访问),并且取决于部署它的各个设置。

Recovery - TxnLogToolkit 恢复 - TxnLogToolkit

TxnLogToolkit is a command line tool shipped with ZooKeeper which is capable of recovering transaction log entries with broken CRC.
TxnLogToolkit 是 ZooKeeper 附带的命令行工具,能够恢复带有损坏 CRC 的事务日志条目。

Running it without any command line parameters or with the -h,--help argument, it outputs the following help page:
在没有任何命令行参数或使用-h,--help参数的情况下运行它,它会输出以下帮助页面:

$ bin/zkTxnLogToolkit.sh
usage: TxnLogToolkit [-dhrv] txn_log_file_name
-d,--dump      Dump mode. Dump all entries of the log file. (this is the default)
-h,--help      Print help message
-r,--recover   Recovery mode. Re-calculate CRC for broken entries.
-v,--verbose   Be verbose in recovery mode: print all entries, not just fixed ones.
-y,--yes       Non-interactive mode: repair all CRC errors without asking

The default behaviour is safe: it dumps the entries of the given transaction log file to the screen: (same as using -d,--dump parameter)
默认行为是安全的:它将给定事务日志文件的条目转储到屏幕:(与使用-d,--dump参数相同)

$ bin/zkTxnLogToolkit.sh log.100000001
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
4/5/18 2:15:58 PM CEST session 0x16295bafcc40000 cxid 0x0 zxid 0x100000001 createSession 30000
CRC ERROR - 4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null
4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null
4/5/18 2:16:12 PM CEST session 0x26295bafcc90000 cxid 0x0 zxid 0x100000003 createSession 30000
4/5/18 2:17:34 PM CEST session 0x26295bafcc90000 cxid 0x0 zxid 0x200000001 closeSession null
4/5/18 2:17:34 PM CEST session 0x16295bd23720000 cxid 0x0 zxid 0x200000002 createSession 30000
4/5/18 2:18:02 PM CEST session 0x16295bd23720000 cxid 0x2 zxid 0x200000003 create '/andor,#626262,v{s{31,s{'world,'anyone}}},F,1
EOF reached after 6 txns.

There's a CRC error in the 2nd entry of the above transaction log file. In dump mode, the toolkit only prints this information to the screen without touching the original file. In recovery mode (-r,--recover flag) the original file still remains untouched and all transactions will be copied over to a new txn log file with ".fixed" suffix. It recalculates CRC values and copies the calculated value, if it doesn't match the original txn entry. By default, the tool works interactively: it asks for confirmation whenever CRC error encountered.
上述事务日志文件的第二个条目中存在 CRC 错误。在转储模式下,工具包仅将此信息打印到屏幕上,而不触及原始文件。在恢复模式( -r,--recover标志)下,原始文件仍然保持不变,所有事务将被复制到带有“.fixed”后缀的新 txn 日志文件中。如果它与原始 txn 条目不匹配,它会重新计算 CRC 值并复制计算值。默认情况下,该工具以交互方式工作:每当遇到 CRC 错误时,它都会要求确认。

$ bin/zkTxnLogToolkit.sh -r log.100000001
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
CRC ERROR - 4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null
Would you like to fix it (Yes/No/Abort) ?

Answering Yes means the newly calculated CRC value will be outputted to the new file. No means that the original CRC value will be copied over. Abort will abort the entire operation and exits. (In this case the ".fixed" will not be deleted and left in a half-complete state: contains only entries which have already been processed or only the header if the operation was aborted at the first entry.)
回答“是”表示新计算的 CRC 值将输出到新文件中。意味着原始 CRC 值将被复制。 Abort将中止整个操作并退出。 (在这种情况下,“.fixed”将不会被删除并处于半完成状态:仅包含已处理的条目,或者如果操作在第一个条目处中止,则仅包含标头。)

$ bin/zkTxnLogToolkit.sh -r log.100000001
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
CRC ERROR - 4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null
Would you like to fix it (Yes/No/Abort) ? y
EOF reached after 6 txns.
Recovery file log.100000001.fixed has been written with 1 fixed CRC error(s)

The default behaviour of recovery is to be silent: only entries with CRC error get printed to the screen. One can turn on verbose mode with the -v,--verbose parameter to see all records. Interactive mode can be turned off with the -y,--yes parameter. In this case all CRC errors will be fixed in the new transaction file.
恢复的默认行为是静默:只有带有 CRC 错误的条目才会打印到屏幕上。可以使用-v,--verbose参数打开详细模式来查看所有记录。可以使用-y,--yes参数关闭交互模式。在这种情况下,所有 CRC 错误都将在新的事务文件中修复。

Things to Avoid 要避免的事情

Here are some common problems you can avoid by configuring ZooKeeper correctly:
以下是通过正确配置 ZooKeeper 可以避免的一些常见问题:

Best Practices 最佳实践

For best results, take note of the following list of good Zookeeper practices:
为了获得最佳结果,请注意以下 Zookeeper 良好实践列表:

For multi-tenant installations see the section detailing ZooKeeper "chroot" support, this can be very useful when deploying many applications/services interfacing to a single ZooKeeper cluster.
对于多租户安装,请参阅详细介绍 ZooKeeper“chroot”支持的部分,这在部署连接到单个 ZooKeeper 集群的许多应用程序/服务时非常有用。