Redis基础知识(学习笔记16--持久化 (2))
三. AOF持久化
AOF,Append Only File,是指Redis将每一次的写操作(命令)都以日志的形式记录到一个AOF文件中的持久化技术。当需要恢复内存数据时,将这些写操作重新执行一次,便会恢复到之前的内存数据状态。
3.1 AOF基础配置
(1)AOF的开启
直接修改配置文件的方式
# Please check https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/ for more information. appendonly no
默认情况下AOF持久化是没有开启的,通过修改配置文件中的appendonly的属性为yes。这种方式需要重启生效。
命令的查看修改方式
状态查看的,可通过下面的命令:
> config get appendonly
修改的话
>config set appendonly yes
注意,这个命令只是修改了内存中的配置。这种命令修改的方式,不需要重启生效。
如果要刷新到配置文件中
>config rewrite
(2)文件名的配置
# - appendonly.aof.1.base.rdb as a base file. # - appendonly.aof.1.incr.aof, appendonly.aof.2.incr.aof as incremental files. # - appendonly.aof.manifest as a manifest file. appendfilename "appendonly.aof" ###注意:这只是定义了这类文件的前缀
Redis 7 在这里发生了重大变化。原来只有一个appendonly.aof文件,现在具有了三类多个文件:
*** 基础文件:可以是RDF格式也可以是AOF格式。其存放的内容是由RDB转为AOF当时内存的快照数据。该文件可以有多个。
*** 增量文件:以操作日志形式记录转为AOF后的写入操作。该文件可以有多个。
*** 清单文件:用于维护AOF文件的创建顺序,保障激活时的应用顺序,该文件只有一个。
(3)混合式持久化开启
# Redis can create append-only base files in either RDB or AOF formats. Using # the RDB format is always faster and more efficient, and disabling it is only # supported for backward compatibility purposes. aof-use-rdb-preamble yes
对于基本文件(例如:appendonly.aof.1.base.rdb as a base file) 可以是RDF格式也可以是AOF格式,通过aof-use-rdb-preamble 属性可以选择。其默认值为yes,即默认AOF持久化的基本文件为rdb格式文件,也就是默认采用混合式持久化。
(4)AOF完整配置参数
############################## APPEND ONLY MODE ############################### # By default Redis asynchronously【异步】 dumps the dataset on disk. This mode is # good enough in many applications, but an issue with the Redis process or # a power outage may result into a few minutes of writes lost (depending on # the configured save points).###概况说RDB存在的风险 # # The Append Only File is an alternative【可供替代的】 persistence mode that provides # much better durability【持久性】. For instance using the default data fsync policy # (see later in the config file) Redis can lose just one second of writes in a # dramatic【巨大的,引人注目的】 event like a server power outage, or a single write if something # wrong with the Redis process itself happens, but the operating system is # still running correctly.###概况说AOF开启带来的好处 # # AOF and RDB persistence can be enabled at the same time without problems. # If the AOF is enabled on startup Redis will load the AOF, that is the file # with the better durability guarantees. # # Note that changing this value in a config file of an existing database and # restarting the server can lead to data loss. A conversion needs to be done # by setting it via CONFIG command on a live server first. # # Please check https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/ for more information. appendonly no # The base name of the append only file. # # Redis 7 and newer use a set of append-only files to persist the dataset # and changes applied to it. There are two basic types of files in use: # # - Base files, which are a snapshot representing the complete state of the # dataset at the time the file was created. Base files can be either in # the form of RDB (binary serialized) or AOF (textual commands). # - Incremental files, which contain additional commands that were applied # to the dataset following the previous file. # # In addition, manifest【货单;清单】 files are used to track the files and the order in # which they were created and should be applied. # # Append-only file names are created by Redis following a specific pattern. # The file name's prefix is based on the 'appendfilename' configuration # parameter, followed by additional information about the sequence【序号】 and type【类型】. # # For example, if appendfilename is set to appendonly.aof, the following file # names could be derived【产生;获得】: # # - appendonly.aof.1.base.rdb as a base file.【大多数情况下,这个类型的文件就一个;如果内存里面的数据量很大时,也可能会不止一个】 # - appendonly.aof.1.incr.aof, appendonly.aof.2.incr.aof as incremental files.【这个类型的文件会同时有多个】 # - appendonly.aof.manifest as a manifest file. appendfilename "appendonly.aof" # For convenience, Redis stores all persistent append-only files in a dedicated【专门的;特定的】 # directory. The name of the directory is determined by the appenddirname # configuration parameter. appenddirname "appendonlydir" # The fsync() call tells the Operating System to actually write data on disk # instead of waiting for more data in the output buffer. Some OS will really flush # data on disk, some other OS will just try to do it ASAP. # # Redis supports three different modes: # # no: don't fsync, just let the OS flush the data when it wants. Faster. # always: fsync after every write to the append only log. Slow, Safest. # everysec: fsync only one time every second. Compromise. # # The default is "everysec", as that's usually the right compromise between # speed and data safety. It's up to you to understand if you can relax this to # "no" that will let the operating system flush the output buffer when # it wants, for better performances (but if you can live with the idea of # some data loss consider the default persistence mode that's snapshotting), # or on the contrary, use "always" that's very slow but a bit safer than # everysec. # # More details please check the following article: # http://antirez.com/post/redis-persistence-demystified.html # # If unsure, use "everysec". # appendfsync always appendfsync everysec # appendfsync no # When the AOF fsync policy is set to always or everysec, and a background # saving process (a background save or AOF log background rewriting) is # performing a lot of I/O against the disk, in some Linux configurations # Redis may block too long on the fsync() call. Note that there is no fix for # this currently, as even performing fsync in a different thread will block # our synchronous write(2) call. # # In order to mitigate this problem it's possible to use the following option # that will prevent fsync() from being called in the main process while a # BGSAVE or BGREWRITEAOF is in progress. # # This means that while another child is saving, the durability of Redis is # the same as "appendfsync no". In practical terms, this means that it is # possible to lose up to 30 seconds of log in the worst scenario (with the # default Linux settings). # # If you have latency problems turn this to "yes". Otherwise leave it as # "no" that is the safest pick from the point of view of durability. no-appendfsync-on-rewrite no # Automatic rewrite of the append only file. # Redis is able to automatically rewrite the log file implicitly calling【隐式调用】 # BGREWRITEAOF when the AOF log size grows by the specified percentage【指定的百分比】. # # This is how it works: Redis remembers the size of the AOF file after the # latest rewrite (if no rewrite has happened since the restart, the size of # the AOF at startup is used). # # This base size is compared to the current size. If the current size is # bigger than the specified percentage, the rewrite is triggered. Also # you need to specify a minimal size for the AOF file to be rewritten, this # is useful to avoid rewriting the AOF file even if the percentage increase # is reached but it is still pretty small. # # Specify a percentage of zero in order to disable the automatic AOF # rewrite feature. auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb # An AOF file may be found to be truncated at the end during the Redis # startup process, when the AOF data gets loaded back into memory. # This may happen when the system where Redis is running # crashes, especially when an ext4 filesystem is mounted without the # data=ordered option (however this can't happen when Redis itself # crashes or aborts but the operating system still works correctly). # # Redis can either exit with an error when this happens, or load as much # data as possible (the default now) and start if the AOF file is found # to be truncated at the end. The following option controls this behavior. # # If aof-load-truncated is set to yes, a truncated AOF file is loaded and # the Redis server starts emitting a log to inform the user of the event. # Otherwise if the option is set to no, the server aborts with an error # and refuses to start. When the option is set to no, the user requires # to fix the AOF file using the "redis-check-aof" utility before to restart # the server. # # Note that if the AOF file will be found to be corrupted in the middle # the server will still exit with an error. This option only applies when # Redis will try to read more data from the AOF file but not enough bytes # will be found. aof-load-truncated yes # Redis can create append-only base files in either RDB or AOF formats. Using # the RDB format is always faster and more efficient, and disabling it is only # supported for backward compatibility purposes. aof-use-rdb-preamble yes # Redis supports recording timestamp annotations in the AOF to support restoring # the data from a specific point-in-time. However, using this capability changes # the AOF format in a way that may not be compatible with existing AOF parsers. aof-timestamp-enabled no ################################ SHUTDOWN #####################################
3.2 AOF 文件格式
AOF文件包含三类文件:基本文件、增量文件与清单文件。其中基本文件一般为rdb格式,不再赘述,下面主要看下增量文件和清单文件的内容格式。
(1)Redis 协议
增量文件扩展名为.aof,采用AOF格式。AOF格式其实就是redis通讯协议格式,AOF持久化文件的本质就是基于redis通讯协议的文本,将命令以纯文本的方式写入到文件中。
redis协议规定,redis文本是以行来划分,每行以\r\n行结束。每一行都有一个消息头,以表示消息类型。消息头由六种不同的符号表示,其意义如下:
- (+)表示一个正确的状态消息;
- (-)表示一个错误的消息;
- (*)表示消息体总共有多少行,不包括当前行;
- ($)表示下一行消息数据的长度,不包括换行符长度\r\n;
- (空)表示一个消息数据;
- (:)表示返回一个数值。
(2)查看AOF文件
打开appendonly.aof.1.incr.aof文件,可以看到格式如下:
(3)清单文件
打开appendonly.aof.manifest文件,其格式如下:
tpye有两种类型:b(base file) 和 i(incremental file)。
该文件首先会按照seq序号列举出所有基本文件,基本文件type类型为b,然后再按照seq序号再列举出所有增量文件,增量文件type为i。
对redis启动时的数据恢复,也会按照该文件从上到下依次加载他们中的数据。
3.3 Rewrite 机制
随着使用时间的推移,AOF文件会越来越大。为了防止AOF文件由于太大而占用大量的磁盘空间,降低性能,redis引入了rewrite机制来对AOF文件进行压缩。
(1) rewrite的定义
所谓rewrite其实就是对AOF文件进行重写整理。当rewrite开启后,主进程redis-server创建出一个子进程bgrewriteaof,由该子进程完成rewrite过程。其首先对现有aof文件进行rewrite计算,将计算结果写入到一个临时文件,写入完毕后,再rename该临时文件为原aof文件名,覆盖原有文件。
(2)rewrite计算
rewrite计算也称为rewrite策略。rewrite计算遵循以下策略:
- 读操作命令不写入文件;
- 无效命令不写入文件;
- 过期数据不写入文件;
- 多条命令合并写入文件。
(3)手动开启rewrite
rewrite过程的执行有两种方式。一种是通过bgrewriteaof命令手动开启,一种是通过设置条件自动开启。
手动启动(触发)的命令:
> bgrewriteaof
该命令会使主进程redis-server创建出一个子进程bgrewriteaof,由该子进程完成rewrite过程。而在rewrite期间,redis-server仍是可以对外提供读写服务的 。
(4)自动开启rewrite
其实就是配置文件中的参数,如下:
# Automatic rewrite of the append only file. # Redis is able to automatically rewrite the log file implicitly calling # BGREWRITEAOF when the AOF log size grows by the specified percentage. # # This is how it works: Redis remembers the size of the AOF file after the # latest rewrite (if no rewrite has happened since the restart, the size of # the AOF at startup is used). # # This base size is compared to the current size. If the current size is # bigger than the specified percentage, the rewrite is triggered. Also # you need to specify a minimal size for the AOF file to be rewritten, this # is useful to avoid rewriting the AOF file even if the percentage increase # is reached but it is still pretty small. # # Specify a percentage of zero in order to disable the automatic AOF # rewrite feature. auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb
学习参阅特别声明
【Redis视频从入门到高级】
【https://www.bilibili.com/video/BV1U24y1y7jF?p=11&vd_source=0e347fbc6c2b049143afaa5a15abfc1c】
热门相关:恐怖复苏 我的女友不可能是怪物 嫡女谋:逆天三小姐 重生嫡女谋天下 性观战辅导