pre/post snapshot polishing. still needs test-scripts

This commit is contained in:
Edwin Eefting 2021-06-18 17:16:18 +02:00
parent 1286bfafd0
commit 84437ee1d0
3 changed files with 31 additions and 19 deletions

View File

@ -279,19 +279,6 @@ root@ws1:~# zfs-autobackup test --verbose
This also allows you to make several snapshots during the day, but only backup the data at night when the server is not busy. This also allows you to make several snapshots during the day, but only backup the data at night when the server is not busy.
## Running custom commands before and after snapshotting
If you need, e.g. to quiesce a couple of mysql databases to make on-disk data consistent before snapshotting, try:
```sh
zfs-autobackup \
--pre-snapshot-cmd 'daemon -f jexec mysqljail1 mysql -s -e "set autocommit=0;flush logs;flush tables with read lock;\\! echo \$\$ > /tmp/mysql_lock.pid && sleep 60"' \
--pre-snapshot-cmd 'daemon -f jexec mysqljail2 mysql -s -e "set autocommit=0;flush logs;flush tables with read lock;\\! echo \$\$ > /tmp/mysql_lock.pid && sleep 60"' \
--post-snapshot-cmd 'pkill -F /jails/mysqljail1/tmp/mysql_lock.pid' \
--post-snapshot-cmd 'pkill -F /jails/mysqljail2/tmp/mysql_lock.pid' \
test
```
## Thinning out obsolete snapshots ## Thinning out obsolete snapshots
The thinner is the thing that destroys old snapshots on the source and target. The thinner is the thing that destroys old snapshots on the source and target.
@ -430,6 +417,25 @@ zfs send -> send buffer -> custom send pipes -> compression -> transfer rate lim
#### On the receiving side: #### On the receiving side:
decompression -> custom recv pipes -> buffer -> zfs recv decompression -> custom recv pipes -> buffer -> zfs recv
## Running custom commands before and after snapshotting
If you need, e.g. to quiesce a couple of mysql databases to make on-disk data consistent before snapshotting, you can use the `--pre-snapshot-cmd` and `--post-snapshot-cmd` options.
For example:
```sh
zfs-autobackup \
--pre-snapshot-cmd 'daemon -f jexec mysqljail1 mysql -s -e "set autocommit=0;flush logs;flush tables with read lock;\\! echo \$\$ > /tmp/mysql_lock.pid && sleep 60"' \
--pre-snapshot-cmd 'daemon -f jexec mysqljail2 mysql -s -e "set autocommit=0;flush logs;flush tables with read lock;\\! echo \$\$ > /tmp/mysql_lock.pid && sleep 60"' \
--post-snapshot-cmd 'pkill -F /jails/mysqljail1/tmp/mysql_lock.pid' \
--post-snapshot-cmd 'pkill -F /jails/mysqljail2/tmp/mysql_lock.pid' \
backupfs1
```
The post-snapshot commands are ALWAYS executed, even if a pre-snapshot command or the actual snapshot fail.
A failure of a post-snapshot command is non-fatal and will be ignored. The remaining post-snapshot commands wont be executed in that case.
## Tips ## Tips
* Use ```--debug``` if something goes wrong and you want to see the commands that are executed. This will also stop at the first error. * Use ```--debug``` if something goes wrong and you want to see the commands that are executed. This will also stop at the first error.

View File

@ -511,8 +511,8 @@ class ZfsAutobackup:
self.set_title("Snapshotting") self.set_title("Snapshotting")
source_node.consistent_snapshot(source_datasets, source_node.new_snapshotname(), source_node.consistent_snapshot(source_datasets, source_node.new_snapshotname(),
min_changed_bytes=self.args.min_change, min_changed_bytes=self.args.min_change,
pre_snapshot_cmd=self.args.pre_snapshot_cmd, pre_snapshot_cmds=self.args.pre_snapshot_cmd,
post_snapshot_cmd=self.args.post_snapshot_cmd) post_snapshot_cmds=self.args.post_snapshot_cmd)
################# sync ################# sync
# if target is specified, we sync the datasets, otherwise we just thin the source. (e.g. snapshot mode) # if target is specified, we sync the datasets, otherwise we just thin the source. (e.g. snapshot mode)

View File

@ -162,7 +162,7 @@ class ZfsNode(ExecuteNode):
"""determine uniq new snapshotname""" """determine uniq new snapshotname"""
return self.backup_name + "-" + time.strftime("%Y%m%d%H%M%S") return self.backup_name + "-" + time.strftime("%Y%m%d%H%M%S")
def consistent_snapshot(self, datasets, snapshot_name, min_changed_bytes, pre_snapshot_cmd=[], post_snapshot_cmd=[]): def consistent_snapshot(self, datasets, snapshot_name, min_changed_bytes, pre_snapshot_cmds=[], post_snapshot_cmds=[]):
"""create a consistent (atomic) snapshot of specified datasets, per pool. """create a consistent (atomic) snapshot of specified datasets, per pool.
""" """
@ -193,7 +193,7 @@ class ZfsNode(ExecuteNode):
return return
try: try:
for cmd in pre_snapshot_cmd: for cmd in pre_snapshot_cmds:
self.verbose("Running pre-snapshot-cmd") self.verbose("Running pre-snapshot-cmd")
self.run(cmd=shlex.split(cmd), readonly=False) self.run(cmd=shlex.split(cmd), readonly=False)
@ -207,9 +207,15 @@ class ZfsNode(ExecuteNode):
self.run(cmd, readonly=False) self.run(cmd, readonly=False)
finally: finally:
for cmd in post_snapshot_cmd: for cmd in post_snapshot_cmds:
self.verbose("Running post-snapshot-cmd") self.verbose("Running post-snapshot-cmd")
self.run(cmd=shlex.split(cmd), readonly=False, valid_exitcodes=[]) try:
self.run(cmd=shlex.split(cmd), readonly=False)
except Exception as e:
pass
self.warning("Post snapshot command failed, ignoring.")
def selected_datasets(self, exclude_received, exclude_paths): def selected_datasets(self, exclude_received, exclude_paths):
"""determine filesystems that should be backed up by looking at the special autobackup-property, systemwide """determine filesystems that should be backed up by looking at the special autobackup-property, systemwide