pre/post snapshot polishing. still needs test-scripts

This commit is contained in:
Edwin Eefting 2021-06-18 17:16:18 +02:00
parent 1286bfafd0
commit 84437ee1d0
3 changed files with 31 additions and 19 deletions

@ -279,19 +279,6 @@ root@ws1:~# zfs-autobackup test --verbose
This also allows you to make several snapshots during the day, but only backup the data at night when the server is not busy.
## Running custom commands before and after snapshotting
If you need, e.g. to quiesce a couple of mysql databases to make on-disk data consistent before snapshotting, try:
```sh
zfs-autobackup \
--pre-snapshot-cmd 'daemon -f jexec mysqljail1 mysql -s -e "set autocommit=0;flush logs;flush tables with read lock;\\! echo \$\$ > /tmp/mysql_lock.pid && sleep 60"' \
--pre-snapshot-cmd 'daemon -f jexec mysqljail2 mysql -s -e "set autocommit=0;flush logs;flush tables with read lock;\\! echo \$\$ > /tmp/mysql_lock.pid && sleep 60"' \
--post-snapshot-cmd 'pkill -F /jails/mysqljail1/tmp/mysql_lock.pid' \
--post-snapshot-cmd 'pkill -F /jails/mysqljail2/tmp/mysql_lock.pid' \
test
```
## Thinning out obsolete snapshots
The thinner is the thing that destroys old snapshots on the source and target.
@ -430,6 +417,25 @@ zfs send -> send buffer -> custom send pipes -> compression -> transfer rate lim
#### On the receiving side:
decompression -> custom recv pipes -> buffer -> zfs recv
## Running custom commands before and after snapshotting
If you need, e.g. to quiesce a couple of mysql databases to make on-disk data consistent before snapshotting, you can use the `--pre-snapshot-cmd` and `--post-snapshot-cmd` options.
For example:
```sh
zfs-autobackup \
--pre-snapshot-cmd 'daemon -f jexec mysqljail1 mysql -s -e "set autocommit=0;flush logs;flush tables with read lock;\\! echo \$\$ > /tmp/mysql_lock.pid && sleep 60"' \
--pre-snapshot-cmd 'daemon -f jexec mysqljail2 mysql -s -e "set autocommit=0;flush logs;flush tables with read lock;\\! echo \$\$ > /tmp/mysql_lock.pid && sleep 60"' \
--post-snapshot-cmd 'pkill -F /jails/mysqljail1/tmp/mysql_lock.pid' \
--post-snapshot-cmd 'pkill -F /jails/mysqljail2/tmp/mysql_lock.pid' \
backupfs1
```
The post-snapshot commands are ALWAYS executed, even if a pre-snapshot command or the actual snapshot fail.
A failure of a post-snapshot command is non-fatal and will be ignored. The remaining post-snapshot commands wont be executed in that case.
## Tips
* Use ```--debug``` if something goes wrong and you want to see the commands that are executed. This will also stop at the first error.

@ -511,8 +511,8 @@ class ZfsAutobackup:
self.set_title("Snapshotting")
source_node.consistent_snapshot(source_datasets, source_node.new_snapshotname(),
min_changed_bytes=self.args.min_change,
pre_snapshot_cmd=self.args.pre_snapshot_cmd,
post_snapshot_cmd=self.args.post_snapshot_cmd)
pre_snapshot_cmds=self.args.pre_snapshot_cmd,
post_snapshot_cmds=self.args.post_snapshot_cmd)
################# sync
# if target is specified, we sync the datasets, otherwise we just thin the source. (e.g. snapshot mode)

@ -162,7 +162,7 @@ class ZfsNode(ExecuteNode):
"""determine uniq new snapshotname"""
return self.backup_name + "-" + time.strftime("%Y%m%d%H%M%S")
def consistent_snapshot(self, datasets, snapshot_name, min_changed_bytes, pre_snapshot_cmd=[], post_snapshot_cmd=[]):
def consistent_snapshot(self, datasets, snapshot_name, min_changed_bytes, pre_snapshot_cmds=[], post_snapshot_cmds=[]):
"""create a consistent (atomic) snapshot of specified datasets, per pool.
"""
@ -193,7 +193,7 @@ class ZfsNode(ExecuteNode):
return
try:
for cmd in pre_snapshot_cmd:
for cmd in pre_snapshot_cmds:
self.verbose("Running pre-snapshot-cmd")
self.run(cmd=shlex.split(cmd), readonly=False)
@ -207,9 +207,15 @@ class ZfsNode(ExecuteNode):
self.run(cmd, readonly=False)
finally:
for cmd in post_snapshot_cmd:
for cmd in post_snapshot_cmds:
self.verbose("Running post-snapshot-cmd")
self.run(cmd=shlex.split(cmd), readonly=False, valid_exitcodes=[])
try:
self.run(cmd=shlex.split(cmd), readonly=False)
except Exception as e:
pass
self.warning("Post snapshot command failed, ignoring.")
def selected_datasets(self, exclude_received, exclude_paths):
"""determine filesystems that should be backed up by looking at the special autobackup-property, systemwide