Snapshots/Rollback with Snapper on OpenSuse 13.2

Snapshots/Rollback with Snapper

Submitted by tbazant on Thu, 11/07/2013 - 15:38

Being able to do file system snapshots on Linux providing the ability to do rollbacks is a feature that was often requested in the past. Snapper, in conjunction with the Btrfs file system now fills that gap.

Btrfs, a new copy-on-write file system for Linux, supports file system snapshots (a copy of the state of a subvolume at a certain point of time) of subvolumes (one or more separately mountable file systems within each physical partition). Snapper lets you manage these snapshots. Snapper comes with a command line and a YaST interface.

By default Snapper and Btrfs on openSUSE are set up to serve as an “undo tool” for system changes made with YaST and zypper. Before and after running a YaST module or zypper, a snapshot is created. Snapper lets you compare the two snapshots and provides means to revert the differences between the two snapshots. The tools also provide system backups by creating hourly snapshots of the system subvolumes.

4.1 Requirements #

Since Btrfs is the only filesystem on openSUSE supporting snapshots, it is required on all partitions or subvolumes you want to “snapshot”.

4.1.1 snapshots and Disk Space #

When a snapshot is created, both the snapshot and the original point to the same blocks in the file system. So, initially a snapshot does not occupy additional disk space. If data in the original file system is modified, changed data blocks are copied while the old data blocks are kept for the snapshot. Therefore, a snapshot occupies the same amount of space as the data modified. So, over time, the amount of space a snapshot allocates, constantly grows. As a consequence, deleting files from a Btrfs file system containing snapshots may not free disk space!

Note: Snapshot Location

Snapshots always reside on the same partition or subvolume that has been snapshotted. It is not possible to store snapshots on a different partition or subvolume.

As a result, partitions containing snapshots need to be larger than “normal” partitions. The exact amount strongly depends on the number of snapshots you keep and the amount of data modifications. As a rule of thumb you should consider using twice the size than you normally would.

Tip: Freeing space / Disk Usage

In order to free space on a Btrfs partition containing snapshots you need to delete unneeded snapshots rather than files. Older snapshots occupy more space than recent ones.

Since the df does not show the correct disk usage on Btrfs file systems, you need to use the command btrfs filesystem df MOUNT_POINT. Displaying the amount of disk space a snapshot allocates is currently not supported by the Btrfs tools.

4.2 Using Snapper to Undo YaST and zypper Changes #

If you set up the root partition with Btrfs during the installation, Snapper—pre-configured for doing rollbacks of YaST or zypper changes—will automatically be installed. Every time you start a YaST module or a zypper transaction, two snapshots are created: a “pre-snapshot” capturing the state of the filesystem before the start of the module and a “post-snapshot” after the module has been finished.

Using the YaST Snapper module or the snapper command line tool, you can undo the changes made by YaST/zypper by restoring files from the “pre-snapshot”. Comparing two snapshots the tools also allow you to see which files have been changed. You can also display the differences between two versions of a file (diff).

Since Linux is a multitasking system, processes other than YaST or zypper may modify data in the timeframe between the pre- and the post-snapshot. If this is the case, completely reverting to the pre-snapshot will also undo these changes by other processes. In most cases this would be unwanted—therefore it is strongly recommended to closely review the changes between two snapshots before starting the rollback. If there are changes from other processes you want to keep, select which files to roll back.

Important: Limitations

Make sure you know about Snapper's limitations before attempting to use its rollback mechanism. See Section 4.7, “Limitations” for details.

Note: Storage Time of Snapshots

By default, the last 100 YaST and zypper snappshots are being kept. If this number is exceeded, the oldest snapshot(s) will be deleted.

Procedure 4.1: Undoing changes using the YaST Snapper module #

1. Start the Snapper module from the Miscellaneous section in YaST or by entering yast2 snapper.
2. Make sure Current Configuration is set to root. This is always the case unless you have manually added own Snapper configurations.
3. Choose a pair of pre- and post-snapshots from the list. Both, YaST and zypper snapshot pairs are of the type Pre & Post. YaST snapshots are labeled as yast module_name in the Description column; zypper snapshots are labeled zypp (zypper).
   
4. Click Show Changes to open the list of files that differ between the two snapshots. The following image shows a list of files that have changed after having added the user tester.
   
5. Review the list of files. To display a “diff” between the pre- and post-version of a file, select it from the list. The following images shows the changes to /etc/passwd after having added the user tester.
   
6. To restore a set of files, select the relevant files or directories by ticking the respective checkbox. Click Restore Selected and confirm the action by clicking Yes.
   
   To restore a single file, activate its diff view by clicking on its name. Click Restore From First and confirm your choice with Yes.

Procedure 4.2: Undoing changes using the snapper command #

1. Get a list of yast and zypper snapshots by running snapper list -t pre-post. YaST snapshots are labeled as yast module_name in theDescription column; zypper snapshots are labeled zypp (zypper).

   ~ # snapper list -t pre-post
   Pre # | Post # | Pre Date                 | Post Date                | Description
   ------+--------+--------------------------+--------------------------+----------------------+
   4     | 5      | Tue Jan 10 14:39:14 2012 | Tue Jan 10 14:39:33 2012 | yast system_settings
   65    | 66     | Thu Jan 12 17:18:10 2012 | Thu Jan 12 17:18:23 2012 | zypp(zypper)
   68    | 69     | Thu Jan 12 17:25:46 2012 | Thu Jan 12 17:27:09 2012 | zypp(zypper)
   73    | 74     | Thu Jan 12 17:32:55 2012 | Thu Jan 12 17:33:13 2012 | yast system_settings
   75    | 76     | Thu Jan 12 17:33:56 2012 | Thu Jan 12 17:34:42 2012 | yast users
   77    | 92     | Thu Jan 12 17:38:36 2012 | Thu Jan 12 23:13:13 2012 | yast snapper
   83    | 84     | Thu Jan 12 22:10:33 2012 | Thu Jan 12 22:10:39 2012 | zypp(zypper)
   85    | 86     | Thu Jan 12 22:16:58 2012 | Thu Jan 12 22:17:09 2012 | zypp(zypper)
   88    | 89     | Thu Jan 12 23:10:42 2012 | Thu Jan 12 23:10:46 2012 | zypp(zypper)
   90    | 91     | Thu Jan 12 23:11:40 2012 | Thu Jan 12 23:11:42 2012 | zypp(zypper)
   108   | 109    | Fri Jan 13 13:01:06 2012 | Fri Jan 13 13:01:10 2012 | zypp(zypper)
   

2. Get a list of changed files for a snapshot pair with snapper status PRE..POST. Files with content changes are marked with c, files that have been added are marked with + and deleted files are marked with -. The following example shows a snapshot pair for the installation of the package ncftp.

   ~ # snapper status 108..109
   +... /usr/bin/ncftp
   +... /usr/bin/ncftpbatch
   +... /usr/bin/ncftpget
   +... /usr/bin/ncftpls
   [...]
   +... /usr/share/man/man1/ncftpspooler.1.gz
   c... /var/cache/zypp/solv/@System/cookie
   c... /var/cache/zypp/solv/@System/solv
   c... /var/lib/rpm/Basenames
   c... /var/lib/rpm/Dirnames
   c... /var/lib/rpm/Filemd5s
   c... /var/lib/rpm/Group
   c... /var/lib/rpm/Installtid
   c... /var/lib/rpm/Name
   c... /var/lib/rpm/Packages
   c... /var/lib/rpm/Providename
   c... /var/lib/rpm/Provideversion
   c... /var/lib/rpm/Requirename
   c... /var/lib/rpm/Requireversion
   c... /var/lib/rpm/Sha1header
   c... /var/lib/rpm/Sigmd5
   c... /var/lib/zypp/SoftLocks

3. To display the diff for a certain file, run snapper diff PRE..POST FILENAME. If you do not specify FILENAME, a diff for all files will be displayed.

   ~ # snapper diff 108..109 /var/lib/zypp/SoftLocks
   --- /.snapshots/108/snapshot/var/lib/zypp/SoftLocks 2012-01-12 23:15:22.408009164 +0100
   +++ /.snapshots/109/snapshot/var/lib/zypp/SoftLocks 2012-01-13 13:01:08.724009131 +0100
   @@ -1,4 +1,2 @@
   -# zypp::SoftLocksFile generated Thu Jan 12 23:10:46 2012
   -#
   -ncftp
   -#
   +# zypp::SoftLocksFile generated Fri Jan 13 13:01:08 2012
   +##

4. To restore one or more files run snapper -v undochange PRE..POST FILENAMES. If you do not specify a FILENAMES, all changed files will be restored.

   ~ # snapper -v undochange 108..109
   create:0 modify:16 delete:21
   undoing change...
   deleting /usr/share/man/man1/ncftpspooler.1.gz
   deleting /usr/share/man/man1/ncftpput.1.gz
   [...]
   deleting /usr/bin/ncftpls
   deleting /usr/bin/ncftpget
   deleting /usr/bin/ncftpbatch
   deleting /usr/bin/ncftp
   modifying /var/cache/zypp/solv/@System/cookie
   modifying /var/cache/zypp/solv/@System/solv
   modifying /var/lib/rpm/Basenames
   modifying /var/lib/rpm/Dirnames
   modifying /var/lib/rpm/Filemd5s
   modifying /var/lib/rpm/Group
   modifying /var/lib/rpm/Installtid
   modifying /var/lib/rpm/Name
   modifying /var/lib/rpm/Packages
   modifying /var/lib/rpm/Providename
   modifying /var/lib/rpm/Provideversion
   modifying /var/lib/rpm/Requirename
   modifying /var/lib/rpm/Requireversion
   modifying /var/lib/rpm/Sha1header
   modifying /var/lib/rpm/Sigmd5
   modifying /var/lib/zypp/SoftLocks
   undoing change done

4.3 Using Snapper to Restore Files from Hourly Backups #

Apart from the YaST and zypper snapshots, Snapper creates hourly snapshots of the system partition (/). You can use these backup snapshots to restore files that have accidentally been deleted or modified beyond recovery. By making use of Snapper's diff feature you can also find out which modifications have been made at a certain point of time.

Hourly backup snapshots are of the type Single and are marked with the description timeline. To restore files from these snapshots proceed as described in Procedure 4.1, “ Undoing changes using the YaST Snapper module ” or Procedure 4.2, “ Undoing changes using the snapper command ”

Note: Storage Time of Snapshots

By default, the last ten snapshots of each day are kept. Furthermore ten snapshots are kept for the last ten days, months, and years. For details seeExample 4.1, “Example timeline configuration”.

4.4 Managing Snapshots #

With Snapper you can not only create and delete snapshots, but also set and modify the metadata. All Snapper operations are carried out for a certain configuration. If you do not specify a configuration, the default one (root) is used. If you want to manage snapshots for your own configuration (seeSection 4.6, “Creating Snapper Configurations” for details), you need to specify it with the global snapper option -c:

snapper -c myconfig list

For details on snapper refer to its manpage (man 1 snapper).

4.4.1 Snapshot Metadata #

Each snapshot consists of the snapshot itself and some metadata. With Snapper you can set and modify some metadata. The following metadata is available for each snapshot:

• Type: Snapshot type, see Section 4.4.1.1, “Snapshot Types” for details. This data can not be changed.
• Number: Unique number of the snapshot. This data can not be changed.
• Pre Number: Specifies the number of the corresponding pre snapshot. For snapshots of type post only. This data can not be changed.
• Description: A description of the snapshot.
• Userdata: An extended description where you can specify custom data in the form or comma-separated key=value list: reason=testing_stuff, user=tux
• Cleanup-Algorithm: Cleanup-algorithm for the snapshot

4.4.1.1 Snapshot Types #

Snapper knows three different types of snapshots: pre, post and single. Physically they do not differ, but Snapper handles them differently.

pre

Snapshot of a filesystem before a modification. Each pre snapshot has got a corresponding post snapshot. Used e.g. for the automatic YaST/zypper snapshots.

post

Snapshot of a filesystem after a modification. Each post snapshot has got a corresponding pre snapshot. Used e.g. for the automatic YaST/zypper snapshots.

single

Stand-alone snapshot. Used e.g. for the automatic hourly snapshots. This is the default type when crating snapshots.

4.4.1.2 Cleanup-algorithms #

Snapper provides three algorithms to cleanup old snapshots. The algorithms are executed in a daily cron-job. The cleanup-frequency itself is defined in the Snapper configuration for the partition or subvolume (see Section 4.6.1, “Adjusting the Config File” for details).

number

Deletes old snapshots when a certain snapshot count is reached.

timeline

Deletes old snapshots having passed a certain age, but keeps a number of hourly, daily, monthly and yearly snapshots.

empty-pre-post

Deletes pre/post snapshot pairs with empty diffs.

4.4.2 Creating Snapshots #

Creating a snapshot is done by running snapper create. You should always specify a meaningful description with -d in order to later be able to identify its purpose.

snapper create -d "Before the Apache config cleanup"

Creates a stand-alone snapshot (type single) for the default (root) configuration with a description. Because no cleanup-algorithm is specified, the snapshot will never be deleted automatically.

snapper -c home create -d ".-file cleanup in ~tux"

Creates a stand-alone snapshot (type single) for a custom configuration named home with a description. Because no cleanup-algorithm is specified, the snapshot will never be deleted automatically.

snapper -c home create -d "daily HOME backup" -c timeline

Creates a stand-alone snapshot (type single) for a custom configuration named home with a description. The file will automatically be deleted when it meets the criteria specified for the timeline cleanup-algorithm in the configuration.

4.4.3 Modifying Snapshot Metadata #

Snapper allows to modify the description, the cleanup algorithm and the userdata of a snapshot. All other metadata cannot be changed.

snapper modify -c "timeline" 10

Modifies the metadata of snapshot 10 for the default (root) configuration. The cleanup algorithm is set to timeline.

snapper -c home modify -d "daily backup" -c "" 120

Modifies the metadata of snapshot 120 for a custom configuration named home. A new description is set and the cleanup algorithm is unset.

4.4.4 Deleting Snapshots #

To delete a snapshot, you need to know its number. Get it by running snapper list. To delete a snapshot, run snapper delete NUMBER. When deleting a pre snapshot, you should always delete its corresponding post snapshot (and vice-versa).

snapper delete 65

Deletes snapshot 65 for the default (root) configuration.

snapper -c home delete 89 90

Deletes snapshots 89 and 90 for a custom configuration named home.

snapper delete 65
snapper -c myconfig delete 89 90

Tip: Old Snapshots Occupy More Disk Space

If you delete snapshots in order to free space on your harddisk (see Section 4.1.1, “snapshots and Disk Space” for details), make sure to delete old snapshots first. The older a snapshot is, the more disk space it occupies.

Snapshots are also automatically deleted by a daily cron-job. Refer to Section 4.4.1.2, “Cleanup-algorithms” for details.

4.5 Disabling Snapper #

If you have set up the root partition with Btrfs during the installation, Snapper automatically creates hourly snapshots of the system, as well as pre- and post-snapshots for YaST and zypper transactions. Each of these tasks can be disabled as follows:

Disabling Hourly Snapshots

Edit /etc/snapper/configs/root and set TIMELINE_CREATE to no:

TIMELINE_CREATE="no"

Disabling zypper snapshots

Uninstall the package snapper-zypp-plugin

Disabling YaST snapshots

Edit /etc/sysconfig/yast2 and set USE_SNAPPER to no:

USE_SNAPPER="no"

4.6 Creating Snapper Configurations #

The way Snapper behaves is defined in a config file that is specific for each partition or Btrfs subvolume. These config files reside under/etc/snapper/configs/. The default config installed with Snapper for the / directory is named root. It creates and manages the YaST and zypper snapshots as well as the hourly backup snapshot for /.

You may create your own configurations for other partitions formatted with Btrfs or existing subvolumes on a Btrfs partition. In the following example we will set up a Snapper configuration for backing up the webserver data residing on a separate, Btrfs-formatted partition mounted at /srv/www.

You can use either snapper itself or the YaST Snapper module to restore files from these snapshots. In YaST you need to select your Current Configuration, while you need to specify your config for snapper with the global switch -c (e.g. snapper -c myconfig list).

To create a new Snapper configuration, run snapper create-config:

snapper -c www-data1 create-config
   /srv/www2

1	Name of config file.
2	Mount point of the partition or Btrfs subvolume to snapshot.

This command will create a new config file /etc/snapper/config-templates/www-data with reasonable default values (taken from/etc/snapper/config-templates/default).

Tip: Config Defaults

Default values for a new config are taken from /etc/snapper/config-templates/default. To use your own set of defaults, create a copy of this file in the same directory and adjust it to your needs. To use it, specify the -t option with the create-config command:

snapper -c www-data create-config -t my_defaults /srv/www

4.6.1 Adjusting the Config File #

To adjust the config file, you need to modify it with an editor. It contains key/value pairs in the form of key=value. You may only change the value.

SUBVOLUME

Mount point of the partition or subvolume to snapshot. Do not change.

FSTYPE

Filesystem type of the partition. Do not change.

NUMBER_CLEANUP

Defines whether to automatically delete old snapshots when the total snapshot count exceeds a number specified with NUMBER_LIMIT and an age specified with NUMBER_MIN_AGE. Valid values: yes, no

Note: Limit and Age

NUMBER_LIMIT and NUMBER_MIN_AGE are always evaluated both. Snapshots are only deleted when both conditions are met. If you always want to keep a certain number of snapshots regardless of their age, set NUMBER_MIN_AGE to 0. On the other hand, if you do not want to keep snapshots beyond a certain age, set NUMBER_LIMIT to 0.

NUMBER_LIMIT

Defines how many snapshots to keep if NUMBER_CLEANUP is set to yes.

NUMBER_MIN_AGE

Defines the minimum age in seconds a snapshot must have before it can automatically be deleted.

TIMELINE_CREATE

If set to yes, hourly snapshots are created.This is currently the only way to automatically create snapshots, therefore setting it to yes is strongly recommended. Valid values: yes, no

TIMELINE_CLEANUP

Defines whether to automatically delete old snapshots when the snapshot count exceeds a number specified with the TIMELINE_LIMIT_*options and an age specified with TIMELINE_MIN_AGE. Valid values: yes, no

TIMELINE_MIN_AGE

Defines the minimum age in seconds a snapshot must have before it can automatically be deleted.

TIMELINE_LIMIT_HOURLY, TIMELINE_LIMIT_DAILY, TIMELINE_LIMIT_MONTHLY, TIMELINE_LIMIT_YEARLY

Number of snapshots to keep for hour, day, month, year.

Example 4.1: Example timeline configuration #

TIMELINE_CREATE="yes"
TIMELINE_CLEANUP="yes"
TIMELINE_MIN_AGE="1800"
TIMELINE_LIMIT_HOURLY="10"
TIMELINE_LIMIT_DAILY="10"
TIMELINE_LIMIT_MONTHLY="10"
TIMELINE_LIMIT_YEARLY="10"
       

This example configuration enables hourly screenshots which are automatically cleaned up. TIMELINE_MIN_AGE and TIMELINE_LIMIT_* are always evaluated both. In this example, the minimum age of a snapshot, before it can be deleted is set to 30 minutes (1800 seconds). Since we create hourly snapshots, this ensures that only the latest snapshots are kept. If TIMELINE_LIMIT_DAILY is set to not zero, this means that the first screenshot of the day is kept, too.

Snapshots to be Kept #

• Hourly: The last ten snapshots that have been made
• Daily: The first daily snapshot that has been made is kept for the last ten days.
• Monthly: The first snapshot made on the last day of the month is kept for the last ten months.
• Yearly: The first snapshot made on Dec 31st is kept for the last ten years.

4.7 Limitations #

Although being ready for production, Btrfs as well as Snapper are constantly developed further. The following limitations exist at the moment. It is planned to solve these issues in future releases.

4.7.1 Data Consistency #

There is no mechanism to ensure data consistency when creating snapshot. Whenever a file is written (e.g. a database) at the same time the snapshot is created, it will result in a broken or partly written file. Restoring such a file will cause problems. Therefore it is strongly recommended to always closely review the list of changed files and their diffs. Only restore files that really need to belonging to the action you want to roll back.

4.7.2 Reverting User Additions #

Usually /home resides on a separate partition. Such a separate partition is not part of the default configuration for doing YaST rollbacks. Therefore the user's home partition will not be deleted when reverting a user addition using Snapper. It is strongly recommended to use the YaST User and Group Management tool to remove users.

4.7.3 No Rollback on /boot and Boot Loader Changes #

Currently openSUSE cannot boot from Btrfs partitions. Therefore a separate partition for /boot is created upon the installation when using Btrfsfor the system partition. Since /boot does not support snapshots, the following restrictions apply for YaST/zypper rollbacks:

no rollback for any configuration changes on the bootloader

The only file that can be rolled back is the bootloader configuration file in /etc. The main configuration files reside under /boot and cannot be rolled back.

no complete rollback for Kernel installations

The Kernel itself and its initrd are installed in the /boot partition, whereas Kernel modules or sources are installed in /var/lib and /usr/src, respectively. Whatsmore, each Kernel installation also changes the bootloader configuration files in /boot. So whenever you do a rollback that involves undoing a Kernel installation, you need to manually remove the Kernel and its initrd from /boot and adjust the bootloader configuration by removing the boot entry for the Kernel.

4.8 Frequently Asked Questions #

Why does Snapper never show changes in /var/log, /tmp and other directories?

For some directories we decided to disable snapshotting, e.g. /var/log since reverting logs makes searching for problems difficult. To exclude a path from snapshotting we create a subvolume for that path. The following mount points are excluded from snapshotting on openSUSE:

• /opt
• /srv
• /tmp
• /var/crash
• /var/log
• /var/run
• /var/spool
• /var/tmp

Can I boot a Snapshot from the Bootloader?

This is currently not possible. The bootloader on openSUSE currently does not support booting from a Btrfs partition.