7. Backups

7.1. Backups of the cloud

Regular backups of the OpenStack control plane and databases are taken and exported to off-site backups in a secondary data centre in West Cambridge, for recovery of the OpenStack control plane in the scenario that the three redundant OpenStack controllers and the databases they host are irreparably damaged.

7.2. Backups in the cloud

Conversely, backups of user data belonging to customers who may be using the cloud service is generally not done automatically or programatically by the service itself, except in the case of replicating users’ own backups to an off-site (see below section on ‘openstack volume backup’)

Higher degrees of data resilience are afforded by the local hypervisor disks for instance’s root disks ( where these are co-located on their compute node) being in a RAID1+0 array. Instance root disks or secondary volumes attached as Cinder block storage volumes are allocated either from a Ceph storage cluster with three-way object replication or as NexentaStor volumes carved from ZFS pools constructed from tray-redundant RAID6 groups.

This high degree of resilience is not in and of itself a backup for user data volumes on the cloud service, and where users desire to make their own backups (on-site or off-site) of important data there are ways to achieve this on the cloud service.

Options for user-operated backups on distinct physical storage are possible for Cinder volumes using either the “openstack volume backup” API calls or by manually attaching a block storage volume from a physically separate storage backend and using this as a backup filesystem target (for e.g. rsync’ed backups over SFTP, etc).

Ceph object storage can also be used as a backup target, bearing in mind that the physical storage backing the object store is the same storage as providing the primary Ceph-based block storage volume backend, and so backing up in this fashion would achieve extra replication but not physically separate backup.

Volume storage backends available and their status are;

  • West Cambridge Data Centre - Ceph (Primary, replicated but not backed up)
  • West Cambridge Data Centre - NexentaStor (Secondary, offered on request for volume backups)
  • Soulsby Data Centre - NexentaStor (Secondary, offered on request for volume backups)

7.3. Backing up with “openstack volume backup”

The “openstack volume backup” API calls make a block-level copy of a Cinder block device and store it on the WCDC NexentaStor. These block volumes are replicated to the Soulsby NexentaStor regularly as a scheduled automatic replication job and so users utilising this method would only need to schedule and manage volume backups, and replication to an off-site will be handled automatically.

“openstack volume backup” will only process a backup call to a volume currently mounted to an instance if invoked with the “–force” flag. Using this flag would pose a relatively high risk of data or filesystem corruption to both the originating volume and its backup, so volumes should generally be detached from a running instance before being backed up with “openstack volume backup”, if a block-level copy of the volume with this method is desired.