Katarina

Main page     Download    

NAME

katarina - rsync based multi profile, multi servers backup system


SYNOPSIS

     help mode :
       [-h] [-H]
     backup mode :
       -t backup -n cyclename -p profilename [-m] 
     retry mode :
       -t retry -n cyclename -p profilename [-m]
     verify clients mode :
       -t check -p profilename
     use: -t mode :
        backup : to actualy backup
        check : to verify clients availability
     use: -h : to read this
     use: -H : to read perldoc
     use: -m : report by mail
     use: -s : report on screen
     use: -n : to specify cycle name (daily, weekly...)
     use: -p profilename : to specify $config_dir/profilename.kat


DESCRIPTION

katarina is a perl backup system.

It reads its config in a text file (perl syntax) to determine global parameters a list of targets with specific parameters

Remote data is backed up using rsync. An rsync agent must be running on the remote system and available for the katarina server.

Data directories are rotated using cp -la which means data is deduplicated server by server (not between different servers). Old directories are recycled if possible to avoid re-creation of the last rotated backup.

Global parameters in config file can be :

    active       : 0 or 1 to globaly (de)activate the profile
    mail         : email address of the Katarina administrators
    forks        : number of concurrent backup threads
    backup_dir   : root directory for backups
    premounted   : abort if this filesystem is not mounted in order
                   to not fill wrong partitions
    cycle        : hash of different cycles and the time to keep data
                   before expiration. This can be used to define daily
                   profiles, weekly or much more complicated policies.
                   See EXAMPLES below.
    exclude      : array of remote directories to exclude
    bwlimit      : bandwith limit (in kByte) to be applied to every
                   server
    rmcmd        : rm command. For example : ionice -c 3 /bin/rm
    cpcmd        : cp command. For example : ionice -c 3 /bin/cp
    rsynccmd     : rsync command. For example : ionice -c 3 /usr/local/bin/rsync
    rsyncretries : number of rsync tries if it fails

Specific server parameters in config file can be :

    active     : 0 or 1 to to (de)activate the server
    ip         : IP address (v4 or v6) to use to connect to server if 
                 servername is not an FQDN
    module     : name of the rsync module to use on remote server. If 
                 not specified, it will be "katarina"
    exclude    : array of remote directories to exclude (in addition to 
                 those given in global parameter)
    bwlimit    : bandwith limit (in kByte). Replace the provided global
                 value (if provided)


EXAMPLES

Configuration files should be placed in /etc/katarina

Here is a .kat example configuration file :

    our %globals = (
        "mail"       => "root@example.com",
        "active"     => 1,
        "forks"      => 2,
                # 2 threads will be used to backup in parallel
        "backup_dir" => "/backup/test",
                # directory used to write servers data
        "premounted" => "/backup", 
                # abort if this filesystem is not mounted in order
                # to not fill wrong partitions
        "cycles"  => {
            "daily" => {
                "method" => "rsync",
                "retention" => (7 * 86400) },
                  # Keep up daily backups younger than 7 days
            "weekly" => {
                "method" => "use daily",
                  # Do not rsync to the server for this backup : use
                  # the existing "daily" archives to build last "weekly"
                "retention" => (30 * 86400) },
                  # Keep up weekly backups younger than 30 days
            },
        "exclude" => [
                "lost+found/", "tmp/", "proc/", "sys/",
                "selinux/", "dev/",
                "media/", "lib/init/rw" ],
    );
    our %targets = (
        "server1.example.com" => {
            "active"  => 1,
            "exclude" => [
                    "/backup",
                    ],
        },
        "server2.example.com" => {
            "active"  => 1,
            "ip" => 192.168.42.42,
        },
        "server3.example.com" => {
            "active"  => 0,
              # This one is in maintenance: no backup will be done
        },
        "server4.example.com" => {
            "active"  => 1,
            "ip" => 192.168.42.43,
            "module" => "export",
              # rsyncd.conf on the backuped host specifies "export" as 
              # a rsync module.
        },
    );

For this configuration file the following crons could be used :

    # Daily backup
    0 6 * * * root katarina.pl -t backup -m -p test -n daily
    # Weekly backup
    0 6 * * 0 root katarina.pl -t backup -m -p test -n weekly

Here is a rsyncd.conf configuration that can be used on the backed-up clients :

    pid file=/var/run/rsyncd.pid
    [katarina]
        comment = Katarina Backup
        path = /
        use chroot = no
        lock file = /var/lock/rsyncd
        read only = yes
        list = yes
        uid = root
        gid = root
        ignore errors = no
        ignore nonreadable = no
        transfer logging = no
        timeout = 600
        refuse options = checksum dry-run
        hosts allow = 192.168.0.1

Cycles can be freely defined :

    "cycles"  => {
        "minutly" => {
            "method" => "rsync",
            "retention" => 60 },
              # Keep up daily backups younger than 1 minute
        "what_I_want" => {
            "method" => "use minutly",
              # Use the las "minutly" backup to generate the last
              # "what_I_want" backup
            "retention" => (42 * 60) },
              # Keep up backups younger than 42 minutes
        },


AUTHOR

Cyril Bellot <jcpp@users.sourceforge.net>

Last news:

zfs snapshots incoming
on Mon, 13 Jan 2014 13:29:16 -0000

A current beta version is running using zfs snapshots. In theory it will possible to use other filesystems with snapshots systems.



New release : 1.8
on Fri, 19 Apr 2013 14:43:11 -0000

V1.8 is out including a new retry mode and fixes



New release : 1.7
on Thu, 11 Oct 2012 21:34:12 -0000
V1.7 is out including a new \(smart\) algorythm for scheduler, retries for rsync commands, fixes

New release : v1.6
on Mon, 28 May 2012 09:56:23 -0000
V1.6 is out including a scheduler, retry scripts, more information in check mode and some fixes

New release : v1.5
on Wed, 18 Jan 2012 12:37:56 -0000
V1.5 is out including statistics improvements; munin plugins have been added.

New release : v1.4
on Wed, 04 Jan 2012 10:20:56 -0000
V1.4 is out. Fixed perl warnings "Using a hash as a reference is deprecated"

New release : v1.3
on Wed, 21 Sep 2011 12:45:56 -0000
V1.3 is out. You can have a volume checked if mounted before backup.

New release : v1.2
on Mon, 18 Jul 2011 15:37:51 -0000
V1.2 is out with a few stats and error handling fixes

New release : v1.1
on Wed, 08 Jun 2011 14:31:43 -0000
V1.1 is out with a few bug fixes

Projet registered
on Fri, 20 May 2011 09:37:04 -0000
Projet has been registered on SF