From owner-freebsd-hackers  Mon Aug 25 11:23:42 1997
Return-Path: <owner-freebsd-hackers>
Received: (from root@localhost)
          by hub.freebsd.org (8.8.7/8.8.7) id LAA28973
          for hackers-outgoing; Mon, 25 Aug 1997 11:23:42 -0700 (PDT)
Received: from sendero-ppp.i-connect.net (sendero-ppp.i-Connect.Net [206.190.143.100])
          by hub.freebsd.org (8.8.7/8.8.7) with SMTP id LAA28957
          for <hackers@FreeBSD.ORG>; Mon, 25 Aug 1997 11:23:36 -0700 (PDT)
Received: (qmail 9425 invoked by uid 1000); 25 Aug 1997 18:23:55 -0000
Message-ID: <XFMail.970825112355.Shimon@i-Connect.Net>
X-Mailer: XFMail 1.2-alpha [p0] on FreeBSD
X-Priority: 3 (Normal)
MIME-Version: 1.0
Content-Type: multipart/mixed;
 boundary="_=XFMail.1.2-alpha.p0.FreeBSD:970825111934:1078=_"
In-Reply-To: <199708250808.KAA17675@dante.iol.it>
Date: Mon, 25 Aug 1997 11:23:55 -0700 (PDT)
Organization: Atlas Telecom
From: Simon Shapiro <Shimon@i-Connect.Net>
To: "=?ISO-8859-1?Q?Dott._Davide_Tom=E8?=" <dtome@iol.it>
Subject: RE: multiple backup on one tape
Cc: hackers@FreeBSD.ORG
Sender: owner-freebsd-hackers@FreeBSD.ORG
X-Loop: FreeBSD.org
Precedence: bulk

This message is in MIME format
--_=XFMail.1.2-alpha.p0.FreeBSD:970825111934:1078=_
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit


Hi "Dott. Davide Tomh";  On 25-Aug-97 you wrote: 
>  Hi,
>  I'd like to know how can I have backups of different
>  machines on the same tape with tar, and how to
>  restore them.

Ah, finally something I can contribute :-)

See the attachment for a program to do just that.
It backs up your system, one filesystem at a time, all on one (or
more) tape.

You restore by simply reversing the process.  On some Unixes, you
may get an error in reading so do the following instead:

dd if=/dev/nrst0 bs=64k | ${whatever_archiver_you_use}

For example, to restore the output of dat_bu, NON_DESTRUCTIVELY,

    cd /
    dd if=/dev/nrst0 bs=64k | cpio -H newc -ovdm

Repeat until you see TWO zero size files restored in a row.

BTW, this has nothing to do with FreeBSD, nor with this list the
way I understand it.  -questions would be more appropriate.  No?

Simon

--_=XFMail.1.2-alpha.p0.FreeBSD:970825111934:1078=_
Content-Disposition: attachment; filename="dat_bu"
Content-Transfer-Encoding: none
Content-Description: Multiple volumes backup on one tape
Content-Type: application/octet-stream; name=dat_bu; SizeOnDisk=16438

#!/bin/bash
#set -x
#*****************************************************************************
#                                                                            *
#       Copyright (c) 1991, 1992, 1995 by Simon Shapiro                      *
#       All Rights Reserved                                                  *
#                                                                            *
#       THIS IS UNPUBLISHED PROPRIETARY SOURCE CODE OF                       *
#                        Simon Shapiro                                       *
#                                                                            *
#       The copyright notice above does not evidence any                     *
#       actual or intended publication of such source code.                  *
#                                                                            *
#*****************************************************************************/

#*****************************************************************************
#                                                                            *
#  dat_bu:       Backup on DAT, each file system in its own archive          *
#                                                                            *
#  arguments:    -e           Pre-Erase the tape before use                  *
#                -F fs_type   Specify file system type to backup             *
#                -V           Verify (ask) each file system is indeed to be  *
#                             backed up                                      *
#                -v           Be verbose about your progress                 *
#                -d  dest_dev Specify where the backup is to go              *
#                -z           Compress output (via gzip)                     *
#                -f           Forces backup to continue regardless of errors *
#                -q           Do not say anything at all unless an error     *
#                -b           Background mode on                             *
#                -p p|P       Make a partial backup of all changes since last*
#                             PARTIAL backup                                 *
#                -p c|C       Make a partial backup of all changes since last*
#                             COMPLETE backup                                *
#                -l no_days   Do a partial backup of files that changed in   *
#                             the last ``no_days'' days.                     *
#                -a           Append to end of exiting tape                  *
#                                                                            *
#  returns:      0 if all is well                                            *
#                non-zero error status                                       *
#                                                                            *
#  caveats:      Writes over /etc/.last*                                     *
#                depends on finfo                                            *
#                order of backup depends on output of dfspace(1)             *
#                Crazy filenames can cause problems                          *
#                First filename to be backed-up is the files list.           *
#                Tape block size is 64kb.                                    *
#                                                                            *
#*****************************************************************************/

#ident  "$Header: /usr/sbin/RCS/dat_bu,v 1.8 1997/01/29 18:36:23 ShimonR Exp $"

ask=false
append_tape=false
valid_options="F:mVSaqfevs:d:p:d:zl:"
force_it=false
bad_option=false
pre_erase=false
tape_dest=${BU_DEVICE:-/dev/nrst0}
tape_block=`expr 1024 '*' 64`
tmpfile=/tmp/dat_bu.${$}
root=${BU_FS:=/BackUp}
total_kilobytes=0
total_files=0
total_fs=0
shut_up=false
verbose=false
fs_list=${tmpfile}.fs_list
back_ground=false
do_partial=false
partial_type=nonsense
partial_flag=""
use_snapshots=true
mount_order=false
compress=false
fs_types=""
verbose_cpio=
last_days=0

trap 'do_cleanup; exit 2' 1 2 3 4 5

substr()
{
	local flag pat str
	local usage="usage: substr -lLrR pat string or substr string pat"

	case "$1" in
	-l | -L | -r | -R)
		flag="$1"
		pat="$2"
		shift 2
		;;
	-*)
		echo "substr: unknown option: $1"
		echo "$usage"
		return 1
		;;
	*)
		flag="-r"
		pat="$2"
		;;
	esac

	if [ "$#" -eq 0 -o "$#" -gt 2 ] ; then
		echo "substr: bad argument count"
		return 2
	fi

	str="$1"

	#
	# We don't want -f, but we don't want to turn it back on if
	# we didn't have it already
	#
	case "$-" in
	"*f*")
		;;
	*)
		fng=1
		set -f
		;;
	esac

	case "$flag" in
	-l)
		str="${str#$pat}"		# substr -l pat string
		;;
	-L)
		str="${str##$pat}"		# substr -L pat string
		;;
	-r)
		str="${str%$pat}"		# substr -r pat string
		;;
	-R)
		str="${str%%$pat}"		# substr -R pat string
		;;
	*)
		str="${str%$2}"			# substr string pat
		;;
	esac

	echo "$str"

	#
	# If we had file name generation when we started, re-enable it
	#
	if [ "$fng" = "1" ] ; then
		set +f
	fi
}

function do_cleanup
{
  trap '' 1 2 3 4 5

	echo "Terminated... Exiting..."
  rm -f ${tmpfile}*
  rm -f ${fs_list}*
  rm -f /etc/.lastbackup

  if [ ${first_backup:-x} = true ]
    then
      (mv /etc/OLD.lastbackup /etc/.lastbackup) > /dev/null 2>&1
  fi
      
  if [ ${first_partial:-x} = true ]
    then
      (mv /etc/OLD.lastpartial /etc/.lastbackup) > /dev/null 2>&1
 fi

  if [ ${do_partial} = false ]
   then
      mt -f ${tape_dest} rewind > /dev/null 2>&1
      mt -f ${tape_dest} offline > /dev/null 2>&1 &
  fi
}

function ask_bu
{
  fsn=${1}
  answer=invalid
  while  [ ${answer} = invalid ]
    do
      echo -e "Do you want to backup file system \"${fsn}\" (Y/N)? \c"
      read answer
      case ${answer} in
        y|Y)
          echo true > ${tmpfile}.ask
          ;;
        n|N)
          echo false > ${tmpfile}.ask
          ;;
        *)
          answer=invalid
          ;;
      esac
    done
}

function setup_list
{
  sul_output=${1}; shift

  > ${tmpfile}.mount

  mount | grep '^/dev/[ws]d' | cut -d ' ' -f 1,3 >> ${tmpfile}.mount
  mount | grep '^/dev/ccd' | cut -d ' ' -f 1,3 >> ${tmpfile}.mount
  while read fs_device fs
    do
      echo "${fs} ${fs_device}" >> ${sul_output}
			echo "${fs}" >> /tmp/FS-LIST
    done < ${tmpfile}.mount
}

if [ -f /proc/filesystems ]
  then
    valid_fs_types="`cat /proc/filesystems | cut -d '	' -f 2`"
  else
    valid_fs_types="ufs ffs nfs"
fi

# Parse the command line
while getopts ${valid_options} c
  do
    case ${c} in
      l)
        do_partial=true
        last_days=${OPTARG:-14}
        expr ${last_days} + 1 > /dev/null 2>&1
        if [ ${?} != 0 ]
          then
            echo "${0} ERROR: -l argument must be a decimal number!"
            exit 1
        fi
        ;;
      F)
        optarg_is_valid=false
        for fst in ${valid_fs_types}
          do
            if [ ${OPTARG} = ${fst} ]
              then
                optarg_is_valid=true
                fs_types="${fs_types} ${OPTARG}"
                break
            fi
          done

        if [ ${optarg_is_valid} = false ]
          then
            echo "\"${OPTARG}\" is not a filesystem supported by this kernel!"
            echo "Only \"${valid_fs_types}\" are supported."
            exit 1
        fi
        ;;
      z)
        compress=true
        ;;
      m )
        mount_order=true
        ;;
      V )
        ask=true
        ;;
      p )
        do_partial=true;
        partial_type=${OPTARG}
        if [ ${partial_type:=x} = 'p' -o ${partial_type} = 'P' ]
          then
            partial_type=partial
          else
            if [ ${partial_type} = 'c' -o ${partial_type} = 'C' ]
              then
                partial_type=complete
              else
                echo "Partial backup (-p) can be type c (since last Complete)"
                echo "                             or p (since last Partial)"
                exit 1
            fi
        fi
        ;;
      b )
        back_ground=true
        ;;
      q )
        shut_up=true
        ;;
      f )
        force_it=true
        ;;
      e )
        pre_erase=false
        ;;
      d )
        tape_dest=${OPTARG}
        ;;
      v )
        verbose=true
        verbose_cpio=v
        ;;
      a )
        append_tape=true
        ;;
      \? )
        bad_option=true
        ;;
    esac
  done

if [ ${bad_option} = true ]
  then
    echo "${0} ERROR:  Correct Usage: ${0} [-options]"
    echo
    echo "Options: -V     Verify (ask) intent for each file system"
		echo "         -m     Backup in mount order (default = alphabetically)"
    echo "         -a     Append to end of data (default=start snew)"
    echo "         -q     Be quite, unless errors (default=moderately verbose"
    echo "         -f     Force continuation (default=terminate on I/O errors)"
    echo "         -e     Pre-erase the tape (default=do not pre-erase)"
    echo "         -v     Be verbose about your progress"
    echo "         -d d_d Use device d_d as destination (default=${tape_dest})"
    echo "         -p p_t Partial Backup (default=Complete Backup);"
    echo "                If p_t=p then backup all since last Partial backup"
    echo "                If p_t=c then backup all since last Complete backup"
    echo "         -z     Compress via gzip"
    exit 1
fi

if [ ${fs_types:-x} = x ]
  then
    fs_types="ufs ffs ext2 nfs"
    echo "No file system types specified.  Using \"${fs_types}\"."
fi


if [ ${do_partial} = true ]
  then
    backup_type=Partial
  else
    backup_type=Complete
fi

if [ -f /etc/.lastbackup ]
  then
    first_backup=false

    if [ ${do_partial} = true ]
      then
        partial_flag="-newer /etc/.OLD.lastbackup"
    fi

    rm -f /etc/.OLD.lastbackup
    mv /etc/.lastbackup /etc/.OLD.lastbackup
  else
    first_backup=true
    if [ ${do_partial} = true ]
      then
        echo "Cannot do a partial backup.  No complete backup done before"
        exit 2
      fi
fi

if [ -f /etc/.lastpartial ]
  then
    first_partial=false
    rm -f /etc/.OLD.lastpartial
    mv /etc/.lastpartial /etc/.OLD.lastpartial 

    if [ ${partial_type} = partial ]
      then
        partial_flag="-newer /etc/.OLD.lastpartial"
    fi

  else
    first_partial=true
fi

if [ ${last_days} != 0 ]
  then
    partial_flag="-ctime ${last_days}"
fi

date > /etc/.lastbackup

if [ ${shut_up} = false ]
  then
    echo "${backup_type} system Backup Started on `date`"
fi

rm -f /tmp/FS-LIST
setup_list ${fs_list} ${fs_types}

fslist=`cat /tmp/FS-LIST`
rm -f /tmp/FS-LIST
for fs in ${fslist}
  do
    if [ ${ask} = true ]
      then
        ask_bu ${fs}
        case `cat ${tmpfile}.ask` in
          true)
            doit="Backed-Up"
            ;;
          false)
            doit=Skipped
            ;;
          *)
            doit="Invalid-Response"
        esac
      else
        doit=Backed-Up
    fi

    echo "${fs} ${doit}" >> /tmp/FS-LIST
  done

# Prepare tape drive
if [ ${back_ground} = true ]
  then
    echo
    echo "Install ${backup_type} Backup tape in drive & strike ENTER \c"
    read yes
fi

# Turn DAT compression ON
#mt -f ${tape_dest} datcompression 1 ; Linux way.
#mt -f ${tape_dest} density 19 # If you figure this out from the man page...


if [ ${append_tape} = true ]
  then
    mt -f ${tape_dest} eom

    if [ ${?} != 0 ]
      then
        echo "${0} ERROR:  Failed to advance DAT tape in drive ${tape_dest}"
        kill -5 ${$}
      else
        echo
    fi
  else
    if [ ${do_partial} = false ]
      then
        mt -f ${tape_dest} rewind
    fi
fi

if [ ${pre_erase} = true ]
  then
    if [ ${verbose}  = true ]
      then
        echo "  Pre-Erasing tape..."
    fi
    mt -f ${tape_dest} erase > /dev/null 2>&1
fi

# Backup the File System List
(cd /tmp;echo FS-LIST | cpio -H newc -o${verbose_cpio} -C 65536 -O ${tape_dest})

echo "0 0 0" > ${tmpfile}.stats

# Backup the file systems one at a time
while read fs fs_device # From ${fs_list}...
  do
    # Do we really want to back it up?
    if [ ${ask} = true ]
      then
        doit="`grep ${fs} /tmp/FS-LIST`"
        doit=`echo ${doit} | awk '{ print $2 }'`
        case ${doit} in
          Backed-Up)
            ;;
          *)
            continue
            ;;
        esac
    fi

    # We have some special cases
    case ${fs} in
      /var/tmp )
        ;;                        # We do not want a backup of this one
      /tmp )
        ;;                        # We do not want a backup of this one
      * )
        total_fs=`expr ${total_fs} + 1`

        if [ ${fs} = "/" ]
          then
            fs_name=/root
          else
            fs_name=${fs}
        fi

        if [ ${verbose} = true ]
          then
            echo "  Building Backup List for the File System \"${fs_name}\"..."
        fi

        # Walk down the file system, do not crss mountpoints, do not follow
        # Symbolic Links
        ( cd /;
           fs_path=`substr -L / ${fs}`
           find ${fs_path:-.} -depth \
                              -xdev \
                              ${partial_flag} \
                              -fstype ufs \
                              -exec ls -ds {} \; | \
           grep -v 'var/spool/news' \
                   > ${tmpfile}.raw
        )

        echo "${tmpfile}.list" > ${tmpfile}.list
        cat ${tmpfile}.raw | awk '{ print $2 }' >> ${tmpfile}.list

        # Compute sizes
        kilobytes=`cat ${tmpfile}.raw | awk '{ print $1 }' | addint`
        kilobytes=`expr ${kilobytes} / 2`
        files=`wc -l ${tmpfile}.raw`;files=`echo ${files} | cut -d ' ' -f 1`

        if [ ${files} -eq 0 ]
          then
            if [ ${verbose} = true ]
              then
                echo "Skipping backup of \"${fs_name}\".  Nothing to backup"
            fi
          else

            if [ ${verbose} = true ]
              then
                if [ ${files} -gt 999 ]
                  then
                    kf=`expr ${files} / 1000`;mf=`expr ${files} % 1000`
                    mf=`printf "%03d" ${mf}`;tf="${kf},${mf}"
                  else
                    tf=${files}
                fi

                mb=`expr ${kilobytes} / 1024`;kb=`expr ${kilobytes} % 1024`
                kb=`expr ${kb}00 / 1024`
                kb=`printf "%03d" ${kb}`;tb="${mb}.${kb} megabytes"
          
                echo "  Backup of \"${fs_name}\" will save"
                echo "  ${tf} files spanning ${tb}"
            fi

            total_files=`expr ${total_files} + ${files}`
            total_kilobytes=`expr ${total_kilobytes} + ${kilobytes}`
            ( cd /;
              if [ ${compress} = true ]
                then
                  cat ${tmpfile}.list | \
                  cpio -H newc -o${verbose_cpio} -C ${tape_block} | 
                  gzip -c --fast | dd ibs=32k of=${tape_dest} obs=1024k
                else
                  cat ${tmpfile}.list | \
                  cpio -H newc -o${verbose_cpio} -C ${tape_block} -O ${tape_dest
}
              fi
            )

            if [ ${?} -ne 0 ]
              then
                echo "Backup of \"${fs_name}\" Failed."
                if [ ${force_it} = false ]
                  then
                    kill -5 ${$}
                fi
            fi
        fi
        ;;
      esac
      echo "${total_kilobytes} ${total_files} ${total_fs}" > ${tmpfile}.stats
  done < ${fs_list}  # End of while--do...

# On complete backups, rewind & eject the tape
if [ ${do_partial} = false ]
  then
    mt -f ${tape_dest} rewind > /dev/null 2>&1
    mt -f ${tape_dest} offline > /dev/null 2>&1 &
fi

# Process overall statistics
read total_kilobytes total_files total_fs < ${tmpfile}.stats
rm -f ${tmpfile}*

if [ ${total_files} -gt 999 ]
  then
    kf=`expr ${total_files} / 1000`;mf=`expr ${total_files} % 1000`
    mf=`printf "%03d" ${mf}`;tf="${kf},${mf}"
  else
    tf=${total_files}
fi

mb=`expr ${total_kilobytes} / 1024`;kb=`expr ${total_kilobytes} % 1024`
kb=`expr ${kb}00 / 1024`
kb=`printf "%03d" ${kb}`;tb="${mb}.${kb} megabytes"
          
if [ ${shut_up} = false ]
  then
    echo "${backup_type} System Backup: ${tb} in"
    echo "                        ${tf} files on"
    echo "                        ${total_fs} file systems"
    echo "completed on `date`"
fi

exit 0


--_=XFMail.1.2-alpha.p0.FreeBSD:970825111934:1078=_--
End of MIME message