Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 11 Jan 2003 01:33:37 +1030
From:      Greg Hurrell <greg@hurrell.cc>
To:        freebsd-isp@FreeBSD.ORG
Subject:   Rsync problems (was: SpamAssassin spawned 700 perl processes)
Message-ID:  <B24D68C0-24AC-11D7-93AF-000393BC25EC@hurrell.cc>
In-Reply-To: <000101c2b8a1$a66a3960$0201a8c0@scorpio>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help
Given today's scare with spamassassin taking my server down, I decided=20=

it's time to get some backups over the network happening a little more=20=

regularly (one of my databases was corrupted during the crash, but=20
thankfully I was able to repair it).

One of the reasons I only backup over the network once per week is that=20=

I use rsync to do the backup, but it often gives me error messages:

> usr/home/cust42/public_html/service/usage/webalizer.current
>        3903 100%    0.00kB/s    0:00:00
> usr/home/cust42/public_html/service/usage/webalizer.hist
>         187 100%    0.00kB/s    0:00:00
> total: matches=3D402  tag_hits=3D2424  false_alarms=3D0 data=3D14308139
> wrote 22337 bytes  read 11811403 bytes  9198.40 bytes/sec
> total size is 123336221  speedup is 10.42
> rsync error: partial transfer (code 23) at=20
> /SourceCache/rsync/rsync-8/rsync/main.c(883)

So I try again and it gets a little further this time:

> usr/home/cust42/public_html/service/usage/webalizer.hist is uptodate
> usr/home/cust42/public_html/spacer.gif is uptodate
> usr/home/cust42/public_html/text1.swf is uptodate
> usr/home/cust42/public_html/yoursay.htm is uptodate
> usr/home/cust39/public_html/resumes/Resume Rosana Garc=EDa.doc
>       41472 100%   19.78MB/s    0:00:00
> usr/home/cust39/public_html/resumes/Resum=E92_antonio.doc
>       87040 100%  243.55kB/s    0:00:00
> total: matches=3D0  tag_hits=3D0  false_alarms=3D0 data=3D128512
> wrote 69 bytes  read 443351 bytes  4243.25 bytes/sec
> total size is 123336221  speedup is 278.15
> rsync error: partial transfer (code 23) at=20
> /SourceCache/rsync/rsync-8/rsync/main.c(883)

Perhaps this one has a clue to the problem... This time I see a special=20=

character (=E9) in one of the filenames...

And so on and so forth, until the backup is completely done.

Here is the script I use to do the backup. It uses a file called=20
"syncdirs" to know which dirs to back up. The file just contains a list=20=

of dirs like this:

> /etc
> /usr/local/etc
> /var/db/mysqldumps
> /usr/home/

Anyway, here is the actual script:

> #!/bin/sh -e
>
> # rsync.sh -- remote host backup script
>
> # list of dirs to sync (stored locally):
> syncdirlist=3D"/usr/backups/rsync/host1/syncdirs"
>
> # remote host to backup from:
> remotehost=3D"host1"
>
> # directory for storing backup on local machine:
> localbackupdir=3D"/usr/backups/rsync/host1/store/"
>
> # check to make sure we are root
> if [ $(whoami) !=3D "root" ]; then
>   echo "You must run this script as root. [Exiting]"
>   exit
> fi
>
> # read dirs to sync from file, ignoring comments and blank lines
> for syncdir in `/bin/cat ${syncdirlist} | /usr/bin/grep -v "#" | \
>     /usr/bin/grep -v "^\$"`
> do
>     echo Backing up $syncdir from remote host:
>     /usr/bin/rsync -avvcRz -e ssh --delete --partial --numeric-ids \
>     --modify-window=3D10 --bwlimit=3D10 --progress --exclude =
"Maildir/" \
>     --exclude "logs/" root@${remotehost}:${syncdir} \
>     ${localbackupdir}
>     echo -e "Backup of $syndir complete.\n\n"
>
> done

Any ideas on what could be causing these failures? Generally if I do=20
the sync again, it will get a little further before failing. After a=20
few tries, the entire backup is done; but I would much prefer it if it=20=

would just work the first time!

I am running rsync locally on Mac OS X 10.2.3. The output of "rsync=20
--version" is:

> rsync  version 2.5.2  protocol version 26
> Copyright (C) 1996-2002 by Andrew Tridgell and others
> <http://rsync.samba.org/>;
> Capabilities: 64-bit files, socketpairs, hard links, symlinks,=20
> batchfiles, IPv6,
>               32-bit system inums, 64-bit internal inums

The machine being backed-up is a FreeBSD box (FreeBSD 4.7-RELEASE-p2=20
#6: Thu Nov 14 21:42:32 CST 2002) and the output of "ssh -V" on that=20
machine is:

> OpenSSH_3.4p1 FreeBSD-20020702, SSH protocols 1.5/2.0, OpenSSL=20
> 0x0090607f

"rsync --version" on the FreeBSD box yields:

> rsync  version 2.5.5  protocol version 26
> Copyright (C) 1996-2002 by Andrew Tridgell and others
> <http://rsync.samba.org/>;
> Capabilities: 64-bit files, socketpairs, hard links, symlinks,=20
> batchfiles,
>               IPv6, 32-bit system inums, 64-bit internal inums

Thanks for any insight you can provide!

Cheers :-)
Greg



El Viernes, 10 enero, 2003, a las 10:43 PM, Jeff Palmer escribi=F3:

> Hate to follow up my own post,  but I left some information out.
>
> Currently,   calling spamassassin with the -L switch allows=20
> spamassassin
> to work,  without doing any of the DNS lookups,  and thereby not=20
> letting
> razor et al affect it's performance.  This of course decreases
> spamassassins ability to accurately tag spam, but for now seems a very
> viable option.  At least the mail server will continue running.  And =
at
> least some spam will still be tagged.
>
> Jeff
>
>
>> -----Original Message-----
>> From: owner-freebsd-isp@FreeBSD.ORG
> [mailto:owner-freebsd-isp@FreeBSD.ORG]
>> On Behalf Of Jeff Palmer
>> Sent: Friday, January 10, 2003 7:03 AM
>> To: freebsd-isp@FreeBSD.ORG
>> Subject: RE: SpamAssassin spawned 700 perl processes
>>
>> Seems yesterday that razor, razor2 and pyzor were down.
>> The current thinking on the SA mailing lists,   is that the razor
>> timeout setting was uneffective.
>>
>> There has been a work-around implemented in 2.50 (due any time now)
>> which should eliminate this problem in the future should the =
situation
>> arise again.
>>
>>
>> Jeff Palmer
>> scorpio@drkshdw.org
>>
>>
>
>
>
>
> To Unsubscribe: send mail to majordomo@FreeBSD.org
> with "unsubscribe freebsd-isp" in the body of the message
>


To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-isp" in the body of the message




Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?B24D68C0-24AC-11D7-93AF-000393BC25EC>