Date: Fri, 16 Jan 1998 16:50:42 +0000 From: Lee McLoughlin <lmjm@icparc.ic.ac.uk> To: Lyndon Nerenberg <lyndon@ve7tcp.ampr.org> Cc: Christiaan Keet <keet@mordor.plig.net>, freebsd-hubs@FreeBSD.ORG Subject: Re: mirror keeps crashes Message-ID: <34BF8FE2.795037E1@icparc.ic.ac.uk> References: <199801161611.JAA26188@ve7tcp.ampr.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Lyndon Nerenberg wrote: > > > Jesper Skriver wrote: > > > I run ftp.dk.freebsd.org, and I have a problem, my mirror keeps crashing, > > > probably due to the wast number of files. > > No, it's hitting a kernel-enforced per-process VM limit. There's a > kernel parameter you need to bump up: > > options MAXDSIZ="536870912UL" #512 MB > > This ups the per-process VM limit to 512MB. You still might have to > fiddle with your shells limit command as well. If you are *really* *really* keen and *really* *really* brave then an experimental copy of mirror 2.9 will appear in sunsite.doc.ic.ac.uk packages/mirror/experimental/ within the next couple of days. This accepts a new option: algorithm=1 which tells mirror to do its work a directory at a time. This means less memory needed but a higher load on the server mirror is pulling back from as it will run a 'dir' command for each directory. I've been using it to mirror a couple of very large sites without problems. -- Lee McLoughlin. Phone: +44 171 594 8388 IC-Parc, Imperial College, Fax: +44 171 594 8432 South Kensington, London. SW7 2AZ. UK. Email: lmjm@icparc.ic.ac.uk
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?34BF8FE2.795037E1>