From owner-freebsd-questions@FreeBSD.ORG Sun Aug 31 17:45:30 2003 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4870916A4BF for ; Sun, 31 Aug 2003 17:45:30 -0700 (PDT) Received: from rutger.owt.com (rutger.owt.com [204.118.6.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1ECD343F3F for ; Sun, 31 Aug 2003 17:45:29 -0700 (PDT) (envelope-from kstewart@owt.com) Received: from topaz-out (owt-207-41-94-233.owt.com [207.41.94.233]) by rutger.owt.com (8.11.6p2/8.9.3) with ESMTP id h810j4216930; Sun, 31 Aug 2003 17:45:04 -0700 From: Kent Stewart To: "Charles Howse" , "'ODHIAMBO Washington'" Date: Sun, 31 Aug 2003 17:45:03 -0700 User-Agent: KMail/1.5.3 References: <003001c37014$6aa06820$04fea8c0@moe> In-Reply-To: <003001c37014$6aa06820$04fea8c0@moe> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200308311745.03944.kstewart@owt.com> cc: freebsd-questions@freebsd.org Subject: Re: scripting the buildworld/installworld process X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Sep 2003 00:45:30 -0000 On Sunday 31 August 2003 04:05 pm, Charles Howse wrote: > > bw w -j5 > > 893.563u 311.353s 30:31.44 65.7% 1302+1591k 53927+138930io > > 2331pf+0w > > > > The wall clock time is really how efficient your process is. The > > fact that your user time or sys time is faster doesn't mean much if > > the wall > > clock time is 1/3 longer. > > Wall clock? Would that be 30:31.44 from above? > > > I think that once you have it setup, I will probably make my > > KISS simple > > script more complicated :). I have one machine that the mouse > > doesn't > > Glad to share, if that's what you mean. That is what I meant. You learn by trying things. > > After re-reading your page about scripting the buildworld, I see an > opportunity to test my script. > > I never thought about putting /usr/src and /usr/obj on different > HDD's. I have 2 ATAPI HDD in my box, on different IDE busses, so that > should work out well. > I'll also need to adjust some slice sizes as well. > > How much space do you recommend for /var if I'm logging like you do? > It's currently: > Disk1 - > / 100M > swap 256M > /var 200M > /usr about 1.8G > Disk2 - > /disk2 7.8G My smallest HD is 13GB and the system had 4 of them. My typical system is on a 3x30GB ATA-100/133 HDs. Since, I have a lot of free space, my partitions are typically 1.5GB. FreeBSD 4.x creates an ../obj of around 300-400MB; however, FreeBSD 5.x produces a 700-800MB. I like to mount a partition as /usr/obj and /usr/src. The builds work just fine with a link to /disk2/obj as /usr/obj. The reason for separate partition is that I have messed up my system to the point that I had to nfs_mount /usr/src and /usr/obj from a different machine. If I mount /disk2/obj as /usr/obj, the make installworld dies because it knows the true path to the obj files. If I redo a system, I create a mount point as /usr/obj. Until then, I just link /usr/obj to /usr2/obj and etc. I think that just moving /usr/obj to your 2nd HD as a link would separate the I/O. For FreeBSD 5.x, you need a lot more space in /. When I installed 5.0-release, I created / as 500MB and it is about 24% used. If you have to manually download some tarballs, you can chew up space in a hurry. The files in /var/log/build can fill up a fs really fast because each system update is around 7-8MB. Since I log everything from cvsup to "make index", I fill up space in a hurry :). If something breaks, I want to be able to point to the commit that broke it. > > So I assume /usr/obj should be on the second HDD? > About how much space should I reserve? > > Looks like I should also adjust my script to remove the -j4, eh? Like you said, play with it. I use write caching and softupdates. They all made it faster. Depending on your system, you may find that -j is faster. From my experience, all of my systems (6 of various speeds and cpus), do a build world faster without a -j. When I was doing benchmarks on a Cray X/MP with 100MB/s & 1000MB/s data channels to stripped disks capable of 20MB/s continuous, write caching doubled the throughput of the system. The gain was not obvious up front and everyone was surprised by the result. After we saw it, it was a shoulder shrug because it appeared obvious that you want a process to have faster access to data that it needs than forcing it to save processed data first. In the commercial world, protecting the processed data has a much higher assigned value. I went for speed. If someone said 4-stable was broken, I wanted to be able to test the builds in less than 30 minutes. I also didn't want to spend a lot of money to be able to do that :). BTW, those build times all had setiathome running in the background. Having it running in the background really doesn't change the elapsed time too much exept in 5.x. I kill seti before I do a timed run on it. Kent -- Kent Stewart Richland, WA http://users.owt.com/kstewart/index.html