Date: Mon, 10 Feb 2014 11:07:36 -0500 From: Glen Barber <gjb@FreeBSD.org> To: Tim Daneliuk <tundra@tundraware.com> Cc: freebsd-stable@freebsd.org Subject: Re: And Here I Thought buildworld/makeworld Was IO Bound Message-ID: <20140210160736.GC1629@glenbarber.us> In-Reply-To: <52F8F426.9000003@tundraware.com> References: <52F84AF8.8050007@tundraware.com> <52F86768.9000109@googlemail.com> <52F8E8C6.2060909@tundraware.com> <20140210152407.GB1629@glenbarber.us> <52F8F426.9000003@tundraware.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--qXCixuLMVvZDruUh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Feb 10, 2014 at 09:45:42AM -0600, Tim Daneliuk wrote: > >>Well ... it has to hit the disk sooner or later. But, if the > >>frequency of physical writes is low because of aggregated > >>IO from the cache, I guess that would tend to make the whole > >>business more CPU bound than IO bound. I just found this surprising. > >> > > > >What is the underlying filesystem? > > >=20 > UFS w/Softupdates, no journaling. >=20 >=20 > [...] > >>Yeah, I wonder what other people are seeing for a full buildworld/kerne= l and/or what > >>the master machines at FreeBSD.org do in this regards. > >> > >>Would anyone else care to share with the class? > >> > > > >If you mean "what do the machines do with regard to tuning", the only > >specific tuning is turning off atime. > > >=20 >=20 > Could you comment a bit more about this please? How you do it, > rationale', etc. >=20 Each build happens in its own ZFS dataset, so atime is disabled during dataset creation (zfs create -o atime=3Doff zroot/11-amd64-GENERIC-snap). The tuning(7) manual has more details, but basically each time a file is read, a separate write happens to update the file access time. Turning off atime eliminates these excessive writes. > >If you mean "what do the machines do as far as overall build time", with > >a clean obj/ directory, 35 minutes sounds about right. I do not have > >exact numbers. > > > >The machine that currently produces weekly snapshot images uses '-j10' > >for buildworld and '-j6' for buildkernel for each build. It also runs > >the builds in parallel. Even with three parallel 'make -j10 buildworld' > >(one for head/, stable/10/, and stable/9/), disk IO is minimal. The > >only time the disk IO becomes a bottleneck is when creating the > >distribution files (base.txz, kernel.txz, etc.). >=20 >=20 > What sort of CPU/Mem/Disk is that machine? >=20 This machine is 48-core (4 x 12 cores) Opteron 6174 (2.2GHz), 128GB RAM, with 5 drives in a raidz1 on a PERC H700 controller. Glen --qXCixuLMVvZDruUh Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBCAAGBQJS+PlIAAoJELls3eqvi17Qfm0QANJxYsvQLV3GZSceGcfYENMf dIUXehF3QAY34LiJKSX2lGbaXkmWMh9+yS0VaviOR96gaiywQeW8ncp86k/lsl3w ywsZQgcAFs6FeD/OrIgNkNQii4hYrlhE+Dhr+v+1glyO8L0HSqpLOgInJBi9C7lx +3GU38+b6TMRXiHi1NNG8KROjXYFYRsLXatNIBmEodCz5k3wUWc037VUQ4NUn7KQ /fTSr+td231VK7tlmdiC2c+gtWQQB4JUbRGmFSru34+Dl+ZWBa0w7Topc648LyRP w66XOcODhkmogyFnJdPkYSkctzZjou1KfGng65+0bE0cP4jnNu89fBHJUpvtoU98 8DeEDWEoerYaL1/gfN51Suo+b3EpL0ao8cf7PK+ioWC1tMZF8HTU8Be4K7wEo5wD 5qFFIs2S9o7Lpr2/ZhdFzmhxAY8AvoiuKMnaVSqnKsZD+VZbFpgw+jx8rOPw7qXQ FQ1c4epiooo2LaDbRVvyKR9mtBbWzramHzvR4t28cbST4a7Q8NnIFnwhKmOOPfzv /tUCYrYDE+CeSUIrGkRA2DL32DxHJP4w3UrkMbtaZRPsyTOTrYxE5+z4RmebAhJW ThagFoWT/t+g19HUrJs9HTjbsl3K4m9xh/d23IQ9heuejKGrNnmTVcPVkHljOHqt z86RnWlEzqoQIEEmDvJp =Wm6a -----END PGP SIGNATURE----- --qXCixuLMVvZDruUh--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140210160736.GC1629>