Date: Thu, 24 Jan 2008 23:35:21 -0500 From: "Maxim Khitrov" <mkhitrov@gmail.com> To: "Aryeh M. Friedman" <aryeh.friedman@gmail.com> Cc: freebsd-questions@freebsd.org Subject: Re: speeding up buildworld/kernel Message-ID: <26ddd1750801242035j3b3cea5ek281fe7e00afbf217@mail.gmail.com> In-Reply-To: <47995F19.7050905@gmail.com> References: <4798F1D0.3090009@gmail.com> <26ddd1750801241232l51792bf3ydc02e17e4be49d6e@mail.gmail.com> <20080124210539.GA50047@owl.midgard.homeip.net> <26ddd1750801241408w7b776c87g14b94cfa1433616a@mail.gmail.com> <47995F19.7050905@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Jan 24, 2008 11:01 PM, Aryeh M. Friedman <aryeh.friedman@gmail.com> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > Maxim Khitrov wrote: > > On Jan 24, 2008 4:05 PM, Erik Trulsson <ertr1013@student.uu.se> > > wrote: > >> On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote: > >>> On Jan 24, 2008 3:15 PM, Aryeh M. Friedman > >>> <aryeh.friedman@gmail.com> wrote: > >>>> I update my sources at least once a day and do > >>>> buildworld/kernel just as often... It seems some stuff that > >>>> needs not be recompiled is on every single run for example > >>>> gcc and kerbos. I have NO_CLEAN in /etc/make.conf is there > >>>> anything else I can do to speed stuff up... for ref here is > >>>> my /etc/make.conf: > >>>> > >>>> CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by > >>>> use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 > >>>> PERL_VERSION=5.8.8 > >>>> > >>>> - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers > >>>> Developer, not business, friendly > >>>> http://www.flosoft-systems.com > >>> I might be wrong, but NO_CLEAN seems like a bad idea except in > >>> special circumstances. Install ccache, but make sure you set > >>> CCACHE_HASH_COMPILER environment variable to 1. That will make > >>> sure that the cache stays valid if the compiler executable is > >>> overwritten by an identical copy (as it would be on > >>> installworld). When the compiler changes the cache will be > >>> repopulated on the next rebuild. > >> You are indeed wrong. NO_CLEAN will work fine almost all the > >> time - except in special circumstances. The few times it does > >> not work one can always do a 'make clean' by hand first. (Or even > >> faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX to some > >> useful value you can do the same thing for the ports tree. > >> Personally I always compile with -DNO_CLEAN and use 'rm -fr' to > >> clean. I have never had problems originating with this. > >> > >> ccache is not very useful for buildworld, since among the first > >> thing buildworld does is to build the compiler and then use the > >> newly built compiler to compile the rest. I.e. the already > >> installed compiler (which is the one ccache will handle) will not > >> be used for most of the build thus removing almost all the > >> advantage of ccache. It is supposed to be possible to use ccache > >> for buildworld as well, but that would require a bit of hackery. > > > > That's not true. I just ran `make buildworld buildkernel` on my > > firewall. Here are ccache stats when the operation finished: > > > > root@cerberus [/root]# ccache -s cache directory > > /srv/.ccache cache hit 12056 cache miss > > 38 called for link 461 multiple source files > > 1 not a C/C++ file 1228 unsupported compiler > > option 7 files in cache 117366 cache > > size 679.6 Mbytes max cache size > > 2.0 Gbytes > > > > Ccache is used through the entire build process and there is no > > hackery involved. Just follow the directions for changing the > > compiler to /usr/local/libexec/ccache/world-cc. On this Celeron D > > 1.8 GHz machine rebuilding world and kernel takes 45 minutes and 40 > > seconds. I don't recall exactly what it was without ccache, but I > > think it was around 3 hours. Just make sure that you set the > > CCACHE_HASH_COMPILER variable, otherwise it will assume that the > > compiler is different just because its modification time has > > changed. > > > > - Max > > > No commands where issued on the terminal except the ones showed and > ccache was installed with default settings (I didn't modify any files > after doing "make install" for ccache) It doesn't work like that. You have to read /usr/local/share/doc/ccache/ccache-howto-freebsd.txt and configure things properly before ccache is used for building the os. In /etc/make.conf you need to add the following: .if exists(/usr/local/libexec/ccache) && !defined(NOCCACHE) && \ (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*)) CC= /usr/local/libexec/ccache/world-cc CXX= /usr/local/libexec/ccache/world-c++ .endif Unless you actually want to use /root/.ccache (and have enough space for it), I would move that directory to some other partition. In my /etc/csh.cshrc I have this configuration: setenv CCACHE_DIR /srv/.ccache setenv CCACHE_PATH /usr/bin:/usr/local/bin setenv CCACHE_HASH_COMPILER 1 Once you've done all of this, rebuild the world. You can run ccache -s during that process. If the hit/miss numbers stay at 0 then ccache is not being used. Check your configuration and try again. The first run will be slower than normal (though not by much), because the cache is being populated for the first time. On the second run, however, clear ccache stats (ccache -z) and you should see the same results as I've posted above. With the exception of only a few files, just about everything should be obtained from the cache and not compiled from scratch. - Max
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?26ddd1750801242035j3b3cea5ek281fe7e00afbf217>