Date: Wed, 1 Jun 2016 12:36:05 -0700 From: Bryan Drewery <bdrewery@FreeBSD.org> To: Andrey Chernov <ache@freebsd.org>, current@freebsd.org Subject: Re: 'make depend' or 'make' bug on recent --current Message-ID: <7f325ac2-f722-c491-fca0-2e582773e780@FreeBSD.org> In-Reply-To: <bec48601-f6b4-6b8a-dc0d-fad47416723b@freebsd.org> References: <092f5e98-dae8-dbc9-2a6e-7068b972278f@freebsd.org> <311f3a82-b702-d375-170f-82ae39236ab0@FreeBSD.org> <bec48601-f6b4-6b8a-dc0d-fad47416723b@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --r6heT2Xl3xlT5im5ACh5dO7QQqH1hCgPb Content-Type: multipart/mixed; boundary="fNu2oeONVv8Kd0hFeRqDIeU09ltsktlmD" From: Bryan Drewery <bdrewery@FreeBSD.org> To: Andrey Chernov <ache@freebsd.org>, current@freebsd.org Message-ID: <7f325ac2-f722-c491-fca0-2e582773e780@FreeBSD.org> Subject: Re: 'make depend' or 'make' bug on recent --current References: <092f5e98-dae8-dbc9-2a6e-7068b972278f@freebsd.org> <311f3a82-b702-d375-170f-82ae39236ab0@FreeBSD.org> <bec48601-f6b4-6b8a-dc0d-fad47416723b@freebsd.org> In-Reply-To: <bec48601-f6b4-6b8a-dc0d-fad47416723b@freebsd.org> --fNu2oeONVv8Kd0hFeRqDIeU09ltsktlmD Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 6/1/16 11:49 AM, Andrey Chernov wrote: > On 01.06.2016 21:18, Bryan Drewery wrote: >> On 6/1/2016 6:11 AM, Andrey Chernov wrote: >>> Steps to reproduce: >>> >>> cd /usr/src/lib/libc/stdlib >>> touch *div*.c >>> cd .. >>> make depend >>> make >>> >>> And see how imaxdiv.o only is recompiled. >>> No div.o ldiv.o lldiv.o are recompiled. Because they never were compiled! lib/libc/stdlib/Makefile.inc MISRCS has div.c ldiv.c lldiv.c However, for at least amd64 it also includes: lib/libc/amd64/stdlib/Makefile.inc which has MDSRCS=3D div.S ldiv.S lldiv.S Note the .depend.lldiv.o file: > ~/git/freebsd/lib/libc/stdlib # cat /usr/obj/root/git/freebsd/lib/libc/= =2Edepend.lldiv.o > lldiv.o: /root/git/freebsd/lib/libc/amd64/stdlib/lldiv.S \ > /usr/obj/root/git/freebsd/tmp/usr/include/machine/asm.h \ > /usr/obj/root/git/freebsd/tmp/usr/include/sys/cdefs.h It's built from lldiv.S, not lldiv.c. No bug here. proof: > cc -O2 -pipe -I/root/git/freebsd/lib/libc/include -I/root/git/freebsd= /lib/libc/../../include -I/root/git/freebsd/lib/libc/amd64 -DNLS -D__DBI= NTERFACE_PRIVATE -I/root/git/freebsd/lib/libc/../../contrib/ > gdtoa -I/root/git/freebsd/lib/libc/../../contrib/libc-vis -DINET6 -I/us= r/obj/root/git/freebsd/lib/libc -I/root/git/freebsd/lib/libc/resolv -D_AC= L_PRIVATE -DPOSIX_MISTAKE -I/root/git/freebsd/lib/libc/../li > bmd -I/root/git/freebsd/lib/libc/../../contrib/jemalloc/include -DMALLO= C_PRODUCTION -I/root/git/freebsd/lib/libc/../../contrib/tzcode/stdtime -I= /root/git/freebsd/lib/libc/stdtime -I/root/git/freebsd/lib/l > ibc/locale -DBROKEN_DES -DPORTMAP -DDES_BUILTIN -I/root/git/freebsd/lib= /libc/rpc -DYP -DNS_CACHING -DSYMBOL_VERSIONING -MD -MF.depend.lldiv.o -= MTlldiv.o -std=3Dgnu99 -fstack-protector-strong -Wsystem-heade > rs -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -= Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tauto= logical-compare -Wno-unused-value -Wno-parentheses-equality > -Wno-unused-function -Wno-enum-conversion -Wno-unused-local-typedef -Wn= o-switch -Wno-switch-enum -Wno-knr-promoted-parameter -fcolor-diagnostic= s -Qunused-arguments -I/root/git/freebsd/lib/libutil -I/roo > t/git/freebsd/lib/msun/amd64 -I/root/git/freebsd/lib/msun/x86 -I/root/g= it/freebsd/lib/msun/src -c /root/git/freebsd/lib/libc/amd64/stdlib/lldi= v.S -o lldiv.o > cc -fpic -DPIC -O2 -pipe -I/root/git/freebsd/lib/libc/include -I/root= /git/freebsd/lib/libc/../../include -I/root/git/freebsd/lib/libc/amd64 -D= NLS -D__DBINTERFACE_PRIVATE -I/root/git/freebsd/lib/libc/.. > /../contrib/gdtoa -I/root/git/freebsd/lib/libc/../../contrib/libc-vis -= DINET6 -I/usr/obj/root/git/freebsd/lib/libc -I/root/git/freebsd/lib/libc/= resolv -D_ACL_PRIVATE -DPOSIX_MISTAKE -I/root/git/freebsd/li > b/libc/../libmd -I/root/git/freebsd/lib/libc/../../contrib/jemalloc/inc= lude -DMALLOC_PRODUCTION -I/root/git/freebsd/lib/libc/../../contrib/tzcod= e/stdtime -I/root/git/freebsd/lib/libc/stdtime -I/root/git/f > reebsd/lib/libc/locale -DBROKEN_DES -DPORTMAP -DDES_BUILTIN -I/root/git= /freebsd/lib/libc/rpc -DYP -DNS_CACHING -DSYMBOL_VERSIONING -MD -MF.depe= nd.lldiv.So -MTlldiv.So -std=3Dgnu99 -fstack-protector-strong > -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wno-= pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-varia= ble -Wno-tautological-compare -Wno-unused-value -Wno-parenth > eses-equality -Wno-unused-function -Wno-enum-conversion -Wno-unused-loc= al-typedef -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -fco= lor-diagnostics -Qunused-arguments -I/root/git/freebsd/lib/ > libutil -I/root/git/freebsd/lib/msun/amd64 -I/root/git/freebsd/lib/msun= /x86 -I/root/git/freebsd/lib/msun/src -c /root/git/freebsd/lib/libc/am= d64/stdlib/lldiv.S -o lldiv.So note lldiv.S in both. >> >> My dev system is busy at the moment. I'll test it and get back to you.= >=20 > I need to add that I do 'make cleandepend/cleandir/cleanobj' + 'make > obj' again and full rebuild with no old files, but the bug repeated aga= in. >=20 >>> P.S. new make depend is simple disgusting. It tends to recompile >>> everything in the system if some minor header file is touched, but >> >> If the header is used by all source files then that is expected. >> >> However if you do not have a .depend.obj.o file then it is quite >> aggressive with building. If you touch any header it will rebuild >> everything. But you shouldn't get into that situation unless you rm -= f >> .depend* first. >> >>> completely forget to recompile source code changes. I suggest to back= >>> out all AI in that area. >>> 'make depend' is not time-consuming task and good old way never made >>> mistakes. >> >> The graph in the original commit for WITH_FAST_DEPEND disagrees. >> https://svnweb.freebsd.org/base?view=3Drevision&revision=3D290433 >> >> We run the preprocessor once now, not twice. >=20 > It sounds good, just implemented bad. You measure some spherical chicke= n > in vacuum, not what really happens. In the old times I almost never hav= e > clang libs rebuild (few files from there max when FreeBSD_version is > increased), but now I got them fully rebuilt with any header change. > That is the biggest slowdown and not what you try to measure. > Don't ever use 'make world'. Try to rebuild the system incrementally an= d > you'll see. I cannot argue with a lack of solid evidence. The old 'make depend' ran 'mkdep' which ran 'cc -E -M' which produces *the same output* as 'cc foo.c -o foo.o -MD -MF .depend.foo.o -MT foo.o'. There's nothing different in the actual .depend file implementation/content. Clang rebuilds often because it is changing often! Just look at recent commit logs and you'll see r300984 which will cause a rebuild of clang. Or r301011 which modified sys/sys/param.h which will rebuild just about everything. These are normal and how the old system worked as well. There certainly are some issues with the new system. 1. Processing the split files can be slow over NFS 2. make cleandepend - will cause the next build to make a lot of guesses and not be very efficient. 3. make cleandepend - There is no way to generate the .depend* files again without rebuilding. 4. It doesn't fix all of the missing dependencies that have been missing forever such as crt, csu, libgcc, etc for static builds. 5. the csu builds don't use it yet but have workarounds for it 6. removing the 'make depend' tree-walk can hurt downstream builds which relied on having a multi-phase build to generate a .mk file to include (odd case I hit at work). No such case exists in the upstream FreeBSD tr= ee. --=20 Regards, Bryan Drewery --fNu2oeONVv8Kd0hFeRqDIeU09ltsktlmD-- --r6heT2Xl3xlT5im5ACh5dO7QQqH1hCgPb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJXTzkmAAoJEDXXcbtuRpfPG3gH/2VJNdGriXkgDvJ/rYpbBSlk EO3TJzWgUicQGIcgyEp+tgLG7wP1k0vZ+fbTZBWA8mlmIfnBBX4TiYavQRzTACYL rb4Tp9VHiI9lFREpev6Lyw4LzDmnOgtc9RL3VZBaHr0T03JVciMfMV5jzo8PIN0f Z7u88XqRLgEk7LrdJJdvu0Qm9g4y9EIRTCuAt/K2gWgEfePW54n5HvCJLa9OCgAo wvAH2tMYL/W4PQg6iG8gd4o7vVvHi+t9Y+++bOvOrlzQJpBdCSiK8qGM5ZmeYKLl odQki1caX2eHCPHun08zPODI/8HBscXGZKQ+HVCUJ0FiVHjG6qR1FodicZTnqrM= =g4CW -----END PGP SIGNATURE----- --r6heT2Xl3xlT5im5ACh5dO7QQqH1hCgPb--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7f325ac2-f722-c491-fca0-2e582773e780>