From owner-freebsd-current@freebsd.org Thu Apr 23 16:45:12 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F10F32BC9EF for ; Thu, 23 Apr 2020 16:45:12 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 497NT413j0z41yV; Thu, 23 Apr 2020 16:45:12 +0000 (UTC) (envelope-from delphij@gmail.com) Received: by mail-qk1-x743.google.com with SMTP id t3so7122821qkg.1; Thu, 23 Apr 2020 09:45:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Fs9z/GBHo0CEXKSrOI5x1A2xbJZn/gjXoPsgRKVTuvI=; b=PhJBKboyCxqEBHIzhsFZIHAeSSDj/+h0PgU66uXBsKlpeLb+zD+m0eDNHLGHV/kR3z a/l8bbThO484qXljoQ6dFrU/PiqVT4U01rHavHcgMFx8s7qmjL9n46qzd/Zz8LFAgzs8 x23mTau8bAKjx+jP52k2rPGG6UcJma3PpSfJSvvoQOBW08f/IwM1E7ZVo8IX2b+LW0aK fZjl+TP5nhFfNctpP4+CuWNDVA/9UyeG+loEH1iDoiTBh4G4kK7LcYEfWEyVhyWObwxV MB7E1XpEM7S2ieJyhDsiv47iBPyjFxqLNm9es3iIjvqwyvYdvYr+NvlEyJjhx/lPsLDB 8kIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Fs9z/GBHo0CEXKSrOI5x1A2xbJZn/gjXoPsgRKVTuvI=; b=RVYWX7I0C7DWHqp426rMKhWAG2j3UF8QixsUeD67EzyxUd1/t2pWMROOxWmAXKFiIc 5UtzSAX+D9JzTiXXUu3iST3+NI8Qq/nuViXrPdkYJ9fv/iLQio3MXrRrE7nNN4kgxWD2 s75B/3agxEfy/ywg6RMM8owBBTta0gN1jJeLWWs9zrJL+KixOCXX+S9w1Y56zHwZaaWF IoggR6M1OJxfoONkqk1VcRbs/cStg5HC8oPSJY3lDw1aDvugxABh8tqkvejxc8dLUj75 l0xd0liLLJA7YgSh/AbSHy4EmS34J93BSQCVzLQtmDM8LtwjpVEOUt7HuOOr3exuo9bc /iUA== X-Gm-Message-State: AGi0PuauNOuqxaFEWMheM4aXy/hjeuun5nqGTxfjrSFlKT7Unn6Q8CCq /6RaNvqw/Vv0klGBTJbkBME8lQZz/tZbNsQxp+UPoyeG1s0= X-Google-Smtp-Source: APiQypIQUcynDnGzI92ILP41bT0adPN17HYfJ7PgQ6beZ/oqEfGbb9bfO8vEHJsIz4G9b429iJojhwMAEc2ySoSQKlI= X-Received: by 2002:a05:620a:16b0:: with SMTP id s16mr4638618qkj.204.1587660311106; Thu, 23 Apr 2020 09:45:11 -0700 (PDT) MIME-Version: 1.0 References: <202004171047.03HAlFk6050161@gndrsh.dnsmgr.net> In-Reply-To: From: Xin LI Date: Thu, 23 Apr 2020 09:44:59 -0700 Message-ID: Subject: Ordering of files in zoneinfo [Was Re: sort.core error doing installworld on Current.] To: Johan Hendriks Cc: FreeBSD Current , Brooks Davis , "M. Warner Losh" , Glen Barber , Bryan Drewery , Ed Maste , fk@fabiankeil.de, Brad Davis X-Rspamd-Queue-Id: 497NT413j0z41yV X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=PhJBKboy; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of delphij@gmail.com designates 2607:f8b0:4864:20::743 as permitted sender) smtp.mailfrom=delphij@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; URI_COUNT_ODD(1.00)[21]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCPT_COUNT_SEVEN(0.00)[9]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (0.01), ipnet: 2607:f8b0::/32(-0.33), asn: 15169(-0.43), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[3.4.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Apr 2020 16:45:13 -0000 Hi, Thanks for raising this. I have took a look at the change history, it seems that the find operation was introduced in r245265 (brooks@, to support packaged base) and sort was initially implemented as find -s in r289451 (ngie@, to make METALOG reproducible) then as sort in r328958 (imp@, for portability). I wonder if we could drop the sort and replace ${TZS} in line 100 with ${TZS:O} instead? By the way, looking at https://github.com/freebsd/pkg/blob/master/libpkg/metalog.c , I wonder if the sort should really happen in pkg(8) instead? On Fri, Apr 17, 2020 at 7:28 AM Johan Hendriks wrote: > Op 17-04-2020 om 13:30 schreef Johan Hendriks: > > > > Op 17-04-2020 om 12:47 schreef Rodney W. Grimes: > >>>> Op 17-04-2020 om 03:31 schreef Rodney W. Grimes: > >>>>>> On Thu, Apr 16, 2020 at 12:39 PM Kevin Oberman > >>>>>> wrote: > >>>>>> > >>>>>>> So you some how had a sort core dump sitting in > >>>>>>> /usr/obj/usr/src/amd64.amd64/share/zoneinfo/builddir. The > >>>>>>> questions, how > >>>>>>> did get there? I'd take a look at the date on the file and, it > >>>>>>> it is older > >>>>>>> than the buildworld, just assume that it was left-over garbage. > >>>>>>> In either > >>>>>>> case, you can delete it and do another installworld. > >>>>>>> > >>>>>>> That should most likely fix things, but, if the buildworld or > >>>>>>> installworld > >>>>>>> had a crash of sort(1) that left the file, further investigation > >>>>>>> might be > >>>>>>> needed. Re-making the zoneinfo would help track it down should > >>>>>>> this be a re > >>>>>>> al bug, but it's my uneducated guess that it's not. > >>>>>>> -- > >>>>>>> Kevin Oberman, Part time kid herder and retired Network Engineer > >>>>>>> E-mail: rkoberman@gmail.com > >>>>>>> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 > >>>>>>> > >>>>>> Please forgive that awful post! I missed a part of your message > >>>>>> by laziness. > >>>>>> > >>>>>> It's odd that the error of sort(1) crashing was not caught by the > >>>>>> script. > >>>>> Yes, that is a Makefile flaw someplace. > >>>>> Further there must be a wildcard being used to decide which files to > >>>>> install, that is a further Makefile flaw. Wildcards should NOT be > >>>>> used > >>>>> in the source of an install list, exactly because of this type of > >>>>> cruft > >>>>> that can be dropped in an obj dir. > >> From src/share/zoneinfo/Makefile at about line 93: > >> 92 if make(*install*) > >> 93 TZS!= cd ${TZBUILDDIR} && find * -type f | LC_ALL=C sort > >> ^^^^ this is a very bad thing to do > >> in a Makefile. > >> > >> 94 .endif > >> > >> Now I still don't know why sort cored, but I am sure this is the line > >> that did it. > >> > >>>>>> Clearly, sort should NOT crash! Again, a re-build of zoneinfo > >>>>>> might catch > >>>>>> something. Looking at the core might tell you which "sort" was > >>>>>> involved... > >>>>>> the one you just built or the one in the base system. This could > >>>>>> be just a > >>>>>> FOTU, but I would not bet on it. > >>>>> I suspect a recent zoneinfo commit as the root cause. > >>>>> > >>>> I have no idea how to bypass this issue. > >>>> I have used sort from the latest snapshot and placed that file on the > >>>> system and in the build dir, but i keep getting the core > >>>> > >>>> How can i test an build and install part for zoneinfo > >>>> > >>>> If i go into the dir /usr/src/share/zoneinfo and do make install it > >>>> does > >>>> not work, do i need to add something? > >>> Can you show us the output from > >>> cd /usr/src/share/zoneinfo > >>> make clean && make depend && make all && make install > >>> Someplace in that we should get to see sort crashing... > >>> > > On both machines my src.conf file is the same. > > > > I will start over from a clean world by doing a make cleanworld and > > see if it then still gives the errors > > Maybe some old artifacts are hanging around. > > > > > > > >>> > >>>> Thank you both for your time > >>>> > >>>>>> -- > >>>>>> Kevin Oberman, Part time kid herder and retired Network Engineer > >>>>>> E-mail: rkoberman@gmail.com > >>>>>> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 > >>>>>> > >>>>>> > >>>>>>> On Thu, Apr 16, 2020 at 11:49 AM Johan Hendriks > >>>>>>> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> I have a machine running FreeBSD head. > >>>>>>>> rev 13.0-CURRENT #11 r360008 > >>>>>>>> > >>>>>>>> This is a quite powerful machine, so i thought it was a good > >>>>>>>> idea to let > >>>>>>>> that server do the build and for my virtualbox machine i can > >>>>>>>> use the > >>>>>>>> powerful machine to do a installword over NFS. > >>>>>>>> > >>>>>>>> But when i did the make installworld step the client so to say > >>>>>>>> gives an > >>>>>>>> error. > >>>>>>>> > >>>>>>>> install -o root -g wheel -m 444 > >>>>>>>> /usr/obj/usr/src/amd64.amd64/share/zoneinfo/builddir/Zulu > >>>>>>>> /usr/share/zoneinfo/Zulu > >>>>>>>> install -o root -g wheel -m 444 > >>>>>>>> /usr/obj/usr/src/amd64.amd64/share/zoneinfo/builddir/posixrules > >>>>>>>> /usr/share/zoneinfo/posixrules > >>>>>>>> install -o root -g wheel -m 444 > >>>>>>>> /usr/obj/usr/src/amd64.amd64/share/zoneinfo/builddir/sort.core > >>>>>>>> /usr/share/zoneinfo/sort.core > >>>>>>>> install: > >>>>>>>> /usr/obj/usr/src/amd64.amd64/share/zoneinfo/builddir/sort.core: > >>>>>>>> Permission denied > >>>>>>>> *** Error code 71 > >>>>>>>> > >>>>>>>> Stop. > >>>>>>>> bmake[5]: stopped in /usr/src/share/zoneinfo > >>>>>>>> *** Error code 1 > >>>>>>>> > >>>>>>>> Stop. > >>>>>>>> bmake[4]: stopped in /usr/src/share > >>>>>>>> *** Error code 1 > >>>>>>>> > >>>>>>>> Stop. > >>>>>>>> bmake[3]: stopped in /usr/src > >>>>>>>> *** Error code 1 > >>>>>>>> > >>>>>>>> Stop. > >>>>>>>> bmake[2]: stopped in /usr/src > >>>>>>>> *** Error code 1 > >>>>>>>> > >>>>>>>> Stop. > >>>>>>>> bmake[1]: stopped in /usr/src > >>>>>>>> *** Error code 1 > >>>>>>>> > >>>>>>>> Stop. > >>>>>>>> make: stopped in /usr/src > >>>>>>>> .ERROR_TARGET='installworld' > >>>>>>>> .ERROR_META_FILE='' > >>>>>>>> .MAKE.LEVEL='0' > >>>>>>>> MAKEFILE='' > >>>>>>>> .MAKE.MODE='normal' > >>>>>>>> _ERROR_CMD='.PHONY' > >>>>>>>> .CURDIR='/usr/src' > >>>>>>>> .MAKE='make' > >>>>>>>> .OBJDIR='/usr/obj/usr/src/amd64.amd64' > >>>>>>>> .TARGETS='installworld' > >>>>>>>> DESTDIR='' > >>>>>>>> LD_LIBRARY_PATH='' > >>>>>>>> MACHINE='amd64' > >>>>>>>> MACHINE_ARCH='amd64' > >>>>>>>> MAKEOBJDIRPREFIX='/usr/obj' > >>>>>>>> MAKESYSPATH='/usr/src/share/mk' > >>>>>>>> MAKE_VERSION='20181221' > >>>>>>>> PATH='/sbin:/bin:/usr/sbin:/usr/bin' > >>>>>>>> SRCTOP='/usr/src' > >>>>>>>> OBJTOP='/usr/obj/usr/src/amd64.amd64' > >>>>>>>> > >>>>>>>> It looks likes sort coredumps in the usr/share/zoneinfo part of > >>>>>>>> the base. > >>>>>>>> As it has no permission on the NFS share it errors out. > >>>>>>>> On the server itself, the installworld goes well, but it leaves a > >>>>>>>> sort.core file behind in /usr/share/zoneinfo > >>>>>>>> > >>>>>>>> cd /usr/share/zoneinfo > >>>>>>>> ls -al > >>>>>>>> > >>>>>>>> > >>>>>> _______________________________________________ > >>>>>> freebsd-current@freebsd.org mailing list > >>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current > >>>>>> To unsubscribe, send any mail to > >>>>>> "freebsd-current-unsubscribe@freebsd.org" > >>>>>> > >>>> _______________________________________________ > >>>> freebsd-current@freebsd.org mailing list > >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current > >>>> To unsubscribe, send any mail to > >>>> "freebsd-current-unsubscribe@freebsd.org" > >>>> > >>> -- > >>> Rod Grimes rgrimes@freebsd.org > >>> _______________________________________________ > >>> freebsd-current@freebsd.org mailing list > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-current > >>> To unsubscribe, send any mail to > >>> "freebsd-current-unsubscribe@freebsd.org" > >>> > I have rebuild everything on the host and did a make cleanworld. > Al is fine now. > I should have done that before i asked here. > Sorry to have wasted your time. > But we did find a Makefile that should be doing things differently. > > regards > Johan > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >