From owner-freebsd-pkgbase@freebsd.org Sat May 25 17:58:42 2019 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60EE715B2472 for ; Sat, 25 May 2019 17:58:42 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9771C82CFA for ; Sat, 25 May 2019 17:58:40 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1558807110; bh=R8dTSe9XBS1XC5AwsBysZ1x1a7tO7flQg495thsvgxc=; h=X-UI-Sender-Class:From:To:Subject:Date; b=iE+cxBRqlpGC6fxB9WCmJUJ982engzWk9RD0uq5Z6DXw7cJ6R1C2J6cTrMArk2uAX GtJ5slrQ5mLIliDTdd2R2zaYwgpRMIy/73GxFzu/TH3doK93oZgcacfOEWNfxTH8sM NJjfKZ5M1347f8s3IF4BcMJkAUfnOyEliAK48Fh8= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [89.245.37.17] ([89.245.37.17]) by web-mail.gmx.net (3c-app-gmx-bs48.server.lan [172.19.170.101]) (via HTTP); Sat, 25 May 2019 19:45:42 +0200 MIME-Version: 1.0 Message-ID: From: mj-mailinglist@gmx.de To: freebsd-pkgbase@freebsd.org Subject: How is the ${WSTAGEDIR}/METALOG file generated? Content-Type: text/plain; charset=UTF-8 Date: Sat, 25 May 2019 19:45:42 +0200 Importance: normal Sensitivity: Normal X-Priority: 3 X-Provags-ID: V03:K1:Is4yocOMsiawKVygheyXC81Qd66F9uvEVl8Ja6cJpTuWoOI6zlUumhaQ7pj4GhWbXJTrf HAgRPYCbpvkJMVrr45xnh5V62HkKqalZ5NljtJZMOiYEqeXgLUun45m64cMP9RJID4gDeeLBfBy/ Xm8/oW+EPKqr31+X2vkqhswY/U2WiAqvuV0mfKZ2J10K9vA58e+TWBLx9gRqH0CtbMMIvleLFbpY dEeOLfDkHz9VQG+Tivl5RWlYLVadqo8B/lMMWWaogXhkLmda46SJxFttUeJyJoyBshnRoq9J5zsw gI= X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:Ks2SVxwsPEs=:hDmFyM8Fqfq4fZL391lQuJ gg/CsAQIDUNSEzAWZ3BmEkP6pQrtZR0Z/jF0bc7bOBBz/njE7n0d5WnX5iLItj8a44tMJ2m2l fRESgU9C/g/E6mWPsPGB2Um1ShT958QXpxPnSPBNqDdEN0MI8HkNoHWcYPI2CByclgC4zy+FB xy4QPXfKQ4qZ9DgEyU8SYnVeYjFL7cNmEX3+3pk+jAfnKLVxQonRKsb0Hc32yiclQxYKlPfDK Thg1EGc9hyi1Lj5nt00ppKOVFKQ/fDrfRrLHiCC8mzQASXiZBQLYyt+enjNVa7ps/GtoHnNob ozT91A7AVfE2N6O/K3sXYI4AN5PURYmvq/Vib3cAnRkOM5wL7ggGfKrPbuhDgcK06uV7vlFk5 /R+at+syCaeLUOr/5Ltf6MGxDyt0Wc3T0DNj8eaHeYqY8RJEjbzM42SJEsqvfg2KTcY8nSlhq dCDOB1Ycrwk/Loir/7wtPbAHiy4fdtQ5Oq/DZSlbuCudanAaEI17iAD3YXEN8D/5u0QU04L85 GMhuVN8+wJ7TGLaHDxOLpvGrl1p098xX42KVk7SWG3jISY+eoW/svbmy/y6XoPRjHSm3xSsNb nBu2AfWePyfyyeX+4gulAL+PCyTub5ohzzJ+iYSI8pzWHWt+fWrXv4BA== Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 9771C82CFA X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=iE+cxBRq; spf=pass (mx1.freebsd.org: domain of mj-mailinglist@gmx.de designates 212.227.17.20 as permitted sender) smtp.mailfrom=mj-mailinglist@gmx.de X-Spamd-Result: default: False [-1.39 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:212.227.17.0/27]; FREEMAIL_FROM(0.00)[gmx.de]; TO_DN_NONE(0.00)[]; SUBJECT_HAS_CURRENCY(1.00)[]; DKIM_TRACE(0.00)[gmx.net:+]; MX_GOOD(-0.01)[mx00.emig.gmx.net,mx01.emig.gmx.net]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_SHORT(-0.23)[-0.235,0]; RCVD_IN_DNSWL_LOW(-0.10)[20.17.227.212.list.dnswl.org : 127.0.3.1]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmx.de]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; RECEIVED_SPAMHAUS_PBL(0.00)[17.37.245.89.zen.spamhaus.org : 127.0.0.10]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; SUBJECT_ENDS_QUESTION(1.00)[]; DMARC_NA(0.00)[gmx.de]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-1.05)[ip: (-6.33), ipnet: 212.227.0.0/16(-1.48), asn: 8560(2.58), country: DE(-0.01)]; FROM_NO_DN(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 May 2019 17:58:42 -0000 As far as i can see, the METALOG file plays an important role in pkgbase, it is used by the mtree-to-plist.awk script, to generate the per package p= list file, e.g runtime.plist, runtime-debug.plist, runtime-manuals.plist, etc. For the assignment of a file to a package the tag package=3D... seems rele= vent. e.g "./etc/mtree/BSD.root.dist type=3Dfile uname=3Droot gname=3Dwheel mode= =3D0444 size=3D1613 tags=3Dpackage=3Druntime" I found some places where these tags are defined (https://reviews.freebsd.= org/D20173), but how the entries end up in the METALOG file is hard to find. Can somebo= dy point me to the relevant places? To the METALOG file itself, here are some stats from a just finished "make packages" run (first line #mtree 2.0 removed): METALOG =2D------ lines total: 35293 with package tag: 33243 without package tag: 2051 type=3Ddir: 3370 type=3Dfile: 30484 type=3Dlink: 1440 there are a lot of duplicate entries in the file, e.g. 10x ./usr/share/snmp/mibs type=3Ddir uname=3Droot gname=3Dwheel mode=3D0= 755 tags=3Dpackage=3Druntime 13x ./usr/share/snmp/defs type=3Ddir uname=3Droot gname=3Dwheel mode=3D0= 755 tags=3Dpackage=3Druntime 15x ./boot type=3Ddir uname=3Droot gname=3Dwheel mode=3D0755 tags=3Dpack= age=3Druntime 79x ./etc type=3Ddir uname=3Droot gname=3Dwheel mode=3D0755 tags=3Dpacka= ge=3Druntime This seems not to be a problem, but pkg complains about files/dirs already added to the package. The stats for a sort -u METALOG look like this: METALOG.sortu =2D------------ lines total: 34640 with package tag: 32598 without package tag: 2042 type=3Ddir: 2884 type=3Dfile: 30316 type=3Dlink: 1440 there are still some semi duplicate entries, they differ only on added tag= s: ./etc type=3Ddir uname=3Droot gname=3Dwheel mode=3D0755 ./etc type=3Ddir uname=3Droot gname=3Dwheel mode=3D0755 tags=3Dpackage=3Dr= untime ... ./boot/lua type=3Ddir uname=3Droot gname=3Dwheel mode=3D0755 ./boot/lua type=3Ddir uname=3Droot gname=3Dwheel mode=3D0755 tags=3Dpackag= e=3Druntime ... ./etc/devd type=3Ddir uname=3Droot gname=3Dwheel mode=3D0755 ./etc/devd type=3Ddir uname=3Droot gname=3Dwheel mode=3D0755 tags=3Dpackag= e=3Druntime And one final stat: the sum of lines in all *.plist files: cat *.plist | wc -l: 33243 cat *.plist | sort -u | wc -l 32598 Is it neccessary to add package=3D... tags to the above 2051/2042 lines ab= ove? Are these files/dirs missing from packages? Directories could be created b= y pkg during installation, but i only found the relevant directive in the runtime.ucl file: ... directories { /dev =3D "y"; /tmp =3D "y"; ... =2D- Martin