From owner-freebsd-pkgbase@freebsd.org Thu Jun 7 15:49:39 2018 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 476A5FE1F63 for ; Thu, 7 Jun 2018 15:49:39 +0000 (UTC) (envelope-from egypcio@gmail.com) Received: from mail-pl0-x243.google.com (mail-pl0-x243.google.com [IPv6:2607:f8b0:400e:c01::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BEAF2708D8; Thu, 7 Jun 2018 15:49:38 +0000 (UTC) (envelope-from egypcio@gmail.com) Received: by mail-pl0-x243.google.com with SMTP id g20-v6so6405645plq.1; Thu, 07 Jun 2018 08:49:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=MkKAFwrZFC0ab90pXCJxSKpTJWr31/T1Xd5tCVeq+1g=; b=uheTBMEFd0AwSZuO50aAzCIK8L1w2rAmxl1E1PO4QoK9vjSgVM3NqKCS6rfau90XIC qwDw4XuX8qaFmlXceZHM+sYzwgMJWOw1uurI5zzU3A+FGOOX6JtR7/LB8/14qUX8jPJE v4kCwP9GKrNQFGMK8l7TLooFOy+R27Gk8kLxmhY7boIC4JNkBIrN2PZZyBgrkMCvC9uB SYfVg52zza5sjIC0s9uHBiQ3/jCbQ+rIPrBVl+Bzi1Ym9csuh1F95+bCYZw5sTqYQWdP CgaNjz8Ft7iq3EMsjUvdQzLjwM0L4TQE6iB0RdsfLspLImGj3P6SG/J5OEBWuKQBHpcq gCvA== X-Gm-Message-State: APt69E3BZo4uOx/ibMXyHMVVAITIEMTEdQUvkOB5ytVQRL7X/wgfK3Gl varamsDt9K40eZBTDMGSm5NaAj3oppGkCD+4ecvQAw== X-Google-Smtp-Source: ADUXVKJS8H7vRK3QSPPDWMdsbtxvk1ZSbKuPrPhBu2q30pBGJjKAswu4Fg/DSetR1tjgQ+AsuCj+R58ZDPL8jJpiOKU= X-Received: by 2002:a17:902:8341:: with SMTP id z1-v6mr2629729pln.40.1528386577713; Thu, 07 Jun 2018 08:49:37 -0700 (PDT) MIME-Version: 1.0 Sender: egypcio@gmail.com Received: by 2002:a17:90a:1fcc:0:0:0:0 with HTTP; Thu, 7 Jun 2018 08:49:37 -0700 (PDT) In-Reply-To: <20171113195144.GH53537@corpmail.liquidneon.com> References: <20170927163012.GA99112@FreeBSD.org> <20171113172054.GA23670@FreeBSD.org> <1972909.hTPcQf8OeH@ralph.baldwin.cx> <20171113195144.GH53537@corpmail.liquidneon.com> From: =?UTF-8?Q?Vin=C3=ADcius_Zavam?= Date: Thu, 7 Jun 2018 17:49:37 +0200 X-Google-Sender-Auth: eUjGTEtOiWpkZKdUZP7FKUoPbPk Message-ID: Subject: Re: Problems with packaging /etc and etcupdate(8) To: Brad Davis Cc: John Baldwin , Glen Barber , freebsd-pkgbase@freebsd.org, Will Andrews Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2018 15:49:39 -0000 2017-11-13 20:51 GMT+01:00 Brad Davis : > > On Mon, Nov 13, 2017 at 09:26:49AM -0800, John Baldwin wrote: > > I had assumed that in a pkgbase setup, one wouldn't use etcupdate or mergemaster. > > Instead, my understanding is that pkg was going to "learn" about configuration > > files and would grow etcupdate-like functionality to merge in changes to configuration > > files during upgrades (and would also support 'pkg confdiff', etc. so that > > 'pkg confdiff -a' would replace 'etcupdate diff' for example, but would also support > > packages built from ports, not just base system conf files). Each pkgbase package > > would contain its associated /etc files as configuration files in this scheme. > > To that end Will and I have been working on a patch to move all files in head/etc/ > to the places they belong. I am probably ~75% of the way through them so > far. Once I get the last bit wrapped up, I will post a review. > > > Regards, > Brad Davis I am sorry for getting back into this thread, but talking to some friends a while ago it occurred me that something I run at home could be useful for others here - related to etcupdate and updates using pkg-base (on HEAD). I'll not cover the delete-files things here, and I must admit that I was not following the mailing list for a while :\ so, after building world, kernel and creating the respective distfiles (base.txz, kernel.txz) my build server generates the "etc.tbz" using the following command: # etcupdate build -B -s /usr/src /dist/etc.tbz in another machine (my laptop, also running HEAD), where I extract the distfiles for base and kernel, here is how I deal with updating/upgrading: ~ fetch *.txz stuff, and etc.tbz ~ extract kernel.txz to / # rm -rf /var/db/etcupdate/previous # mkdir -p /var/db/etcupdate/current # mkdir -p /var/db/meu # tar --unlink -xjpf /whatever/etc.tbz -C /var/db/meu/etc for FILE in $(find /var/db/meu/etc -type f | cut -d\/ -f6-9); do d=3D$(dirname ${FILE}) f=3D$(basename ${FILE}) mkdir -p /var/db/etcupdate/previous/$d cp -a /$d/$f /var/db/etcupdate/previous/$d/$f 2>/dev/null done # etcupdate -p -F -t /var/db/meu/etc.tbz # etcupdate resolve -p # etcupdate diff > /var/db/meu/etc.diff.pre ~ extract base.txz to / # etcupdate -F -t /var/db/meu/etc.tbz # etcupdate resolve # etcupdate diff > /var/db/meu/etc.diff ~ reboot. well, it's indeed NOT an .ucl to be used direclty with pkg, but the idea of scripting it is there. right? -- Vin=C3=ADcius Zavam keybase.io/egypcio/key.asc From owner-freebsd-pkgbase@freebsd.org Thu Jun 7 15:54:58 2018 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D7FBFE27B1 for ; Thu, 7 Jun 2018 15:54:58 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-it0-x241.google.com (mail-it0-x241.google.com [IPv6:2607:f8b0:4001:c0b::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A8AF71001 for ; Thu, 7 Jun 2018 15:54:57 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-it0-x241.google.com with SMTP id l6-v6so13289495iti.2 for ; Thu, 07 Jun 2018 08:54:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:openpgp:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=fi3HlJbE2S6xFv3zWfM94M8NwWxoW2nBOARW3TRFXfg=; b=q7Y2+mK/gnbsg4xKsXoS5XF/1Y1765inKuwAf6BaUwAo5eY5+SZDLuN+HATPCV8bbC O7JAUNl2YjoMe3Ny2RZXHtnkBu8qQEOc/Vu1CTlQDsOX3+HihnPSKRAobHL8yEscXJrR Qzse8SNFC4R/tIn9+nQKFmIoPzNeC9pGZZtCXgWAf3tedYHJchEk7u3IK3o7tOXDrhwK 9kgEuR7/kO09N2iW0wMGpjiA0oSdWkER3D4X7557MDkKVa/raPnASSPJnGNWWhlPib1i aHJaeUQgGDuzo2tmKP9xaRDo/xqGravfp2Memf8OznZqoNJLRev0Ej9nqb6zJNN1iFqQ 03Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding:content-language; bh=fi3HlJbE2S6xFv3zWfM94M8NwWxoW2nBOARW3TRFXfg=; b=pws3y4O0y0zZpP1G/wQs+P+VgU8FCgDIfO4bCAjpdAZSo/5yQpBVyG6RgSBdsKmpJV YhuIExFUj2wDfPEf9QqLtCxyTvZiejcsdgOC0n0BJMPiVHGpbD9A0R/95qQEetjDkmbn DE7twDGX4zskwWOk/20RW8UxQg2QGxfuOlm5U1Gc0L+uPvQ2yG7yhKJunv/0udOIBGkO 7KRenGXeuqQ00+AaBp22llZrylimI0oRamVajyMYcD+DsjoGLyty1y3jD1eHCVDHbujH aOC1ec8SQLjrywiekSpLlee/cwPa3U1X8W3P2f3wLI7iWEwPlC1FnWnGbuUSVoNvD9S1 6Qnw== X-Gm-Message-State: APt69E2chO2v/VDIUMUixlLsMPq5xCFevHH/Vcv/Vm0rd7pF1Tof5tn0 UOhuPlJPzRl3cV+4udhgoAB8s9PvXOUGvyOQkg3LAfZ6BqOJakC5fMKMy6oQvNo8jaH9dfpGQue dyqnjHtyDmpSLwOrY/+DpLFK0mKR88yZXaWQHF3zg2dKyZzO8VUleh/Ig8YURRqKxeFSl X-Google-Smtp-Source: ADUXVKJlHNtBj1jsMPLdJy1Pl/8PmzBwjWu0C7lb62ck2qaicQ1nQCMmWF9PuW2Y5Xg+v3J4/iieXw== X-Received: by 2002:a24:4689:: with SMTP id j131-v6mr2418939itb.111.1528386895963; Thu, 07 Jun 2018 08:54:55 -0700 (PDT) Received: from [10.200.5.24] ([137.122.64.159]) by smtp.gmail.com with ESMTPSA id o9-v6sm873235itf.6.2018.06.07.08.54.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 07 Jun 2018 08:54:55 -0700 (PDT) Subject: Re: Problems with packaging /etc and etcupdate(8) To: freebsd-pkgbase@freebsd.org References: <20170927163012.GA99112@FreeBSD.org> <20171113172054.GA23670@FreeBSD.org> <1972909.hTPcQf8OeH@ralph.baldwin.cx> <20171113195144.GH53537@corpmail.liquidneon.com> From: Kris Moore Openpgp: preference=signencrypt Autocrypt: addr=kris@ixsystems.com; prefer-encrypt=mutual; keydata= xsBNBE5nfNMBCADxo/scoVqCZbXXeJTFET/xl/TWfjP5HtlP65F2LLzmjCcGz8/6B7lyCYDp yMawhEael7QPRBt/PCbc6fKspq3Rei+3IniPKkxfxfpUtsr1AIA6iMntzRWTa0DrW2C1NpDH 8MU5VXu91dNKE86umFZYqO54GUGmCEM06q0ZMCPBooUgQchNs8YztiEJzg6GokYQm4/3EHRJ ddQcyB7jupZYSNG49Hqm84QDeGBuXKJDcn7a6dn5ZvStFQO523Clv+83qYLdnaMPsQVAyj+w ifcC28jyGegvxK2gz5j2zcqmlK//3UEKiTwqoxsQM0McbE4PsVotpBDR5rSiydvRlimRABEB AAHNKUtyaXMgTW9vcmUgKG5vIHBhc3N3b3JkKSA8a3Jpc0BwY2JzZC5vcmc+wsB4BBMBAgAi AhsjBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAUCVHM5MAAKCRB/3CIMMFd81zyiB/9rKgfK 9i4pwSB6qYPkrNOavRqu3cLDL4H86k9PUBWeGWDtHYVqu6yhSxz3rUSyipNltmxl2IardlQl AJj2YCe+lrdRMejW4DpPhc8Yx1/OBcYlZ31SE98ns9tqrQVPNg7/rGazO/4osW8bwYvI61w2 W82fS7UzDb9++Qg2prsXo1YA5pKxCM52NX7UwkhREvBjDBlW2R3CubAyVcKqDXVb6ILCisFA mx0s78NucK6nbuHCfot31IIpD5wdnyY2qkLsLeFPnMpkrC8XpAh5mRkrlBw862vGjT2/GVR6 LwCGnHZNLawPcU53H1U3Nq50dDayhwk0ZCYuqSVC/rOQVaRezsBNBE5nfNMBCADD6cD19M4r W5hUuGnmrKhNH0d5j0xSUjQS0jM2Y7c52jra+qEYUstPr1oJX5IPG5fvi0ItIpQ1kU9O9GbA oLlL0rP16HmrWV+lzDK71L3m3jZg35pvXrhBjdygR5jozd8FpMH5FNCRtjjn7iiQQWv2u82A udJunI8xdB+HPtg0l/HdhWX5cf2Av/96G1zbA+r5bW6+k7pUM/XCMVYozRrScW7paK6PSkg5 XL3OdtA/PBUg0RoEdomeJlKU87Phs0qIJw6pG/Ng8ThoLLhtqFGCsPnj+UqBklzGQaC3kPeg dEHsf/xeiFdWo/AyKc9R6R4a1Kr0N8lQxCJBzvVz9BDrABEBAAHCwF8EGAECAAkCGwwFAlRz OTAACgkQf9wiDDBXfNf0SQf/bsIeG9s+lIC1muXPTjfZH5k1U1V0pFI2i4jjBOL+OfgqTpNx XeIs2NclW1A/xBRt9YuoTQJRZMOXZBxeMCEm1n5MLRj+zxCJmUWnQvEBE6gC/cadB7efqc70 /UVjmgyxKZTJzmUuJ2ZKbe/LekfZuzSQ83lX1pVT/gfCe2zJBfZwoNoNQ8lDBwT/fdleChLe qVlArrhxYtTdQCAfG4M3Uxs6OWQK3OQJO15aNjybQl514xLnQrLmLMD95zIuckGbGlSEJ+HO 0BZYPdcT3C1sIRnhxifbEd4wose9ydOsxxdVZr3mYarTjPlAVxAsX4nP2+K75ymFi/YH+ARE dxQWXA== Message-ID: <06be6c14-1a57-f508-77c7-dc81359b8135@ixsystems.com> Date: Thu, 7 Jun 2018 11:54:54 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2018 15:54:58 -0000 On 06/07/2018 11:49, Vinícius Zavam via freebsd-pkgbase wrote: > 2017-11-13 20:51 GMT+01:00 Brad Davis : >> On Mon, Nov 13, 2017 at 09:26:49AM -0800, John Baldwin wrote: >>> I had assumed that in a pkgbase setup, one wouldn't use etcupdate or > mergemaster. >>> Instead, my understanding is that pkg was going to "learn" about > configuration >>> files and would grow etcupdate-like functionality to merge in changes > to configuration >>> files during upgrades (and would also support 'pkg confdiff', etc. so > that >>> 'pkg confdiff -a' would replace 'etcupdate diff' for example, but would > also support >>> packages built from ports, not just base system conf files). Each > pkgbase package >>> would contain its associated /etc files as configuration files in this > scheme. >> To that end Will and I have been working on a patch to move all files in > head/etc/ >> to the places they belong. I am probably ~75% of the way through them so >> far. Once I get the last bit wrapped up, I will post a review. >> >> >> Regards, >> Brad Davis > I am sorry for getting back into this thread, but talking to some friends a > while ago it occurred me that something I run at home could be useful for > others here - related to etcupdate and updates using pkg-base (on HEAD). > I'll not cover the delete-files things here, and I must admit that I was > not following the mailing list for a while :\ > > so, after building world, kernel and creating the respective distfiles > (base.txz, kernel.txz) my build server generates the "etc.tbz" using the > following command: > > # etcupdate build -B -s /usr/src /dist/etc.tbz > > in another machine (my laptop, also running HEAD), where I extract the > distfiles for base and kernel, here is how I deal with updating/upgrading: > > ~ fetch *.txz stuff, and etc.tbz > ~ extract kernel.txz to / > > # rm -rf /var/db/etcupdate/previous > # mkdir -p /var/db/etcupdate/current > # mkdir -p /var/db/meu > > # tar --unlink -xjpf /whatever/etc.tbz -C /var/db/meu/etc > > for FILE in $(find /var/db/meu/etc -type f | cut -d\/ -f6-9); do > d=$(dirname ${FILE}) > f=$(basename ${FILE}) > mkdir -p /var/db/etcupdate/previous/$d > cp -a /$d/$f /var/db/etcupdate/previous/$d/$f 2>/dev/null > done > > # etcupdate -p -F -t /var/db/meu/etc.tbz > # etcupdate resolve -p > # etcupdate diff > /var/db/meu/etc.diff.pre > > ~ extract base.txz to / > > # etcupdate -F -t /var/db/meu/etc.tbz > # etcupdate resolve > # etcupdate diff > /var/db/meu/etc.diff > > ~ reboot. > > well, it's indeed NOT an .ucl to be used direclty with pkg, but the idea of > scripting it is there. right? > > > -- > Vinícius Zavam > keybase.io/egypcio/key.asc > _______________________________________________ > freebsd-pkgbase@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-pkgbase > To unsubscribe, send any mail to "freebsd-pkgbase-unsubscribe@freebsd.org" This is more or less how we are doing it for TrueOS. It works remarkably well. https://github.com/trueos/trueos/blob/trueos-master/Makefile.inc1#L1687 https://github.com/trueos/trueos/blob/trueos-master/release/packages/runtime.ucl -- Kris Moore Vice President of Engineering iXsystems Enterprise Storage & Servers Driven By Open Source