From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 3 14:19:46 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A004F1065672 for ; Sun, 3 Oct 2010 14:19:46 +0000 (UTC) (envelope-from onemda@gmail.com) Received: from mail-qy0-f177.google.com (mail-qy0-f177.google.com [209.85.216.177]) by mx1.freebsd.org (Postfix) with ESMTP id 5B7858FC13 for ; Sun, 3 Oct 2010 14:19:45 +0000 (UTC) Received: by qyk31 with SMTP id 31so4448265qyk.15 for ; Sun, 03 Oct 2010 07:19:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=JArGa0MIj7tyx5S0xN0xjvqPVItEoSJnzX9K05fpfdI=; b=tCkPxOPje+yVRznxGhkogptrvIRQr23oTjeeat3Z8e58hcz8ZCIpM/AX92hAG9ClAN AVnIvUlJSByjx1m60IhDWYoyc/arWa3Ni41xmJKynNMFQVyr49HIhbfWOR1TfEWzPO07 9gkUQXlEtLOzvxp6S+VG5cBq5xaUxr0DiOlyw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=MA5SrmrAM0mvAvP120LaNup/u3e4sHXWKKEFBYoy7Ebi1I7COMWo7Q5A6gHemQSBSm YiFMfzZLiPVwqR3ZvLEMULwqyfWroHNvnQFTZdpFSbfR+GUVJk7Mp5dQhEYb/EGDyFtO 32R54W2NZktmMs9jRW9DvmOGctsHHQPIE0OTw= MIME-Version: 1.0 Received: by 10.220.48.206 with SMTP id s14mr872825vcf.43.1286114006762; Sun, 03 Oct 2010 06:53:26 -0700 (PDT) Received: by 10.220.200.1 with HTTP; Sun, 3 Oct 2010 06:53:26 -0700 (PDT) Date: Sun, 3 Oct 2010 13:53:26 +0000 Message-ID: From: Paul B Mahol To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Oct 2010 14:19:46 -0000 Hi, diff --git a/tools/build/mk/OptionalObsoleteFiles.inc b/tools/build/mk/OptionalObsoleteFiles.inc index d3aa4b2..2123107 100644 --- a/tools/build/mk/OptionalObsoleteFiles.inc +++ b/tools/build/mk/OptionalObsoleteFiles.inc @@ -141,6 +141,7 @@ OLD_FILES+=usr/share/man/man8/authpf.8.gz .endif .if ${MK_BIND} == no +OLD_FILES+=etc/periodic/daily/470.status-named OLD_FILES+=usr/bin/dig OLD_FILES+=usr/bin/host OLD_FILES+=usr/bin/nslookup @@ -943,6 +944,7 @@ OLD_FILES+=rescue/ping6 #.endif .if ${MK_IPFILTER} == no +OLD_FILES+=etc/periodic/security/510.ipfdenied OLD_FILES+=rescue/ipf OLD_FILES+=sbin/ipf OLD_FILES+=sbin/ipfs @@ -1517,6 +1519,20 @@ OLD_FILES+=usr/share/man/man8/verify_krb5_conf.8.gz # to be filled in #.endif +.if ${MK_LOCATE} == no +OLD_FILES+=etc/locate.rc +OLD_FILES+=etc/periodic/weekly/310.locate +OLD_FILES+=usr/bin/locate +OLD_FILES+=usr/libexec/locate.bigram +OLD_FILES+=usr/libexec/locate.code +OLD_FILES+=usr/libexec/locate.concatdb +OLD_FILES+=usr/libexec/locate.mklocatedb +OLD_FILES+=usr/libexec/locate.updatedb +OLD_FILES+=usr/share/man/man1/locate.1.gz +OLD_FILES+=usr/share/man/man8/locate.updatedb.8.gz +OLD_FILES+=usr/share/man/man8/updatedb.8.gz +.endif + .if ${MK_LPR} == no OLD_FILES+=etc/hosts.lpd OLD_FILES+=etc/printcap @@ -1549,6 +1565,7 @@ OLD_FILES+=usr/share/man/man8/pac.8.gz .endif .if ${MK_MAIL} == no +OLD_FILES+=etc/periodic/daily/130.clean-msgs OLD_FILES+=usr/bin/Mail OLD_FILES+=usr/bin/biff OLD_FILES+=usr/bin/from @@ -1772,7 +1789,12 @@ OLD_FILES+=usr/share/man/man8/tftp-proxy.8.gz .endif .if ${MK_PKGTOOLS} == no +OLD_FILES+=etc/periodic/daily/490.status-pkg-changes OLD_FILES+=etc/periodic/weekly/400.status-pkg +OLD_FILES+=usr/include/pkg.h +OLD_FILES+=usr/lib/libpkg.a +OLD_FILES+=usr/lib/libpkg.so +OLD_LIBS+=usr/lib/libpkg.so.0 OLD_FILES+=usr/sbin/pkg_add OLD_FILES+=usr/sbin/pkg_create OLD_FILES+=usr/sbin/pkg_delete @@ -1976,6 +1998,11 @@ OLD_FILES+=usr/share/man/man8/rtquery.8.gz .endif .if ${MK_SENDMAIL} == no +OLD_FILES+=etc/periodic/daily/150.clean-hoststat +OLD_FILES+=etc/periodic/daily/210.backup-aliases +OLD_FILES+=etc/periodic/daily/440.status-mailq +OLD_FILES+=etc/periodic/daily/460.status-mail-rejects +OLD_FILES+=etc/periodic/daily/500.queuerun OLD_FILES+=bin/rmail OLD_FILES+=usr/bin/vacation OLD_FILES+=usr/include/libmilter/mfapi.h From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 3 18:25:08 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 712BE1065693 for ; Sun, 3 Oct 2010 18:25:08 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 12AD38FC1F for ; Sun, 3 Oct 2010 18:25:07 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1P2TFA-0000kI-4a for freebsd-hackers@freebsd.org; Sun, 03 Oct 2010 20:25:04 +0200 Received: from 93-141-115-47.adsl.net.t-com.hr ([93.141.115.47]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 03 Oct 2010 20:25:04 +0200 Received: from ivoras by 93-141-115-47.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 03 Oct 2010 20:25:04 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Sun, 03 Oct 2010 20:24:27 +0200 Lines: 72 Message-ID: References: <4CA4BCD2.4070303@freebsd.org> <4CA5A1B1.7050902@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 93-141-115-47.adsl.net.t-com.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100620 Thunderbird/3.0.4 In-Reply-To: <4CA5A1B1.7050902@freebsd.org> Cc: freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Oct 2010 18:25:08 -0000 On 10/01/10 10:54, Andre Oppermann wrote: > On 30.09.2010 19:51, Ivan Voras wrote: >> On 09/30/10 18:37, Andre Oppermann wrote: >> >>> Both the vmmap and page table make use of splay trees to manage the >>> entries and to speed up lookups compared to long to traverse linked >>> lists or more memory expensive hash tables. Some structures though >>> do have an additional linked list to simplify ordered traversals. >> >> The property of splay tree requiring *writes* for nearly every read >> really is a thorn in the eye for SMP. It seems to me that even if the >> immediate benefits from converting to something else are not directly >> observable, it will still be worth doing it. > > Fully agreed. > >> It's a shame that RCU is still a patent minefield :/ >> >> http://mirror.leaseweb.com/kernel/people/npiggin/patches/lockless/2.6.16-rc5/radix-intro.pdf >> > > I'm not convinced that RCU is the only scalable way of sharing a > data structure across a possibly large number of CPU's. Of course, it's just well understood currently. > The term "lockless" is often used and frequently misunderstood. > Having a lockess data structure *doesn't* mean that is either > performant, scalable or both. It heavily depends on a number > of additional factors. Many times "lockless" just replaces a > simple lock/unlock cycle with a number of atomic operations on > the data structure. This can easily backfire because an atomic Yes. > operation just hides the computational complexity and also dirties > the CPU's cache lines. Generally on cache coherent architectures > almost all of the atomic operation is in HW with bus lock cycles, > bus snooping and whatnot. While seemingly simple form the programmers > point of view, the overhead and latency is still there. Needless Yes, you basically just offload the operation to hardware but the steps it needs to make are the same in concept. > a) make sure the lock is held for only a small amount of time > to avoid lock contention. > b) do everything you can outside of the lock. > c) if the lock is found to be heavily contended rethink the > whole approach and check if other data structures can be used. > d) minimize write accesses to memory in the lock protected > shared data structure. > e) PROFILE, DON'T SPECULATE! Measure the access pattern and > measure the locking/data access strategy's cost in terms > of CPU cycles consumed. > > f) on lookup heavy data structures avoid writing to memory and > by it dirtying CPU caches. > g) on modify heavy data structures avoid touching too many > elements. > h) on lookup and modify heavy data structure that are used > across many CPU's all bets are off and a different data > structure approach should be considered resulting ideally > in case f). > > It all starts with the hypothesis that a data structure is not > optimally locked. This looks like material for a developer-centric wiki page :) There is a lot of dispersed wisdom in this thread which would be nice if gathered in one place. From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 3 19:42:45 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4BE41106566B for ; Sun, 3 Oct 2010 19:42:45 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 0443F8FC16 for ; Sun, 3 Oct 2010 19:42:44 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1P2USF-0008K5-Ra for freebsd-hackers@freebsd.org; Sun, 03 Oct 2010 21:42:39 +0200 Received: from 93-141-115-47.adsl.net.t-com.hr ([93.141.115.47]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 03 Oct 2010 21:42:39 +0200 Received: from ivoras by 93-141-115-47.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 03 Oct 2010 21:42:39 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Sun, 03 Oct 2010 21:42:29 +0200 Lines: 27 Message-ID: References: <4CA4BCD2.4070303@freebsd.org> <20101001182856.GF87427@hoeg.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 93-141-115-47.adsl.net.t-com.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100620 Thunderbird/3.0.4 In-Reply-To: <20101001182856.GF87427@hoeg.nl> Cc: freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Oct 2010 19:42:45 -0000 On 10/01/10 20:28, Ed Schouten wrote: > Andre, > > * Andre Oppermann wrote: >> A splay tree is an interesting binary search tree with insertion, >> lookup and removal performed in O(log n) *amortized* time. With >> the *amortized* time being the crucial difference to other binary trees. >> On every access *including* lookup it rotates the tree to make the >> just found element the new root node. For all gory details see: >> http://en.wikipedia.org/wiki/Splay_tree > > Even though a red-black tree is quite good since it guarantees a $2 \log > n$ upperbound, the problem is that it's quite computationally intensive. > > Maybe it would be worth looking at other types of balanced trees? For > example, another type of tree which has only $O(\log n)$ amortized > insertion/removal/lookup time, but could already be a lot better in > practice, is a Treap. How many elements are held in vm_map trees? From the source it looks like one tree with all pages in the system and then one per-process? Trees have very varied real-time characteristics, e.g.: http://attractivechaos.awardspace.com/udb.html http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.83.6795&rep=rep1&type=pdf From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 3 20:44:50 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18C971065670 for ; Sun, 3 Oct 2010 20:44:50 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id D0D688FC17 for ; Sun, 3 Oct 2010 20:44:49 +0000 (UTC) Received: by iwn10 with SMTP id 10so1439498iwn.13 for ; Sun, 03 Oct 2010 13:44:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=WWFImXhORzyxKdky/hfBzlj5jqpRGmrpi9YSEljC/+0=; b=VkqACXXQOc3wNn4eEXD508BgMyBvtJ5OJ6Ztpt8JBKm24XKOMnSo9xgIRSC8nBgX5A FFmj7QIPIbCs0fJBo84U7aiRjw/gh96iaYbL0KC2t1V3ljh8UnnQCwNDCELXsnVTXQzH GVGtn3PbJ89imu8eeuC+jHtAuqDfxE7huH14M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=XekneSFTBwf3bIHlEKyzzo9yRneSz23ERpxNtFIb9pw+tuWa3aHMQnZzI2FNkB2cj/ cAnxAujBtxgoxMj9QRw3QGfEgw1mUSZhxe+x8e85YkNPAsODt327yDPKVmrqD+TQgXIY WGyG24MV0pp1cKTyHp+6sj6HgO70VY0Fth7Y4= MIME-Version: 1.0 Received: by 10.231.172.70 with SMTP id k6mr9012552ibz.125.1286138688956; Sun, 03 Oct 2010 13:44:48 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.190.68 with HTTP; Sun, 3 Oct 2010 13:44:48 -0700 (PDT) In-Reply-To: References: Date: Sun, 3 Oct 2010 13:44:48 -0700 X-Google-Sender-Auth: 3Kj3I67n-e864SqqOXQzSBAmf_g Message-ID: From: Garrett Cooper To: Paul B Mahol Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Oct 2010 20:44:50 -0000 On Sun, Oct 3, 2010 at 6:53 AM, Paul B Mahol wrote: > Hi, > > > diff --git a/tools/build/mk/OptionalObsoleteFiles.inc > b/tools/build/mk/OptionalObsoleteFiles.inc > index d3aa4b2..2123107 100644 > --- a/tools/build/mk/OptionalObsoleteFiles.inc > +++ b/tools/build/mk/OptionalObsoleteFiles.inc > @@ -141,6 +141,7 @@ OLD_FILES+=3Dusr/share/man/man8/authpf.8.gz > =A0.endif > > =A0.if ${MK_BIND} =3D=3D no > +OLD_FILES+=3Detc/periodic/daily/470.status-named > =A0OLD_FILES+=3Dusr/bin/dig > =A0OLD_FILES+=3Dusr/bin/host > =A0OLD_FILES+=3Dusr/bin/nslookup > @@ -943,6 +944,7 @@ OLD_FILES+=3Drescue/ping6 > =A0#.endif > > =A0.if ${MK_IPFILTER} =3D=3D no > +OLD_FILES+=3Detc/periodic/security/510.ipfdenied > =A0OLD_FILES+=3Drescue/ipf > =A0OLD_FILES+=3Dsbin/ipf > =A0OLD_FILES+=3Dsbin/ipfs > @@ -1517,6 +1519,20 @@ OLD_FILES+=3Dusr/share/man/man8/verify_krb5_conf.8= .gz > =A0# to be filled in > =A0#.endif > > +.if ${MK_LOCATE} =3D=3D no > +OLD_FILES+=3Detc/locate.rc > +OLD_FILES+=3Detc/periodic/weekly/310.locate > +OLD_FILES+=3Dusr/bin/locate > +OLD_FILES+=3Dusr/libexec/locate.bigram > +OLD_FILES+=3Dusr/libexec/locate.code > +OLD_FILES+=3Dusr/libexec/locate.concatdb > +OLD_FILES+=3Dusr/libexec/locate.mklocatedb > +OLD_FILES+=3Dusr/libexec/locate.updatedb > +OLD_FILES+=3Dusr/share/man/man1/locate.1.gz > +OLD_FILES+=3Dusr/share/man/man8/locate.updatedb.8.gz > +OLD_FILES+=3Dusr/share/man/man8/updatedb.8.gz > +.endif > + > =A0.if ${MK_LPR} =3D=3D no > =A0OLD_FILES+=3Detc/hosts.lpd > =A0OLD_FILES+=3Detc/printcap > @@ -1549,6 +1565,7 @@ OLD_FILES+=3Dusr/share/man/man8/pac.8.gz > =A0.endif > > =A0.if ${MK_MAIL} =3D=3D no > +OLD_FILES+=3Detc/periodic/daily/130.clean-msgs > =A0OLD_FILES+=3Dusr/bin/Mail > =A0OLD_FILES+=3Dusr/bin/biff > =A0OLD_FILES+=3Dusr/bin/from > @@ -1772,7 +1789,12 @@ OLD_FILES+=3Dusr/share/man/man8/tftp-proxy.8.gz > =A0.endif > > =A0.if ${MK_PKGTOOLS} =3D=3D no > +OLD_FILES+=3Detc/periodic/daily/490.status-pkg-changes > =A0OLD_FILES+=3Detc/periodic/weekly/400.status-pkg > +OLD_FILES+=3Dusr/include/pkg.h > +OLD_FILES+=3Dusr/lib/libpkg.a > +OLD_FILES+=3Dusr/lib/libpkg.so > +OLD_LIBS+=3Dusr/lib/libpkg.so.0 > =A0OLD_FILES+=3Dusr/sbin/pkg_add > =A0OLD_FILES+=3Dusr/sbin/pkg_create > =A0OLD_FILES+=3Dusr/sbin/pkg_delete > @@ -1976,6 +1998,11 @@ OLD_FILES+=3Dusr/share/man/man8/rtquery.8.gz > =A0.endif > > =A0.if ${MK_SENDMAIL} =3D=3D no > +OLD_FILES+=3Detc/periodic/daily/150.clean-hoststat > +OLD_FILES+=3Detc/periodic/daily/210.backup-aliases > +OLD_FILES+=3Detc/periodic/daily/440.status-mailq > +OLD_FILES+=3Detc/periodic/daily/460.status-mail-rejects > +OLD_FILES+=3Detc/periodic/daily/500.queuerun > =A0OLD_FILES+=3Dbin/rmail > =A0OLD_FILES+=3Dusr/bin/vacation > =A0OLD_FILES+=3Dusr/include/libmilter/mfapi.h There are other things that need fixing in this file, but yes... I agree with the above. -Garrett From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 4 00:03:58 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E606A106564A for ; Mon, 4 Oct 2010 00:03:57 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id A8BB38FC1C for ; Mon, 4 Oct 2010 00:03:57 +0000 (UTC) Received: by iwn10 with SMTP id 10so1599031iwn.13 for ; Sun, 03 Oct 2010 17:03:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:reply-to :in-reply-to:references:date:message-id:subject:from:to:cc :content-type; bh=oqn9RR5Cp3DZ/mg4QfSkfgCO0MlGluLgrGVhsVzMMw4=; b=UPGqeN66PJyik3NZZ1+Iy8n+sJU1+1qxphpVVfhp1t37z6V8en6OR3D37Rx1tkcBi8 vSvJ4Fs46ufi299fgNlP3xEapyZWZySowtclV/b3lRBh4FfVPuOf21FaAx4tWaq/POA8 oHkrepEpwuWWK5C0k9KD6aFRPM/hmmcFD+SSQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; b=ifp7AE9TtpxMicVTYmsZ0yClgcLHefZrABll9E7KJf8CMWNSaubArpB81wwdzJGNo5 /jundiUrtFAXs1FLeWzIwXfc4zGSbxfUjcAdHQ8rAg8XXfYmyEC6VzerosW3fP16Y0/P zRf0UMwrgDqOjYBvarozeP7CMADKZzNhNWspI= MIME-Version: 1.0 Received: by 10.42.5.140 with SMTP id 12mr10568247icw.28.1286150636179; Sun, 03 Oct 2010 17:03:56 -0700 (PDT) Received: by 10.42.170.136 with HTTP; Sun, 3 Oct 2010 17:03:56 -0700 (PDT) In-Reply-To: References: <29760054.post@talk.nabble.com> Date: Sun, 3 Oct 2010 19:03:56 -0500 Message-ID: From: Alan Cox To: Svatopluk Kraus Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: page table fault, which should map kernel virtual address space X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alc@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Oct 2010 00:03:58 -0000 On Thu, Sep 30, 2010 at 6:28 AM, Svatopluk Kraus wrote: > On Tue, Sep 21, 2010 at 7:38 PM, Alan Cox wrote: > > On Mon, Sep 20, 2010 at 9:32 AM, Svatopluk Kraus > wrote: > >> Beyond 'kernel_map', some submaps of 'kernel_map' (buffer_map, > >> pager_map,...) exist as result of 'kmem_suballoc' function call. > >> When this submaps are used (for example 'kmem_alloc_nofault' > >> function) and its virtual address subspace is at the end of > >> used kernel virtual address space at the moment (and above 'NKPT' > >> preallocation), then missing page tables are not allocated > >> and double fault can happen. > >> > > > > No, the page tables are allocated. If you create a submap X of the > kernel > > map using kmem_suballoc(), then a vm_map_findspace() is performed by > > vm_map_find() on the kernel map to find space for the submap X. As you > note > > above, the call to vm_map_findspace() on the kernel map will call > > pmap_growkernel() if needed to extend the kernel page table. > > > > If you create another submap X' of X, then that submap X' can only map > > addresses that fall within the range for X. So, any necessary page table > > pages were allocated when X was created. > > You are right. Mea culpa. I was focused on a solution and made > too quick conclusion. The page table fault hitted in 'pager_map', > which is submap of 'clean_map' and when I debugged the problem > I didn't see a submap stuff as a whole. > > > That said, there may actually be a problem with the implementation of the > > superpage_align parameter to kmem_suballoc(). If a submap is created > with > > superpage_align equal to TRUE, but the submap's size is not a multiple of > > the superpage size, then vm_map_find() may not allocate a page table page > > for the last megabyte or so of the submap. > > > > There are only a few places where kmem_suballoc() is called with > > superpage_align set to TRUE. If you changed them to FALSE, that is an > easy > > way to test this hypothesis. > > Yes, it helps. > > My story is that the problem started up when I updated a project > ('coldfire' port) > based on FreeBSD 8.0. to FreeBSD current version. In the current version > the 'clean_map' submap is created with superpage_align set to TRUE. > > I have looked at vm_map_find() and debugged the page table fault once > again. > IMO, it looks that a do-while loop does not work in the function as > intended. > A vm_map_findspace() finds a space and calls pmap_growkernel() if needed. > A pmap_align_superpage() arrange the space but never call > pmap_growkernel(). > A vm_map_insert() inserts the aligned space into a map without error > and never call pmap_growkernel() and does not invoke loop iteration. > > I don't know too much about an idea how a virtual memory model is > implemented > and used in other modules. But it seems that it could be more reasonable to > align address space in vm_map_findspace() internally and not to loop > externally. > > I have tried to add a check in vm_map_insert() that checks the 'end' > parameter > against 'kernel_vm_end' variable and returns KERN_NO_SPACE error if needed. > In this case the loop in vm_map_find() works and I have no problem with > the page table fault. But 'kernel_vm_end' variable must be initializated > properly before first use of vm_map_insert(). The 'kernel_vm_end' variable > can be self-initializated in pmap_growkernel() in FreeBSD 8.0 (it is too > late), > but it was changed in current version ('i386' port). > > Thanks for your answer, but I'm still looking for permanent > and approved solution. > I have a patch that implements one possible fix for this problem. I'll probably commit that patch in the next day or two. Regards, Alan From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 4 02:33:19 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 27F521065672; Mon, 4 Oct 2010 02:33:19 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id BBD268FC0A; Mon, 4 Oct 2010 02:33:18 +0000 (UTC) Received: by iwn10 with SMTP id 10so1741516iwn.13 for ; Sun, 03 Oct 2010 19:33:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:cc:content-type; bh=ldcL971yTjEN8/KnZIsClufVTDv7Zo9gFzP3yKQUnAM=; b=O8oRCE0bbtS6oNA/rv3/VzSUYwS3wrzAr+/ey7Dn2IOazzcHxROPJXFi9mQe6oEGsP 4Ww3RfYyyXZqDsD56UlNvrKWFz/PBYx/lJnSfSHj04RIM9CtkzWR/I15eiaXNZPDcPW3 HK5M+edLx2lyRoAI8lSjGn0wKAcjm+x0V+QHo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=RC0ZGiQNXUxqioJ/utTHukwEBglrVv8ddW8VFOVinXrH3mUi2kE6VOsa3Mtdl7vF6u 5ffmmYu2hMjlKcpXQRCSfcfi0lbljaKNyOt0XayQRdwbHuQDpWJAlOPhx6ehuz1KSEx3 WUUdUFi4eIRH0zVdDgeZeevS3U9RAutMFfFKk= MIME-Version: 1.0 Received: by 10.231.162.13 with SMTP id t13mr9331231ibx.160.1286159597965; Sun, 03 Oct 2010 19:33:17 -0700 (PDT) Received: by 10.231.190.68 with HTTP; Sun, 3 Oct 2010 19:33:17 -0700 (PDT) Date: Sun, 3 Oct 2010 19:33:17 -0700 Message-ID: From: Garrett Cooper To: John Baldwin , scottl@freebsd.org, sbruno@freebsd.org Content-Type: multipart/mixed; boundary=00504501400cf612690491c15d0f X-Mailman-Approved-At: Mon, 04 Oct 2010 03:05:37 +0000 Cc: freebsd-hackers@freebsd.org Subject: Fix mfiutil compile with -DDEBUG X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Oct 2010 02:33:19 -0000 --00504501400cf612690491c15d0f Content-Type: text/plain; charset=ISO-8859-1 make -DDEBUG is broken in mfiutil: $ make -DDEBUG cc -O2 -pipe -fno-strict-aliasing -pipe -O2 -march=nocona -fno-builtin-strftime -DDEBUG -Wall -Wcast-align -Woverflow -Wsign-compare -Wunused -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-uninitialized -Wno-pointer-sign -c /usr/src/usr.sbin/mfiutil/mfi_config.c /usr/src/usr.sbin/mfiutil/mfi_config.c: In function 'dump_config': /usr/src/usr.sbin/mfiutil/mfi_config.c:1027: error: 'union mfi_pd_ref' has no member named 'device_id' /usr/src/usr.sbin/mfiutil/mfi_config.c:1083: error: 'union mfi_pd_ref' has no member named 'device_id' *** Error code 1 Stop in /usr/src/usr.sbin/mfiutil. $ device_id is a field in the v field in the mfi_pd_ref union (/sys/dev/mfi/mfireg.h): union mfi_pd_ref { struct { uint16_t device_id; uint16_t seq_num; } v; uint32_t ref; } __packed; The attached patch fixes the compile (and produces logical results) with make -DDEBUG: $ sudo /usr/obj/usr/src/usr.sbin/mfiutil/mfiutil debug mfi0 Configuration (Debug): 1 arrays, 1 volumes, 0 spares array size: 288 volume size: 256 spare size: 40 array 0 of 4 drives: size = 1951170560 drive 4 ONLINE raw size: 1953525168 non-coerced size: 1952476592 coerced size: 1951170560 drive 5 ONLINE raw size: 1953525168 non-coerced size: 1952476592 coerced size: 1951170560 drive 6 ONLINE raw size: 1953525168 non-coerced size: 1952476592 coerced size: 1951170560 drive 7 ONLINE raw size: 1953525168 non-coerced size: 1952476592 coerced size: 1951170560 volume mfid0 RAID-6 OPTIMAL primary raid level: 6 raid level qualifier: 3 secondary raid level: 0 stripe size: 7 num drives: 4 init state: 0 consistent: 1 no bgi: 0 spans: array 0 @ 0 : 1951170560 Thanks! -Garrett --00504501400cf612690491c15d0f Content-Type: application/octet-stream; name="fix-mfiutil-debug.diff" Content-Disposition: attachment; filename="fix-mfiutil-debug.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_geupz4eb0 SW5kZXg6IHVzci5zYmluL21maXV0aWwvbWZpX2NvbmZpZy5jCj09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHVzci5z YmluL21maXV0aWwvbWZpX2NvbmZpZy5jCShyZXZpc2lvbiAyMTMzNzcpCisrKyB1c3Iuc2Jpbi9t Zml1dGlsL21maV9jb25maWcuYwkod29ya2luZyBjb3B5KQpAQCAtMTAyNCw3ICsxMDI0LDcgQEAK IAkJICAgIGFyLT5udW1fZHJpdmVzKTsKIAkJcHJpbnRmKCIgICAgICBzaXplID0gJWp1XG4iLCAo dWludG1heF90KWFyLT5zaXplKTsKIAkJZm9yIChqID0gMDsgaiA8IGFyLT5udW1fZHJpdmVzOyBq KyspIHsKLQkJCWRldmljZV9pZCA9IGFyLT5wZFtqXS5yZWYuZGV2aWNlX2lkOworCQkJZGV2aWNl X2lkID0gYXItPnBkW2pdLnJlZi52LmRldmljZV9pZDsKIAkJCWlmIChkZXZpY2VfaWQgPT0gMHhm ZmZmKQogCQkJCXByaW50ZigiICAgICAgICBkcml2ZSBNSVNTSU5HXG4iKTsKIAkJCWVsc2UgewpA QCAtMTA4MCw3ICsxMDgwLDcgQEAKIAkJc3AgPSAoc3RydWN0IG1maV9zcGFyZSAqKXA7CiAJCXBy aW50ZigiICAgICVzIHNwYXJlICV1ICIsCiAJCSAgICBzcC0+c3BhcmVfdHlwZSAmIE1GSV9TUEFS RV9ERURJQ0FURUQgPyAiZGVkaWNhdGVkIiA6Ci0JCSAgICAiZ2xvYmFsIiwgc3AtPnJlZi5kZXZp Y2VfaWQpOworCQkgICAgImdsb2JhbCIsIHNwLT5yZWYudi5kZXZpY2VfaWQpOwogCQlwcmludGYo IiVzIiwgbWZpX3Bkc3RhdGUoTUZJX1BEX1NUQVRFX0hPVF9TUEFSRSkpOwogCQlwcmludGYoIiBi YWNrczpcbiIpOwogCQlmb3IgKGogPSAwOyBqIDwgc3AtPmFycmF5X2NvdW50OyBqKyspCg== --00504501400cf612690491c15d0f-- From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 4 03:32:02 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B8228106564A; Mon, 4 Oct 2010 03:32:02 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4A65C8FC13; Mon, 4 Oct 2010 03:32:02 +0000 (UTC) Received: by iwn10 with SMTP id 10so1797311iwn.13 for ; Sun, 03 Oct 2010 20:32:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=xXoZFGzbx/xzBIKzdFRDbET2vQwXnCFyMHUmLsCzKAw=; b=ZnSrndG7TzOWOcG/eKWdRTyD3ZI1rCi5rjH9WXKvjlr9YZXnbuZlXn7ApTdka3mwg2 w+HPc4KSfOPKwQY68qDSQ/iFz7D4bQ02aghPVeb0IAvDEfnuSA9x5WPb9kVOcY1qi/Le RTDoqJVVMXrZeOsY3pRGEjLKVyf8WAWPXBPYQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=snrnqYNjNtgol1EuryBIuTIuRdcRL9XQ24OI2bxaw1HbdjIdZ/m1bMMvh0ynorCyXW b0h7aMfZ54Fw+GRIQiAJw5r14AfQqbpJ4gR6YtBvWolmDFCvjVRdhlbBBMA88TSkf+xj EBD3McUZa2nLziNNVwGcxTGqPBjyOUz0sKbKE= MIME-Version: 1.0 Received: by 10.231.152.143 with SMTP id g15mr9328983ibw.76.1286163121805; Sun, 03 Oct 2010 20:32:01 -0700 (PDT) Received: by 10.231.190.68 with HTTP; Sun, 3 Oct 2010 20:32:01 -0700 (PDT) In-Reply-To: References: Date: Sun, 3 Oct 2010 20:32:01 -0700 Message-ID: From: Garrett Cooper To: John Baldwin , scottl@freebsd.org, sbruno@freebsd.org Content-Type: multipart/mixed; boundary=001636d34573ffaadf0491c22f70 Cc: freebsd-hackers@freebsd.org Subject: Re: Make mfiutil(8) more robust X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Oct 2010 03:32:02 -0000 --001636d34573ffaadf0491c22f70 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Sun, Oct 3, 2010 at 8:30 PM, Garrett Cooper wrote: > =A0 =A0As discussed offlist with some of the Yahoo! FreeBSD folks, > mfiutil catches errors, but doesn't communicate it back up to the > executing process. Examples follow... > =A0 =A0Before: > > $ ./mfiutil show adapter > mfiutil: mfi_open: Permission denied > $ echo $? > 0 > > =A0 =A0(I would expect it to exit with a non-zero exit code) > =A0 =A0After: > > $ ./mfiutil show adapter > mfiutil: mfi_open: Permission denied > $ echo $? > 1 > > =A0 =A0(Much better!) > > $ sudo ./mfiutil show adapter > mfi0 Adapter: > =A0 =A0Product Name: MegaRAID SAS 8704ELP > =A0 Serial Number: P391734409 > =A0 =A0 =A0 =A0Firmware: 11.0.1-0026 > =A0 =A0 RAID Levels: JBOD, RAID0, RAID1, RAID5, RAID6, RAID10, RAID50 > =A0Battery Backup: present > =A0 =A0 =A0 =A0 =A0 NVRAM: 32K > =A0Onboard Memory: 128M > =A0Minimum Stripe: 8K > =A0Maximum Stripe: 1M > $ echo $? > 0 > > =A0 =A0(Looks good too!) > > =A0 =A0After (mptutil): > > $ ../mptutil/mptutil show adapter > mptutil: mpt_open: No such file or directory > $ echo $? > 1 > > =A0 =A0(Well, I don't have a mpt(4) card, so...) > > =A0 =A0This trivial patch fixes this code to report valid errors in the > handlers. I've added code to catch errors with mptutil as well as the > logic is basically the same, and it functions as expected in the > negative case; I don't have the hardware to test the mptutil command > in the positive case though to ensure that functions as expected. > Thanks! Adding hackers@ for visibility with mptutil. Thanks, -Garrett --001636d34573ffaadf0491c22f70 Content-Type: application/octet-stream; name="mptutil-capture-handler-errors.diff" Content-Disposition: attachment; filename="mptutil-capture-handler-errors.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_geus8v4s2 SW5kZXg6IG1wdHV0aWwuYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBtcHR1dGlsLmMJKHJldmlzaW9uIDIxMzM4 MCkKKysrIG1wdHV0aWwuYwkod29ya2luZyBjb3B5KQpAQCAtMTE0LDEwICsxMTQsMTIgQEAKIAog CVNFVF9GT1JFQUNIKGNtZCwgTVBUX0RBVEFTRVQodG9wKSkgewogCQlpZiAoc3RyY21wKCgqY21k KS0+bmFtZSwgYXZbMF0pID09IDApIHsKLQkJCSgqY21kKS0+aGFuZGxlcihhYywgYXYpOwotCQkJ cmV0dXJuICgwKTsKKwkJCWlmICgoKmNtZCktPmhhbmRsZXIoYWMsIGF2KSkKKwkJCQlyZXR1cm4g KDEpOworCQkJZWxzZQorCQkJCXJldHVybiAoMCk7CiAJCX0KIAl9CiAJd2FybngoIlVua25vd24g Y29tbWFuZCAlcy4iLCBhdlswXSk7Ci0JcmV0dXJuICgwKTsKKwlyZXR1cm4gKDEpOwogfQo= --001636d34573ffaadf0491c22f70 Content-Type: application/octet-stream; name="mfiutil-capture-handler-errors.diff" Content-Disposition: attachment; filename="mfiutil-capture-handler-errors.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_geus95as3 SW5kZXg6IG1maXV0aWwuYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBtZml1dGlsLmMJKHJldmlzaW9uIDIxMTc2 NykKKysrIG1maXV0aWwuYwkod29ya2luZyBjb3B5KQpAQCAtMTI1LDEwICsxMjUsMTIgQEAKIAog CVNFVF9GT1JFQUNIKGNtZCwgTUZJX0RBVEFTRVQodG9wKSkgewogCQlpZiAoc3RyY21wKCgqY21k KS0+bmFtZSwgYXZbMF0pID09IDApIHsKLQkJCSgqY21kKS0+aGFuZGxlcihhYywgYXYpOwotCQkJ cmV0dXJuICgwKTsKKwkJCWlmICgoKmNtZCktPmhhbmRsZXIoYWMsIGF2KSkKKwkJCQlyZXR1cm4g KDEpOworCQkJZWxzZQorCQkJCXJldHVybiAoMCk7CiAJCX0KIAl9CiAJd2FybngoIlVua25vd24g Y29tbWFuZCAlcy4iLCBhdlswXSk7Ci0JcmV0dXJuICgwKTsKKwlyZXR1cm4gKDEpOwogfQo= --001636d34573ffaadf0491c22f70-- From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 4 10:23:37 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE33A106564A for ; Mon, 4 Oct 2010 10:23:37 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 637948FC19 for ; Mon, 4 Oct 2010 10:23:37 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3B4E4.dip.t-dialin.net [87.179.180.228]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 49FE384400C; Mon, 4 Oct 2010 12:23:34 +0200 (CEST) Received: from webmail.leidinger.net (unknown [IPv6:fd73:10c7:2053:1::2:102]) by outgoing.leidinger.net (Postfix) with ESMTP id 59487163F; Mon, 4 Oct 2010 12:23:31 +0200 (CEST) Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.13.8/Submit) id o94ANP7o085584; Mon, 4 Oct 2010 12:23:25 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Mon, 04 Oct 2010 12:23:25 +0200 Message-ID: <20101004122325.11342p9v1rnwp6w4@webmail.leidinger.net> Date: Mon, 04 Oct 2010 12:23:25 +0200 From: Alexander Leidinger To: Paul B Mahol References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 49FE384400C.A5859 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=1.274, required 6, autolearn=disabled, RDNS_NONE 1.27) X-EBL-MailScanner-SpamScore: s X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1286792615.13301@xJY/nc5Ls4abVwCxPgIebw X-EBL-Spam-Status: No X-Mailman-Approved-At: Mon, 04 Oct 2010 10:45:47 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Oct 2010 10:23:37 -0000 Quoting Paul B Mahol (from Sun, 3 Oct 2010 13:53:26 +0000): > Hi, > > > diff --git a/tools/build/mk/OptionalObsoleteFiles.inc > b/tools/build/mk/OptionalObsoleteFiles.inc > index d3aa4b2..2123107 100644 > --- a/tools/build/mk/OptionalObsoleteFiles.inc > +++ b/tools/build/mk/OptionalObsoleteFiles.inc > @@ -141,6 +141,7 @@ OLD_FILES+=usr/share/man/man8/authpf.8.gz > .endif > > .if ${MK_BIND} == no > +OLD_FILES+=etc/periodic/daily/470.status-named If bind is installed from ports instead, this file could be useful, isn't it? > OLD_FILES+=usr/bin/dig > OLD_FILES+=usr/bin/host > OLD_FILES+=usr/bin/nslookup [...] > @@ -1976,6 +1998,11 @@ OLD_FILES+=usr/share/man/man8/rtquery.8.gz > .endif > > .if ${MK_SENDMAIL} == no > +OLD_FILES+=etc/periodic/daily/150.clean-hoststat > +OLD_FILES+=etc/periodic/daily/210.backup-aliases > +OLD_FILES+=etc/periodic/daily/440.status-mailq > +OLD_FILES+=etc/periodic/daily/460.status-mail-rejects > +OLD_FILES+=etc/periodic/daily/500.queuerun > OLD_FILES+=bin/rmail > OLD_FILES+=usr/bin/vacation > OLD_FILES+=usr/include/libmilter/mfapi.h At least status-mailq is still useful with other MTAs, e.g. if you use postfix (and assuming the mailwrapper is configured correctly), this should still work. Depending on the setup of the aliases, backup-aliases will still work too. I haven't checked for the other ones. In general: please submit via send-pr (or similar) to GNATS. This way the patch should not be lost. Bye, Alexander. -- There are two ways of disliking art. One is to dislike it. The other is to like it rationally. -- Oscar Wilde http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 4 11:02:21 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A43D106566C for ; Mon, 4 Oct 2010 11:02:21 +0000 (UTC) (envelope-from onemda@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id D99908FC1A for ; Mon, 4 Oct 2010 11:02:19 +0000 (UTC) Received: by pzk1 with SMTP id 1so622356pzk.13 for ; Mon, 04 Oct 2010 04:02:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=QhaEjdtD7RzF2fJ8GL1AmhCgwkxDGnI4NYrSVMHL7vM=; b=xLaPMClZc50K++wtzT28CALBD5WPVhHdG1/f285b1eOHV5KOkhyj0L/AQ+s6X2DS8U 95g17YJJB2PhWsJsFiuVKgnBeXJjVDLBxzp5qFZwr/00ES9iKNzlVp0+qet5xbnV4FpU hXI1KdKse6dkNpZEBbQOU8mjCcj8wXKr91PVI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=H7aRngnT5+1QqC12cP8YAuS8bDAfatQOLW0xRx2i50elvghLUnmejNTmMkjKZ+IC7l uZK7gOGKrabbbh2ABuYPTiMD4eo7lIiVxLL80mR45Wr50+MKju/nDT2ciVVcQ7C9FyzQ 6CvUKn3/+W7mydDw4RebQRs4Prl+/hu8GoVUI= MIME-Version: 1.0 Received: by 10.142.134.11 with SMTP id h11mr4132753wfd.302.1286190106074; Mon, 04 Oct 2010 04:01:46 -0700 (PDT) Received: by 10.220.200.1 with HTTP; Mon, 4 Oct 2010 04:01:45 -0700 (PDT) In-Reply-To: <20101004122325.11342p9v1rnwp6w4@webmail.leidinger.net> References: <20101004122325.11342p9v1rnwp6w4@webmail.leidinger.net> Date: Mon, 4 Oct 2010 11:01:45 +0000 Message-ID: From: Paul B Mahol To: Alexander Leidinger Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org Subject: Re: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Oct 2010 11:02:21 -0000 On 10/4/10, Alexander Leidinger wrote: > Quoting Paul B Mahol (from Sun, 3 Oct 2010 13:53:26 > +0000): > >> Hi, >> >> >> diff --git a/tools/build/mk/OptionalObsoleteFiles.inc >> b/tools/build/mk/OptionalObsoleteFiles.inc >> index d3aa4b2..2123107 100644 >> --- a/tools/build/mk/OptionalObsoleteFiles.inc >> +++ b/tools/build/mk/OptionalObsoleteFiles.inc >> @@ -141,6 +141,7 @@ OLD_FILES+=usr/share/man/man8/authpf.8.gz >> .endif >> >> .if ${MK_BIND} == no >> +OLD_FILES+=etc/periodic/daily/470.status-named > > If bind is installed from ports instead, this file could be useful, isn't > it? > >> OLD_FILES+=usr/bin/dig >> OLD_FILES+=usr/bin/host >> OLD_FILES+=usr/bin/nslookup > > [...] > >> @@ -1976,6 +1998,11 @@ OLD_FILES+=usr/share/man/man8/rtquery.8.gz >> .endif >> >> .if ${MK_SENDMAIL} == no >> +OLD_FILES+=etc/periodic/daily/150.clean-hoststat >> +OLD_FILES+=etc/periodic/daily/210.backup-aliases >> +OLD_FILES+=etc/periodic/daily/440.status-mailq >> +OLD_FILES+=etc/periodic/daily/460.status-mail-rejects >> +OLD_FILES+=etc/periodic/daily/500.queuerun >> OLD_FILES+=bin/rmail >> OLD_FILES+=usr/bin/vacation >> OLD_FILES+=usr/include/libmilter/mfapi.h > > At least status-mailq is still useful with other MTAs, e.g. if you use > postfix (and assuming the mailwrapper is configured correctly), this > should still work. Depending on the setup of the aliases, > backup-aliases will still work too. I haven't checked for the other > ones. If you want to keep some "old" files, you just need to press `n' instead of `y' when deleting files. You are not forced to run make delete-old at all. When installing world in new environment such files are not installed anyway. > > In general: please submit via send-pr (or similar) to GNATS. This way > the patch should not be lost. > > Bye, > Alexander. > > -- > There are two ways of disliking art. > One is to dislike it. > The other is to like it rationally. > -- Oscar Wilde > > http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 > http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 4 12:36:39 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CFA201065695 for ; Mon, 4 Oct 2010 12:36:39 +0000 (UTC) (envelope-from roam@ringlet.net) Received: from praag.hoster.bg (praag.hoster.bg [77.77.142.10]) by mx1.freebsd.org (Postfix) with ESMTP id 4E06D8FC15 for ; Mon, 4 Oct 2010 12:36:38 +0000 (UTC) Received: from middenheim.hoster.bg (middenheim.hoster.bg [77.77.142.11]) by praag.hoster.bg (Postfix) with ESMTP id 2543B8CACC for ; Mon, 4 Oct 2010 15:36:37 +0300 (EEST) Received: from straylight.ringlet.net (nat90.cnsys.bg [85.95.80.90]) (Authenticated sender: roam@hoster.bg) by mail.hoster.bg (Postfix) with ESMTP id 301945C283 for ; Mon, 4 Oct 2010 15:36:30 +0300 (EEST) Received: from roam (uid 1000) (envelope-from roam@ringlet.net) id 416069 by straylight.ringlet.net (DragonFly Mail Agent) Mon, 04 Oct 2010 15:36:29 +0300 Date: Mon, 4 Oct 2010 15:36:29 +0300 From: Peter Pentchev To: Paul B Mahol Message-ID: <20101004123629.GA4308@straylight.ringlet.net> Mail-Followup-To: Paul B Mahol , Alexander Leidinger , freebsd-hackers@freebsd.org References: <20101004122325.11342p9v1rnwp6w4@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="NzB8fVQJ5HfG6fxh" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-MailScanner-ID: 301945C283.AA137 X-hoster-MailScanner: Found to be clean X-hoster-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.001, required 10, autolearn=disabled, UNPARSEABLE_RELAY 0.00) X-hoster-MailScanner-From: roam@ringlet.net X-hoster-MailScanner-To: freebsd-hackers@freebsd.org X-Spam-Status: No Cc: Alexander Leidinger , freebsd-hackers@freebsd.org Subject: Re: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Oct 2010 12:36:39 -0000 --NzB8fVQJ5HfG6fxh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 04, 2010 at 11:01:45AM +0000, Paul B Mahol wrote: > On 10/4/10, Alexander Leidinger wrote: > > Quoting Paul B Mahol (from Sun, 3 Oct 2010 13:53:26 > > +0000): > > > >> Hi, > >> > >> > >> diff --git a/tools/build/mk/OptionalObsoleteFiles.inc > >> b/tools/build/mk/OptionalObsoleteFiles.inc > >> index d3aa4b2..2123107 100644 > >> --- a/tools/build/mk/OptionalObsoleteFiles.inc > >> +++ b/tools/build/mk/OptionalObsoleteFiles.inc > >> @@ -141,6 +141,7 @@ OLD_FILES+=3Dusr/share/man/man8/authpf.8.gz > >> .endif > >> > >> .if ${MK_BIND} =3D=3D no > >> +OLD_FILES+=3Detc/periodic/daily/470.status-named > > > > If bind is installed from ports instead, this file could be useful, isn= 't > > it? > > > >> OLD_FILES+=3Dusr/bin/dig > >> OLD_FILES+=3Dusr/bin/host > >> OLD_FILES+=3Dusr/bin/nslookup > > > > [...] > > > >> @@ -1976,6 +1998,11 @@ OLD_FILES+=3Dusr/share/man/man8/rtquery.8.gz > >> .endif > >> > >> .if ${MK_SENDMAIL} =3D=3D no > >> +OLD_FILES+=3Detc/periodic/daily/150.clean-hoststat > >> +OLD_FILES+=3Detc/periodic/daily/210.backup-aliases > >> +OLD_FILES+=3Detc/periodic/daily/440.status-mailq > >> +OLD_FILES+=3Detc/periodic/daily/460.status-mail-rejects > >> +OLD_FILES+=3Detc/periodic/daily/500.queuerun > >> OLD_FILES+=3Dbin/rmail > >> OLD_FILES+=3Dusr/bin/vacation > >> OLD_FILES+=3Dusr/include/libmilter/mfapi.h > > > > At least status-mailq is still useful with other MTAs, e.g. if you use > > postfix (and assuming the mailwrapper is configured correctly), this > > should still work. Depending on the setup of the aliases, > > backup-aliases will still work too. I haven't checked for the other > > ones. >=20 > If you want to keep some "old" files, you just need to press `n' instead = of `y' > when deleting files. You are not forced to run make delete-old at all. On the other hand, doing this (skipping those files) on each and every syst= em update (say, on a development or testing machine) could get a bit... boring= - not to mention that at some point you'll learn to "oh, just say no", and th= en forget to remove something important. G'luck, Peter --=20 Peter Pentchev roam@space.bg roam@ringlet.net roam@FreeBSD.org PGP key: http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint FDBA FD79 C26F 3C51 C95E DF9E ED18 B68D 1619 4553 Do you think anybody has ever had *precisely this thought* before? --NzB8fVQJ5HfG6fxh Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iQIcBAEBCgAGBQJMqcpJAAoJEGUe77AlJ98TDFkP/0+tlP8DW92HFB+LQVX7QyTr WYac0+AFfIAwc2bSFtKoI78KaeEU6uBnMCG2Y++eQgPF95gsFZRxyYJ0KV3ipl2s oRrqjzjAYw8Q3LFSEoyUv+EUIWWJyohHFAZMVPDUwjKRBuWd3k5Rf2vKkvYOBUUd x8IZPIMBpt45sU8C4SlDMZ3PLyjsmFjdkNGDCDxMeghQGB3JX9tE4ck93BM52d0S /2Vd6+fj85PH8fGb4eN2cGM/T9+yn+QNopwINbgNi8nVmB36BRsilCrVWh/JINbl Ks+6dONcsZCwbugCWnBYGAohpZJNdZIjVgaoYq9ddhRcwi2eBriROShMzp1bPo3e C0rvddTCu0Lqz0HzIoV7/1nTaAPst4QEV4Tuqtx80DP7Hq3/zlRJFPD5QINW1ggx 3KmiJ8YUiibgWse3tt7ws08Bdq0WGi2ii84dW4bUA/f6pc8KNGEsnqPRUKzIY/Ow 62qfYmWI4RzMAxpwrUgrIfdNEqk2MalUbuTd6D2Awd3Kbv40yBGNj2dm+AhwSaWA plapFjm8fl9uAoeY1PGeBCnJFG5/aR0D7MQx807px69UzhX8Ob37ndLVkt33Jhai rjnG5Iem6lV5fMoUSNXDWJWyYjejfBF2gpGH+9Id19yNmfDguio40QMWHxmeezMd qlOlRU5sC+9+7poNBtQm =EMdK -----END PGP SIGNATURE----- --NzB8fVQJ5HfG6fxh-- From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 4 15:31:32 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 666D51065693 for ; Mon, 4 Oct 2010 15:31:32 +0000 (UTC) (envelope-from onemda@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id E4F848FC17 for ; Mon, 4 Oct 2010 15:31:31 +0000 (UTC) Received: by fxm9 with SMTP id 9so4297802fxm.13 for ; Mon, 04 Oct 2010 08:31:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=O2QNQ3VKUhgtn6t7lHPcakkfaSh5yy9nXM2zx8dwRzE=; b=FRfnBjrLL1FEFrHv+ChZxo7yi3Ks4HclFQJj0xYZmU5u+bVtmU62BBnYCqEq+VfT3r p5HNJzNrJpr/dCRsjfP9F7/Ah1Nc75vK7VlwYcu/ojF7sJrg1ls+6vCVVZxhNyYjQjZI p0+sqkE2m3t0AVdFXSt06i3d0fqIvulMRiGw8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=xrVq28IY1jLK6gvxnfMqgaIwbN690PxbDvE4nXMyfY8V4SZaARUCXk7OxEo7yS/oo1 wj3ip9pligv7cXOcQy7bbeGU22pacahgFSFd9kO0w/+Uxc/YehHAOlXWeKh8t08MJwA2 GS03Cuivcsvi0N6/I08xXwfGECg2R2X57ZpF4= MIME-Version: 1.0 Received: by 10.239.158.194 with SMTP id v2mr1062027hbc.109.1286206290398; Mon, 04 Oct 2010 08:31:30 -0700 (PDT) Received: by 10.220.200.1 with HTTP; Mon, 4 Oct 2010 08:31:29 -0700 (PDT) In-Reply-To: <20101004123629.GA4308@straylight.ringlet.net> References: <20101004122325.11342p9v1rnwp6w4@webmail.leidinger.net> <20101004123629.GA4308@straylight.ringlet.net> Date: Mon, 4 Oct 2010 15:31:29 +0000 Message-ID: From: Paul B Mahol To: Paul B Mahol , Alexander Leidinger , freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: Re: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Oct 2010 15:31:32 -0000 On 10/4/10, Peter Pentchev wrote: > On Mon, Oct 04, 2010 at 11:01:45AM +0000, Paul B Mahol wrote: >> On 10/4/10, Alexander Leidinger wrote: >> > Quoting Paul B Mahol (from Sun, 3 Oct 2010 13:53:26 >> > +0000): >> > >> >> Hi, >> >> >> >> >> >> diff --git a/tools/build/mk/OptionalObsoleteFiles.inc >> >> b/tools/build/mk/OptionalObsoleteFiles.inc >> >> index d3aa4b2..2123107 100644 >> >> --- a/tools/build/mk/OptionalObsoleteFiles.inc >> >> +++ b/tools/build/mk/OptionalObsoleteFiles.inc >> >> @@ -141,6 +141,7 @@ OLD_FILES+=usr/share/man/man8/authpf.8.gz >> >> .endif >> >> >> >> .if ${MK_BIND} == no >> >> +OLD_FILES+=etc/periodic/daily/470.status-named >> > >> > If bind is installed from ports instead, this file could be useful, >> > isn't >> > it? >> > >> >> OLD_FILES+=usr/bin/dig >> >> OLD_FILES+=usr/bin/host >> >> OLD_FILES+=usr/bin/nslookup >> > >> > [...] >> > >> >> @@ -1976,6 +1998,11 @@ OLD_FILES+=usr/share/man/man8/rtquery.8.gz >> >> .endif >> >> >> >> .if ${MK_SENDMAIL} == no >> >> +OLD_FILES+=etc/periodic/daily/150.clean-hoststat >> >> +OLD_FILES+=etc/periodic/daily/210.backup-aliases >> >> +OLD_FILES+=etc/periodic/daily/440.status-mailq >> >> +OLD_FILES+=etc/periodic/daily/460.status-mail-rejects >> >> +OLD_FILES+=etc/periodic/daily/500.queuerun >> >> OLD_FILES+=bin/rmail >> >> OLD_FILES+=usr/bin/vacation >> >> OLD_FILES+=usr/include/libmilter/mfapi.h >> > >> > At least status-mailq is still useful with other MTAs, e.g. if you use >> > postfix (and assuming the mailwrapper is configured correctly), this >> > should still work. Depending on the setup of the aliases, >> > backup-aliases will still work too. I haven't checked for the other >> > ones. >> >> If you want to keep some "old" files, you just need to press `n' instead >> of `y' >> when deleting files. You are not forced to run make delete-old at all. > > On the other hand, doing this (skipping those files) on each and every > system > update (say, on a development or testing machine) could get a bit... boring > - > not to mention that at some point you'll learn to "oh, just say no", and > then > forget to remove something important. Oh! I see... I want it in black. > > G'luck, > Peter > > -- > Peter Pentchev roam@space.bg roam@ringlet.net roam@FreeBSD.org > PGP key: http://people.FreeBSD.org/~roam/roam.key.asc > Key fingerprint FDBA FD79 C26F 3C51 C95E DF9E ED18 B68D 1619 4553 > Do you think anybody has ever had *precisely this thought* before? > From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 4 15:43:06 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 361C110656A4; Mon, 4 Oct 2010 15:43:06 +0000 (UTC) (envelope-from me@zan.st) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0F15F8FC1B; Mon, 4 Oct 2010 15:43:05 +0000 (UTC) Received: by pwi8 with SMTP id 8so1726452pwi.13 for ; Mon, 04 Oct 2010 08:43:05 -0700 (PDT) Received: by 10.142.132.15 with SMTP id f15mr8783372wfd.299.1286205540554; Mon, 04 Oct 2010 08:19:00 -0700 (PDT) MIME-Version: 1.0 Received: by 10.42.165.4 with HTTP; Mon, 4 Oct 2010 08:18:40 -0700 (PDT) X-Originating-IP: [187.35.28.248] In-Reply-To: <4CA630F5.9060500@networx.ch> References: <4CA630F5.9060500@networx.ch> From: stephano zanzin Date: Mon, 4 Oct 2010 12:18:40 -0300 Message-ID: To: Andre Oppermann Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, freebsd-current@freebsd.org Subject: Re: Very interesting paper: An Analysis of Linux Scalability to many Cores X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Oct 2010 15:43:06 -0000 Very interesting! Thanks On Fri, Oct 1, 2010 at 4:05 PM, Andre Oppermann wrote= : > Just saw the link to a very interesting paper on SMP scalability. > A very good read and highly relevant for our efforts as well. In > certain areas we may already fare better, in others we still have > some work to do. > > An Analysis of Linux Scalability to many Cores > > ABSTRACT > This paper analyzes the scalability of seven system applications > (Exim, memcached, Apache, PostgreSQL, gmake, Psearchy, and MapReduce) > running on Linux on a 48-core computer. Except for gmake, all > applications trigger scalability bottlenecks inside a recent Linux > kernel. Using mostly standard parallel programming techniques=97 > this paper introduces one new technique, sloppy counters=97 > these bottlenecks can be removed from the kernel or avoided by > changing the applications slightly. Modifying the kernel required > in total 3002 lines of code changes. A speculative conclusion from > this analysis is that there is no scalability reason to give up on > traditional operating system organizations just yet. > > http://pdos.csail.mit.edu/papers/linux:osdi10.pdf > > -- > Andre > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > --=20 stephano From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 00:00:28 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 54889106566B for ; Tue, 5 Oct 2010 00:00:28 +0000 (UTC) (envelope-from gnemmi@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 0BDD78FC19 for ; Tue, 5 Oct 2010 00:00:27 +0000 (UTC) Received: by qyk8 with SMTP id 8so3933150qyk.13 for ; Mon, 04 Oct 2010 17:00:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=xsp/uxk39MFIb0SrUQURTqwpsLgirlPqFHVXgF3+ClQ=; b=BWQXvprIZ8B5AeWGfLakWpI/WvF+NQPLaM+MuYmAP3d+VrHIYeotmcUuRDm40w9EfQ S6yAqxAO3W2+jlKTOCaY2/4svN7jNUluP19Baj2QsuiivLY2gKouW4iAC+lQlL9BMcRS M9Uzfxh57Gk3HDNKWSWbxRnXjwBSeH16ktQGg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=H+YwyZVBu+i2/jgYr1HLvlw6as7jWE9ZXSBI0CG7F+NLTIXL4Rron0M3lFGglCZXqT oFG4jnudLcN/CZwAhhCUzyDnD9QH01eOfg6tWgXqqqJLmCmq3zShliAsv7daj5o59D+J mG7HH7I5LtyA/BeA9TJcdbgad6Y0znsENl+Y0= MIME-Version: 1.0 Received: by 10.224.69.16 with SMTP id x16mr7432534qai.284.1286235001032; Mon, 04 Oct 2010 16:30:01 -0700 (PDT) Received: by 10.220.183.65 with HTTP; Mon, 4 Oct 2010 16:30:00 -0700 (PDT) In-Reply-To: References: <20101004122325.11342p9v1rnwp6w4@webmail.leidinger.net> <20101004123629.GA4308@straylight.ringlet.net> Date: Tue, 5 Oct 2010 01:30:00 +0200 Message-ID: From: Gonzalo Nemmi To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 00:00:28 -0000 On Mon, Oct 4, 2010 at 5:31 PM, Paul B Mahol wrote: > On 10/4/10, Peter Pentchev wrote: >> On Mon, Oct 04, 2010 at 11:01:45AM +0000, Paul B Mahol wrote: >>> On 10/4/10, Alexander Leidinger wrote: >>> > Quoting Paul B Mahol (from Sun, 3 Oct 2010 13:53:2= 6 >>> > +0000): >>> > >>> >> Hi, >>> >> >>> >> >>> >> diff --git a/tools/build/mk/OptionalObsoleteFiles.inc >>> >> b/tools/build/mk/OptionalObsoleteFiles.inc >>> >> index d3aa4b2..2123107 100644 >>> >> --- a/tools/build/mk/OptionalObsoleteFiles.inc >>> >> +++ b/tools/build/mk/OptionalObsoleteFiles.inc >>> >> @@ -141,6 +141,7 @@ OLD_FILES+=3Dusr/share/man/man8/authpf.8.gz >>> >> =A0.endif >>> >> >>> >> =A0.if ${MK_BIND} =3D=3D no >>> >> +OLD_FILES+=3Detc/periodic/daily/470.status-named >>> > >>> > If bind is installed from ports instead, this file could be useful, >>> > isn't >>> > it? >>> > >>> >> =A0OLD_FILES+=3Dusr/bin/dig >>> >> =A0OLD_FILES+=3Dusr/bin/host >>> >> =A0OLD_FILES+=3Dusr/bin/nslookup >>> > >>> > [...] >>> > >>> >> @@ -1976,6 +1998,11 @@ OLD_FILES+=3Dusr/share/man/man8/rtquery.8.gz >>> >> =A0.endif >>> >> >>> >> =A0.if ${MK_SENDMAIL} =3D=3D no >>> >> +OLD_FILES+=3Detc/periodic/daily/150.clean-hoststat >>> >> +OLD_FILES+=3Detc/periodic/daily/210.backup-aliases >>> >> +OLD_FILES+=3Detc/periodic/daily/440.status-mailq >>> >> +OLD_FILES+=3Detc/periodic/daily/460.status-mail-rejects >>> >> +OLD_FILES+=3Detc/periodic/daily/500.queuerun >>> >> =A0OLD_FILES+=3Dbin/rmail >>> >> =A0OLD_FILES+=3Dusr/bin/vacation >>> >> =A0OLD_FILES+=3Dusr/include/libmilter/mfapi.h >>> > >>> > At least status-mailq is still useful with other MTAs, e.g. if you us= e >>> > postfix (and assuming the mailwrapper is configured correctly), this >>> > should still work. Depending on the setup of the aliases, >>> > backup-aliases will still work too. I haven't checked for the other >>> > ones. >>> >>> If you want to keep some "old" files, you just need to press `n' instea= d >>> of `y' >>> when deleting files. You are not forced to run make delete-old at all. >> >> On the other hand, doing this (skipping those files) on each and every >> system >> update (say, on a development or testing machine) could get a bit... bor= ing >> - >> not to mention that at some point you'll learn to "oh, just say no", and >> then >> forget to remove something important. > > Oh! I see... I want it in black. Hold your horses there ... get it to work as expected and then, _and_only_then_ you can start raising your finger and accussing poster of bikesheding ... As it stands, _it_sucks_ and I have already pointed it out before without much success (something as useless as telnet wasn=B4t deleted a not so long time ago) ... Peter Pentchev=B4s point was a completely valid one. >> G'luck, >> Peter >> >> -- >> Peter Pentchev =A0 =A0 =A0 =A0roam@space.bg =A0 =A0roam@ringlet.net =A0 = =A0roam@FreeBSD.org >> PGP key: =A0 =A0 =A0http://people.FreeBSD.org/~roam/roam.key.asc >> Key fingerprint =A0 =A0 =A0 FDBA FD79 C26F 3C51 C95E =A0DF9E ED18 B68D 1= 619 4553 >> Do you think anybody has ever had *precisely this thought* before? >> > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 02:58:42 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8536B1065673 for ; Tue, 5 Oct 2010 02:58:42 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4F8408FC19 for ; Tue, 5 Oct 2010 02:58:41 +0000 (UTC) Received: by iwn34 with SMTP id 34so911034iwn.13 for ; Mon, 04 Oct 2010 19:58:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=YDMfhSeE/oQjBVWzUhN5ZO4kBOk3ehQMBrp0YB7QcqE=; b=XNfS/MnoCGfFQCV9OMYYE/OMqwWBAU3oabgdId0RWUMJKE173mSHYeAUGL/OykwXun PK7Im0BneOAKL/KbOJu9xEfNE7QSCh/BbSq3X0d2Psgg5hEpjp8zvzCiodzLNAdoaoQ7 qK3KVo9LrfzoypZOi39lbH0eqo3PVDYs6ZjgQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=fheVAm6zeRXNtA1+ZLZKmnGkFmslrvBXMKSrX+n9e0hrMTVdV7+4wXpQCvsnvPHnj3 YBOjdBgEzK+/MFWAlRjex1Y6ojJGePce9rjvEBxI0Gvm01f1JOEJNZZcWHNzaMN/jA3/ zNwPqzie8/0RGhowtOvYzU3Gg/lAcCcuVaefA= MIME-Version: 1.0 Received: by 10.231.10.139 with SMTP id p11mr11203144ibp.179.1286246191663; Mon, 04 Oct 2010 19:36:31 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.231.171.203 with HTTP; Mon, 4 Oct 2010 19:36:31 -0700 (PDT) Date: Tue, 5 Oct 2010 10:36:31 +0800 X-Google-Sender-Auth: y9UKIQ6lbDEhr8d9-3x3q2UqugU Message-ID: From: Adrian Chadd To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: breaking the crunchgen logic into a share/mk file X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 02:58:42 -0000 Hi, I've broken out the crunchgen logic from src/rescue/rescue into a share/mk file - that way it can be reused in other areas. The diff is here: http://people.freebsd.org/~adrian/crunchgen-mk.diff This bsd.crunchgen.mk file is generic enough to use in my busybox-style thing as well as for src/rescue/rescue/. Comments, feedback, etc welcome! Adrian From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 03:19:56 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB04F106566C for ; Tue, 5 Oct 2010 03:19:56 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 84BBD8FC08 for ; Tue, 5 Oct 2010 03:19:56 +0000 (UTC) Received: by iwn34 with SMTP id 34so934275iwn.13 for ; Mon, 04 Oct 2010 20:19:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=lJ2WeLZdbs/vAoPOmC8N7iQnKmdVRHpy65RFEs72qlU=; b=sdK9B0XdtG/QJOAYopbt95driSPAa0tBA426jk8VTslPFPVv5Ncxn+50XpmhtMiVov Jjg+65vw70V2EoomzGBJw4TcNHZVs+Gktva3/6VRLFeYkuml50qkf2PCmjwmwEwIs4yd 5eFC48E+qTy548eJ0Z/LRZlNvWsidYAp6Jjhg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=KxUxG2j4aEfe7P4zjIu6eKw17nKM1EWt8To0iXjU3U9MmuMGAtT301GoOB0LapPZTc CKNgnb8EABZ0T3K+VINjbuiLBnLqSPFihdZtDjt+vC0COYsYTuZljm9fj++1y5Q0HNvm QjfMiDIhHTO7GLHzGZvQqfNCJGV88s65IW70M= MIME-Version: 1.0 Received: by 10.231.35.138 with SMTP id p10mr11335053ibd.33.1286248788047; Mon, 04 Oct 2010 20:19:48 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.184.3 with HTTP; Mon, 4 Oct 2010 20:19:48 -0700 (PDT) In-Reply-To: References: Date: Mon, 4 Oct 2010 20:19:48 -0700 X-Google-Sender-Auth: ET8FMiIlu3tnKPPaZhVWlxxxsg4 Message-ID: From: Garrett Cooper To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org Subject: Re: breaking the crunchgen logic into a share/mk file X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 03:19:57 -0000 On Mon, Oct 4, 2010 at 7:36 PM, Adrian Chadd wrote: > Hi, > > I've broken out the crunchgen logic from src/rescue/rescue into a > share/mk file - that way it can be reused in other areas. > > The diff is here: http://people.freebsd.org/~adrian/crunchgen-mk.diff > > This bsd.crunchgen.mk file is generic enough to use in my > busybox-style thing as well as for src/rescue/rescue/. > > Comments, feedback, etc welcome! Hi Adrian, Is there a functional difference between the original file and bsd.crunchgen.mk ? Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 09:21:47 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1DE71065672 for ; Tue, 5 Oct 2010 09:21:47 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 2F9FB8FC17 for ; Tue, 5 Oct 2010 09:21:47 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3B163.dip.t-dialin.net [87.179.177.99]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id BA1F484400A; Tue, 5 Oct 2010 11:21:44 +0200 (CEST) Received: from webmail.leidinger.net (unknown [IPv6:fd73:10c7:2053:1::2:102]) by outgoing.leidinger.net (Postfix) with ESMTP id B7E771708; Tue, 5 Oct 2010 11:21:41 +0200 (CEST) Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.13.8/Submit) id o959Lf1O008012; Tue, 5 Oct 2010 11:21:41 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Tue, 05 Oct 2010 11:21:41 +0200 Message-ID: <20101005112141.7373391sd2jwsd9g@webmail.leidinger.net> Date: Tue, 05 Oct 2010 11:21:41 +0200 From: Alexander Leidinger To: Paul B Mahol References: <20101004122325.11342p9v1rnwp6w4@webmail.leidinger.net> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: BA1F484400A.A4288 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=1.274, required 6, autolearn=disabled, RDNS_NONE 1.27) X-EBL-MailScanner-SpamScore: s X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1286875305.17098@1X9W3gfmQVLLGDwfctsg4A X-EBL-Spam-Status: No X-Mailman-Approved-At: Tue, 05 Oct 2010 10:58:15 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 09:21:47 -0000 Quoting Paul B Mahol (from Mon, 4 Oct 2010 11:01:45 +0000): > On 10/4/10, Alexander Leidinger wrote: >> Quoting Paul B Mahol (from Sun, 3 Oct 2010 13:53:26 >> +0000): >> >>> Hi, >>> >>> >>> diff --git a/tools/build/mk/OptionalObsoleteFiles.inc >>> b/tools/build/mk/OptionalObsoleteFiles.inc >>> index d3aa4b2..2123107 100644 >>> --- a/tools/build/mk/OptionalObsoleteFiles.inc >>> +++ b/tools/build/mk/OptionalObsoleteFiles.inc >>> @@ -141,6 +141,7 @@ OLD_FILES+=usr/share/man/man8/authpf.8.gz >>> .endif >>> >>> .if ${MK_BIND} == no >>> +OLD_FILES+=etc/periodic/daily/470.status-named >> >> If bind is installed from ports instead, this file could be useful, isn't >> it? >> >>> OLD_FILES+=usr/bin/dig >>> OLD_FILES+=usr/bin/host >>> OLD_FILES+=usr/bin/nslookup >> >> [...] >> >>> @@ -1976,6 +1998,11 @@ OLD_FILES+=usr/share/man/man8/rtquery.8.gz >>> .endif >>> >>> .if ${MK_SENDMAIL} == no >>> +OLD_FILES+=etc/periodic/daily/150.clean-hoststat >>> +OLD_FILES+=etc/periodic/daily/210.backup-aliases >>> +OLD_FILES+=etc/periodic/daily/440.status-mailq >>> +OLD_FILES+=etc/periodic/daily/460.status-mail-rejects >>> +OLD_FILES+=etc/periodic/daily/500.queuerun >>> OLD_FILES+=bin/rmail >>> OLD_FILES+=usr/bin/vacation >>> OLD_FILES+=usr/include/libmilter/mfapi.h >> >> At least status-mailq is still useful with other MTAs, e.g. if you use >> postfix (and assuming the mailwrapper is configured correctly), this >> should still work. Depending on the setup of the aliases, >> backup-aliases will still work too. I haven't checked for the other >> ones. > > If you want to keep some "old" files, you just need to press `n' > instead of `y' > when deleting files. You are not forced to run make delete-old at all. I know, I wrote delete-old and related ones. :) > When installing world in new environment such files are not installed anyway. That's an argument (and your patch is OK). So I should discuss (somewhere else) if this needs to be changed. Bye, Alexander. -- Stone's Law: One man's "simple" is another man's "huh?" http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 09:34:33 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6CFEB1065670 for ; Tue, 5 Oct 2010 09:34:33 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id F007E8FC18 for ; Tue, 5 Oct 2010 09:34:32 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3B163.dip.t-dialin.net [87.179.177.99]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id AF17984400A; Tue, 5 Oct 2010 11:34:29 +0200 (CEST) Received: from webmail.leidinger.net (unknown [IPv6:fd73:10c7:2053:1::2:102]) by outgoing.leidinger.net (Postfix) with ESMTP id BFB89170B; Tue, 5 Oct 2010 11:34:26 +0200 (CEST) Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.13.8/Submit) id o959YNtq011006; Tue, 5 Oct 2010 11:34:23 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Tue, 05 Oct 2010 11:34:23 +0200 Message-ID: <20101005113423.10801orcw6xef9s0@webmail.leidinger.net> Date: Tue, 05 Oct 2010 11:34:23 +0200 From: Alexander Leidinger To: Gonzalo Nemmi References: <20101004122325.11342p9v1rnwp6w4@webmail.leidinger.net> <20101004123629.GA4308@straylight.ringlet.net> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: AF17984400A.A548E X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=1.274, required 6, autolearn=disabled, RDNS_NONE 1.27) X-EBL-MailScanner-SpamScore: s X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1286876070.90725@disOupjXTP6721O3wyos+A X-EBL-Spam-Status: No X-Mailman-Approved-At: Tue, 05 Oct 2010 10:58:29 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 09:34:33 -0000 Quoting Gonzalo Nemmi (from Tue, 5 Oct 2010 =20 01:30:00 +0200): > On Mon, Oct 4, 2010 at 5:31 PM, Paul B Mahol wrote: >> On 10/4/10, Peter Pentchev wrote: >>> On Mon, Oct 04, 2010 at 11:01:45AM +0000, Paul B Mahol wrote: >>>> On 10/4/10, Alexander Leidinger wrote: >>>> > At least status-mailq is still useful with other MTAs, e.g. if you u= se >>>> > postfix (and assuming the mailwrapper is configured correctly), this >>>> > should still work. Depending on the setup of the aliases, >>>> > backup-aliases will still work too. I haven't checked for the other >>>> > ones. >>>> >>>> If you want to keep some "old" files, you just need to press `n' inste= ad >>>> of `y' >>>> when deleting files. You are not forced to run make delete-old at all. >>> >>> On the other hand, doing this (skipping those files) on each and every >>> system >>> update (say, on a development or testing machine) could get a bit... bo= ring >>> - >>> not to mention that at some point you'll learn to "oh, just say no", an= d >>> then >>> forget to remove something important. >> >> Oh! I see... I want it in black. > > Hold your horses there ... get it to work as expected and then, > _and_only_then_ you can start raising your finger and accussing poster > of bikesheding ... > As it stands, _it_sucks_ and I have already pointed it out before > without much success (something as useless as telnet wasn=C2=B4t deleted = a > not so long time ago) ... Peter Pentchev=C2=B4s point was a completely > valid one. As the one who wrote the functionailty: The delete-old stuff is =20 designed to delete stuff which was installed by an installworld (or =20 similar) before, but is not installed by installworld (or similar) =20 now. As there may be old programs around which depend upon such a file =20 or directory, you have to explicitely tell if you are ok to delete it =20 or not (it is the responsability of the administrator to make sure =20 everything is OK). If you are sure no program still depends upon it =20 (make check-old), you can specify -DBATCH_DELETE_OLD_FILES to delete =20 everything without being asked (see "man build"). As we've read in this thread that the files I would like to keep are =20 not installed by an installworld if the corresponding option is =20 specified, the patch is 100% correct and Peter and you want to use =20 delete-old for something it was not designed for (and I should discuss =20 somewhere else to install the files I talked about even if the =20 corresponding option is specified). Bye, Alexander. --=20 Detroit is Cleveland without the glitter. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 12:16:52 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DAB5D106566B for ; Tue, 5 Oct 2010 12:16:52 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6CBF68FC14 for ; Tue, 5 Oct 2010 12:16:51 +0000 (UTC) Received: by wyb29 with SMTP id 29so5527453wyb.13 for ; Tue, 05 Oct 2010 05:16:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=01RRSsRRz/FO+RwEKxrxQJ2g6ndyTGycQSFCWKcIwio=; b=MukF2cPgSewkbFA1RrQ9c3q5PThBNDRgDJ9GAas0z5prJp5EdgJhJvE2yYHKmY22WE /2bzFVhj3ABfr/k0V6+gr0sB5uGMwJedghFkeJFX0KDixtrsk0peMClbR+GDEvTHEyYB 93ItTY4ms3r2EsaSNSeooh92mJ36ANdFgFTog= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=PPdlRZ6VsuJEz9gnlYVlSPF7x5XEeaZFfA8XpZoywgSCgHkNGABY30cOysRtWidXph CkcEw3JhbEWn5CJSz5cpAm0M1ng4V7z9s5kh18t0aDK84wluOQJjJkrqfckMXkBCWBb3 P+vAPDJpzLST2KS4ebiynbgbaY4mi/6vWb4bg= MIME-Version: 1.0 Received: by 10.216.187.130 with SMTP id y2mr945137wem.60.1286281011162; Tue, 05 Oct 2010 05:16:51 -0700 (PDT) Received: by 10.216.172.210 with HTTP; Tue, 5 Oct 2010 05:16:51 -0700 (PDT) In-Reply-To: References: <29760054.post@talk.nabble.com> Date: Tue, 5 Oct 2010 14:16:51 +0200 Message-ID: From: Svatopluk Kraus To: alc@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: page table fault, which should map kernel virtual address space X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 12:16:52 -0000 On Mon, Oct 4, 2010 at 2:03 AM, Alan Cox wrote: > On Thu, Sep 30, 2010 at 6:28 AM, Svatopluk Kraus wrote= : >> >> On Tue, Sep 21, 2010 at 7:38 PM, Alan Cox wrote: >> > On Mon, Sep 20, 2010 at 9:32 AM, Svatopluk Kraus >> > wrote: >> >> Beyond 'kernel_map', some submaps of 'kernel_map' (buffer_map, >> >> pager_map,...) exist as result of 'kmem_suballoc' function call. >> >> When this submaps are used (for example 'kmem_alloc_nofault' >> >> function) and its virtual address subspace is at the end of >> >> used kernel virtual address space at the moment (and above 'NKPT' >> >> preallocation), then missing page tables are not allocated >> >> and double fault can happen. >> >> >> > >> > No, the page tables are allocated.=A0 If you create a submap X of the >> > kernel >> > map using kmem_suballoc(), then a vm_map_findspace() is performed by >> > vm_map_find() on the kernel map to find space for the submap X.=A0 As = you >> > note >> > above, the call to vm_map_findspace() on the kernel map will call >> > pmap_growkernel() if needed to extend the kernel page table. >> > >> > If you create another submap X' of X, then that submap X' can only map >> > addresses that fall within the range for X.=A0 So, any necessary page >> > table >> > pages were allocated when X was created. >> >> You are right. Mea culpa. I was focused on a solution and made >> too quick conclusion. The page table fault hitted in 'pager_map', >> which is submap of 'clean_map' and when I debugged the problem >> I didn't see a submap stuff as a whole. >> >> > That said, there may actually be a problem with the implementation of >> > the >> > superpage_align parameter to kmem_suballoc().=A0 If a submap is create= d >> > with >> > superpage_align equal to TRUE, but the submap's size is not a multiple >> > of >> > the superpage size, then vm_map_find() may not allocate a page table >> > page >> > for the last megabyte or so of the submap. >> > >> > There are only a few places where kmem_suballoc() is called with >> > superpage_align set to TRUE.=A0 If you changed them to FALSE, that is = an >> > easy >> > way to test this hypothesis. >> >> Yes, it helps. >> >> My story is that the problem started up when I updated a project >> ('coldfire' port) >> based on FreeBSD 8.0. to FreeBSD current version. In the current version >> the 'clean_map' submap is created with superpage_align set to TRUE. >> >> I have looked at vm_map_find() and debugged the page table fault once >> again. >> IMO, it looks that a do-while loop does not work in the function as >> intended. >> A vm_map_findspace() finds a space and calls pmap_growkernel() if needed= . >> A pmap_align_superpage() arrange the space but never call >> pmap_growkernel(). >> A vm_map_insert() inserts the aligned space into a map without error >> and never call pmap_growkernel() and does not invoke loop iteration. >> >> I don't know too much about an idea how a virtual memory model is >> implemented >> and used in other modules. But it seems that it could be more reasonable >> to >> align address space in vm_map_findspace() internally and not to loop >> externally. >> >> I have tried to add a check in vm_map_insert() that checks the 'end' >> parameter >> against 'kernel_vm_end' variable and returns KERN_NO_SPACE error if >> needed. >> In this case the loop in vm_map_find() works and I have no problem with >> the page table fault. But 'kernel_vm_end' variable must be initializated >> properly before first use of vm_map_insert(). The 'kernel_vm_end' variab= le >> can be self-initializated in pmap_growkernel() in FreeBSD 8.0 (it is too >> late), >> but it was changed in current version ('i386' port). >> >> Thanks for your answer, but I'm still looking for permanent >> and approved solution. > > I have a patch that implements one possible fix for this problem.=A0 I'll > probably commit that patch in the next day or two. > > Regards, > Alan I tried your patch and it works. Many thanks. Regards, Svata From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 13:09:59 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09166106566B for ; Tue, 5 Oct 2010 13:09:59 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 886F18FC12 for ; Tue, 5 Oct 2010 13:09:58 +0000 (UTC) Received: by fxm9 with SMTP id 9so4750432fxm.13 for ; Tue, 05 Oct 2010 06:09:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:from:to:subject :date:content-type:content-transfer-encoding:in-reply-to:references :x-mailer; bh=Kg0JMMo6T9ZI0rW5HyaPkTMZyslUlTWq0d4gBmSMP10=; b=WjEm8NYLw7FjrIeSrAwgX0O9pIzNHC3vheKYTLiZyh6fYstZ+MIpgJt8QT/P2tUiim dmZGqJ/vYhxP84q7TGtH/CU1LQVCrWc4XoIxpwpDa7w9VbJDQwahAQpdVy+1JAicBdpw rEJ6kPV94tIzN6W2u9j0SJKSeQq1XaHl5rh0M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:subject:date:content-type :content-transfer-encoding:in-reply-to:references:x-mailer; b=Bu/AZzi4RPr0evncnIDbwUKFWtgZZZUpgYIkfUTclMFn5RomSoq/1/6d6R6zz50vNG MuuFj4DEnfyMuAO77CZzQXSLqWhQ1dxXQa1m5SIrvnYFIXtLLtTWC0XswxyvZJdsCRLQ Ca1R+2d223S3l1JJ90A5paJlYk/iiAFfejEPA= Received: by 10.223.104.11 with SMTP id m11mr10557810fao.50.1286282864507; Tue, 05 Oct 2010 05:47:44 -0700 (PDT) Received: from DEV ([82.193.208.173]) by mx.google.com with ESMTPS id a6sm2829125faa.20.2010.10.05.05.47.06 (version=SSLv3 cipher=RC4-MD5); Tue, 05 Oct 2010 05:47:43 -0700 (PDT) Message-ID: <20101005.124823.093.1@DEV> From: rank1seeker@gmail.com To: freebsd-hackers@freebsd.org, "Alexander Leidinger" Date: Tue, 05 Oct 2010 14:48:23 +0200 Content-Type: text/plain; charset="Windows-1250" Content-Transfer-Encoding: quoted-printable In-Reply-To: <20101005113423.10801orcw6xef9s0@webmail.leidinger.net> References: <20101004122325.11342p9v1rnwp6w4@webmail.leidinger.net> <20101004123629.GA4308@straylight.ringlet.net> <20101005113423.10801orcw6xef9s0@webmail.leidinger.net> X-Mailer: POP Peeper (3.7.0.0) Cc: Subject: Re: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 13:09:59 -0000 ----- Original Message -----=0D=0AFrom: Alexander Leidinger = =0D=0ATo: Gonzalo Nemmi = =0D=0ACc: freebsd-hackers@freebsd.org=0D=0ADate: Tue, = 05 Oct 2010 11:34:23 +0200=0D=0ASubject: Re: Improve = OptionalObsoleteFiles.inc=0D=0A=0D=0A> Quoting Gonzalo Nemmi = (from Tue, 5 Oct 2010 =0D=0A> 01:30:00 +0200):=0D=0A> = =0D=0A> > On Mon, Oct 4, 2010 at 5:31 PM, Paul B Mahol = wrote:=0D=0A> >> On 10/4/10, Peter Pentchev = wrote:=0D=0A> >>> On Mon, Oct 04, 2010 at 11:01:45AM +0000, Paul B Mahol = wrote:=0D=0A> >>>> On 10/4/10, Alexander Leidinger = wrote:=0D=0A> =0D=0A> >>>> > At least = status-mailq is still useful with other MTAs, e.g. if you use=0D=0A> >>>> = > postfix (and assuming the mailwrapper is configured correctly), = this=0D=0A> >>>> > should still work. Depending on the setup of the = aliases,=0D=0A> >>>> > backup-aliases will still work too. I haven't = checked for the other=0D=0A> >>>> > ones.=0D=0A> >>>>=0D=0A> >>>> If you = want to keep some "old" files, you just need to press `n' instead=0D=0A> = >>>> of `y'=0D=0A> >>>> when deleting files. You are not forced to run = make delete-old at all.=0D=0A> >>>=0D=0A> >>> On the other hand, doing = this (skipping those files) on each and every=0D=0A> >>> system=0D=0A> = >>> update (say, on a development or testing machine) could get a bit... = boring=0D=0A> >>> -=0D=0A> >>> not to mention that at some point you'll = learn to "oh, just say no", and=0D=0A> >>> then=0D=0A> >>> forget to = remove something important.=0D=0A> >>=0D=0A> >> Oh! I see... I want it in = black.=0D=0A> >=0D=0A> > Hold your horses there ... get it to work as = expected and then,=0D=0A> > _and_only_then_ you can start raising your = finger and accussing poster=0D=0A> > of bikesheding ...=0D=0A> > As it = stands, _it_sucks_ and I have already pointed it out before=0D=0A> > = without much success (something as useless as telnet wasn=B4t deleted = a=0D=0A> > not so long time ago) ... Peter Pentchev=B4s point was a = completely=0D=0A> > valid one.=0D=0A> =0D=0A> As the one who wrote the = functionailty: The delete-old stuff is =0D=0A> designed to delete stuff = which was installed by an installworld (or =0D=0A> similar) before, but = is not installed by installworld (or similar) =0D=0A> now. As there may = be old programs around which depend upon such a file =0D=0A> or = directory, you have to explicitely tell if you are ok to delete it = =0D=0A> or not (it is the responsability of the administrator to make = sure =0D=0A> everything is OK). If you are sure no program still depends = upon it =0D=0A> (make check-old), you can specify = -DBATCH_DELETE_OLD_FILES to delete =0D=0A> everything without being = asked (see "man build").=0D=0A> =0D=0A> As we've read in this thread that = the files I would like to keep are =0D=0A> not installed by an = installworld if the corresponding option is =0D=0A> specified, the patch = is 100% correct and Peter and you want to use =0D=0A> delete-old for = something it was not designed for (and I should discuss =0D=0A> = somewhere else to install the files I talked about even if the =0D=0A> = corresponding option is specified).=0D=0A> =0D=0A> Bye,=0D=0A> = Alexander.=0D=0A=0D=0A=0D=0A~2 months ago I've opened a thread called: = Deinstalling parts of base=0D=0ATake a look here:=0D=0A = http://forums.freebsd.org/showthread.php?t=3D17860=0D=0A=0D=0ADomagoj=0D=0A From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 13:39:03 2010 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 13049106566B for ; Tue, 5 Oct 2010 13:39:03 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) by mx1.freebsd.org (Postfix) with ESMTP id 9788A8FC1A for ; Tue, 5 Oct 2010 13:39:02 +0000 (UTC) Received: from [10.0.1.48] (unknown [217.157.7.211]) by csmtp2.one.com (Postfix) with ESMTPA id 7B60ED702A360 for ; Tue, 5 Oct 2010 13:28:37 +0000 (UTC) From: Erik Cederstrand Content-Type: multipart/signed; boundary=Apple-Mail-1444-887955366; protocol="application/pkcs7-signature"; micalg=sha1 Date: Tue, 5 Oct 2010 15:28:36 +0200 Message-Id: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> To: FreeBSD Hackers Mime-Version: 1.0 (Apple Message framework v1081) X-Mailer: Apple Mail (2.1081) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 13:39:03 -0000 --Apple-Mail-1444-887955366 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hello hackers, I got reminded of a problem I had a couple of years back compressing = FreeBSD jails. I was using bsdiff for the compression and found out that = md5 sums of static libraries (.a files) in /usr/lib and /usr/local/lib = didn't match between jails, even though the source code used to create = the jails hadn't changed. One of my goals is to detect which files in a = distribution change between two commits. It turns out that timestamps are stored in the library: # strings DIR1/usr/lib/libfetch.a > tmp1 # strings DIR2/usr/lib/libfetch.a > tmp2 # diff tmp1 tmp2 2c2 < / 1200728973 0 0 0 954 ` --- > / 1200723259 0 0 0 954 ` 57c57 < file.o/ 1200728973 0 0 100644 2356 ` --- > file.o/ 1200723259 0 0 100644 2356 ` 86c86 < http.o/ 1200728973 0 0 100644 17180 ` --- > http.o/ 1200723258 0 0 100644 17180 ` [...] I'm wondering if this is necessary, or if this can possibly be turned of = with a knob somewhere. Thanks, Erik= --Apple-Mail-1444-887955366-- From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 13:30:03 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 867451065672 for ; Tue, 5 Oct 2010 13:30:03 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 10AFE8FC13 for ; Tue, 5 Oct 2010 13:30:02 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3B163.dip.t-dialin.net [87.179.177.99]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id A93E184400C; Tue, 5 Oct 2010 15:29:57 +0200 (CEST) Received: from webmail.leidinger.net (unknown [IPv6:fd73:10c7:2053:1::2:102]) by outgoing.leidinger.net (Postfix) with ESMTP id 83B5A1726; Tue, 5 Oct 2010 15:29:51 +0200 (CEST) Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.13.8/Submit) id o95DTpht064591; Tue, 5 Oct 2010 15:29:51 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Tue, 05 Oct 2010 15:29:50 +0200 Message-ID: <20101005152950.16797j1h6ygogi88@webmail.leidinger.net> Date: Tue, 05 Oct 2010 15:29:50 +0200 From: Alexander Leidinger To: rank1seeker@gmail.com References: <20101004122325.11342p9v1rnwp6w4@webmail.leidinger.net> <20101004123629.GA4308@straylight.ringlet.net> <20101005113423.10801orcw6xef9s0@webmail.leidinger.net> <20101005.124823.093.1@DEV> In-Reply-To: <20101005.124823.093.1@DEV> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: A93E184400C.A63D9 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=1.274, required 6, autolearn=disabled, RDNS_NONE 1.27) X-EBL-MailScanner-SpamScore: s X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1286890199.24479@her5jXqYxqact/cVl4mjVA X-EBL-Spam-Status: No X-Mailman-Approved-At: Tue, 05 Oct 2010 14:03:15 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 13:30:03 -0000 Quoting rank1seeker@gmail.com (from Tue, 05 Oct 2010 14:48:23 +0200): > ----- Original Message ----- > From: Alexander Leidinger > To: Gonzalo Nemmi > Cc: freebsd-hackers@freebsd.org > Date: Tue, 05 Oct 2010 11:34:23 +0200 > Subject: Re: Improve OptionalObsoleteFiles.inc >> As the one who wrote the functionailty: The delete-old stuff is >> designed to delete stuff which was installed by an installworld (or >> similar) before, but is not installed by installworld (or similar) >> now. As there may be old programs around which depend upon such a file >> or directory, you have to explicitely tell if you are ok to delete it >> or not (it is the responsability of the administrator to make sure >> everything is OK). If you are sure no program still depends upon it >> (make check-old), you can specify -DBATCH_DELETE_OLD_FILES to delete >> everything without being asked (see "man build"). >> >> As we've read in this thread that the files I would like to keep are >> not installed by an installworld if the corresponding option is >> specified, the patch is 100% correct and Peter and you want to use >> delete-old for something it was not designed for (and I should discuss >> somewhere else to install the files I talked about even if the >> corresponding option is specified). > ~2 months ago I've opened a thread called: Deinstalling parts of base > Take a look here: > http://forums.freebsd.org/showthread.php?t=17860 We do not have a target which allows to go to src/bin/XXX and run "make deinstall" or similar. What is correctly written in this thread is that "make check-old" and "make delete-old" (and "make delete-old-libs") is supposed to do what you want (given that there is already support in the src to exclude this part of FreeBSD during installation). The most up-to-date version is in -current, I do not know about the stable branches (I do not monitor if people which change the corresponding file have merged the changes to stable branches or not). What may not be 100% correct in src is the list of files to delete (= a bug). For this the suggestion in the thread is also correct, if there is something which does not work please send a PR. It would be great if you specify which files are left over or even better provide a patch for the /usr/src/tools/build/mk/OptionalObsoleteFiles.inc file (it's not hard to add files there, you just have to follow the real-world-examples there). Attention: only stuff which is already supported to be skipped during installworld belongs there. So if you specify WITHOUT_XXX and you do an installworld into a scratch-area (with "make installworld DESTDIR=/path/to/scratch" it shall not show up there). Bye, Alexander. -- Haste makes waste. -- John Heywood http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 14:13:40 2010 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A507A106566B for ; Tue, 5 Oct 2010 14:13:40 +0000 (UTC) (envelope-from erikt@midgard.homeip.net) Received: from ch-smtp01.sth.basefarm.net (ch-smtp01.sth.basefarm.net [80.76.149.212]) by mx1.freebsd.org (Postfix) with ESMTP id 26DF18FC12 for ; Tue, 5 Oct 2010 14:13:39 +0000 (UTC) Received: from c83-255-61-120.bredband.comhem.se ([83.255.61.120]:50674 helo=falcon.midgard.homeip.net) by ch-smtp01.sth.basefarm.net with esmtp (Exim 4.71) (envelope-from ) id 1P382t-00031F-4r for hackers@FreeBSD.org; Tue, 05 Oct 2010 15:59:09 +0200 Received: (qmail 4374 invoked from network); 5 Oct 2010 15:59:06 +0200 Received: from owl.midgard.homeip.net (10.1.5.7) by falcon.midgard.homeip.net with ESMTP; 5 Oct 2010 15:59:06 +0200 Received: (qmail 3514 invoked by uid 1001); 5 Oct 2010 15:59:06 +0200 Date: Tue, 5 Oct 2010 15:59:06 +0200 From: Erik Trulsson To: Erik Cederstrand Message-ID: <20101005135906.GA3430@owl.midgard.homeip.net> References: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> User-Agent: Mutt/1.5.20 (2009-06-14) X-Originating-IP: 83.255.61.120 X-Scan-Result: No virus found in message 1P382t-00031F-4r. X-Scan-Signature: ch-smtp01.sth.basefarm.net 1P382t-00031F-4r ff869574345385dc8b0043e818980f7b Cc: FreeBSD Hackers Subject: Re: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 14:13:40 -0000 On Tue, Oct 05, 2010 at 03:28:36PM +0200, Erik Cederstrand wrote: > Hello hackers, > > I got reminded of a problem I had a couple of years back compressing > FreeBSD jails. I was using bsdiff for the compression and found out > that md5 sums of static libraries (.a files) in /usr/lib and > /usr/local/lib didn't match between jails, even though the source > code used to create the jails hadn't changed. One of my goals is to > detect which files in a distribution change between two commits. > > It turns out that timestamps are stored in the library: Yes, they are. That is because the file format used for static libraries include a timestamp for each object file stored in the archive. You can use 'ar -tv /PATH/TO/LIBRARY/libfoo.a' to get a list of the objects stored in the archive and the corresponding timestamps. See the ar(5) manpage for a description of the file format, and the ar(1) manpage for information on how to manage such files. -- Erik Trulsson ertr1013@student.uu.se From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 15:13:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B474010656A5 for ; Tue, 5 Oct 2010 15:13:31 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from www.sonnenberger.org (www.sonnenberger.org [92.79.50.50]) by mx1.freebsd.org (Postfix) with ESMTP id 3F2068FC1F for ; Tue, 5 Oct 2010 15:13:29 +0000 (UTC) Received: from britannica.bec.de (www.sonnenberger.org [192.168.1.10]) by www.sonnenberger.org (Postfix) with ESMTP id E57CE66663; Tue, 5 Oct 2010 17:13:26 +0200 (CEST) Received: by britannica.bec.de (Postfix, from userid 1000) id DE3C9117EF4; Tue, 5 Oct 2010 17:13:48 +0200 (CEST) Date: Tue, 5 Oct 2010 17:13:48 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org, FreeBSD Hackers Message-ID: <20101005151348.GA27763@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org, FreeBSD Hackers References: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Subject: Re: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 15:13:31 -0000 On Tue, Oct 05, 2010 at 03:28:36PM +0200, Erik Cederstrand wrote: > I'm wondering if this is necessary, or if this can possibly be turned > of with a knob somewhere. Newer binutils got a flag after a discussion about this and related issues in NetBSD. It basically stores 0 for the uid/gid/time fields. Joerg From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 15:22:13 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3C861065670 for ; Tue, 5 Oct 2010 15:22:13 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-2.mit.edu (DMZ-MAILSEC-SCANNER-2.MIT.EDU [18.9.25.13]) by mx1.freebsd.org (Postfix) with ESMTP id 6F12E8FC17 for ; Tue, 5 Oct 2010 15:22:13 +0000 (UTC) X-AuditID: 1209190d-b7cacae000000a14-41-4cab42a62f93 Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) by dmz-mailsec-scanner-2.mit.edu (Symantec Brightmail Gateway) with SMTP id 57.36.02580.6A24BAC4; Tue, 5 Oct 2010 11:22:14 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id o95FMCXk010758; Tue, 5 Oct 2010 11:22:12 -0400 Received: from multics.mit.edu (MULTICS.MIT.EDU [18.187.1.73]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id o95FM9EA013514 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 5 Oct 2010 11:22:11 -0400 (EDT) Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id o95FM8MX003813; Tue, 5 Oct 2010 11:22:08 -0400 (EDT) Date: Tue, 5 Oct 2010 11:22:07 -0400 (EDT) From: Benjamin Kaduk To: Joerg Sonnenberger In-Reply-To: <20101005151348.GA27763@britannica.bec.de> Message-ID: References: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> <20101005151348.GA27763@britannica.bec.de> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: AAAAAA== Cc: freebsd-hackers@freebsd.org Subject: Re: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 15:22:13 -0000 On Tue, 5 Oct 2010, Joerg Sonnenberger wrote: > On Tue, Oct 05, 2010 at 03:28:36PM +0200, Erik Cederstrand wrote: >> I'm wondering if this is necessary, or if this can possibly be turned >> of with a knob somewhere. > > Newer binutils got a flag after a discussion about this and related > issues in NetBSD. It basically stores 0 for the uid/gid/time fields. This also came up recently in the context of producing static libraries from object files with large uid's (such as in AFS). A couple of patches to ar(1) and libarchive were bandied about, though I don't think Tim Kientzle has committed any of them, yet. (As it turns out, I may also need to patch ranlib(1) in order to get a complete buildworld in my large-uid environment, and possibly other things.) -Ben Kaduk From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 15:27:48 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E92621065763 for ; Tue, 5 Oct 2010 15:27:47 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4ECBE8FC33 for ; Tue, 5 Oct 2010 15:27:45 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA28533; Tue, 05 Oct 2010 18:27:43 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4CAB43EF.8010807@freebsd.org> Date: Tue, 05 Oct 2010 18:27:43 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4CA4C63F.4070503@icyb.net.ua> In-Reply-To: <4CA4C63F.4070503@icyb.net.ua> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Ed Maste Subject: Re: sysctl for querying kmem_map->size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 15:27:49 -0000 on 30/09/2010 20:17 Andriy Gapon said the following: > > Here's a patch that adds a sysctl for querying kmem_map->size, which may be useful > for system state/resources monitoring: > http://people.freebsd.org/~avg/sysctl-kmem_map_size.diff > > I am quite unsure about sizeof(kmem_map->size) == sizeof(int) hack, but I couldn't > think of other way to decide whether to use SYSCTL_ADD_UINT or SYSCTL_ADD_ULONG > depending on real type behind vm_size_t. Here's an updated patch: http://people.freebsd.org/~avg/sysctl-kmem_map_size2.diff The new code wraps kmem_map->size into SYSCTL_PROC() to handle vm_size_t type differences for different platforms. The idea is suggested by Ed Maste. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 15:32:55 2010 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2C4EB106566C for ; Tue, 5 Oct 2010 15:32:55 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from www.sonnenberger.org (www.sonnenberger.org [92.79.50.50]) by mx1.freebsd.org (Postfix) with ESMTP id DAFB58FC12 for ; Tue, 5 Oct 2010 15:32:54 +0000 (UTC) Received: from britannica.bec.de (www.sonnenberger.org [192.168.1.10]) by www.sonnenberger.org (Postfix) with ESMTP id E57CE66663; Tue, 5 Oct 2010 17:13:26 +0200 (CEST) Received: by britannica.bec.de (Postfix, from userid 1000) id DE3C9117EF4; Tue, 5 Oct 2010 17:13:48 +0200 (CEST) Date: Tue, 5 Oct 2010 17:13:48 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org, FreeBSD Hackers Message-ID: <20101005151348.GA27763@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org, FreeBSD Hackers References: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Subject: Re: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 15:32:55 -0000 On Tue, Oct 05, 2010 at 03:28:36PM +0200, Erik Cederstrand wrote: > I'm wondering if this is necessary, or if this can possibly be turned > of with a knob somewhere. Newer binutils got a flag after a discussion about this and related issues in NetBSD. It basically stores 0 for the uid/gid/time fields. Joerg From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 18:18:14 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F1E21065673; Tue, 5 Oct 2010 18:18:14 +0000 (UTC) (envelope-from jeremie@le-hen.org) Received: from smtp5-g21.free.fr (smtp5-g21.free.fr [212.27.42.5]) by mx1.freebsd.org (Postfix) with ESMTP id 41FC38FC12; Tue, 5 Oct 2010 18:18:11 +0000 (UTC) Received: from endor.tataz.chchile.org (unknown [82.233.239.98]) by smtp5-g21.free.fr (Postfix) with ESMTP id C4EBDD4818E; Tue, 5 Oct 2010 20:18:05 +0200 (CEST) Received: from felucia.tataz.chchile.org (felucia.tataz.chchile.org [192.168.1.9]) by endor.tataz.chchile.org (Postfix) with ESMTP id A2FEF33D26; Tue, 5 Oct 2010 18:18:04 +0000 (UTC) Received: by felucia.tataz.chchile.org (Postfix, from userid 1000) id 8F38BA128C; Tue, 5 Oct 2010 18:18:04 +0000 (UTC) Date: Tue, 5 Oct 2010 20:18:04 +0200 From: Jeremie Le Hen To: Kostik Belousov Message-ID: <20101005181804.GJ7536@felucia.tataz.chchile.org> References: <20100803150545.GH14016@felucia.tataz.chchile.org> <20100803114651.651e0ea4@kan.dnsalias.net> <20100805191446.GJ14016@felucia.tataz.chchile.org> <20100919081406.GH6864@felucia.tataz.chchile.org> <20100919184146.GE2389@deviant.kiev.zoral.com.ua> <20100920162925.GL6864@felucia.tataz.chchile.org> <20100920192708.GK2389@deviant.kiev.zoral.com.ua> <20100927094651.GB57265@felucia.tataz.chchile.org> <20100927154457.GJ43070@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100927154457.GJ43070@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: kan@freebsd.org, freebsd-hackers@freebsd.org, Jeremie Le Hen Subject: Re: [PATCH] Add -lssp_nonshared to GCC's LIB_SPEC unconditionally X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 18:18:14 -0000 Hi Kib, On Mon, Sep 27, 2010 at 06:44:57PM +0300, Kostik Belousov wrote: > Hardcoding /usr/lib as the path to the library in the script looks > problematic. For the buidlworld, you are linking resulting binaries > with the host library, instead of the buildworld-produced one. For > lib32, it makes non-working combination of 32/64 bit. Sorry for the late reply, but I had to collect various evidences for my sayings and my development machine is reaaaaaaaaaaally slow. In fact it seems the toolchain built for buildworld contains a ld(1) binary which invariably bases lookups for libraries in ${WORLDTMP}, even in case of an absolute path. I have two evidences of this: - Putting /usr/obj/usr/src/tmp/usr/lib/libssp_nonshared.a in /usr/obj/usr/src/tmp/usr/lib/libc.ld leads toolchain's ld(1) to use /usr/obj/usr/src/tmp/usr/obj/usr/src/tmp/usr/lib/libssp_nonshared.a; - I also verified this with a hand-wrought opensnoop-like DTrace script. If it's enough for you, can you please go forward and commit my patch? I can make a cleaner one if you want. Thanks. Regards, -- Jeremie Le Hen Humans are born free and equal. But some are more equal than others. Coluche From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 21:19:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C133E106564A for ; Tue, 5 Oct 2010 21:19:35 +0000 (UTC) (envelope-from eknath.iyer@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 59AA18FC16 for ; Tue, 5 Oct 2010 21:19:34 +0000 (UTC) Received: by wwb24 with SMTP id 24so70314wwb.31 for ; Tue, 05 Oct 2010 14:19:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=OCD2u70yD3hoHmNiSRXu+m932T1T19dyRZAGQ4XOqEo=; b=j4UdToAfYfpl2ymehZoiYXKMGFH+c+NdBZC3TYeoJGooAP/0jZG43o6xzWa01LWIuO vz93QuNl9Z3z4xJxGNV4JzeSM5P3O6dUMuMuQxm0PrWSkrxuE5Xp9he06xnMGlCyihQ5 iXY0vKDtNQlcJGWhZ3Swdy/bIcpzMFGNJzwQM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=Y9UlqL9LATguyddLYOs72sphxsKD5GpNIDufwfqUE0Z0TlWRIpQf/Msl8O5bNrKyEK Z6xcLlBkGjkzj5B9m76U+Rc2/buDZ3PZg7HBOVhotlbS9SCt3kf91wlMY6/SJr3LQ1KE 2G+GJq151Opd9npfNNKuo6X4jqgPjiv2QhFdo= MIME-Version: 1.0 Received: by 10.216.7.210 with SMTP id 60mr1603614wep.30.1286313573538; Tue, 05 Oct 2010 14:19:33 -0700 (PDT) Received: by 10.216.26.65 with HTTP; Tue, 5 Oct 2010 14:19:33 -0700 (PDT) Date: Tue, 5 Oct 2010 17:19:33 -0400 Message-ID: From: Eknath Venkataramani To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Thread priorities X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 21:19:35 -0000 In kern_switch.c, It is said that, Free threads are 'preassigned' to the KSEs. and that the KSEs derive their priority from threads and are put on the run queue. 1. Where do the threads receive their priorities from? 2. Does it recieve the priority from the KSEG? -- Eknath Venkataramani From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 21:47:44 2010 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CEEF8106564A for ; Tue, 5 Oct 2010 21:47:44 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) by mx1.freebsd.org (Postfix) with ESMTP id 714D98FC08 for ; Tue, 5 Oct 2010 21:47:44 +0000 (UTC) Received: from [192.168.10.202] (0x573b9942.cpe.ge-1-2-0-1101.ronqu1.customer.tele.dk [87.59.153.66]) by csmtp2.one.com (Postfix) with ESMTPA id 315E6CC2DCBBD; Tue, 5 Oct 2010 21:47:42 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: multipart/signed; boundary=Apple-Mail-1468-917900132; protocol="application/pkcs7-signature"; micalg=sha1 From: Erik Cederstrand In-Reply-To: <20101005135906.GA3430@owl.midgard.homeip.net> Date: Tue, 5 Oct 2010 23:47:41 +0200 Message-Id: References: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> <20101005135906.GA3430@owl.midgard.homeip.net> To: Erik Trulsson X-Mailer: Apple Mail (2.1081) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FreeBSD Hackers Subject: Re: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 21:47:44 -0000 --Apple-Mail-1468-917900132 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Den 05/10/2010 kl. 15.59 skrev Erik Trulsson: > On Tue, Oct 05, 2010 at 03:28:36PM +0200, Erik Cederstrand wrote: >> Hello hackers, >>=20 >> I got reminded of a problem I had a couple of years back compressing >> FreeBSD jails. I was using bsdiff for the compression and found out >> that md5 sums of static libraries (.a files) in /usr/lib and >> /usr/local/lib didn't match between jails, even though the source >> code used to create the jails hadn't changed. One of my goals is to >> detect which files in a distribution change between two commits. >>=20 >> It turns out that timestamps are stored in the library: >=20 > Yes, they are. That is because the file format used for static > libraries include a timestamp for each object file stored in the > archive. >=20 > You can use 'ar -tv /PATH/TO/LIBRARY/libfoo.a' to get a list of the > objects stored in the archive and the corresponding timestamps. >=20 > See the ar(5) manpage for a description of the file format, and the > ar(1) manpage for information on how to manage such files. Thanks, that was very helpful. It seems I can at least normalize the .a = files using something like the following to weed out timestamps and = uid/gid: % ar -x /usr/lib/libfetch.a % chown 0:0 * % touch -t 197001010000 * % ar -r libfetch.a `ar -t /usr/lib/libfetch.a` The above takes ca. 10 seconds for all static libraries on my aging = machine, which is OK in my case. Unfortunately it seems there's still a = creation time of the archive itself that I cant alter using the above, = so the md5 sums still don't match: % diff mod.strings orig.strings 2c2 < / 1286312209 0 0 0 958 ` --- > / 1269146263 0 0 0 958 ` Using Python to replace 1286312209 with 1269146263 in the binary file = now gives me matching md5 sums. It seems a bit complicated just to get = rid of some timestamps, though. Thanks, Erik= --Apple-Mail-1468-917900132-- From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 22:29:32 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F37D3106566C for ; Tue, 5 Oct 2010 22:29:32 +0000 (UTC) (envelope-from giovanni.trematerra@gmail.com) Received: from mail-ww0-f42.google.com (mail-ww0-f42.google.com [74.125.82.42]) by mx1.freebsd.org (Postfix) with ESMTP id 8C78C8FC0A for ; Tue, 5 Oct 2010 22:29:32 +0000 (UTC) Received: by wwb13 with SMTP id 13so142036wwb.1 for ; Tue, 05 Oct 2010 15:29:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=vpudEbN478f0sSislrslp/TFTyRUHtm088WWIVsDQM8=; b=wFhlwHetkCJYvXczIOdEXJbbtXJlvWaS/xyHoPo9s9sV0JAJF4moPtYVq86TkkuAVq M4u8TUfUTpTbdhj46kHGUw4KLQyqwV17pVfiJUMF5/KrqsRKLsrGPT9SYLivDR72c5xI j4KcPp4YrC956SJDyHDsgY4quvyCYloEgvjVA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=mAe1Mohvk8oy/A+gu/XAkmU0Le9ponhwGlEP3x9OkOaICd4JACWc6l0v6qcDexGbKu o9R/tcQViMfsswX9DZ4zI9k+L1Gpm2Vq1pzvMO2KLGrR1bONieriN4nXGkbB37zKscsn afa2DfttW8QZzE1fYtFKBbERruz1npbSnOMlo= MIME-Version: 1.0 Received: by 10.227.129.12 with SMTP id m12mr10154449wbs.102.1286317770876; Tue, 05 Oct 2010 15:29:30 -0700 (PDT) Sender: giovanni.trematerra@gmail.com Received: by 10.227.144.203 with HTTP; Tue, 5 Oct 2010 15:29:30 -0700 (PDT) Date: Wed, 6 Oct 2010 00:29:30 +0200 X-Google-Sender-Auth: S7iyWq_zIwvvbRVmPq9btJ1JkZg Message-ID: From: Giovanni Trematerra To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: kernel micro-benchmarking framework X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 22:29:33 -0000 Hi all, based on a work of rwatson@ about micro-benchmarking, I managed to have a kernel module that exposes some sysctls. Reading sysctl associated to test start the benchmark and print the results. The code is split up in this way: test.h, test.c where the infrastructure work lives. test_sync_timing.c test_mem_timing.c where the actual micro-benchmarks live: I wrote some macros to simplify adding more benchmarks. (test.h) The idea is to have a struct for every benchmark/test struct timing_proc { void (*setup)(void *); /* called before starting timing */ void (*test)(void *); /* what we want microbenchmark */ void (*tear_down)(void *); /* called after the end of timing */ void *args; /* pointer passed to the above funcs */ }; and let an agnostic code deals with it. Every test can specify a setup and tear_down function for allocate/deallocate resources and a test function to benchmark things. The great difference with Robert's code is that the test function cannot be inline as it's a pointer to function. I don't know if that could influence the results. The test function is called with interrupt disabled. We could further extent this framework to add regression test support. You could get the code here: http://www.trematerra.net/patches/timing.tbz Feedback and reviews are welcome. Thanks -- Gianni From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 22:46:56 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8989106564A for ; Tue, 5 Oct 2010 22:46:56 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) by mx1.freebsd.org (Postfix) with ESMTP id 7FF9F8FC12 for ; Tue, 5 Oct 2010 22:46:56 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.4/8.14.4) with ESMTP id o95MktaW096672; Tue, 5 Oct 2010 16:46:55 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.4/8.14.4/Submit) with ESMTP id o95Mkt2d096669; Tue, 5 Oct 2010 16:46:55 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Tue, 5 Oct 2010 16:46:55 -0600 (MDT) From: Warren Block To: Alexander Leidinger In-Reply-To: <20101005152950.16797j1h6ygogi88@webmail.leidinger.net> Message-ID: References: <20101004122325.11342p9v1rnwp6w4@webmail.leidinger.net> <20101004123629.GA4308@straylight.ringlet.net> <20101005113423.10801orcw6xef9s0@webmail.leidinger.net> <20101005.124823.093.1@DEV> <20101005152950.16797j1h6ygogi88@webmail.leidinger.net> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (wonkity.com [127.0.0.1]); Tue, 05 Oct 2010 16:46:55 -0600 (MDT) Cc: rank1seeker@gmail.com, freebsd-hackers@freebsd.org Subject: Re: Improve OptionalObsoleteFiles.inc X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 22:46:56 -0000 On Tue, 5 Oct 2010, Alexander Leidinger wrote: > > We do not have a target which allows to go to src/bin/XXX and run "make > deinstall" or similar. What is correctly written in this thread is that "make > check-old" and "make delete-old" (and "make delete-old-libs") is supposed to > do what you want (given that there is already support in the src to exclude > this part of FreeBSD during installation). The most up-to-date version is in > -current, I do not know about the stable branches (I do not monitor if people > which change the corresponding file have merged the changes to stable > branches or not). > > What may not be 100% correct in src is the list of files to delete (= a bug). > For this the suggestion in the thread is also correct, if there is something > which does not work please send a PR. This would be more accurate and require less maintenance if OptionalObsoleteFiles.inc were auto-generated ala genplist. Don't know if that's practical, just wanted to suggest it. From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 23:00:05 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 372AA106566C for ; Tue, 5 Oct 2010 23:00:05 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from out-0.mx.aerioconnect.net (out-0-19.mx.aerioconnect.net [216.240.47.79]) by mx1.freebsd.org (Postfix) with ESMTP id 116368FC1A for ; Tue, 5 Oct 2010 23:00:04 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o95Mdmdl009566; Tue, 5 Oct 2010 15:39:48 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 406722D6015; Tue, 5 Oct 2010 15:39:47 -0700 (PDT) Message-ID: <4CABA963.40709@freebsd.org> Date: Tue, 05 Oct 2010 15:40:35 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.9) Gecko/20100915 Thunderbird/3.1.4 MIME-Version: 1.0 To: Eknath Venkataramani References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: freebsd-hackers@freebsd.org Subject: Re: Thread priorities X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 23:00:05 -0000 On 10/5/10 2:19 PM, Eknath Venkataramani wrote: > In kern_switch.c, > > It is said that, Free threads are 'preassigned' to the KSEs. and that the > KSEs derive their priority from threads and are put on the run queue. > 1. Where do the threads receive their priorities from? from the process they are assigned to. > 2. Does it recieve the priority from the KSEG? well, it got some modification of it's priority due to KSEG behaviour. (or at least that was the plan) this must be very old code because ksegs were removed some time ago. please From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 5 23:50:54 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id EA90A1065673; Tue, 5 Oct 2010 23:50:54 +0000 (UTC) Date: Tue, 5 Oct 2010 23:50:54 +0000 From: Alexander Best To: freebsd-hackers@freebsd.org Message-ID: <20101005235054.GA45827@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Subject: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Oct 2010 23:50:55 -0000 hi there, i think the following example shows the problem better than a long explanation: `touch ftest && chflags arch ftest && chflags -vv 0 ftest`. ^^non-root ^^root ^^non-root chflags claims to have cleared the 'arch' flag (which should be impossible as non-root user), but indeed has done nothing. i've tried the same with 'sappnd' and that works as can be expected. The issue was confirmed to exist in HEAD (me), stable/8 (pgollucc1, jpaetzel) and stable/7 (nox). On stable/6 it does NOT exist (jpaetzel). chflags properly fails with EPERM. cheers. alex -- a13x From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 05:36:59 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 929821065747 for ; Wed, 6 Oct 2010 05:36:59 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 48C408FC13 for ; Wed, 6 Oct 2010 05:36:59 +0000 (UTC) Received: by iwn8 with SMTP id 8so617290iwn.13 for ; Tue, 05 Oct 2010 22:36:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=KX0+2or9Zz9H3lOtlb3EncgymJbdmH9sKNuL3E4OSNI=; b=JbM6XDcpAMXUN2ynrEima5LbdyKb4gNd7a2l/97JROhMyHvWUGnUQyxkukILG4A4Ob PLPTh2C7bo3lW9b0kUQ3/J9nSva71SAEQUYim575R1zTdW5EOr8sG58qcTBXD2K8Js2d 3iVGiSp7ORn6sXZTLKFpG2wY0uXO0GgW1iP18= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=CSEKv4SM9LVn8vUbmiHBP81CdV7d56nW47exqsfjf97OMsIPMrf+pptyv/GBb2RlAt yLraaVstfifILdj1EzZHedWjwxNv+9zbR+LYYtFrfl8FZX8UuwY/tQ3A7ozpFkXz1oRq wcBUN2LbJcrAVq4zAUHRebI6kbUXCT4kTlwRQ= MIME-Version: 1.0 Received: by 10.231.157.205 with SMTP id c13mr13348072ibx.71.1286343417465; Tue, 05 Oct 2010 22:36:57 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.231.171.203 with HTTP; Tue, 5 Oct 2010 22:36:57 -0700 (PDT) In-Reply-To: References: Date: Wed, 6 Oct 2010 13:36:57 +0800 X-Google-Sender-Auth: 05qDm20Cst2aWQZlpo1cExV8IwE Message-ID: From: Adrian Chadd To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: breaking the crunchgen logic into a share/mk file X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 05:36:59 -0000 On 5 October 2010 11:19, Garrett Cooper wrote: > Hi Adrian, > =A0 =A0Is there a functional difference between the original file and > bsd.crunchgen.mk ? Only the bare minimum needed to make it useful: * added shared library as well as fully static library support * added a method for listing sourcedirs that need "build-tools" target built first. These were hard-coded. I specifically didn't want to introduce too much in this first pass. Adrian From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 06:19:11 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 03FAB1065693 for ; Wed, 6 Oct 2010 06:19:11 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id D84888FC12 for ; Wed, 6 Oct 2010 06:19:10 +0000 (UTC) Received: from [10.123.2.179] ([192.168.1.65]) by monday.kientzle.com (8.14.3/8.14.3) with ESMTP id o9660mZV002132; Wed, 6 Oct 2010 06:00:48 GMT (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: text/plain; charset=us-ascii From: Tim Kientzle In-Reply-To: Date: Tue, 5 Oct 2010 23:00:47 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <140AD250-ECD1-4BCF-806B-F5901B3BFD87@kientzle.com> References: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> <20101005135906.GA3430@owl.midgard.homeip.net> To: Erik Cederstrand X-Mailer: Apple Mail (2.1081) Cc: FreeBSD Hackers Subject: Re: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 06:19:11 -0000 On Oct 5, 2010, at 2:47 PM, Erik Cederstrand wrote: > Den 05/10/2010 kl. 15.59 skrev Erik Trulsson: > On Tue, Oct 05, 2010 at 03:28:36PM +0200, Erik Cederstrand wrote: >>>=20 >>> I was using bsdiff for the compression and found out >>> that md5 sums of static libraries (.a files) in /usr/lib and >>> /usr/local/lib didn't match between jails, even though the source >>> code used to create the jails hadn't changed. One of my goals is to >>> detect which files in a distribution change between two commits. >>>=20 >>> It turns out that timestamps are stored in the library: >>=20 >=20 > Thanks, that was very helpful. It seems I can at least normalize the = .a files ... Unfortunately it seems there's still a creation time of the = archive itself that I cant alter using the above, so the md5 sums still = don't match: >=20 > % diff mod.strings orig.strings > 2c2 > < / 1286312209 0 0 0 958 ` > --- >> / 1269146263 0 0 0 958 ` That's the timestamp on the pseudo-entry used to store the archive symbol table. (GNU/SysV ar files use a pseudo-entry named "/" to store the symbol table. Since '/' is added to end of names in this format, this is essentially an entry with an empty name.) The current ar code generates this entry and sets the timestamp around line 624 of usr.bin/ar/write.c. As far as I can tell, ar itself doesn't care about the timestamp here; it's possible that ranlib or ld do care. If they don't, we could set this timestamp to zero always. Tim From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 07:20:49 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F1AEC106567A for ; Wed, 6 Oct 2010 07:20:49 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (unknown [IPv6:2a01:348:0:15:5d59:5c40:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 885158FC14 for ; Wed, 6 Oct 2010 07:20:49 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id A3C17E60C0 for ; Wed, 6 Oct 2010 08:20:48 +0100 (BST) Received: from core.nessbank (client-82-31-12-174.midd.adsl.virginmedia.com [82.31.12.174]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA for ; Wed, 6 Oct 2010 08:20:48 +0100 (BST) From: Bruce Cran To: freebsd-hackers@freebsd.org Date: Wed, 6 Oct 2010 08:20:47 +0100 User-Agent: KMail/1.13.5 (FreeBSD/9.0-CURRENT; KDE/4.5.1; amd64; ; ) References: <20101005235054.GA45827@freebsd.org> In-Reply-To: <20101005235054.GA45827@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201010060820.47559.bruce@cran.org.uk> Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 07:20:50 -0000 On Wednesday 06 October 2010 00:50:54 Alexander Best wrote: > `touch ftest && chflags arch ftest && chflags -vv 0 ftest`. > ^^non-root ^^root ^^non-root > > chflags claims to have cleared the 'arch' flag (which should be impossible > as non-root user), but indeed has done nothing. I guess that should be "sudo chflags arch ftest"? -- Bruce Cran From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 07:49:24 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F15D5106564A for ; Wed, 6 Oct 2010 07:49:23 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) by mx1.freebsd.org (Postfix) with ESMTP id 9282B8FC1B for ; Wed, 6 Oct 2010 07:49:23 +0000 (UTC) Received: from [192.168.0.22] (0x573fa596.cpe.ge-1-1-0-1109.ronqu1.customer.tele.dk [87.63.165.150]) by csmtp2.one.com (Postfix) with ESMTPA id CFB965E36103A; Wed, 6 Oct 2010 07:49:21 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: multipart/signed; boundary=Apple-Mail-1524-954000664; protocol="application/pkcs7-signature"; micalg=sha1 From: Erik Cederstrand In-Reply-To: <140AD250-ECD1-4BCF-806B-F5901B3BFD87@kientzle.com> Date: Wed, 6 Oct 2010 09:49:22 +0200 Message-Id: References: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> <20101005135906.GA3430@owl.midgard.homeip.net> <140AD250-ECD1-4BCF-806B-F5901B3BFD87@kientzle.com> To: Tim Kientzle X-Mailer: Apple Mail (2.1081) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FreeBSD Hackers Subject: Re: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 07:49:24 -0000 --Apple-Mail-1524-954000664 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Den 06/10/2010 kl. 08.00 skrev Tim Kientzle: >=20 > That's the timestamp on the pseudo-entry used to store the archive > symbol table. (GNU/SysV ar files use a pseudo-entry named "/" to > store the symbol table. Since '/' is added to end of names in this > format, this is essentially an entry with an empty name.) >=20 > The current ar code generates this entry and sets the timestamp > around line 624 of usr.bin/ar/write.c. >=20 > As far as I can tell, ar itself doesn't care about the timestamp here; > it's possible that ranlib or ld do care. If they don't, we could set > this timestamp to zero always. Yeah, I had a look at the code too. I was thinking maybe it would help = to add a modifier to replace timestamps, uids and gids with '0' when = inserting a file, to respect POLA. Thanks, Erik= --Apple-Mail-1524-954000664-- From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 08:11:41 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A055F1065674 for ; Wed, 6 Oct 2010 08:11:41 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id 7FC658FC1E for ; Wed, 6 Oct 2010 08:11:41 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id o968BXNp067062 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 6 Oct 2010 01:11:34 -0700 (PDT) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id o968BXZR067061; Wed, 6 Oct 2010 01:11:33 -0700 (PDT) Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA06711; Wed, 6 Oct 10 01:06:20 PDT Date: Wed, 06 Oct 2010 01:06:19 -0700 From: perryh@pluto.rain.com To: erik@cederstrand.dk Message-Id: <4cac2dfb.tcWd/AxzVjqyrL09%perryh@pluto.rain.com> References: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> <20101005135906.GA3430@owl.midgard.homeip.net> In-Reply-To: User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org Subject: Re: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 08:11:41 -0000 Erik Cederstrand wrote: > It seems I can at least normalize the .a files using something > like the following to weed out timestamps and uid/gid: > > % ar -x /usr/lib/libfetch.a > % chown 0:0 * > % touch -t 197001010000 * > % ar -r libfetch.a `ar -t /usr/lib/libfetch.a` > > ... Unfortunately it seems there's still a creation time of the > archive itself that I cant alter using the above, so the md5 sums > still don't match: > > % diff mod.strings orig.strings > 2c2 > < / 1286312209 0 0 0 958 ` > --- > > / 1269146263 0 0 0 958 ` Any particular reason to recollect them into an archive, if the point is just to check md5 signatures? I'm pretty sure collecting them with tar instead will avoid this problem. From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 09:58:01 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A8AF71065670 for ; Wed, 6 Oct 2010 09:58:01 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp1.one.com (csmtp1.one.com [195.47.247.21]) by mx1.freebsd.org (Postfix) with ESMTP id 430F48FC13 for ; Wed, 6 Oct 2010 09:58:01 +0000 (UTC) Received: from [192.168.0.22] (0x573fa596.cpe.ge-1-1-0-1109.ronqu1.customer.tele.dk [87.63.165.150]) by csmtp1.one.com (Postfix) with ESMTP id 8E5BF1BC054B5; Wed, 6 Oct 2010 09:57:59 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: multipart/signed; boundary=Apple-Mail-1545-961717679; protocol="application/pkcs7-signature"; micalg=sha1 From: Erik Cederstrand In-Reply-To: <4cac2dfb.tcWd/AxzVjqyrL09%perryh@pluto.rain.com> Date: Wed, 6 Oct 2010 11:57:59 +0200 Message-Id: <9C53E6F3-82AB-4D95-B61A-ED49AC030B7A@cederstrand.dk> References: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> <20101005135906.GA3430@owl.midgard.homeip.net> <4cac2dfb.tcWd/AxzVjqyrL09%perryh@pluto.rain.com> To: perryh@pluto.rain.com X-Mailer: Apple Mail (2.1081) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: hackers@freebsd.org Subject: Re: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 09:58:01 -0000 --Apple-Mail-1545-961717679 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Den 06/10/2010 kl. 10.06 skrev perryh@pluto.rain.com: > Erik Cederstrand wrote: >=20 >> It seems I can at least normalize the .a files using something >> like the following to weed out timestamps and uid/gid: >>=20 >> % ar -x /usr/lib/libfetch.a >> % chown 0:0 * >> % touch -t 197001010000 * >> % ar -r libfetch.a `ar -t /usr/lib/libfetch.a` >>=20 >> ... Unfortunately it seems there's still a creation time of the >> archive itself that I cant alter using the above, so the md5 sums >> still don't match: >>=20 >> % diff mod.strings orig.strings >> 2c2 >> < / 1286312209 0 0 0 958 ` >> --- >>> / 1269146263 0 0 0 958 ` >=20 > Any particular reason to recollect them into an archive, if the > point is just to check md5 signatures? I'm pretty sure collecting > them with tar instead will avoid this problem. That's of course another option. I could unpack the archive and be = satisfied if all contained files have matching md5's. I guess the perfectionist in me is protesting. If I build FreeBSD twice = from the same source code, the result should be exactly the same. It = would be useful in a number of cases, e.g. IDS. ccache also relies on = the assumption that checksums will match on identical source code and = compiler version, although I know that's a level beneath where ar = operates. Thanks, Erik= --Apple-Mail-1545-961717679-- From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 11:07:41 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3D53106564A for ; Wed, 6 Oct 2010 11:07:41 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp3.one.com (csmtp3.one.com [91.198.169.23]) by mx1.freebsd.org (Postfix) with ESMTP id 43A978FC1F for ; Wed, 6 Oct 2010 11:07:41 +0000 (UTC) Received: from [192.168.0.22] (0x573fa596.cpe.ge-1-1-0-1109.ronqu1.customer.tele.dk [87.63.165.150]) by csmtp3.one.com (Postfix) with ESMTP id 1FEE9240B1C3; Wed, 6 Oct 2010 11:07:39 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: multipart/signed; boundary=Apple-Mail-1552-965896787; protocol="application/pkcs7-signature"; micalg=sha1 From: Erik Cederstrand In-Reply-To: <140AD250-ECD1-4BCF-806B-F5901B3BFD87@kientzle.com> Date: Wed, 6 Oct 2010 13:07:38 +0200 Message-Id: References: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> <20101005135906.GA3430@owl.midgard.homeip.net> <140AD250-ECD1-4BCF-806B-F5901B3BFD87@kientzle.com> To: Tim Kientzle X-Mailer: Apple Mail (2.1081) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FreeBSD Hackers Subject: Re: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 11:07:41 -0000 --Apple-Mail-1552-965896787 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Den 06/10/2010 kl. 08.00 skrev Tim Kientzle: >>=20 >> % diff mod.strings orig.strings >> 2c2 >> < / 1286312209 0 0 0 958 ` >> --- >>> / 1269146263 0 0 0 958 ` >=20 > That's the timestamp on the pseudo-entry used to store the archive > symbol table. (GNU/SysV ar files use a pseudo-entry named "/" to > store the symbol table. Since '/' is added to end of names in this > format, this is essentially an entry with an empty name.) >=20 > The current ar code generates this entry and sets the timestamp > around line 624 of usr.bin/ar/write.c. >=20 > As far as I can tell, ar itself doesn't care about the timestamp here; > it's possible that ranlib or ld do care. If they don't, we could set > this timestamp to zero always. Is something like the following acceptable? Without risking changes to = buildworld/distribution just now, this would allow me to dump contents = of an archive and re-insert them with '0' for mtime, uid and gid before = checking checksums, without affecting normal ar behaviour. erik% svn diff Index: ar.1 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- ar.1 (revision 213478) +++ ar.1 (working copy) @@ -62,6 +62,7 @@ .Op Fl a Ar position-after .Op Fl b Ar position-before .Op Fl c +.Op Fl D .Op Fl i Ar position-before .Op Fl j .Op Fl s @@ -179,6 +180,14 @@ .Ar archive . The archive's symbol table, if present, is updated to reflect the new contents of the archive. +.It Fl D +When used in combination with the=20 +.Fl r +option, insterts 0's instead of the real mtime, uid and gid values +from the members named by arguments +.Ar files ... . +This ensures that checksums on the resulting archives are reproducible +when member contents are identical. .It Fl f Synonymous with option .Fl T . Index: ar.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- ar.c (revision 213478) +++ ar.c (working copy) @@ -154,7 +154,7 @@ } } =20 - while ((opt =3D getopt_long(argc, argv, = "abCcdfijlMmopqrSsTtuVvxz", + while ((opt =3D getopt_long(argc, argv, = "abCcdDfijlMmopqrSsTtuVvxz", longopts, NULL)) !=3D -1) { switch(opt) { case 'a': @@ -173,6 +173,9 @@ case 'd': set_mode(bsdar, opt); break; + case 'D': + bsdar->options |=3D AR_D; + break; case 'f': case 'T': bsdar->options |=3D AR_TR; @@ -357,8 +360,8 @@ (void)fprintf(stderr, "\tar -m [-Tabijsvz] position archive file = ...\n"); (void)fprintf(stderr, "\tar -p [-Tv] archive [file ...]\n"); (void)fprintf(stderr, "\tar -q [-Tcjsvz] archive file ...\n"); - (void)fprintf(stderr, "\tar -r [-Tcjsuvz] archive file ...\n"); - (void)fprintf(stderr, "\tar -r [-Tabcijsuvz] position archive = file ...\n"); + (void)fprintf(stderr, "\tar -r [-TcDjsuvz] archive file ...\n"); + (void)fprintf(stderr, "\tar -r [-TabcDijsuvz] position archive = file ...\n"); (void)fprintf(stderr, "\tar -s [-jz] archive\n"); (void)fprintf(stderr, "\tar -t [-Tv] archive [file ...]\n"); (void)fprintf(stderr, "\tar -x [-CTouv] archive [file ...]\n"); Index: ar.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- ar.h (revision 213478) +++ ar.h (working copy) @@ -43,6 +43,7 @@ #define AR_U 0x0200 /* only extract or update newer = members.*/ #define AR_V 0x0400 /* verbose mode */ #define AR_Z 0x0800 /* gzip compression */ +#define AR_D 0x1000 /* insert members with dummy mtime, uid = and gid */ =20 #define DEF_BLKSZ 10240 /* default block size */ =20 Index: write.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- write.c (revision 213478) +++ write.c (working copy) @@ -163,11 +163,22 @@ if (mtime !=3D 0 && bsdar->options & AR_U && sb.st_mtime <=3D = mtime) goto giveup; =20 - obj->uid =3D sb.st_uid; - obj->gid =3D sb.st_gid; + /* + * When modifier -D is specified, mtime and UID / GID from the = file + * will be replaced with 0. This ensures that checksums will = match=20 + * for two archives containing the exact same files. + */ + if (bsdar->options & AR_D) { + obj->uid =3D 0; + obj->gid =3D 0; + obj->mtime =3D 0; + } else { + obj->uid =3D sb.st_uid; + obj->gid =3D sb.st_gid; + obj->mtime =3D sb.st_mtime; + } obj->md =3D sb.st_mode; obj->size =3D sb.st_size; - obj->mtime =3D sb.st_mtime; obj->dev =3D sb.st_dev; obj->ino =3D sb.st_ino; =20 @@ -621,7 +632,10 @@ bsdar->options & AR_S) { entry =3D archive_entry_new(); archive_entry_copy_pathname(entry, "/"); - archive_entry_set_mtime(entry, time(NULL), 0); + if (bsdar->options & AR_O) + archive_entry_set_mtime(entry, 0, 0); + else + archive_entry_set_mtime(entry, time(NULL), 0); archive_entry_set_size(entry, (bsdar->s_cnt + 1) * sizeof(uint32_t) + bsdar->s_sn_sz); AC(archive_write_header(a, entry)); Thanks, Erik --Apple-Mail-1552-965896787-- From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 12:35:56 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 076DA1065670 for ; Wed, 6 Oct 2010 12:35:56 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) by mx1.freebsd.org (Postfix) with ESMTP id 6E65B8FC08 for ; Wed, 6 Oct 2010 12:35:55 +0000 (UTC) Received: from [192.168.0.22] (0x573fa596.cpe.ge-1-1-0-1109.ronqu1.customer.tele.dk [87.63.165.150]) by csmtp2.one.com (Postfix) with ESMTPA id DF249E1C6C533; Wed, 6 Oct 2010 12:35:52 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1081) Content-Type: multipart/signed; boundary=Apple-Mail-1574-971190574; protocol="application/pkcs7-signature"; micalg=sha1 From: Erik Cederstrand In-Reply-To: Date: Wed, 6 Oct 2010 14:35:52 +0200 Message-Id: References: <52C32EA5-A380-4237-A27C-9E2DF2D4E022@cederstrand.dk> <20101005135906.GA3430@owl.midgard.homeip.net> <140AD250-ECD1-4BCF-806B-F5901B3BFD87@kientzle.com> To: Erik Cederstrand X-Mailer: Apple Mail (2.1081) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FreeBSD Hackers , Kai Wang Subject: Re: Timestamps in static libraries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 12:35:56 -0000 --Apple-Mail-1574-971190574 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Den 06/10/2010 kl. 13.07 skrev Erik Cederstrand: > Is something like the following acceptable? Without risking changes to = buildworld/distribution just now, this would allow me to dump contents = of an archive and re-insert them with '0' for mtime, uid and gid before = checking checksums, without affecting normal ar behaviour. Great, I didn't even see the discussion on this list recently: = http://lists.freebsd.org/pipermail/freebsd-hackers/2010-September/033005.h= tml Anyway, I added file mode to the patch. Apparently recent binutils also = added this feature. I haven't looked at their patch, but chance has it I = used the same option '-D'. I'm sure the file mode I'm setting in line 175 of write.c is wrong = (obj->md =3D 100644), but I couldn't quite figure out how to set the = value to rw-r--r--. Here's the new patch: Index: ar.1 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- ar.1 (revision 213478) +++ ar.1 (working copy) @@ -62,6 +62,7 @@ .Op Fl a Ar position-after .Op Fl b Ar position-before .Op Fl c +.Op Fl D .Op Fl i Ar position-before .Op Fl j .Op Fl s @@ -179,6 +180,14 @@ .Ar archive . The archive's symbol table, if present, is updated to reflect the new contents of the archive. +.It Fl D +When used in combination with the=20 +.Fl r +option, insterts 0's instead of the real mtime, uid and gid values=20 +and 644 instead of file mode from the members named by arguments +.Ar files ... . +This ensures that checksums on the resulting archives are reproducible +when member contents are identical. .It Fl f Synonymous with option .Fl T . Index: ar.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- ar.c (revision 213478) +++ ar.c (working copy) @@ -154,7 +154,7 @@ } } =20 - while ((opt =3D getopt_long(argc, argv, = "abCcdfijlMmopqrSsTtuVvxz", + while ((opt =3D getopt_long(argc, argv, = "abCcdDfijlMmopqrSsTtuVvxz", longopts, NULL)) !=3D -1) { switch(opt) { case 'a': @@ -173,6 +173,9 @@ case 'd': set_mode(bsdar, opt); break; + case 'D': + bsdar->options |=3D AR_D; + break; case 'f': case 'T': bsdar->options |=3D AR_TR; @@ -357,8 +360,8 @@ (void)fprintf(stderr, "\tar -m [-Tabijsvz] position archive file = ...\n"); (void)fprintf(stderr, "\tar -p [-Tv] archive [file ...]\n"); (void)fprintf(stderr, "\tar -q [-Tcjsvz] archive file ...\n"); - (void)fprintf(stderr, "\tar -r [-Tcjsuvz] archive file ...\n"); - (void)fprintf(stderr, "\tar -r [-Tabcijsuvz] position archive = file ...\n"); + (void)fprintf(stderr, "\tar -r [-TcDjsuvz] archive file ...\n"); + (void)fprintf(stderr, "\tar -r [-TabcDijsuvz] position archive = file ...\n"); (void)fprintf(stderr, "\tar -s [-jz] archive\n"); (void)fprintf(stderr, "\tar -t [-Tv] archive [file ...]\n"); (void)fprintf(stderr, "\tar -x [-CTouv] archive [file ...]\n"); Index: ar.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- ar.h (revision 213478) +++ ar.h (working copy) @@ -43,6 +43,7 @@ #define AR_U 0x0200 /* only extract or update newer = members.*/ #define AR_V 0x0400 /* verbose mode */ #define AR_Z 0x0800 /* gzip compression */ +#define AR_D 0x1000 /* insert members with dummy mode, = mtime, uid and gid */ =20 #define DEF_BLKSZ 10240 /* default block size */ =20 Index: write.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- write.c (revision 213478) +++ write.c (working copy) @@ -163,11 +163,23 @@ if (mtime !=3D 0 && bsdar->options & AR_U && sb.st_mtime <=3D = mtime) goto giveup; =20 - obj->uid =3D sb.st_uid; - obj->gid =3D sb.st_gid; - obj->md =3D sb.st_mode; + /* + * When option '-D' is specified, mtime and UID / GID from the = file + * will be replaced with 0, and file mode with 644. This ensures = that=20 + * checksums will match for two archives containing the exact = same files. + */ + if (bsdar->options & AR_D) { + obj->uid =3D 0; + obj->gid =3D 0; + obj->mtime =3D 0; + obj->md =3D 100644; + } else { + obj->uid =3D sb.st_uid; + obj->gid =3D sb.st_gid; + obj->mtime =3D sb.st_mtime; + obj->md =3D sb.st_mode; + } obj->size =3D sb.st_size; - obj->mtime =3D sb.st_mtime; obj->dev =3D sb.st_dev; obj->ino =3D sb.st_ino; =20 @@ -621,7 +633,10 @@ bsdar->options & AR_S) { entry =3D archive_entry_new(); archive_entry_copy_pathname(entry, "/"); - archive_entry_set_mtime(entry, time(NULL), 0); + if (bsdar->options & AR_O) + archive_entry_set_mtime(entry, 0, 0); + else + archive_entry_set_mtime(entry, time(NULL), 0); archive_entry_set_size(entry, (bsdar->s_cnt + 1) * sizeof(uint32_t) + bsdar->s_sn_sz); AC(archive_write_header(a, entry)); Thanks, Erik= --Apple-Mail-1574-971190574-- From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 17:09:23 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6C7F106564A; Wed, 6 Oct 2010 17:09:22 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 889998FC0A; Wed, 6 Oct 2010 17:09:22 +0000 (UTC) Received: by gyg4 with SMTP id 4so3268077gyg.13 for ; Wed, 06 Oct 2010 10:09:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=rJKoO6d7/XWbYQOnMWTYp2gjzcMlDO1/RPWNS49lcoM=; b=rTPmzJyyGhfRNkk3HGoRe/96BqyKFjqQwCHVEbn6xDQ3jKVez3pwOgwUSt0W36W/dJ 0/CKbKjNkSWzAxlHS6kMoOOv2TWvopzcCowWIwxomQnAA1RSE7XjyWs/peqxUP6oXVwL R9G3CmVzzTrdYU+a4gUNed6mVO5SBPrfdjxog= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=xKxKHSkvamlp4yc1HJS50Oi596Bh1s8RX98Vtp0cm8acno2TIfgr8vq620ZnonQNA+ qykNuK3dyMjmgAQSszOh5I90WyvmdZ/Na9OKrMKJ5iJIxNRPGoawj4jGiJRAdGpXkoxg Zi+war5XJSS/RnhEv8cRxLcmFdkzQwnswPFJk= MIME-Version: 1.0 Received: by 10.42.1.75 with SMTP id 11mr3043873icf.6.1286384961290; Wed, 06 Oct 2010 10:09:21 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.184.3 with HTTP; Wed, 6 Oct 2010 10:09:21 -0700 (PDT) In-Reply-To: <20101005235054.GA45827@freebsd.org> References: <20101005235054.GA45827@freebsd.org> Date: Wed, 6 Oct 2010 10:09:21 -0700 X-Google-Sender-Auth: aplxqdWYXEEajF4d-6JDuufpcOI Message-ID: From: Garrett Cooper To: Alexander Best Content-Type: multipart/mixed; boundary=90e6ba181872a98a1d0491f5d630 Cc: freebsd-hackers@freebsd.org Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 17:09:23 -0000 --90e6ba181872a98a1d0491f5d630 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Tue, Oct 5, 2010 at 4:50 PM, Alexander Best wrote: > hi there, > > i think the following example shows the problem better than a long explan= ation: > > `touch ftest && chflags arch ftest && chflags -vv 0 ftest`. > =A0^^non-root =A0 =A0 ^^root =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0^^non-root > > chflags claims to have cleared the 'arch' flag (which should be impossibl= e as > non-root user), but indeed has done nothing. > > i've tried the same with 'sappnd' and that works as can be expected. > > The issue was confirmed to exist in HEAD (me), stable/8 (pgollucc1, jpaet= zel) > and stable/7 (nox). > On stable/6 it does NOT exist (jpaetzel). chflags properly fails with EPE= RM. Fails for me when I call the syscall directly, as I would expect, and passes when I'm superuser: $ ./test_chflags (uid, euid) =3D (1000, 1000) test_chflags: chflags: Operation not permitted test_chflags: lchflags: Operation not permitted $ sudo ./test_chflags (uid, euid) =3D (0, 0) According to my basic inspection in strtofflags (.../lib/libc/gen/strtofflags.c), it works as well. And last but not least, executing the commands directly on the CLI work= : $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` $ chflags arch $tmpfile chflags: /tmp/chflags.nQm1IL: Operation not permitted $ rm $tmpfile $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` $ sudo chflags arch $tmpfile $ sudo chflags noarch $tmpfile $ rm $tmpfile Your results may (but shouldn't) vary [unless your environment is setup differently]... Please note that I'm using UFS2 with SUJ... not all filesystems support this (ext2/3/4? msdosfs? ZFS?), so I would be careful about which filesystem you pick and whether or not there's a bug where it's not properly identifying that the operation you're attempting to perform is valid. Thanks, -Garrett $ uname -a FreeBSD bayonetta.local 9.0-CURRENT FreeBSD 9.0-CURRENT #9 r211309M: Thu Aug 19 22:50:36 PDT 2010 root@bayonetta.local:/usr/obj/usr/src/sys/BAYONETTA amd64 --90e6ba181872a98a1d0491f5d630 Content-Type: application/octet-stream; name="test_chflags.c" Content-Disposition: attachment; filename="test_chflags.c" Content-Transfer-Encoding: base64 X-Attachment-Id: f_geyg1gi71 I2luY2x1ZGUgPHN5cy9zdGF0Lmg+CiNpbmNsdWRlIDxlcnIuaD4KI2luY2x1ZGUgPGZjbnRsLmg+ CiNpbmNsdWRlIDxzdGRpby5oPgojaW5jbHVkZSA8dW5pc3RkLmg+CgppbnQgbWFpbihpbnQgYXJn YywgY2hhciphcmd2W10pCnsKCglwcmludGYoIih1aWQsIGV1aWQpID0gKCVkLCAlZClcbiIsIGdl dHVpZCgpLCBnZXRldWlkKCkpOwoKCWNoYXIgdGVtcGxhdGVbXSA9ICIvdG1wL2NoZmxhZ3MuWFhY WFhYIjsKCWNoYXIgKnRtcGZpbGU7CgoJdG1wZmlsZSA9IG1rdGVtcCh0ZW1wbGF0ZSk7CgoJaWYg KHRtcGZpbGUgPT0gTlVMTCkKCQllcnIoMSwgIm1rdGVtcCIpOwoKCWlmIChjbG9zZShvcGVuKHRt cGZpbGUsIE9fQ1JFQVQpKSkKCQllcnIoMSwgImNsb3NlKG9wZW4oLi4pKSIpOwoKCWlmIChjaGZs YWdzKHRtcGZpbGUsIFNGX0FSQ0hJVkVEKSAhPSAwKQoJCXdhcm4oImNoZmxhZ3MiKTsKCWlmIChs Y2hmbGFncyh0bXBmaWxlLCBTRl9BUkNISVZFRCkgIT0gMCkKCQl3YXJuKCJsY2hmbGFncyIpOwoK CWlmIChjaGZsYWdzKHRtcGZpbGUsIDApICE9IDApCgkJZXJyKDEsICJjaGZsYWdzKDApIik7Cgll bHNlCgkJdW5saW5rKHRtcGZpbGUpOwoKCXJldHVybiAoMCk7Cn0K --90e6ba181872a98a1d0491f5d630-- From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 17:35:22 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id BF43C106566C; Wed, 6 Oct 2010 17:35:22 +0000 (UTC) Date: Wed, 6 Oct 2010 17:35:22 +0000 From: Alexander Best To: Garrett Cooper Message-ID: <20101006173522.GA92402@freebsd.org> References: <20101005235054.GA45827@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Cc: freebsd-hackers@freebsd.org Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 17:35:22 -0000 On Wed Oct 6 10, Garrett Cooper wrote: > On Tue, Oct 5, 2010 at 4:50 PM, Alexander Best wrote: > > hi there, > > > > i think the following example shows the problem better than a long explanation: > > > > `touch ftest && chflags arch ftest && chflags -vv 0 ftest`. > >  ^^non-root     ^^root                ^^non-root > > > > chflags claims to have cleared the 'arch' flag (which should be impossible as > > non-root user), but indeed has done nothing. > > > > i've tried the same with 'sappnd' and that works as can be expected. > > > > The issue was confirmed to exist in HEAD (me), stable/8 (pgollucc1, jpaetzel) > > and stable/7 (nox). > > On stable/6 it does NOT exist (jpaetzel). chflags properly fails with EPERM. > > Fails for me when I call the syscall directly, as I would expect, > and passes when I'm superuser: > > $ ./test_chflags > (uid, euid) = (1000, 1000) > test_chflags: chflags: Operation not permitted > test_chflags: lchflags: Operation not permitted > $ sudo ./test_chflags > (uid, euid) = (0, 0) > > According to my basic inspection in strtofflags > (.../lib/libc/gen/strtofflags.c), it works as well. > And last but not least, executing the commands directly on the CLI work: > > $ tmpfile=`mktemp /tmp/chflags.XXXXXX` > $ chflags arch $tmpfile > chflags: /tmp/chflags.nQm1IL: Operation not permitted > $ rm $tmpfile > $ tmpfile=`mktemp /tmp/chflags.XXXXXX` > $ sudo chflags arch $tmpfile > $ sudo chflags noarch $tmpfile > $ rm $tmpfile thanks for your test app and helping out with this problem. i'm not sure however you understood the problem. probably i didn't explain it right: $ sudo rm -d /tmp/chflags.XXXXXX $ tmpfile=`mktemp /tmp/chflags.XXXXXX` $ sudo chflags arch $tmpfile $ chflags noarch $tmpfile is what's causing the problem. the last chflags call should fail, but it doesn't. cheers. alex > > Your results may (but shouldn't) vary [unless your environment is > setup differently]... > Please note that I'm using UFS2 with SUJ... not all filesystems > support this (ext2/3/4? msdosfs? ZFS?), so I would be careful about > which filesystem you pick and whether or not there's a bug where it's > not properly identifying that the operation you're attempting to > perform is valid. > Thanks, > -Garrett > > $ uname -a > FreeBSD bayonetta.local 9.0-CURRENT FreeBSD 9.0-CURRENT #9 r211309M: > Thu Aug 19 22:50:36 PDT 2010 > root@bayonetta.local:/usr/obj/usr/src/sys/BAYONETTA amd64 -- a13x From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 17:58:22 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3EAFA106566B; Wed, 6 Oct 2010 17:58:22 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id E8A948FC18; Wed, 6 Oct 2010 17:58:21 +0000 (UTC) Received: by iwn8 with SMTP id 8so1498680iwn.13 for ; Wed, 06 Oct 2010 10:58:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=WMePUYtMdqeTLJxva3zMGA7zReFSFcYt8u44Pjnd/fE=; b=XFaeYU+DX+oMH1krJKNIUaXxaT7N5YMmBV+bebJ9XdSOMDcRYcC51bjksujoD/wP4w Vg1PED4Xqogn6XoJFhPLBP1Jc4J29U93h4aijKXWtUCqPl3pVk87dLZRmgsCm6NfoDpO EqtcPpE1mlK127tFpkGwxW6z4kje5FBn9AGCA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=kRW2RNDAoC4558fvZVgr2ubIxNlP+DBNDZhWdGyKnHzAgb+TWMX9HjUJimD8jz/U67 DWUnArmxCPXcr6d9IAdF8b463VS5gemmmYXNp5yMIOFB0dj3I8Q1vsuFf8KZcrECwRwP Sy9nqyZvRsyPpEaUPYrIwSYD3dVDj+edGNMM4= MIME-Version: 1.0 Received: by 10.231.19.74 with SMTP id z10mr14311557iba.120.1286387901089; Wed, 06 Oct 2010 10:58:21 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.184.3 with HTTP; Wed, 6 Oct 2010 10:58:21 -0700 (PDT) In-Reply-To: <20101006173522.GA92402@freebsd.org> References: <20101005235054.GA45827@freebsd.org> <20101006173522.GA92402@freebsd.org> Date: Wed, 6 Oct 2010 10:58:21 -0700 X-Google-Sender-Auth: 5vGnhKdf4-Y_umTZMHCQ3YCHQXM Message-ID: From: Garrett Cooper To: Alexander Best Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 17:58:22 -0000 On Wed, Oct 6, 2010 at 10:35 AM, Alexander Best wrote= : > On Wed Oct =A06 10, Garrett Cooper wrote: >> On Tue, Oct 5, 2010 at 4:50 PM, Alexander Best wro= te: >> > hi there, >> > >> > i think the following example shows the problem better than a long exp= lanation: >> > >> > `touch ftest && chflags arch ftest && chflags -vv 0 ftest`. >> > =A0^^non-root =A0 =A0 ^^root =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0^^non-root >> > >> > chflags claims to have cleared the 'arch' flag (which should be imposs= ible as >> > non-root user), but indeed has done nothing. >> > >> > i've tried the same with 'sappnd' and that works as can be expected. >> > >> > The issue was confirmed to exist in HEAD (me), stable/8 (pgollucc1, jp= aetzel) >> > and stable/7 (nox). >> > On stable/6 it does NOT exist (jpaetzel). chflags properly fails with = EPERM. >> >> =A0 =A0 Fails for me when I call the syscall directly, as I would expect= , >> and passes when I'm superuser: >> >> $ ./test_chflags >> (uid, euid) =3D (1000, 1000) >> test_chflags: chflags: Operation not permitted >> test_chflags: lchflags: Operation not permitted >> $ sudo ./test_chflags >> (uid, euid) =3D (0, 0) >> >> =A0 =A0 According to my basic inspection in strtofflags >> (.../lib/libc/gen/strtofflags.c), it works as well. >> =A0 =A0 And last but not least, executing the commands directly on the C= LI work: >> >> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >> $ chflags arch $tmpfile >> chflags: /tmp/chflags.nQm1IL: Operation not permitted >> $ rm $tmpfile >> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >> $ sudo chflags arch $tmpfile >> $ sudo chflags noarch $tmpfile >> $ rm $tmpfile > > thanks for your test app and helping out with this problem. i'm not sure > however you understood the problem. probably i didn't explain it right: > > $ sudo rm -d /tmp/chflags.XXXXXX > $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` > $ sudo chflags arch $tmpfile > $ chflags noarch $tmpfile > > is what's causing the problem. the last chflags call should fail, but it > doesn't. Sorry... my CLI based example was stupid. I meant: $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` $ chflags arch $tmpfile chflags: /tmp/chflags.V2NpXR: Operation not permitted $ chflags noarch $tmpfile $ rm $tmpfile Currently chflags(2) states: The SF_IMMUTABLE, SF_APPEND, SF_NOUNLINK, and SF_ARCHIVED flags may on= ly be set or unset by the super-user. Attempts to set these flags by non= - super-users are rejected, >>> attempts by non-superusers to clear flags that are already unset are silently ignored. <<< These flags may be set at= any time, but normally may only be unset when the system is in single-user mode. (See init(8) for details.) So this behavior is already well documented :). The EPERM section should really note SF_ARCHIVED though (whoever added the flag forgot to add that particular item to the ERRORS section). >> =A0 =A0 Your results may (but shouldn't) vary [unless your environment i= s >> setup differently]... >> =A0 =A0 Please note that I'm using UFS2 with SUJ... not all filesystems >> support this (ext2/3/4? msdosfs? ZFS?), so I would be careful about >> which filesystem you pick and whether or not there's a bug where it's >> not properly identifying that the operation you're attempting to >> perform is valid. >> Thanks, >> -Garrett >> >> $ uname -a >> FreeBSD bayonetta.local 9.0-CURRENT FreeBSD 9.0-CURRENT #9 r211309M: >> Thu Aug 19 22:50:36 PDT 2010 >> root@bayonetta.local:/usr/obj/usr/src/sys/BAYONETTA =A0amd64 Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 19:38:27 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 3EF0D1065674; Wed, 6 Oct 2010 19:38:27 +0000 (UTC) Date: Wed, 6 Oct 2010 19:38:27 +0000 From: Alexander Best To: Garrett Cooper Message-ID: <20101006193827.GA13528@freebsd.org> References: <20101005235054.GA45827@freebsd.org> <20101006173522.GA92402@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Cc: freebsd-hackers@freebsd.org Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 19:38:27 -0000 On Wed Oct 6 10, Garrett Cooper wrote: > On Wed, Oct 6, 2010 at 10:35 AM, Alexander Best wrote: > > On Wed Oct  6 10, Garrett Cooper wrote: > >> On Tue, Oct 5, 2010 at 4:50 PM, Alexander Best wrote: > >> > hi there, > >> > > >> > i think the following example shows the problem better than a long explanation: > >> > > >> > `touch ftest && chflags arch ftest && chflags -vv 0 ftest`. > >> >  ^^non-root     ^^root                ^^non-root > >> > > >> > chflags claims to have cleared the 'arch' flag (which should be impossible as > >> > non-root user), but indeed has done nothing. > >> > > >> > i've tried the same with 'sappnd' and that works as can be expected. > >> > > >> > The issue was confirmed to exist in HEAD (me), stable/8 (pgollucc1, jpaetzel) > >> > and stable/7 (nox). > >> > On stable/6 it does NOT exist (jpaetzel). chflags properly fails with EPERM. > >> > >>     Fails for me when I call the syscall directly, as I would expect, > >> and passes when I'm superuser: > >> > >> $ ./test_chflags > >> (uid, euid) = (1000, 1000) > >> test_chflags: chflags: Operation not permitted > >> test_chflags: lchflags: Operation not permitted > >> $ sudo ./test_chflags > >> (uid, euid) = (0, 0) > >> > >>     According to my basic inspection in strtofflags > >> (.../lib/libc/gen/strtofflags.c), it works as well. > >>     And last but not least, executing the commands directly on the CLI work: > >> > >> $ tmpfile=`mktemp /tmp/chflags.XXXXXX` > >> $ chflags arch $tmpfile > >> chflags: /tmp/chflags.nQm1IL: Operation not permitted > >> $ rm $tmpfile > >> $ tmpfile=`mktemp /tmp/chflags.XXXXXX` > >> $ sudo chflags arch $tmpfile > >> $ sudo chflags noarch $tmpfile > >> $ rm $tmpfile > > > > thanks for your test app and helping out with this problem. i'm not sure > > however you understood the problem. probably i didn't explain it right: > > > > $ sudo rm -d /tmp/chflags.XXXXXX > > $ tmpfile=`mktemp /tmp/chflags.XXXXXX` > > $ sudo chflags arch $tmpfile > > $ chflags noarch $tmpfile > > > > is what's causing the problem. the last chflags call should fail, but it > > doesn't. > > Sorry... my CLI based example was stupid. I meant: > > $ tmpfile=`mktemp /tmp/chflags.XXXXXX` > $ chflags arch $tmpfile > chflags: /tmp/chflags.V2NpXR: Operation not permitted > $ chflags noarch $tmpfile > $ rm $tmpfile > > Currently chflags(2) states: > > The SF_IMMUTABLE, SF_APPEND, SF_NOUNLINK, and SF_ARCHIVED flags may only > be set or unset by the super-user. Attempts to set these flags by non- > super-users are rejected, >>> attempts by non-superusers to clear > flags that > are already unset are silently ignored. <<< These flags may be set at any > time, but normally may only be unset when the system is in single-user > mode. (See init(8) for details.) > > So this behavior is already well documented :). The EPERM section > should really note SF_ARCHIVED though (whoever added the flag forgot > to add that particular item to the ERRORS section). that's perfectly alright. clearing an unset flag shouldn't cause any error to be returned. however in my example arch *does* get set and still trying to unset it as normal user doesn't return an error. please make sure to *only* run the first chflags instance as root. all other command must be run as normal user. > > >>     Your results may (but shouldn't) vary [unless your environment is > >> setup differently]... > >>     Please note that I'm using UFS2 with SUJ... not all filesystems > >> support this (ext2/3/4? msdosfs? ZFS?), so I would be careful about > >> which filesystem you pick and whether or not there's a bug where it's > >> not properly identifying that the operation you're attempting to > >> perform is valid. > >> Thanks, > >> -Garrett > >> > >> $ uname -a > >> FreeBSD bayonetta.local 9.0-CURRENT FreeBSD 9.0-CURRENT #9 r211309M: > >> Thu Aug 19 22:50:36 PDT 2010 > >> root@bayonetta.local:/usr/obj/usr/src/sys/BAYONETTA  amd64 > > Thanks, > -Garrett -- a13x From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 21:21:40 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAA6B106564A for ; Wed, 6 Oct 2010 21:21:40 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id B3BC58FC0A for ; Wed, 6 Oct 2010 21:21:40 +0000 (UTC) Received: from [208.206.78.30] (port=35153 helo=dt.vicor.com) by postoffice.vicor.com with esmtpsa (SSLv3:RC4-MD5:128) (Exim 4.71) (envelope-from ) id 1P3arQ-0000dD-AH; Wed, 06 Oct 2010 13:45:15 -0700 From: Devin Teske To: freebsd-hackers@freebsd.org Content-Type: multipart/mixed; boundary="=-AzjrFZJgCNhWf0axqpuC" Organization: Vicor, Inc Date: Wed, 06 Oct 2010 13:45:12 -0700 Message-Id: <1286397912.27308.40.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.0.2 (2.0.2-41.el4) X-Scan-Signature: 9bf7d650e31550ecb77f6085ae8ebd02 X-Scan-Host: postoffice.vicor.com X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Devin Teske Subject: sysconf -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 21:21:41 -0000 --=-AzjrFZJgCNhWf0axqpuC Content-Type: text/plain Content-Transfer-Encoding: 7bit Hello fellow freebsd-hackers, Long-time hacker, first-time poster. I'd like to share a shell script that I wrote for FreeBSD system administration. The attached shell script works similar to sysctl(8), but rather than querying or working on sysctl MIBs, it instead works on the /etc/rc.conf (and /etc/rc.conf.local) files. Syntax shown below (available by running the script without arguments): Usage: sysconf [OPTIONS] name[=value] ... OPTIONS: -h --help Print this message to stderr and exit. -d Print list of internal dependencies before exit. -e Print query results as `var=value' (useful for producing output to be fed back in). Ignored if -n is specified. -n Show only variable values, not their names. ENVIRONMENT: SYSCONF_SHOW_DEPS Dump list of dependencies. Must be zero or one (default: `0') RC_DEFAULTS Location of `/etc/defaults/rc.conf' file. Examples: root@mymachine ~ $ sysconf hostname hostname: mymachine.localdomain root@mymachine ~ $ sysconf hostname=foomaster.deluxe.com hostname: mymachine.localdomain -> foomaster.deluex.com root@mymachine ~ $ sysconf -e hostname hostname="foomaster.deluxe.com" root@mymachine ~ $ sysconf -n hostname foomaster.deluxe.com The script has zero external dependencies. None. Period. It will use grep(1) if installed, but it is not required. Tested extensively on FreeBSD-4.8, FreeBSD-4.11, FreeBSD-8.0, and FreeBSD-8.1. -- Cheers, Devin Teske -> CONTACT INFORMATION <- Business Solutions Consultant II FIS - fisglobal.com 510-735-5650 Mobile 510-621-2038 Office 510-621-2020 Office Fax 909-477-4578 Home/Fax devin.teske@fisglobal.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> FUN STUFF <- -----BEGIN GEEK CODE BLOCK----- Version 3.1 GAT/CS d(+) s: a- C++(++++) UB++++$ P++(++++) L++(++++) !E--- W++ N? o? K- w O M+ V- PS+ PE Y+ PGP- t(+) 5? X+(++) R>++ tv(+) b+(++) DI+(++) D(+) G+>++ e>+ h r>++ y+ ------END GEEK CODE BLOCK------ http://www.geekcode.com/ -> END TRANSMISSION <- --=-AzjrFZJgCNhWf0axqpuC-- From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 21:21:41 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 287F81065675 for ; Wed, 6 Oct 2010 21:21:41 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id E8C528FC15 for ; Wed, 6 Oct 2010 21:21:40 +0000 (UTC) Received: from [208.206.78.30] (port=35164 helo=dt.vicor.com) by postoffice.vicor.com with esmtpsa (SSLv3:RC4-MD5:128) (Exim 4.71) (envelope-from ) id 1P3azP-0000l6-7u; Wed, 06 Oct 2010 13:53:29 -0700 From: Devin Teske To: freebsd-hackers@freebsd.org In-Reply-To: <1286397912.27308.40.camel@localhost.localdomain> References: <1286397912.27308.40.camel@localhost.localdomain> Content-Type: multipart/mixed; boundary="=-KH81D1WZUAaMnfmeqsg6" Organization: Vicor, Inc Date: Wed, 06 Oct 2010 13:53:26 -0700 Message-Id: <1286398406.27308.43.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.0.2 (2.0.2-41.el4) X-Scan-Signature: 7baab9a83caf3e211b5cfdc7b9cc7c15 X-Scan-Host: postoffice.vicor.com X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Devin Teske Subject: Re: sysconf -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 21:21:41 -0000 --=-KH81D1WZUAaMnfmeqsg6 Content-Type: text/plain Content-Transfer-Encoding: 7bit Oops =) This version will use grep(1) if installed. Diff below. dteske@dt.vicor.com Desktop $ diff -pub sysconf.txt{.orig,} --- sysconf.txt.orig 2010-10-06 13:50:29.000000000 -0700 +++ sysconf.txt 2010-10-06 13:50:45.000000000 -0700 @@ -349,8 +349,7 @@ depend sysconf_set 'local' 'sysconf' '[' local found= local regex="^[[:space:]]*$varname=" for file in $conf_files; do - #if have grep; then - if false; then + if have grep; then grep -q "$regex" $file && found=1 else while read LINE; do \ @@ -386,8 +385,7 @@ depend sysconf_set 'local' 'sysconf' '[' ( found= while read -r LINE; do if [ ! "$found" ]; then - #if have grep; then - if false; then + if have grep; then match="$( echo "$LINE" | grep "$regex" )" else case "$LINE" in On Wed, 2010-10-06 at 13:45 -0700, Devin Teske wrote: > Hello fellow freebsd-hackers, > > Long-time hacker, first-time poster. > > I'd like to share a shell script that I wrote for FreeBSD system > administration. > > The attached shell script works similar to sysctl(8), but rather than > querying or working on sysctl MIBs, it instead works on the /etc/rc.conf > (and /etc/rc.conf.local) files. > > Syntax shown below (available by running the script without arguments): > > Usage: sysconf [OPTIONS] name[=value] ... > OPTIONS: > -h --help Print this message to stderr and exit. > -d Print list of internal dependencies before exit. > -e Print query results as `var=value' (useful for producing > output to be fed back in). Ignored if -n is specified. > -n Show only variable values, not their names. > > ENVIRONMENT: > SYSCONF_SHOW_DEPS Dump list of dependencies. Must be zero or one > (default: `0') > RC_DEFAULTS Location of `/etc/defaults/rc.conf' file. > > Examples: > > root@mymachine ~ $ sysconf hostname > hostname: mymachine.localdomain > root@mymachine ~ $ sysconf hostname=foomaster.deluxe.com > hostname: mymachine.localdomain -> foomaster.deluex.com > root@mymachine ~ $ sysconf -e hostname > hostname="foomaster.deluxe.com" > root@mymachine ~ $ sysconf -n hostname > foomaster.deluxe.com > > The script has zero external dependencies. None. Period. It will use > grep(1) if installed, but it is not required. > > Tested extensively on FreeBSD-4.8, FreeBSD-4.11, FreeBSD-8.0, and > FreeBSD-8.1. -- Cheers, Devin Teske -> CONTACT INFORMATION <- Business Solutions Consultant II FIS - fisglobal.com 510-735-5650 Mobile 510-621-2038 Office 510-621-2020 Office Fax 909-477-4578 Home/Fax devin.teske@fisglobal.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> END TRANSMISSION <- --=-KH81D1WZUAaMnfmeqsg6-- From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 22:01:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B81BC106566C for ; Wed, 6 Oct 2010 22:01:36 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 43A358FC08 for ; Wed, 6 Oct 2010 22:01:36 +0000 (UTC) Received: by vws2 with SMTP id 2so52921vws.13 for ; Wed, 06 Oct 2010 15:01:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=k12gVRm1ZC92JH8wH+B2oFAhMd7zyR+gjRrXlbUbo5A=; b=HXCizDSp54NS2mNjW9Mroy9E0K34Q+qUmzv+2ko+qAietKcMzUhYXEppTvsY/qtxCA X8XcnGZgov8GBueclHUNErdIT0NszSdwICmL1Ufw4YNeDWC2+X4JDWmlzvw2fmt81pB/ lGa0Ron+CBSl5vq3b12U/DhN4XV0Uk0jtMIrE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=DKlh4e1sAGek/B0lhbRcPYCiMY/1tATvdFkUxFlolkGJ6jgnYTd6Cg6JWsQUnaWduz puPWNOChRahFG5AptKXJhuH/gfSEMK5wRkZRyCQLrCissnUbfmwlEy2txOdZXyIgQzqk OeMqWzk3nUZWjbW6tMgwTE1ZGnS9mfcGkae2g= MIME-Version: 1.0 Received: by 10.229.95.73 with SMTP id c9mr9279803qcn.111.1286402494618; Wed, 06 Oct 2010 15:01:34 -0700 (PDT) Received: by 10.229.50.8 with HTTP; Wed, 6 Oct 2010 15:01:34 -0700 (PDT) In-Reply-To: <20101006193827.GA13528@freebsd.org> References: <20101005235054.GA45827@freebsd.org> <20101006173522.GA92402@freebsd.org> <20101006193827.GA13528@freebsd.org> Date: Thu, 7 Oct 2010 02:01:34 +0400 Message-ID: From: Sergey Kandaurov To: Alexander Best Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Garrett Cooper Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 22:01:36 -0000 On 6 October 2010 23:38, Alexander Best wrote: > On Wed Oct =A06 10, Garrett Cooper wrote: >> On Wed, Oct 6, 2010 at 10:35 AM, Alexander Best wr= ote: >> > On Wed Oct =A06 10, Garrett Cooper wrote: >> >> On Tue, Oct 5, 2010 at 4:50 PM, Alexander Best = wrote: >> >> > hi there, >> >> > >> >> > i think the following example shows the problem better than a long = explanation: >> >> > >> >> > `touch ftest && chflags arch ftest && chflags -vv 0 ftest`. >> >> > =A0^^non-root =A0 =A0 ^^root =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0^^non-r= oot >> >> > >> >> > chflags claims to have cleared the 'arch' flag (which should be imp= ossible as >> >> > non-root user), but indeed has done nothing. >> >> > >> >> > i've tried the same with 'sappnd' and that works as can be expected= . >> >> > >> >> > The issue was confirmed to exist in HEAD (me), stable/8 (pgollucc1,= jpaetzel) >> >> > and stable/7 (nox). >> >> > On stable/6 it does NOT exist (jpaetzel). chflags properly fails wi= th EPERM. >> >> >> >> =A0 =A0 Fails for me when I call the syscall directly, as I would exp= ect, >> >> and passes when I'm superuser: >> >> >> >> $ ./test_chflags >> >> (uid, euid) =3D (1000, 1000) >> >> test_chflags: chflags: Operation not permitted >> >> test_chflags: lchflags: Operation not permitted >> >> $ sudo ./test_chflags >> >> (uid, euid) =3D (0, 0) >> >> >> >> =A0 =A0 According to my basic inspection in strtofflags >> >> (.../lib/libc/gen/strtofflags.c), it works as well. >> >> =A0 =A0 And last but not least, executing the commands directly on th= e CLI work: >> >> >> >> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >> >> $ chflags arch $tmpfile >> >> chflags: /tmp/chflags.nQm1IL: Operation not permitted >> >> $ rm $tmpfile >> >> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >> >> $ sudo chflags arch $tmpfile >> >> $ sudo chflags noarch $tmpfile >> >> $ rm $tmpfile >> > >> > thanks for your test app and helping out with this problem. i'm not su= re >> > however you understood the problem. probably i didn't explain it right= : >> > >> > $ sudo rm -d /tmp/chflags.XXXXXX >> > $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >> > $ sudo chflags arch $tmpfile >> > $ chflags noarch $tmpfile >> > >> > is what's causing the problem. the last chflags call should fail, but = it >> > doesn't. >> >> Sorry... my CLI based example was stupid. I meant: >> >> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >> $ chflags arch $tmpfile >> chflags: /tmp/chflags.V2NpXR: Operation not permitted >> $ chflags noarch $tmpfile >> $ rm $tmpfile >> >> Currently chflags(2) states: >> >> =A0 =A0 =A0The SF_IMMUTABLE, SF_APPEND, SF_NOUNLINK, and SF_ARCHIVED fla= gs may only >> =A0 =A0 =A0be set or unset by the super-user. =A0Attempts to set these f= lags by non- >> =A0 =A0 =A0super-users are rejected, >>> attempts by non-superusers to c= lear >> flags that >> =A0 =A0 =A0are already unset are silently ignored. <<< =A0These flags ma= y be set at any >> =A0 =A0 =A0time, but normally may only be unset when the system is in si= ngle-user >> =A0 =A0 =A0mode. =A0(See init(8) for details.) >> >> So this behavior is already well documented :). The EPERM section >> should really note SF_ARCHIVED though (whoever added the flag forgot >> to add that particular item to the ERRORS section). > > that's perfectly alright. clearing an unset flag shouldn't cause any erro= r to > be returned. however in my example arch *does* get set and still trying t= o > unset it as normal user doesn't return an error. > It's even more interesting. As far as I could parse the code: - UFS has no special handling for SF_ARCHIVED (I found only it for msdosfs) - ufs_setattr() does not handle unsetting SF_ARCHIVED, so all what it does is simply return zero. - /bin/chflags doesn't check the actual flags value from inode after calling chflags() syscall, and blindly assumes all is well, if chflags() returns with zero, --=20 wbr, pluknet From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 22:12:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24199106566B for ; Wed, 6 Oct 2010 22:12:36 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id DD33D8FC20 for ; Wed, 6 Oct 2010 22:12:35 +0000 (UTC) Received: by iwn8 with SMTP id 8so87522iwn.13 for ; Wed, 06 Oct 2010 15:12:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=yKRAcAUe2yBXlP4nxy79KVeO+D2l/JwFprfWyapA/zU=; b=n0G5KjP5vBBxg4PNbONOkz2uSj8E92yA3Yiu3qExcZYQ+VORVbxq2oI7QN1yHydZ38 fx2giQEZkjWk7n9cF4rNwv14dnjoFZct/XcRnSAbKPQ8g8TcgqCmsMM5lvuO9RCETPAu CpN3wo9zJOnmo4BoXgTYnj1UeCZrvwVdN6rek= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=Sy2DFqD8one/Yjh8bxUci6ov8YumM30auu7v5VU4G8kFfbXAlIf28+2eBj0V2toKhP pXEuekB6vjfiD/3zBJClkC0BC6kVn/BMcLUQe2DubjNqiPyWyzHMds4ABoqdaOuTAfDv Dc8DfhK2m7/U1z2be1NNwizXsVhYbvS/JcEN4= MIME-Version: 1.0 Received: by 10.42.99.3 with SMTP id u3mr1641450icn.124.1286403154995; Wed, 06 Oct 2010 15:12:34 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.184.3 with HTTP; Wed, 6 Oct 2010 15:12:34 -0700 (PDT) In-Reply-To: <1286397912.27308.40.camel@localhost.localdomain> References: <1286397912.27308.40.camel@localhost.localdomain> Date: Wed, 6 Oct 2010 15:12:34 -0700 X-Google-Sender-Auth: vorfeRJ15IzHk4FSdT4ruQnaHu8 Message-ID: From: Garrett Cooper To: Devin Teske Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: sysconf -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 22:12:36 -0000 On Wed, Oct 6, 2010 at 1:45 PM, Devin Teske wrote: > Hello fellow freebsd-hackers, > > Long-time hacker, first-time poster. > > I'd like to share a shell script that I wrote for FreeBSD system > administration. > > The attached shell script works similar to sysctl(8), but rather than > querying or working on sysctl MIBs, it instead works on the /etc/rc.conf > (and /etc/rc.conf.local) files. > > Syntax shown below (available by running the script without arguments): > > Usage: sysconf [OPTIONS] name[=3Dvalue] ... > OPTIONS: > =A0 =A0 =A0 =A0-h --help =A0 Print this message to stderr and exit. > =A0 =A0 =A0 =A0-d =A0 =A0 =A0 =A0 =A0Print list of internal dependencies = before exit. > =A0 =A0 =A0 =A0-e =A0 =A0 =A0 =A0 =A0Print query results as `var=3Dvalue'= (useful for producing > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0output to be fed back in). Ignored= if -n is specified. > =A0 =A0 =A0 =A0-n =A0 =A0 =A0 =A0 =A0Show only variable values, not their= names. > > ENVIRONMENT: > =A0 =A0 =A0 =A0SYSCONF_SHOW_DEPS =A0 =A0 Dump list of dependencies. Must = be zero or one > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(default: `0') > =A0 =A0 =A0 =A0RC_DEFAULTS =A0 =A0 =A0 =A0 =A0 Location of `/etc/defaults= /rc.conf' file. > > Examples: > > root@mymachine ~ $ sysconf hostname > hostname: mymachine.localdomain > root@mymachine ~ $ sysconf hostname=3Dfoomaster.deluxe.com > hostname: mymachine.localdomain -> foomaster.deluex.com > root@mymachine ~ $ sysconf -e hostname > hostname=3D"foomaster.deluxe.com" > root@mymachine ~ $ sysconf -n hostname > foomaster.deluxe.com > > The script has zero external dependencies. None. Period. It will use > grep(1) if installed, but it is not required. > > Tested extensively on FreeBSD-4.8, FreeBSD-4.11, FreeBSD-8.0, and > FreeBSD-8.1. I haven't looked at this yet, but this will result in user confusion as it differs in purpose from sysconf(3) (which is a POSIX API that helps determine system configuration tunable variables)... Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 23:03:01 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EDB42106566C; Wed, 6 Oct 2010 23:03:01 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9D6D38FC13; Wed, 6 Oct 2010 23:03:01 +0000 (UTC) Received: by iwn8 with SMTP id 8so147495iwn.13 for ; Wed, 06 Oct 2010 16:03:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=OT9MocMOFKh746L3QIUTO3o+CEln2CH9q0EDmsarrTY=; b=W93G5NEBatX6mZw28sOJstGk/Xz/si1osh2p6dwP9tzErakF0hQrfvn8ROeZ3f27qF /aGkBLUCEMAGJOGOQPBc4d9a6zZ34IEplygmATqa2AeO1f+aQEAv7gj33O8/Yn3bCy4V 20rVouGqVpiebqhPTHduwY018gb66e+d4c/QE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=EaJYS8Ij2MEWhcK1H4kmU8DPpRb8pkZWBEZNAk4BaNCkLXOv7Pa1YpDoLQnWeq8NEU 4QBtfZsnyX6rhVAkryAtCdDFVnqkTpB46eJPz4BxPfQIb5X3cG2ZYHNjaIdx7rtmpSkF Pg0TWwznZmanu4ayc6euAf9AgcMRCtBvgzQ7Q= MIME-Version: 1.0 Received: by 10.231.149.198 with SMTP id u6mr14770364ibv.7.1286406180844; Wed, 06 Oct 2010 16:03:00 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.184.3 with HTTP; Wed, 6 Oct 2010 16:03:00 -0700 (PDT) In-Reply-To: References: <20101005235054.GA45827@freebsd.org> <20101006173522.GA92402@freebsd.org> <20101006193827.GA13528@freebsd.org> Date: Wed, 6 Oct 2010 16:03:00 -0700 X-Google-Sender-Auth: ZnnEC0SXHaRIg4P3YR2DnT0wyqo Message-ID: From: Garrett Cooper To: Sergey Kandaurov Content-Type: multipart/mixed; boundary=0050450157037235620491fac75b Cc: Alexander Best , freebsd-hackers@freebsd.org Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 23:03:02 -0000 --0050450157037235620491fac75b Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Wed, Oct 6, 2010 at 3:01 PM, Sergey Kandaurov wrote: > On 6 October 2010 23:38, Alexander Best wrote: >> On Wed Oct =A06 10, Garrett Cooper wrote: >>> On Wed, Oct 6, 2010 at 10:35 AM, Alexander Best w= rote: >>> > On Wed Oct =A06 10, Garrett Cooper wrote: >>> >> On Tue, Oct 5, 2010 at 4:50 PM, Alexander Best = wrote: >>> >> > hi there, >>> >> > >>> >> > i think the following example shows the problem better than a long= explanation: >>> >> > >>> >> > `touch ftest && chflags arch ftest && chflags -vv 0 ftest`. >>> >> > =A0^^non-root =A0 =A0 ^^root =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0^^non-= root >>> >> > >>> >> > chflags claims to have cleared the 'arch' flag (which should be im= possible as >>> >> > non-root user), but indeed has done nothing. >>> >> > >>> >> > i've tried the same with 'sappnd' and that works as can be expecte= d. >>> >> > >>> >> > The issue was confirmed to exist in HEAD (me), stable/8 (pgollucc1= , jpaetzel) >>> >> > and stable/7 (nox). >>> >> > On stable/6 it does NOT exist (jpaetzel). chflags properly fails w= ith EPERM. >>> >> >>> >> =A0 =A0 Fails for me when I call the syscall directly, as I would ex= pect, >>> >> and passes when I'm superuser: >>> >> >>> >> $ ./test_chflags >>> >> (uid, euid) =3D (1000, 1000) >>> >> test_chflags: chflags: Operation not permitted >>> >> test_chflags: lchflags: Operation not permitted >>> >> $ sudo ./test_chflags >>> >> (uid, euid) =3D (0, 0) >>> >> >>> >> =A0 =A0 According to my basic inspection in strtofflags >>> >> (.../lib/libc/gen/strtofflags.c), it works as well. >>> >> =A0 =A0 And last but not least, executing the commands directly on t= he CLI work: >>> >> >>> >> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >>> >> $ chflags arch $tmpfile >>> >> chflags: /tmp/chflags.nQm1IL: Operation not permitted >>> >> $ rm $tmpfile >>> >> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >>> >> $ sudo chflags arch $tmpfile >>> >> $ sudo chflags noarch $tmpfile >>> >> $ rm $tmpfile >>> > >>> > thanks for your test app and helping out with this problem. i'm not s= ure >>> > however you understood the problem. probably i didn't explain it righ= t: >>> > >>> > $ sudo rm -d /tmp/chflags.XXXXXX >>> > $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >>> > $ sudo chflags arch $tmpfile >>> > $ chflags noarch $tmpfile >>> > >>> > is what's causing the problem. the last chflags call should fail, but= it >>> > doesn't. >>> >>> Sorry... my CLI based example was stupid. I meant: >>> >>> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >>> $ chflags arch $tmpfile >>> chflags: /tmp/chflags.V2NpXR: Operation not permitted >>> $ chflags noarch $tmpfile >>> $ rm $tmpfile >>> >>> Currently chflags(2) states: >>> >>> =A0 =A0 =A0The SF_IMMUTABLE, SF_APPEND, SF_NOUNLINK, and SF_ARCHIVED fl= ags may only >>> =A0 =A0 =A0be set or unset by the super-user. =A0Attempts to set these = flags by non- >>> =A0 =A0 =A0super-users are rejected, >>> attempts by non-superusers to = clear >>> flags that >>> =A0 =A0 =A0are already unset are silently ignored. <<< =A0These flags m= ay be set at any >>> =A0 =A0 =A0time, but normally may only be unset when the system is in s= ingle-user >>> =A0 =A0 =A0mode. =A0(See init(8) for details.) >>> >>> So this behavior is already well documented :). The EPERM section >>> should really note SF_ARCHIVED though (whoever added the flag forgot >>> to add that particular item to the ERRORS section). >> >> that's perfectly alright. clearing an unset flag shouldn't cause any err= or to >> be returned. however in my example arch *does* get set and still trying = to >> unset it as normal user doesn't return an error. >> > > It's even more interesting. > > As far as I could parse the code: > - UFS has no special handling for SF_ARCHIVED (I found only it for msdosf= s) _very_ interesting: [/sys]$ grep -r SF_ARCHIVED kern/ fs/ ufs/ | grep -v svn fs/msdosfs/msdosfs_vnops.c: vap->va_flags |=3D SF_ARCHIVED; fs/msdosfs/msdosfs_vnops.c: if (vap->va_flags & ~SF_ARCHIVED) fs/msdosfs/msdosfs_vnops.c: if (vap->va_flags & SF_ARCHIVED) The commit that introduced this change probably wasn't doing the right thing: http://svn.freebsd.org/viewvc/base/head/sys/fs/msdosfs/msdosfs= _vnops.c?revision=3D5241&view=3Dmarkup ; cp(1) probably should have been fixed in lieu of `fixing' msdosfs. > - ufs_setattr() does not handle unsetting SF_ARCHIVED, > =A0so all what it does is simply return zero. [EOPNOTSUPP] The underlying file system does not support file flags. So I would expect for invalid flags to return EOPNOTSUPP. ... $ ~/test_chflags_negative test_chflags_negative: should not get here $ sudo ~/test_chflags_negative test_chflags_negative: should not get here *facepalm* I think the problem in part is here (sys/stat.h): * * Super-user and owner changeable flags. */ #define UF_SETTABLE 0x0000ffff /* mask of owner changeable flags *= / #define UF_NODUMP 0x00000001 /* do not dump file */ #define UF_IMMUTABLE 0x00000002 /* file may not be changed */ #define UF_APPEND 0x00000004 /* writes to file may only append *= / #define UF_OPAQUE 0x00000008 /* directory is opaque wrt. union *= / #define UF_NOUNLINK 0x00000010 /* file may not be removed or renam= ed */ /* * Super-user changeable flags. */ #define SF_SETTABLE 0xffff0000 /* mask of superuser changeable fla= gs */ #define SF_ARCHIVED 0x00010000 /* file is archived */ #define SF_IMMUTABLE 0x00020000 /* file may not be changed */ #define SF_APPEND 0x00040000 /* writes to file may only append *= / #define SF_NOUNLINK 0x00100000 /* file may not be removed or renam= ed */ #define SF_SNAPSHOT 0x00200000 /* snapshot inode */ Note the *_SETTABLE macros, and the fact that they allow for more functionality than what's currently slotted with the one-hot encoded flags currently available. SF_ARCHIVED is not present in the other BSDs or Mac OSX either (I did some hunting for a python bug related to chflags a few weeks ago)... and I'm not even sure what this functionality really buys us because it's not well described (but I'd be happy to get an explanation/history lesson). > - /bin/chflags doesn't check the actual flags value from inode after > calling chflags() syscall, and blindly assumes all is well, if chflags() > returns with zero, Yeah... but ideally tests should be written for this stuff and exercised on all filesystems and exercised whenever code in this particular path is changed, because that would potentially turn into a noticeable performance hit [depending on how it's implemented in chflags(1)]. And lo and behold it already does exist under .../tools/regression/fstest/tests/chflags . I'll audit this once I get back home... Thanks, -Garrett --0050450157037235620491fac75b Content-Type: application/octet-stream; name="test_chflags_negative.c" Content-Disposition: attachment; filename="test_chflags_negative.c" Content-Transfer-Encoding: base64 X-Attachment-Id: f_geyss8wh0 I2luY2x1ZGUgPHN5cy9zdGF0Lmg+CiNpbmNsdWRlIDxlcnIuaD4KI2luY2x1ZGUgPHVuaXN0ZC5o PgoKaW50Cm1haW4odm9pZCkKewoJaWYgKGNoZmxhZ3MoIi90bXAiLCAtMSkpIHsKCQl3YXJuKCJP ayIpOwoJCXJldHVybiAoMCk7Cgl9Cgl3YXJueCgic2hvdWxkIG5vdCBnZXQgaGVyZSIpOwoJcmV0 dXJuICgxKTsKfQo= --0050450157037235620491fac75b-- From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 23:09:50 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A0DC1065673 for ; Wed, 6 Oct 2010 23:09:50 +0000 (UTC) (envelope-from jamesbrandongooch@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 006BE8FC1A for ; Wed, 6 Oct 2010 23:09:49 +0000 (UTC) Received: by wyb29 with SMTP id 29so131953wyb.13 for ; Wed, 06 Oct 2010 16:09:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=dkr7TXN90dSDB+PV1e4QThw+XSHZgQrF1/i/9H69NJs=; b=hYPsuAMooT1FNL3ixMd4yzQIHp+MhxmF7G7lMkI6C0pE4szTrxMuGcgch9vBiCtwja uTpZOmXfe13TFNvFNdgzMHyOzqi+mAYIZlPdyFcC/Dq/Zy7X+9CkVrdveBQ8UKZqbkBI bXqpbAUTXiZ67pfKcjaG3aMyuW1I1TW9LPb7Q= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=rTRZ1+qqoueulM0N3PQppgWKcxPu03wcqS394mce8uWedGE+0Q0eR2Rhrr3gIF6SQz GbUm2iD6TCUaC0RXNVlw6cbMW1TEJDXvYRQxLXrYnGw3bPYEaSFRnUZvlgxjjbKp5mk+ IaqB4HZ7W40rrjU9p9JknmCZRIWvYNJ2eISXM= MIME-Version: 1.0 Received: by 10.216.176.8 with SMTP id a8mr1490056wem.93.1286406588985; Wed, 06 Oct 2010 16:09:48 -0700 (PDT) Received: by 10.216.133.133 with HTTP; Wed, 6 Oct 2010 16:09:48 -0700 (PDT) In-Reply-To: <1286397912.27308.40.camel@localhost.localdomain> References: <1286397912.27308.40.camel@localhost.localdomain> Date: Wed, 6 Oct 2010 18:09:48 -0500 Message-ID: From: Brandon Gooch To: Devin Teske Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: sysconf -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 23:09:50 -0000 On Wed, Oct 6, 2010 at 3:45 PM, Devin Teske wrote: > Hello fellow freebsd-hackers, > > Long-time hacker, first-time poster. > > I'd like to share a shell script that I wrote for FreeBSD system > administration. > > The attached shell script works similar to sysctl(8), but rather than > querying or working on sysctl MIBs, it instead works on the /etc/rc.conf > (and /etc/rc.conf.local) files. > > Syntax shown below (available by running the script without arguments): > > Usage: sysconf [OPTIONS] name[=3Dvalue] ... > OPTIONS: > =A0 =A0 =A0 =A0-h --help =A0 Print this message to stderr and exit. > =A0 =A0 =A0 =A0-d =A0 =A0 =A0 =A0 =A0Print list of internal dependencies = before exit. > =A0 =A0 =A0 =A0-e =A0 =A0 =A0 =A0 =A0Print query results as `var=3Dvalue'= (useful for producing > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0output to be fed back in). Ignored= if -n is specified. > =A0 =A0 =A0 =A0-n =A0 =A0 =A0 =A0 =A0Show only variable values, not their= names. > > ENVIRONMENT: > =A0 =A0 =A0 =A0SYSCONF_SHOW_DEPS =A0 =A0 Dump list of dependencies. Must = be zero or one > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(default: `0') > =A0 =A0 =A0 =A0RC_DEFAULTS =A0 =A0 =A0 =A0 =A0 Location of `/etc/defaults= /rc.conf' file. > > Examples: > > root@mymachine ~ $ sysconf hostname > hostname: mymachine.localdomain > root@mymachine ~ $ sysconf hostname=3Dfoomaster.deluxe.com > hostname: mymachine.localdomain -> foomaster.deluex.com > root@mymachine ~ $ sysconf -e hostname > hostname=3D"foomaster.deluxe.com" > root@mymachine ~ $ sysconf -n hostname > foomaster.deluxe.com > > The script has zero external dependencies. None. Period. It will use > grep(1) if installed, but it is not required. > > Tested extensively on FreeBSD-4.8, FreeBSD-4.11, FreeBSD-8.0, and > FreeBSD-8.1. It seems the list ate the attachment :( Also, take a look at sysconf(3). I think there may be a certain amount of hesitation to welcome a utility with the same name as a POSIX-compliant C library function... ...how about a name like 'sysrc' or 'rcconfig' or something indicating a relationship with the rc.conf file or rc system? Anyway, sounds cool, as I've had a need recently to do some automated configuring of several rc.conf files -- my method was very hackish, and will never see the light of public scrutiny :) -Brandon From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 6 23:53:44 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C24C01065670 for ; Wed, 6 Oct 2010 23:53:44 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id ABF038FC16 for ; Wed, 6 Oct 2010 23:53:44 +0000 (UTC) Received: from [208.206.78.30] (port=35510 helo=dt.vicor.com) by postoffice.vicor.com with esmtpsa (SSLv3:RC4-MD5:128) (Exim 4.71) (envelope-from ) id 1P3dng-00037R-1f; Wed, 06 Oct 2010 16:53:36 -0700 From: Devin Teske To: Brandon Gooch In-Reply-To: References: <1286397912.27308.40.camel@localhost.localdomain> Content-Type: text/plain Organization: Vicor, Inc Date: Wed, 06 Oct 2010 16:53:31 -0700 Message-Id: <1286409211.27308.48.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.0.2 (2.0.2-41.el4) Content-Transfer-Encoding: 7bit X-Scan-Signature: 69a93e6f3710b3f43879ac8614e31bd1 X-Scan-Host: postoffice.vicor.com Cc: freebsd-hackers@freebsd.org Subject: Re: sysconf -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Oct 2010 23:53:44 -0000 On Wed, 2010-10-06 at 18:09 -0500, Brandon Gooch wrote: > On Wed, Oct 6, 2010 at 3:45 PM, Devin Teske wrote: > > Hello fellow freebsd-hackers, > > > > Long-time hacker, first-time poster. > > > > I'd like to share a shell script that I wrote for FreeBSD system > > administration. > > > > The attached shell script works similar to sysctl(8), but rather than > > querying or working on sysctl MIBs, it instead works on the /etc/rc.conf > > (and /etc/rc.conf.local) files. > > > > Syntax shown below (available by running the script without arguments): > > > > Usage: sysconf [OPTIONS] name[=value] ... > > OPTIONS: > > -h --help Print this message to stderr and exit. > > -d Print list of internal dependencies before exit. > > -e Print query results as `var=value' (useful for producing > > output to be fed back in). Ignored if -n is specified. > > -n Show only variable values, not their names. > > > > ENVIRONMENT: > > SYSCONF_SHOW_DEPS Dump list of dependencies. Must be zero or one > > (default: `0') > > RC_DEFAULTS Location of `/etc/defaults/rc.conf' file. > > > > Examples: > > > > root@mymachine ~ $ sysconf hostname > > hostname: mymachine.localdomain > > root@mymachine ~ $ sysconf hostname=foomaster.deluxe.com > > hostname: mymachine.localdomain -> foomaster.deluex.com > > root@mymachine ~ $ sysconf -e hostname > > hostname="foomaster.deluxe.com" > > root@mymachine ~ $ sysconf -n hostname > > foomaster.deluxe.com > > > > The script has zero external dependencies. None. Period. It will use > > grep(1) if installed, but it is not required. > > > > Tested extensively on FreeBSD-4.8, FreeBSD-4.11, FreeBSD-8.0, and > > FreeBSD-8.1. > > It seems the list ate the attachment :( I'll post it as the body of the e-mail tomorrow (can't do it right now, just got a call from the wife -- the car broke down and I have to rush out to get her). > Also, take a look at sysconf(3). I think there may be a certain amount > of hesitation to welcome a utility with the same name as a > POSIX-compliant C library function... > > ...how about a name like 'sysrc' or 'rcconfig' or something indicating > a relationship with the rc.conf file or rc system? > > Anyway, sounds cool, as I've had a need recently to do some automated > configuring of several rc.conf files -- my method was very hackish, > and will never see the light of public scrutiny :) Thanks (to yourself and Garrett Cooper) for pointing out sysconf(3). I do agree with the possible confusion. I like the name suggestions. Let me throw out a couple more before sleeping on them tonight. rcctl (like sysctl but for rc stuffs) rcconf (pretty succinct -- too succinct?) sysrc was a great suggestion. > > -Brandon > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Cheers, Devin Teske -> CONTACT INFORMATION <- Business Solutions Consultant II FIS - fisglobal.com 510-735-5650 Mobile 510-621-2038 Office 510-621-2020 Office Fax 909-477-4578 Home/Fax devin.teske@fisglobal.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> FUN STUFF <- -----BEGIN GEEK CODE BLOCK----- Version 3.1 GAT/CS d(+) s: a- C++(++++) UB++++$ P++(++++) L++(++++) !E--- W++ N? o? K- w O M+ V- PS+ PE Y+ PGP- t(+) 5? X+(++) R>++ tv(+) b+(++) DI+(++) D(+) G+>++ e>+ h r>++ y+ ------END GEEK CODE BLOCK------ http://www.geekcode.com/ -> END TRANSMISSION <- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 03:29:29 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C8DE1065672 for ; Thu, 7 Oct 2010 03:29:29 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id 613638FC0A for ; Thu, 7 Oct 2010 03:29:29 +0000 (UTC) Received: from [173.241.24.124] (port=61355 helo=[10.0.0.109]) by postoffice.vicor.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.71) (envelope-from ) id 1P3hAW-0005Fp-Ql; Wed, 06 Oct 2010 20:29:28 -0700 Mime-Version: 1.0 (Apple Message framework v1081) From: Devin Teske In-Reply-To: Date: Wed, 6 Oct 2010 20:29:20 -0700 Message-Id: <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> References: <1286397912.27308.40.camel@localhost.localdomain> To: Brandon Gooch X-Mailer: Apple Mail (2.1081) X-Scan-Signature: 00db300a15ad7123586a66efd5ef50a6 X-Scan-Host: postoffice.vicor.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 03:29:29 -0000 On Oct 6, 2010, at 4:09 PM, Brandon Gooch wrote: > On Wed, Oct 6, 2010 at 3:45 PM, Devin Teske wrote: >> Hello fellow freebsd-hackers, >>=20 >> Long-time hacker, first-time poster. >>=20 >> I'd like to share a shell script that I wrote for FreeBSD system >> administration. >>=20 >=20 > It seems the list ate the attachment :( Here she is ^_^ Comments welcome. #!/bin/sh # -*- tab-width: 4 -*- ;; Emacs # vi: set tabstop=3D4 :: Vi/ViM # # Revision: 1.0 # Last Modified: October 6th, 2010 ############################################################ COPYRIGHT # # (c)2010. Devin Teske. All Rights Reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the = distribution. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR = PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE = LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR = CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE = GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, = STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY = WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # AUTHOR DATE DESCRIPTION # dteske 2010.09.29 Initial version. # ############################################################ INFORMATION # # Command Usage: # # sysrc [OPTIONS] name[=3Dvalue] ... # # OPTIONS: # -h --help Print this message to stderr and exit. # -d Print list of internal dependencies before exit. # -e Print query results as `var=3Dvalue' (useful for = producing # output to be fed back in). Ignored if -n is = specified. # -n Show only variable values, not their names. #=20 # ENVIRONMENT: # SYSRC_SHOW_DEPS Dump list of dependencies. Must be zero or = one # (default: `0') # RC_DEFAULTS Location of `/etc/defaults/rc.conf' file. # # About the Utility: # # This utility works similar to sysctl(8). It shares the `-e' and `-n' # options (detailed above and in sysctl(8)) and also has the same # `name[=3Dvalue]' syntax for querying/setting configuration options. # # Fundamentally it could be said that sysctl(8) munges sysctl MIBs in = the # entrant kernel, while sysrc(8) -- this utility -- munges values in = the # system rc.conf(5) configuration files. # # NOTE: System configuration files are by-default configured in the = file # `/etc/defaults/rc.conf' in the variable `rc_conf_files', which = by-default # contains a space-separated list of pathnames. On all FreeBSD = systems, this # defaults to the value "/etc/rc.conf /etc/rc.conf.local". Each = pathname is # sourced in-order upon startup. It is in the same fashion that this = script # sources the configuration files before returning the value of the = given # variable by-name. # # When you supply this utility with a variable name, it will return = the value # of the variable. If the variable does not appear in any of the = configured # rc_conf_files, an error is printed and we return with error status. # # When changing values of a given variable it does not matter if the = variable # appears in any of the rc_conf_files or not. If the variable does not = appear # in any of the files, it is appended to the end of the first pathname = in the # rc_conf_files variable. Otherwise, the sysrc_set() function will = replace # only the last-occurrence in the last-file found to contain the = variable. # This effectively gets the value to take effect next reboot without = heavily # modifying these integral files (yet taking care not to allow the = file to # grow unweildy should the script be called repeatedly infinitum). # # About the Code: # # I hope that you enjoy the cleanliness, verbosity (but at the same = time, # utilitarian brevity) and -- dare I say -- beauty of the code of this # utility. I have been refining my = cross-platform/POSIX-compliant/baseline- # compatible method of pure-bourne shell scripting for 10 years now. = Shell is # not the only language that I could have chosen to written this = utility in, # but it assuredly was the most satisfying. Satisfying in the sense = that such # a utility is infinitely more useful to the masses if programmed in = the # baseline-compatibility mindset. # # This fundamental utility takes a grass-roots approach to warming its = way # into the hearts and minds of each and every FreeBSD system = administrator, # power-user, and developer. Designed to be a reliable, stable, core = utility # that is depended upon for daily-use without failing in any way. This = shell # script is unique in that it has absolutely no external dependencies. = The # POSIX bourne-shell made available in FreeBSD (/bin/sh) is all that = is # needed. Though, if grep(1) is available in the shell's search path = ($PATH; # search "Path Search" in sh(1)), the script will take advantage of it = for # better efficiency and use of regular expressions rather than glob(7) # patterns. # # This approach makes this utility uniquely adept at functioning in an # embedded environment where very little may be available (for = example, in # the mfsroot ``floppy images'' produced by the release(7) processs; = also # detailed in the ``FreeBSD Release Engineering'' portion of the = online # handbook available at = http://www.freebsd.org/doc/en_US.ISO8859-1/articles\ # /releng/release-build.html). # # If space is an issue for the embedded environment, the first place I = would # start stripping would be comments. There are enough that trimming = all # comments from this file would be enough to drop multiples of the = storage # container's filesystem fragment size (usually 2KB in UFS -- as is = the # format of the mfsroot that FreeBSD boots off of these days). The = second # place I would start trimming would be the depend() function and all = of its # many uses. Next, I would see the whitespace go before finally = attacking # code structure. # # Within, you'll find that the code is very lean. Though, I admit that = there # are a few places that could be enhanced for CPU efficiency. The = lrev() # function is particularly slow as it must use the shell's internal = buffering # to buffer the entire input before finally returning the = line-reversed # output. This leads to significant (100s of milliseconds) delays when # writing out the changes when the "varname=3Dnew_value" syntax is = invoked. # If we give up the stringent requirement of requiring no external = executable # then we can significantly improve the performance of the lrev() = function by # instead relying on awk(1) to reverse the lines of a file (as awk is # infinitely faster than buffering lines of a file into a local shell # variable -- twice). Though, through the power of the have() = function, we # quite simply add the power of awk(1) without relying on its = presence. # # On a liter note, the sysrc() function is perhaps at it's tenth = generation # for myself and has taken on a quintiscential beauty in its awe = inspiring # simplicity and undulating flexibility. I could only dream that it be = one- # day find a home directly in `/etc/rc.subr'. # # Alas, if you've read this far, you might be the type to wonder what = those # two lines at the top (after the shell interpreter invocation-line) = do: # # # -*- tab-width: 4 -*- ;; Emacs # # vi: set tabstop=3D4 :: Vi/ViM # # The first achieves the amazing feat of getting Emacs to always use = real- # tab characters while maintaining the visual display of = 4-character-wide # indentation. Seasoned Emacs users are blissfully unaware that as = they # modify the file, they are using real tabs versus = 4-spaces-then-tab-then-4- # spaces-then-tab-... # # The second line gets vim (and later versions of vi -- but not nvi = which # is used for vi(1) in FreeBSD) to render real tabs as = 4-characters-wide. # # The two lines together make it so that the file can be passed = back-and- # forth between different developers using Emacs, Vim and vi. Real = tabs are # preserved in all editors and developers are encouraged to use real = tabs. # # Vim/vi users are then given the flexibility of redefining at-will = the tab # width by setting the "ts" internal variable, though Emacs users are # discouraged from changing the tab-width since any value other than 4 = will # cause mixtures of tab-and-space to entered into the file during = auto-indent # events. # # Last, my personal coding technique is to code to 80-character-wide = terminal # widths (while I have tab-width set to 8 in vim -- via ``:set = ts=3D8''). # That's just personal preference (because I find resizing a terminal = with # the mouse to be a heinously-inefficient motion that takes up too = much of # the time that I could be typing into my shell, coding, or navigating = with # the keyboard). # # About Copying (the License): # # Those that are familiar with the license (displayed above in the = COPYING # section) will recognize the Berkeley Open License. It is only = slightly more # restrictive than the in/famous Beer-ware license. You must adhere to = the # two restrictions above which are simple enough. Display the = copyright in # any facsimile representations, modified or un-modified. # # http://www.plainlanguage.gov/ # ############################################################ = CONFIGURATION # # Default setting whether to dump a list of internal dependencies upon = exit # : ${SYSRC_SHOW_DEPS:=3D0} ############################################################ GLOBALS # Global exit status variables : ${SUCCESS:=3D0} : ${FAILURE:=3D1} # # Program name # progname=3D"${0##*/}" # # Options # SHOW_EQUALS=3D SHOW_NAME=3D1 # Reserved for internal use _depend=3D ############################################################ FUNCTION # fprintf $fd $fmt [ $opts ... ] # # Like printf, except allows you to print to a specific file-descriptor. = Useful # for printing to stderr (fd=3D2) or some other known file-descriptor. # : dependency checks performed after depend-function declaration : function ; fprintf ( ) # $fd $fmt [ $opts ... ] { local fd=3D$1 [ $# -gt 1 ] || return ${FAILURE-1} shift 1 printf "$@" >&$fd } # eprintf $fmt [ $opts ... ] # # Print a message to stderr (fd=3D2). # : dependency checks performed after depend-function declaration : function ; eprintf ( ) # $fmt [ $opts ... ] { fprintf 2 "$@" } # show_deps # # Print the current list of dependencies. # : dependency checks performed after depend-function declaration : function ; show_deps ( ) # { if [ "$SYSRC_SHOW_DEPS" =3D "1" ]; then eprintf "Running internal dependency list:\n" local d for d in $_depend; do eprintf "\t%-15ss%s\n" "$d" "$( type "$d" )" done fi } # die [ $err_msg ... ] # # Optionally print a message to stderr before exiting with failure = status. # : dependency checks performed after depend-function declaration : function ; die ( ) # [ $err_msg ... ] { local fmt=3D"$1" [ $# -gt 0 ] && shift 1 [ "$fmt" ] && eprintf "$fmt\n" "$@" show_deps exit ${FAILURE-1} } # have $anything # # Used for dependency calculations. Arguments are passed to the `type' = built-in # to determine if a given command is available (either as a shell = built-in or # as an external binary). The return status is true if the given = argument is # for an existing built-in or executable, otherwise false. # # This is a convenient method for building dependency lists and it aids = in the # readability of a script. For example, # # Example 1: have sed || die "sed is missing" # Example 2: if have awk; then # # We have awk... # else # # We DON'T have awk... # fi # Example 3: have reboot && reboot # : dependency checks performed after depend-function declaration : function ; have ( ) # $anything { type "$@" > /dev/null 2>&1 } # depend $name [ $dependency ... ] # # Add a dependency. Die with error if dependency is not met. # : dependency checks performed after depend-function declaration : function ; depend ( ) # $name [ $dependency ... ] { local by=3D"$1" arg shift 1 for arg in "$@"; do local d for d in $_depend ""; do [ "$d" =3D "$arg" ] && break done if [ ! "$d" ]; then have "$arg" || die \ "%s: Missing dependency '%s' required by = %s" \ "${progname:-$0}" "$arg" "$by" _depend=3D"$_depend${_depend:+ }$arg" fi done } # # Perform dependency calculations for above rudimentary functions. # NOTE: Beyond this point, use the depend-function BEFORE dependency-use # depend fprintf 'local' '[' 'return' 'shift' 'printf' depend eprintf 'fprintf' depend show_deps 'if' '[' 'then' 'eprintf' 'local' 'for' 'do' 'done' = 'fi' depend die 'local' '[' 'shift' 'eprintf' 'show_deps' 'exit' depend have 'local' 'type' 'return' depend depend 'local' 'shift' 'for' 'do' '[' 'break' 'done' 'if' = 'then' \ 'have' 'die' 'fi' # usage # # Prints a short syntax statement and exits. # depend usage 'local' 'eprintf' 'die' : function ; usage ( ) # { local optfmt=3D"\t%-12s%s\n" local envfmt=3D"\t%-22s%s\n" eprintf "Usage: %s [OPTIONS] name[=3Dvalue] ...\n" = "${progname:-$0}" eprintf "OPTIONS:\n" eprintf "$optfmt" "-h --help" \ "Print this message to stderr and exit." eprintf "$optfmt" "-d" \ "Print list of internal dependencies before exit." eprintf "$optfmt" "-e" \ "Print query results as \`var=3Dvalue' (useful for = producing" eprintf "$optfmt" "" \ "output to be fed back in). Ignored if -n is specified." eprintf "$optfmt" "-n" \ "Show only variable values, not their names." eprintf "\n" eprintf "ENVIRONMENT:\n" eprintf "$envfmt" "SYSRC_SHOW_DEPS" \ "Dump list of dependencies. Must be zero or one" eprintf "$envfmt" "" \ "(default: \`0')" eprintf "$envfmt" "RC_DEFAULTS" \ "Location of \`/etc/defaults/rc.conf' file." die } # sysrc $setting # # Get a system configuration setting from the collection of system- # configuration files (in order: /etc/defaults/rc.conf /etc/rc.conf # and /etc/rc.conf). # # Examples: # # sysrc sshd_enable # returns YES or NO # sysrc defaultrouter # returns IP address of default router (if configured) # sysrc 'hostname%%.*' # returns $hostname up to (but not including) first `.' # sysrc 'network_interfaces%%[$IFS]*' # returns first word of $network_interfaces # sysrc 'ntpdate_flags##*[$IFS]' # returns last word of $ntpdate_flags (time server = address) # sysrc usbd_flags-"default" # returns $usbd_flags or "default" if unset # sysrc usbd_flags:-"default" # returns $usbd_flags or "default" if unset or NULL # sysrc cloned_interfaces+"alternate" # returns "alternate" if $cloned_interfaces is set # sysrc cloned_interfaces:+"alternate" # returns "alternate" if $cloned_interfaces is set and = non-NULL # sysrc '#kern_securelevel' # returns length in characters of $kern_securelevel # sysrc 'hostname?' # returns NULL and error status 2 if $hostname is unset = (or if # set, returns the value of $hostname with no error = status) # sysrc 'hostname:?' # returns NULL and error status 2 if $hostname is unset or = NULL # (or if set and non-NULL, returns value without error = status) # depend sysrc 'local' '[' 'return' '.' 'have' 'eval' 'echo' : function ; sysrc ( ) # $varname { : ${RC_DEFAULTS:=3D"/etc/defaults/rc.conf"} local defaults=3D"$RC_DEFAULTS" local varname=3D"$1" # Check arguments [ -r "$defaults" ] || return [ "$varname" ] || return ( # Execute within sub-shell to protect parent environment [ -f "$defaults" -a -r "$defaults" ] && . "$defaults" have source_rc_confs && source_rc_confs eval echo '"${'"$varname"'}"' 2> /dev/null ) } # ... | lrev # lrev $file ... # # Reverse lines of input. Unlike rev(1) which reverses the ordering of # characters on a single line, this function instead reverses the line # sequencing. # # For example, the following input: # # Line 1 # Line 2 # Line 3 # # Becomes reversed in the following manner: # # Line 3 # Line 2 # Line 1 # depend lrev 'local' 'if' '[' 'then' 'while' 'do' 'shift' 'done' 'else' = 'read' \ 'fi' 'echo' : function ; lrev ( ) # $file ... { local stdin_rev=3D if [ $# -gt 0 ]; then # # Reverse lines from files passed as positional = arguments. # while [ $# -gt 0 ]; do local file=3D"$1" [ -f "$file" ] && lrev < "$file" shift 1 done else # # Reverse lines from standard input # while read -r LINE; do stdin_rev=3D"$LINE $stdin_rev" done fi echo -n "$stdin_rev" } # sysrc_set $setting $new_value # # Change a setting in the system configuration files (edits the files = in-place # to change the value in the last assignment to the variable). If the = variable # does not appear in the source file, it is appended to the end of the = primary # system configuration file `/etc/rc.conf'. # depend sysrc_set 'local' 'sysrc' '[' 'return' 'for' 'do' 'done' 'if' = 'have' \ 'then' 'else' 'while' 'read' 'case' 'esac' 'fi' 'break' = \ 'eprintf' 'echo' 'lrev' : function ; sysrc_set ( ) # $varname $new_value { local rc_conf_files=3D"$( sysrc rc_conf_files )" local varname=3D"$1" new_value=3D"$2" local file conf_files=3D # Check arguments [ "$rc_conf_files" ] || return ${FAILURE-1} [ "$varname" ] || return ${FAILURE-1} # Reverse the order of files in rc_conf_files for file in $rc_conf_files; do conf_files=3D"$file${conf_files:+ }$conf_files" done # # Determine which file we are to operate on. If no files match, = we'll # simply append to the last file in the list (`/etc/rc.conf'). # local found=3D local regex=3D"^[[:space:]]*$varname=3D" for file in $conf_files; do #if have grep; then if false; then grep -q "$regex" $file && found=3D1 else while read LINE; do \ case "$LINE" in \ $varname=3D*) found=3D1;; \ esac; \ done < $file fi [ "$found" ] && break done # # Perform sanity checks. # if [ ! -w $file ]; then eprintf "\n%s: cannot create %s: permission denied\n" \ "${progname:-$0}" "$file" return ${FAILURE-1} fi # # If not found, append new value to last file and return. # if [ ! "$found" ]; then echo "$varname=3D\"$new_value\"" >> $file return ${SUCCESS-0} fi # # Operate on the matching file, replacing only the last = occurrence. # local new_contents=3D"`lrev $file 2> /dev/null | \ ( found=3D while read -r LINE; do if [ ! "$found" ]; then #if have grep; then if false; then match=3D"$( echo "$LINE" | grep "$regex" = )" else case "$LINE" in $varname=3D*) match=3D1;; *) match=3D;; esac fi if [ "$match" ]; then LINE=3D"$varname"'=3D"'"$new_value"'"' found=3D1 fi fi echo "$LINE" done ) | lrev`" [ "$new_contents" ] \ && echo "$new_contents" > $file } ############################################################ MAIN SOURCE # # Perform sanity checks # depend main '[' 'usage' [ $# -gt 0 ] || usage # # Process command-line options # depend main 'while' '[' 'do' 'case' 'usage' 'eprintf' \ 'break' 'esac' 'shift' 'done' while [ $# -gt 0 ]; do case "$1" in -h|--help) usage;; -d) SYSRC_SHOW_DEPS=3D1;; -e) SHOW_EQUALS=3D1;; -n) SHOW_NAME=3D;; -*) eprintf "%s: unrecognized option \`$1'\n" "${progname:-$0}" usage;; *) # Since it is impossible (in many shells, including bourne, = c, # tennex-c, and bourne-again) to name a variable beginning = with a # dash/hyphen [-], we will terminate the option-list at the = first # item that doesn't begin with a dash. break;; esac shift 1 done [ "$SHOW_NAME" ] || SHOW_EQUALS=3D # # Process command-line arguments # depend main '[' 'while' 'do' 'case' 'echo' 'sysrc' 'if' 'sysrc_set' = 'then' \ 'fi' 'esac' 'shift' 'done' SEP=3D': ' [ "$SHOW_EQUALS" ] && SEP=3D'=3D"' while [ $# -gt 0 ]; do NAME=3D"${1%%=3D*}" case "$1" in *=3D*) echo -n "${SHOW_NAME:+$NAME$SEP}$( sysrc "$1" )${SHOW_EQUALS:+\"}" if sysrc_set "$NAME" "${1#*=3D}"; then echo " -> $( sysrc "$NAME" )" fi ;; *) if ! IGNORED=3D"$( sysrc "$NAME?" )"; then echo "${progname:-$0}: unknown variable '$NAME'" else echo "${SHOW_NAME:+$NAME$SEP}$( sysrc "$1" )${SHOW_EQUALS:+\"}" fi esac shift 1 done # # Ending routines # depend main 'show_deps' show_deps = ##########################################################################= ###### # END = ##########################################################################= ###### # # $Header$ # # $Copyright: 1999-2010. Vicor Inc. All Rights Reserved. $ # # $Log$ # = ##########################################################################= ###### -- Cheers, Devin Teske -> CONTACT INFORMATION <- Business Solutions Consultant II FIS - fisglobal.com 510-735-5650 Mobile 510-621-2038 Office 510-621-2020 Office Fax 909-477-4578 Home/Fax devin.teske@fisglobal.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> FUN STUFF <- -----BEGIN GEEK CODE BLOCK----- Version 3.1 GAT/CS d(+) s: a- C++(++++) UB++++$ P++(++++) L++(++++) !E--- W++ N? o? = K- w O M+ V- PS+ PE Y+ PGP- t(+) 5? X+(++) R>++ tv(+) b+(++) DI+(++) D(+) G+>++ = e>+ h r>++ y+=20 ------END GEEK CODE BLOCK------ http://www.geekcode.com/ -> END TRANSMISSION <- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 03:50:23 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2C80E1065670 for ; Thu, 7 Oct 2010 03:50:23 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id 100CA8FC18 for ; Thu, 7 Oct 2010 03:50:22 +0000 (UTC) Received: from [173.241.24.124] (port=61411 helo=[10.0.0.109]) by postoffice.vicor.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.71) (envelope-from ) id 1P3hUn-0005S0-0v; Wed, 06 Oct 2010 20:50:22 -0700 Mime-Version: 1.0 (Apple Message framework v1081) From: Devin Teske In-Reply-To: <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> Date: Wed, 6 Oct 2010 20:50:16 -0700 Message-Id: References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> To: Devin Teske X-Mailer: Apple Mail (2.1081) X-Scan-Signature: f2a272f89cf8a714e6f8b25f736cd8fd X-Scan-Host: postoffice.vicor.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Brandon Gooch , freebsd-hackers@freebsd.org Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 03:50:23 -0000 On Oct 6, 2010, at 8:29 PM, Devin Teske wrote: >=20 > On Oct 6, 2010, at 4:09 PM, Brandon Gooch wrote: >=20 >> On Wed, Oct 6, 2010 at 3:45 PM, Devin Teske wrote: >>> Hello fellow freebsd-hackers, >>>=20 >>> Long-time hacker, first-time poster. >>>=20 >>> I'd like to share a shell script that I wrote for FreeBSD system >>> administration. >>>=20 >>=20 >> It seems the list ate the attachment :( >=20 >=20 > Here she is ^_^ Comments welcome. >=20 Again with the patch needed... :( Was testing embedded environment sans-grep(1). dteske@dt.vicor.com bin $ diff -pu sysrc{.orig,} --- sysrc.orig 2010-10-06 20:47:55.000000000 -0700 +++ sysrc 2010-10-06 20:48:06.000000000 -0700 @@ -492,8 +492,7 @@ depend sysrc_set 'local' 'sysrc' '[' 're local found=3D local regex=3D"^[[:space:]]*$varname=3D" for file in $conf_files; do - #if have grep; then - if false; then + if have grep; then grep -q "$regex" $file && found=3D1 else while read LINE; do \ @@ -529,8 +528,7 @@ depend sysrc_set 'local' 'sysrc' '[' 're ( found=3D while read -r LINE; do if [ ! "$found" ]; then - #if have grep; then - if false; then + if have grep; then match=3D"$( echo "$LINE" | grep "$regex" = )" else case "$LINE" in > #!/bin/sh > # -*- tab-width: 4 -*- ;; Emacs > # vi: set tabstop=3D4 :: Vi/ViM > # > # Revision: 1.0 > # Last Modified: October 6th, 2010 > ############################################################ COPYRIGHT > # > # (c)2010. Devin Teske. All Rights Reserved. > # > # Redistribution and use in source and binary forms, with or without > # modification, are permitted provided that the following conditions > # are met: > # 1. Redistributions of source code must retain the above copyright > # notice, this list of conditions and the following disclaimer. > # 2. Redistributions in binary form must reproduce the above copyright > # notice, this list of conditions and the following disclaimer in = the > # documentation and/or other materials provided with the = distribution. > # > # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' = AND > # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING BUT NOT LIMITED TO, THE > # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR = PURPOSE > # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE = LIABLE > # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR = CONSEQUENTIAL > # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE = GOODS > # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS = INTERRUPTION) > # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, = STRICT > # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN = ANY WAY > # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY = OF > # SUCH DAMAGE. > # > # AUTHOR DATE DESCRIPTION > # dteske 2010.09.29 Initial version. > # > ############################################################ = INFORMATION > # > # Command Usage: > # > # sysrc [OPTIONS] name[=3Dvalue] ... > # > # OPTIONS: > # -h --help Print this message to stderr and exit. > # -d Print list of internal dependencies before exit. > # -e Print query results as `var=3Dvalue' (useful for = producing > # output to be fed back in). Ignored if -n is = specified. > # -n Show only variable values, not their names. > #=20 > # ENVIRONMENT: > # SYSRC_SHOW_DEPS Dump list of dependencies. Must be zero or = one > # (default: `0') > # RC_DEFAULTS Location of `/etc/defaults/rc.conf' file. > # > # About the Utility: > # > # This utility works similar to sysctl(8). It shares the `-e' and = `-n' > # options (detailed above and in sysctl(8)) and also has the same > # `name[=3Dvalue]' syntax for querying/setting configuration = options. > # > # Fundamentally it could be said that sysctl(8) munges sysctl MIBs = in the > # entrant kernel, while sysrc(8) -- this utility -- munges values in = the > # system rc.conf(5) configuration files. > # > # NOTE: System configuration files are by-default configured in the = file > # `/etc/defaults/rc.conf' in the variable `rc_conf_files', which = by-default > # contains a space-separated list of pathnames. On all FreeBSD = systems, this > # defaults to the value "/etc/rc.conf /etc/rc.conf.local". Each = pathname is > # sourced in-order upon startup. It is in the same fashion that this = script > # sources the configuration files before returning the value of the = given > # variable by-name. > # > # When you supply this utility with a variable name, it will return = the value > # of the variable. If the variable does not appear in any of the = configured > # rc_conf_files, an error is printed and we return with error = status. > # > # When changing values of a given variable it does not matter if the = variable > # appears in any of the rc_conf_files or not. If the variable does = not appear > # in any of the files, it is appended to the end of the first = pathname in the > # rc_conf_files variable. Otherwise, the sysrc_set() function will = replace > # only the last-occurrence in the last-file found to contain the = variable. > # This effectively gets the value to take effect next reboot without = heavily > # modifying these integral files (yet taking care not to allow the = file to > # grow unweildy should the script be called repeatedly infinitum). > # > # About the Code: > # > # I hope that you enjoy the cleanliness, verbosity (but at the same = time, > # utilitarian brevity) and -- dare I say -- beauty of the code of = this > # utility. I have been refining my = cross-platform/POSIX-compliant/baseline- > # compatible method of pure-bourne shell scripting for 10 years now. = Shell is > # not the only language that I could have chosen to written this = utility in, > # but it assuredly was the most satisfying. Satisfying in the sense = that such > # a utility is infinitely more useful to the masses if programmed in = the > # baseline-compatibility mindset. > # > # This fundamental utility takes a grass-roots approach to warming = its way > # into the hearts and minds of each and every FreeBSD system = administrator, > # power-user, and developer. Designed to be a reliable, stable, core = utility > # that is depended upon for daily-use without failing in any way. = This shell > # script is unique in that it has absolutely no external = dependencies. The > # POSIX bourne-shell made available in FreeBSD (/bin/sh) is all that = is > # needed. Though, if grep(1) is available in the shell's search path = ($PATH; > # search "Path Search" in sh(1)), the script will take advantage of = it for > # better efficiency and use of regular expressions rather than = glob(7) > # patterns. > # > # This approach makes this utility uniquely adept at functioning in = an > # embedded environment where very little may be available (for = example, in > # the mfsroot ``floppy images'' produced by the release(7) processs; = also > # detailed in the ``FreeBSD Release Engineering'' portion of the = online > # handbook available at = http://www.freebsd.org/doc/en_US.ISO8859-1/articles\ > # /releng/release-build.html). > # > # If space is an issue for the embedded environment, the first place = I would > # start stripping would be comments. There are enough that trimming = all > # comments from this file would be enough to drop multiples of the = storage > # container's filesystem fragment size (usually 2KB in UFS -- as is = the > # format of the mfsroot that FreeBSD boots off of these days). The = second > # place I would start trimming would be the depend() function and = all of its > # many uses. Next, I would see the whitespace go before finally = attacking > # code structure. > # > # Within, you'll find that the code is very lean. Though, I admit = that there > # are a few places that could be enhanced for CPU efficiency. The = lrev() > # function is particularly slow as it must use the shell's internal = buffering > # to buffer the entire input before finally returning the = line-reversed > # output. This leads to significant (100s of milliseconds) delays = when > # writing out the changes when the "varname=3Dnew_value" syntax is = invoked. > # If we give up the stringent requirement of requiring no external = executable > # then we can significantly improve the performance of the lrev() = function by > # instead relying on awk(1) to reverse the lines of a file (as awk = is > # infinitely faster than buffering lines of a file into a local = shell > # variable -- twice). Though, through the power of the have() = function, we > # quite simply add the power of awk(1) without relying on its = presence. > # > # On a liter note, the sysrc() function is perhaps at it's tenth = generation > # for myself and has taken on a quintiscential beauty in its awe = inspiring > # simplicity and undulating flexibility. I could only dream that it = be one- > # day find a home directly in `/etc/rc.subr'. > # > # Alas, if you've read this far, you might be the type to wonder = what those > # two lines at the top (after the shell interpreter invocation-line) = do: > # > # # -*- tab-width: 4 -*- ;; Emacs > # # vi: set tabstop=3D4 :: Vi/ViM > # > # The first achieves the amazing feat of getting Emacs to always use = real- > # tab characters while maintaining the visual display of = 4-character-wide > # indentation. Seasoned Emacs users are blissfully unaware that as = they > # modify the file, they are using real tabs versus = 4-spaces-then-tab-then-4- > # spaces-then-tab-... > # > # The second line gets vim (and later versions of vi -- but not nvi = which > # is used for vi(1) in FreeBSD) to render real tabs as = 4-characters-wide. > # > # The two lines together make it so that the file can be passed = back-and- > # forth between different developers using Emacs, Vim and vi. Real = tabs are > # preserved in all editors and developers are encouraged to use real = tabs. > # > # Vim/vi users are then given the flexibility of redefining at-will = the tab > # width by setting the "ts" internal variable, though Emacs users = are > # discouraged from changing the tab-width since any value other than = 4 will > # cause mixtures of tab-and-space to entered into the file during = auto-indent > # events. > # > # Last, my personal coding technique is to code to 80-character-wide = terminal > # widths (while I have tab-width set to 8 in vim -- via ``:set = ts=3D8''). > # That's just personal preference (because I find resizing a = terminal with > # the mouse to be a heinously-inefficient motion that takes up too = much of > # the time that I could be typing into my shell, coding, or = navigating with > # the keyboard). > # > # About Copying (the License): > # > # Those that are familiar with the license (displayed above in the = COPYING > # section) will recognize the Berkeley Open License. It is only = slightly more > # restrictive than the in/famous Beer-ware license. You must adhere = to the > # two restrictions above which are simple enough. Display the = copyright in > # any facsimile representations, modified or un-modified. > # > # http://www.plainlanguage.gov/ > # > ############################################################ = CONFIGURATION >=20 > # > # Default setting whether to dump a list of internal dependencies upon = exit > # > : ${SYSRC_SHOW_DEPS:=3D0} >=20 > ############################################################ GLOBALS >=20 > # Global exit status variables > : ${SUCCESS:=3D0} > : ${FAILURE:=3D1} >=20 > # > # Program name > # > progname=3D"${0##*/}" >=20 > # > # Options > # > SHOW_EQUALS=3D > SHOW_NAME=3D1 >=20 > # Reserved for internal use > _depend=3D >=20 > ############################################################ FUNCTION >=20 > # fprintf $fd $fmt [ $opts ... ] > # > # Like printf, except allows you to print to a specific = file-descriptor. Useful > # for printing to stderr (fd=3D2) or some other known file-descriptor. > # > : dependency checks performed after depend-function declaration > : function ; fprintf ( ) # $fd $fmt [ $opts ... ] > { > local fd=3D$1 > [ $# -gt 1 ] || return ${FAILURE-1} > shift 1 > printf "$@" >&$fd > } >=20 > # eprintf $fmt [ $opts ... ] > # > # Print a message to stderr (fd=3D2). > # > : dependency checks performed after depend-function declaration > : function ; eprintf ( ) # $fmt [ $opts ... ] > { > fprintf 2 "$@" > } >=20 > # show_deps > # > # Print the current list of dependencies. > # > : dependency checks performed after depend-function declaration > : function ; show_deps ( ) # > { > if [ "$SYSRC_SHOW_DEPS" =3D "1" ]; then > eprintf "Running internal dependency list:\n" >=20 > local d > for d in $_depend; do > eprintf "\t%-15ss%s\n" "$d" "$( type "$d" )" > done > fi > } >=20 > # die [ $err_msg ... ] > # > # Optionally print a message to stderr before exiting with failure = status. > # > : dependency checks performed after depend-function declaration > : function ; die ( ) # [ $err_msg ... ] > { > local fmt=3D"$1" > [ $# -gt 0 ] && shift 1 > [ "$fmt" ] && eprintf "$fmt\n" "$@" >=20 > show_deps > exit ${FAILURE-1} > } >=20 > # have $anything > # > # Used for dependency calculations. Arguments are passed to the `type' = built-in > # to determine if a given command is available (either as a shell = built-in or > # as an external binary). The return status is true if the given = argument is > # for an existing built-in or executable, otherwise false. > # > # This is a convenient method for building dependency lists and it = aids in the > # readability of a script. For example, > # > # Example 1: have sed || die "sed is missing" > # Example 2: if have awk; then > # # We have awk... > # else > # # We DON'T have awk... > # fi > # Example 3: have reboot && reboot > # > : dependency checks performed after depend-function declaration > : function ; have ( ) # $anything > { > type "$@" > /dev/null 2>&1 > } >=20 > # depend $name [ $dependency ... ] > # > # Add a dependency. Die with error if dependency is not met. > # > : dependency checks performed after depend-function declaration > : function ; depend ( ) # $name [ $dependency ... ] > { > local by=3D"$1" arg > shift 1 >=20 > for arg in "$@"; do > local d > for d in $_depend ""; do > [ "$d" =3D "$arg" ] && break > done > if [ ! "$d" ]; then > have "$arg" || die \ > "%s: Missing dependency '%s' required by = %s" \ > "${progname:-$0}" "$arg" "$by" > _depend=3D"$_depend${_depend:+ }$arg" > fi > done > } >=20 > # > # Perform dependency calculations for above rudimentary functions. > # NOTE: Beyond this point, use the depend-function BEFORE = dependency-use > # > depend fprintf 'local' '[' 'return' 'shift' 'printf' > depend eprintf 'fprintf' > depend show_deps 'if' '[' 'then' 'eprintf' 'local' 'for' 'do' 'done' = 'fi' > depend die 'local' '[' 'shift' 'eprintf' 'show_deps' 'exit' > depend have 'local' 'type' 'return' > depend depend 'local' 'shift' 'for' 'do' '[' 'break' 'done' 'if' = 'then' \ > 'have' 'die' 'fi' >=20 > # usage > # > # Prints a short syntax statement and exits. > # > depend usage 'local' 'eprintf' 'die' > : function ; usage ( ) # > { > local optfmt=3D"\t%-12s%s\n" > local envfmt=3D"\t%-22s%s\n" >=20 > eprintf "Usage: %s [OPTIONS] name[=3Dvalue] ...\n" = "${progname:-$0}" >=20 > eprintf "OPTIONS:\n" > eprintf "$optfmt" "-h --help" \ > "Print this message to stderr and exit." > eprintf "$optfmt" "-d" \ > "Print list of internal dependencies before exit." > eprintf "$optfmt" "-e" \ > "Print query results as \`var=3Dvalue' (useful for = producing" > eprintf "$optfmt" "" \ > "output to be fed back in). Ignored if -n is specified." > eprintf "$optfmt" "-n" \ > "Show only variable values, not their names." > eprintf "\n" >=20 > eprintf "ENVIRONMENT:\n" > eprintf "$envfmt" "SYSRC_SHOW_DEPS" \ > "Dump list of dependencies. Must be zero or one" > eprintf "$envfmt" "" \ > "(default: \`0')" > eprintf "$envfmt" "RC_DEFAULTS" \ > "Location of \`/etc/defaults/rc.conf' file." >=20 > die > } >=20 > # sysrc $setting > # > # Get a system configuration setting from the collection of system- > # configuration files (in order: /etc/defaults/rc.conf /etc/rc.conf > # and /etc/rc.conf). > # > # Examples: > # > # sysrc sshd_enable > # returns YES or NO > # sysrc defaultrouter > # returns IP address of default router (if configured) > # sysrc 'hostname%%.*' > # returns $hostname up to (but not including) first `.' > # sysrc 'network_interfaces%%[$IFS]*' > # returns first word of $network_interfaces > # sysrc 'ntpdate_flags##*[$IFS]' > # returns last word of $ntpdate_flags (time server = address) > # sysrc usbd_flags-"default" > # returns $usbd_flags or "default" if unset > # sysrc usbd_flags:-"default" > # returns $usbd_flags or "default" if unset or NULL > # sysrc cloned_interfaces+"alternate" > # returns "alternate" if $cloned_interfaces is set > # sysrc cloned_interfaces:+"alternate" > # returns "alternate" if $cloned_interfaces is set and = non-NULL > # sysrc '#kern_securelevel' > # returns length in characters of $kern_securelevel > # sysrc 'hostname?' > # returns NULL and error status 2 if $hostname is unset = (or if > # set, returns the value of $hostname with no error = status) > # sysrc 'hostname:?' > # returns NULL and error status 2 if $hostname is unset or = NULL > # (or if set and non-NULL, returns value without error = status) > # > depend sysrc 'local' '[' 'return' '.' 'have' 'eval' 'echo' > : function ; sysrc ( ) # $varname > { > : ${RC_DEFAULTS:=3D"/etc/defaults/rc.conf"} >=20 > local defaults=3D"$RC_DEFAULTS" > local varname=3D"$1" >=20 > # Check arguments > [ -r "$defaults" ] || return > [ "$varname" ] || return >=20 > ( # Execute within sub-shell to protect parent environment > [ -f "$defaults" -a -r "$defaults" ] && . "$defaults" > have source_rc_confs && source_rc_confs > eval echo '"${'"$varname"'}"' 2> /dev/null > ) > } >=20 > # ... | lrev > # lrev $file ... > # > # Reverse lines of input. Unlike rev(1) which reverses the ordering of > # characters on a single line, this function instead reverses the line > # sequencing. > # > # For example, the following input: > # > # Line 1 > # Line 2 > # Line 3 > # > # Becomes reversed in the following manner: > # > # Line 3 > # Line 2 > # Line 1 > # > depend lrev 'local' 'if' '[' 'then' 'while' 'do' 'shift' 'done' 'else' = 'read' \ > 'fi' 'echo' > : function ; lrev ( ) # $file ... > { > local stdin_rev=3D > if [ $# -gt 0 ]; then > # > # Reverse lines from files passed as positional = arguments. > # > while [ $# -gt 0 ]; do > local file=3D"$1" > [ -f "$file" ] && lrev < "$file" > shift 1 > done > else > # > # Reverse lines from standard input > # > while read -r LINE; do > stdin_rev=3D"$LINE > $stdin_rev" > done > fi >=20 > echo -n "$stdin_rev" > } >=20 > # sysrc_set $setting $new_value > # > # Change a setting in the system configuration files (edits the files = in-place > # to change the value in the last assignment to the variable). If the = variable > # does not appear in the source file, it is appended to the end of the = primary > # system configuration file `/etc/rc.conf'. > # > depend sysrc_set 'local' 'sysrc' '[' 'return' 'for' 'do' 'done' 'if' = 'have' \ > 'then' 'else' 'while' 'read' 'case' 'esac' 'fi' = 'break' \ > 'eprintf' 'echo' 'lrev' > : function ; sysrc_set ( ) # $varname $new_value > { > local rc_conf_files=3D"$( sysrc rc_conf_files )" > local varname=3D"$1" new_value=3D"$2" > local file conf_files=3D >=20 > # Check arguments > [ "$rc_conf_files" ] || return ${FAILURE-1} > [ "$varname" ] || return ${FAILURE-1} >=20 > # Reverse the order of files in rc_conf_files > for file in $rc_conf_files; do > conf_files=3D"$file${conf_files:+ }$conf_files" > done >=20 > # > # Determine which file we are to operate on. If no files match, = we'll > # simply append to the last file in the list (`/etc/rc.conf'). > # > local found=3D > local regex=3D"^[[:space:]]*$varname=3D" > for file in $conf_files; do > #if have grep; then > if false; then > grep -q "$regex" $file && found=3D1 > else > while read LINE; do \ > case "$LINE" in \ > $varname=3D*) found=3D1;; \ > esac; \ > done < $file > fi > [ "$found" ] && break > done >=20 > # > # Perform sanity checks. > # > if [ ! -w $file ]; then > eprintf "\n%s: cannot create %s: permission denied\n" \ > "${progname:-$0}" "$file" > return ${FAILURE-1} > fi >=20 > # > # If not found, append new value to last file and return. > # > if [ ! "$found" ]; then > echo "$varname=3D\"$new_value\"" >> $file > return ${SUCCESS-0} > fi >=20 > # > # Operate on the matching file, replacing only the last = occurrence. > # > local new_contents=3D"`lrev $file 2> /dev/null | \ > ( found=3D > while read -r LINE; do > if [ ! "$found" ]; then > #if have grep; then > if false; then > match=3D"$( echo "$LINE" | grep "$regex" = )" > else > case "$LINE" in > $varname=3D*) match=3D1;; > *) match=3D;; > esac > fi > if [ "$match" ]; then > LINE=3D"$varname"'=3D"'"$new_value"'"' > found=3D1 > fi > fi > echo "$LINE" > done > ) | lrev`" >=20 > [ "$new_contents" ] \ > && echo "$new_contents" > $file > } >=20 > ############################################################ MAIN = SOURCE >=20 > # > # Perform sanity checks > # > depend main '[' 'usage' > [ $# -gt 0 ] || usage >=20 > # > # Process command-line options > # > depend main 'while' '[' 'do' 'case' 'usage' 'eprintf' \ > 'break' 'esac' 'shift' 'done' > while [ $# -gt 0 ]; do > case "$1" in > -h|--help) usage;; > -d) SYSRC_SHOW_DEPS=3D1;; > -e) SHOW_EQUALS=3D1;; > -n) SHOW_NAME=3D;; > -*) eprintf "%s: unrecognized option \`$1'\n" "${progname:-$0}" > usage;; > *) # Since it is impossible (in many shells, including bourne, = c, > # tennex-c, and bourne-again) to name a variable beginning = with a > # dash/hyphen [-], we will terminate the option-list at the = first > # item that doesn't begin with a dash. > break;; > esac > shift 1 > done > [ "$SHOW_NAME" ] || SHOW_EQUALS=3D >=20 > # > # Process command-line arguments > # > depend main '[' 'while' 'do' 'case' 'echo' 'sysrc' 'if' 'sysrc_set' = 'then' \ > 'fi' 'esac' 'shift' 'done' > SEP=3D': ' > [ "$SHOW_EQUALS" ] && SEP=3D'=3D"' > while [ $# -gt 0 ]; do > NAME=3D"${1%%=3D*}" > case "$1" in > *=3D*) > echo -n "${SHOW_NAME:+$NAME$SEP}$( > sysrc "$1" )${SHOW_EQUALS:+\"}" > if sysrc_set "$NAME" "${1#*=3D}"; then > echo " -> $( sysrc "$NAME" )" > fi > ;; > *) > if ! IGNORED=3D"$( sysrc "$NAME?" )"; then > echo "${progname:-$0}: unknown variable '$NAME'" > else > echo "${SHOW_NAME:+$NAME$SEP}$( > sysrc "$1" )${SHOW_EQUALS:+\"}" > fi > esac > shift 1 > done >=20 > # > # Ending routines > # > depend main 'show_deps' > show_deps >=20 > = ##########################################################################= ###### > # END > = ##########################################################################= ###### > # > # $Header$ > # > # $Copyright: 1999-2010. Vicor Inc. All Rights Reserved. $ > # > # $Log$ > # > = ##########################################################################= ###### >=20 > -- > Cheers, > Devin Teske >=20 > -> CONTACT INFORMATION <- > Business Solutions Consultant II > FIS - fisglobal.com > 510-735-5650 Mobile > 510-621-2038 Office > 510-621-2020 Office Fax > 909-477-4578 Home/Fax > devin.teske@fisglobal.com >=20 > -> LEGAL DISCLAIMER <- > This message contains confidential and proprietary information > of the sender, and is intended only for the person(s) to whom it > is addressed. Any use, distribution, copying or disclosure by any > other person is strictly prohibited. If you have received this > message in error, please notify the e-mail sender immediately, > and delete the original message without making a copy. >=20 > -> FUN STUFF <- > -----BEGIN GEEK CODE BLOCK----- > Version 3.1 > GAT/CS d(+) s: a- C++(++++) UB++++$ P++(++++) L++(++++) !E--- W++ N? = o? K- w O > M+ V- PS+ PE Y+ PGP- t(+) 5? X+(++) R>++ tv(+) b+(++) DI+(++) D(+) = G+>++ e>+ h > r>++ y+=20 > ------END GEEK CODE BLOCK------ > http://www.geekcode.com/ >=20 > -> END TRANSMISSION <- >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" -- Cheers, Devin Teske -> CONTACT INFORMATION <- Business Solutions Consultant II FIS - fisglobal.com 510-735-5650 Mobile 510-621-2038 Office 510-621-2020 Office Fax 909-477-4578 Home/Fax devin.teske@fisglobal.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> END TRANSMISSION <- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 04:49:07 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58A7F106566C for ; Thu, 7 Oct 2010 04:49:07 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0A0198FC19 for ; Thu, 7 Oct 2010 04:49:06 +0000 (UTC) Received: by gxk8 with SMTP id 8so158615gxk.13 for ; Wed, 06 Oct 2010 21:49:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :organization:user-agent:mime-version:to:cc:subject:references :in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; bh=kEW7ZWFPbslyRamwDbsX0dudzeAY5CKxad8bHVwzlqU=; b=ZO7xHbAhKfuOeRHDJ1oumnOTDy6R8MgWJ+iXRdJSfHUDnIRx974MuLe5vWR99B29ks yGQCXS6lszmR/9/pRj+mCT6eSoOVC7QXAywTUYLJexYSJbxkITcQ5CEBAWrVhgk0cfZn UePxLV3Pk1pvdwqdTTRnaLy5nBF6LDItkiKYc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:organization:user-agent:mime-version:to :cc:subject:references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=JKZrW1y7Cr5sQTCjhoN2h0DYELb1SpIh8O1rCxee7wjdi/c5Okm2DqAbB74ddyfsPt XcUi7F544BAiliDpgYvpOzRZgJV6B5t1CnIkfNMW5Qy58CsVo2Fpm/xWBBW+rLwJSH4n WyeEcEtdRzIG5+/Txc/Jq+9iYIfu/dzAQlA6Q= Received: by 10.236.95.162 with SMTP id p22mr434505yhf.97.1286426946140; Wed, 06 Oct 2010 21:49:06 -0700 (PDT) Received: from centel.dataix.local ([99.181.144.115]) by mx.google.com with ESMTPS id x45sm951839yhc.45.2010.10.06.21.49.04 (version=SSLv3 cipher=RC4-MD5); Wed, 06 Oct 2010 21:49:05 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4CAD513F.3010903@DataIX.net> Date: Thu, 07 Oct 2010 00:49:03 -0400 From: jhell Organization: http://www.DataIX.net User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.9) Gecko/20100917 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Devin Teske References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> In-Reply-To: <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Brandon Gooch , freebsd-hackers@freebsd.org Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 04:49:07 -0000 On 10/06/2010 23:29, Devin Teske wrote: I am not saying this to sound like I am shooting down this script but from the looks of it, I see no real advantage over the way a FreeBSD system is configured already. You could probably point these out though. With that noted what advantage does this script bring compared to a script like service(8) that is already in the base system? How much time does a end user spend enabling/disabling services for a system? How much of a difference in time would this make to the end user as per say just having to echo a variable into an rc.conf? If this would be put in place into the base system where would it be and who would maintain it ? Is it feasible to expect an end user to read rc.conf(5) services(8) while also consuming knowledge of etc/defaults/rc.conf & boot(8) as well sysctl.conf(5) and loader.conf(5). Also you search for grep and awk in your script. If this were to be in base, then utilities like this would not need to be searched for as base utilities are generally static to where they live in the file-system already... /usr/bin/grep /usr/bin/awk for example. I do not think it makes much sense for a base utility to search outside of its world for a executable especially for grep(1) and awk(1). Call these directly and let the end user modify their PATH as to where the location of these would actually be called from rather than adding extra complicity. > -> LEGAL DISCLAIMER <- > This message contains confidential and proprietary information > of the sender, and is intended only for the person(s) to whom it > is addressed. Any use, distribution, copying or disclosure by any > other person is strictly prohibited. If you have received this > message in error, please notify the e-mail sender immediately, > and delete the original message without making a copy. Due to this disclaimer I am not sure that even the implicit BSD license can override this. Though there is a thin gray line here, it might be suitable to remove this from future email unless you are specifically addressing something to a specific person. Regards, -- jhell,v From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 05:38:12 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3B8D106564A; Thu, 7 Oct 2010 05:38:12 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5DDA78FC0A; Thu, 7 Oct 2010 05:38:12 +0000 (UTC) Received: by iwn8 with SMTP id 8so15205iwn.13 for ; Wed, 06 Oct 2010 22:38:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=KquTfLp1PnwhowBAxIQ7rJ3AMWoab/23J3XytufV5jA=; b=JPtkac8ZBiQSoCPw9XypPROwDohmtW2Qk0csKnrJ4WLFvcMV5KvXoKGMIlMxfhTnzm ChjHtB24Xjffa1Z4xB4Lij16llDbfhygW3A1c7mrnD2CXPZAygsQWMRmen7+9+9N3BtB sSDVr6t7MiRQ7eTA2T5JhSCJlqYHZhV5dLV2U= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=VOZ9o7SUq5t44QB85SfAbxzL3lkp6QcnorFEPiHjURDOdOWxGTDHM3PGg+WNG58PVq UFu0mecsis2ayVArCDlKZ3KENFl5ddzOJ0ojC+xHKBtp7G6fBT0YNioH33lbf+FcNfaK 0HG6px34LOESQdXin+wV5vSV3mifVW3Z3/yZs= MIME-Version: 1.0 Received: by 10.231.193.81 with SMTP id dt17mr284736ibb.177.1286429863838; Wed, 06 Oct 2010 22:37:43 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.184.3 with HTTP; Wed, 6 Oct 2010 22:37:43 -0700 (PDT) In-Reply-To: References: <20101005235054.GA45827@freebsd.org> <20101006173522.GA92402@freebsd.org> <20101006193827.GA13528@freebsd.org> Date: Wed, 6 Oct 2010 22:37:43 -0700 X-Google-Sender-Auth: EOt7Aq_nsrXxrUCnfqQmRs1KZlk Message-ID: From: Garrett Cooper To: Garrett Cooper Content-Type: multipart/mixed; boundary=0050450172391002ee0492004b28 Cc: Alexander Best , Sergey Kandaurov , freebsd-hackers@freebsd.org Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 05:38:12 -0000 --0050450172391002ee0492004b28 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Wed, Oct 6, 2010 at 4:03 PM, Garrett Cooper wrote: > On Wed, Oct 6, 2010 at 3:01 PM, Sergey Kandaurov wrot= e: >> On 6 October 2010 23:38, Alexander Best wrote: >>> On Wed Oct =A06 10, Garrett Cooper wrote: >>>> On Wed, Oct 6, 2010 at 10:35 AM, Alexander Best = wrote: >>>> > On Wed Oct =A06 10, Garrett Cooper wrote: >>>> >> On Tue, Oct 5, 2010 at 4:50 PM, Alexander Best wrote: >>>> >> > hi there, >>>> >> > >>>> >> > i think the following example shows the problem better than a lon= g explanation: >>>> >> > >>>> >> > `touch ftest && chflags arch ftest && chflags -vv 0 ftest`. >>>> >> > =A0^^non-root =A0 =A0 ^^root =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0^^non= -root >>>> >> > >>>> >> > chflags claims to have cleared the 'arch' flag (which should be i= mpossible as >>>> >> > non-root user), but indeed has done nothing. >>>> >> > >>>> >> > i've tried the same with 'sappnd' and that works as can be expect= ed. >>>> >> > >>>> >> > The issue was confirmed to exist in HEAD (me), stable/8 (pgollucc= 1, jpaetzel) >>>> >> > and stable/7 (nox). >>>> >> > On stable/6 it does NOT exist (jpaetzel). chflags properly fails = with EPERM. >>>> >> >>>> >> =A0 =A0 Fails for me when I call the syscall directly, as I would e= xpect, >>>> >> and passes when I'm superuser: >>>> >> >>>> >> $ ./test_chflags >>>> >> (uid, euid) =3D (1000, 1000) >>>> >> test_chflags: chflags: Operation not permitted >>>> >> test_chflags: lchflags: Operation not permitted >>>> >> $ sudo ./test_chflags >>>> >> (uid, euid) =3D (0, 0) >>>> >> >>>> >> =A0 =A0 According to my basic inspection in strtofflags >>>> >> (.../lib/libc/gen/strtofflags.c), it works as well. >>>> >> =A0 =A0 And last but not least, executing the commands directly on = the CLI work: >>>> >> >>>> >> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >>>> >> $ chflags arch $tmpfile >>>> >> chflags: /tmp/chflags.nQm1IL: Operation not permitted >>>> >> $ rm $tmpfile >>>> >> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >>>> >> $ sudo chflags arch $tmpfile >>>> >> $ sudo chflags noarch $tmpfile >>>> >> $ rm $tmpfile >>>> > >>>> > thanks for your test app and helping out with this problem. i'm not = sure >>>> > however you understood the problem. probably i didn't explain it rig= ht: >>>> > >>>> > $ sudo rm -d /tmp/chflags.XXXXXX >>>> > $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >>>> > $ sudo chflags arch $tmpfile >>>> > $ chflags noarch $tmpfile >>>> > >>>> > is what's causing the problem. the last chflags call should fail, bu= t it >>>> > doesn't. >>>> >>>> Sorry... my CLI based example was stupid. I meant: >>>> >>>> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >>>> $ chflags arch $tmpfile >>>> chflags: /tmp/chflags.V2NpXR: Operation not permitted >>>> $ chflags noarch $tmpfile >>>> $ rm $tmpfile >>>> >>>> Currently chflags(2) states: >>>> >>>> =A0 =A0 =A0The SF_IMMUTABLE, SF_APPEND, SF_NOUNLINK, and SF_ARCHIVED f= lags may only >>>> =A0 =A0 =A0be set or unset by the super-user. =A0Attempts to set these= flags by non- >>>> =A0 =A0 =A0super-users are rejected, >>> attempts by non-superusers to= clear >>>> flags that >>>> =A0 =A0 =A0are already unset are silently ignored. <<< =A0These flags = may be set at any >>>> =A0 =A0 =A0time, but normally may only be unset when the system is in = single-user >>>> =A0 =A0 =A0mode. =A0(See init(8) for details.) >>>> >>>> So this behavior is already well documented :). The EPERM section >>>> should really note SF_ARCHIVED though (whoever added the flag forgot >>>> to add that particular item to the ERRORS section). >>> >>> that's perfectly alright. clearing an unset flag shouldn't cause any er= ror to >>> be returned. however in my example arch *does* get set and still trying= to >>> unset it as normal user doesn't return an error. >>> >> >> It's even more interesting. >> >> As far as I could parse the code: >> - UFS has no special handling for SF_ARCHIVED (I found only it for msdos= fs) > > =A0 =A0_very_ interesting: > > [/sys]$ grep -r SF_ARCHIVED kern/ fs/ ufs/ | grep -v svn > fs/msdosfs/msdosfs_vnops.c: =A0 =A0 =A0 =A0 =A0 =A0 vap->va_flags |=3D SF= _ARCHIVED; > fs/msdosfs/msdosfs_vnops.c: =A0 =A0 =A0 =A0 =A0 =A0 if (vap->va_flags & ~= SF_ARCHIVED) > fs/msdosfs/msdosfs_vnops.c: =A0 =A0 =A0 =A0 =A0 =A0 if (vap->va_flags & S= F_ARCHIVED) > > =A0 =A0The commit that introduced this change probably wasn't doing the > right thing: http://svn.freebsd.org/viewvc/base/head/sys/fs/msdosfs/msdos= fs_vnops.c?revision=3D5241&view=3Dmarkup > ; cp(1) probably should have been fixed in lieu of `fixing' msdosfs. > >> - ufs_setattr() does not handle unsetting SF_ARCHIVED, >> =A0so all what it does is simply return zero. > > =A0 =A0 [EOPNOTSUPP] =A0 =A0 =A0 The underlying file system does not supp= ort file > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0flags. > > =A0 =A0So I would expect for invalid flags to return EOPNOTSUPP. > > ... > > $ ~/test_chflags_negative > test_chflags_negative: should not get here > $ sudo ~/test_chflags_negative > test_chflags_negative: should not get here > > *facepalm* > > =A0 =A0I think the problem in part is here (sys/stat.h): > > =A0* > =A0* Super-user and owner changeable flags. > =A0*/ > #define UF_SETTABLE =A0 =A0 0x0000ffff =A0 =A0 =A0/* mask of owner change= able flags */ > #define UF_NODUMP =A0 =A0 =A0 0x00000001 =A0 =A0 =A0/* do not dump file *= / > #define UF_IMMUTABLE =A0 =A00x00000002 =A0 =A0 =A0/* file may not be chan= ged */ > #define UF_APPEND =A0 =A0 =A0 0x00000004 =A0 =A0 =A0/* writes to file may= only append */ > #define UF_OPAQUE =A0 =A0 =A0 0x00000008 =A0 =A0 =A0/* directory is opaqu= e wrt. union */ > #define UF_NOUNLINK =A0 =A0 0x00000010 =A0 =A0 =A0/* file may not be remo= ved or renamed */ > /* > =A0* Super-user changeable flags. > =A0*/ > #define SF_SETTABLE =A0 =A0 0xffff0000 =A0 =A0 =A0/* mask of superuser ch= angeable flags */ > #define SF_ARCHIVED =A0 =A0 0x00010000 =A0 =A0 =A0/* file is archived */ > #define SF_IMMUTABLE =A0 =A00x00020000 =A0 =A0 =A0/* file may not be chan= ged */ > #define SF_APPEND =A0 =A0 =A0 0x00040000 =A0 =A0 =A0/* writes to file may= only append */ > #define SF_NOUNLINK =A0 =A0 0x00100000 =A0 =A0 =A0/* file may not be remo= ved or renamed */ > #define SF_SNAPSHOT =A0 =A0 0x00200000 =A0 =A0 =A0/* snapshot inode */ > > =A0 =A0Note the *_SETTABLE macros, and the fact that they allow for more > functionality than what's currently slotted with the one-hot encoded > flags currently available. > =A0 =A0SF_ARCHIVED is not present in the other BSDs or Mac OSX either (I > did some hunting for a python bug related to chflags a few weeks > ago)... and I'm not even sure what this functionality really buys us > because it's not well described (but I'd be happy to get an > explanation/history lesson). > >> - /bin/chflags doesn't check the actual flags value from inode after >> calling chflags() syscall, and blindly assumes all is well, if chflags() >> returns with zero, > > =A0 =A0Yeah... but ideally tests should be written for this stuff and > exercised on all filesystems and exercised whenever code in this > particular path is changed, because that would potentially turn into a > noticeable performance hit [depending on how it's implemented in > chflags(1)]. And lo and behold it already does exist under > .../tools/regression/fstest/tests/chflags . I'll audit this once I get > back home... For starters, the tests were moved to .../tools/regression/pjdfstest . This fixes the manpage and the negative flags testcase at least. I ran the pjdfstest on a UFS2 partition on my test machine and tmpfs, and it passed chflags with flying colors. msdosfs unfortunately isn't supported yet, but I did some manual testing and everything seemed ok. I also need to check and see whether or not pjdfstest is doing the right job with negative testcases. I didn't have a ext2/3 or zfs pool to test with, so if someone could poke around with those filesystems it would be much appreciated :). And finally, here are all of the references in the sourcebase to SF_ARCHIVED: # /usr/local/bin/svnversion 213377M # grep -r SF_ARCHIVED /usr/src/ | grep -v svn grep: /usr/src/tools/regression/pjdfstest/pjdfstest_5aaec5b222b60945b16daa0= e8d61313d/pjdfstest_b4353ca81458e0bfc9ec5be8ff741eb2/usr/src/tools/regressi= on/priv/priv_vfs_chflags.c: flags |=3D SF_ARCHIVED; /usr/src/tools/regression/priv/priv_vfs_chflags.c: flags |=3D SF_ARCHIVED; /usr/src/tools/regression/priv/priv_vfs_chflags.c: flags |=3D SF_ARCHIVED; /usr/src/tools/regression/pjdfstest/tests/chflags/00.t: allflags=3D"UF_NODU= MP,UF_IMMUTABLE,UF_APPEND,UF_NOUNLINK,UF_OPAQUE,SF_ARCHIVED,SF_IMMUTABLE,SF= _APPEND,SF_NOUNLINK" /usr/src/tools/regression/pjdfstest/tests/chflags/00.t: systemflags=3D"SF_A= RCHIVED,SF_IMMUTABLE,SF_APPEND,SF_NOUNLINK" Binary file /usr/src/tools/regression/pjdfstest/pjdfstest matches /usr/src/tools/regression/pjdfstest/pjdfstest.c:#ifdef SF_ARCHIVED /usr/src/tools/regression/pjdfstest/pjdfstest.c: { SF_ARCHIVED, "SF_ARCHIVE= D" }, : Operation not supported grep: warning: /usr/src/sys/modules/tmpfs/@: recursive directory loop /usr/src/lib/libc/gen/strtofflags.c: { "noarch", SF_ARCHIVED, 0 }, /usr/src/lib/libc/gen/strtofflags.c: { "noarchived", SF_ARCHIVED, 0 }, /usr/src/lib/libc/sys/chflags.2:.It Dv SF_ARCHIVED /usr/src/lib/libc/sys/chflags.2:.Dv SF_ARCHIVED /usr/src/lib/libc/sys/chflags.2:.Dv SF_ARCHIVED , SF_IMMUTABLE , SF_APPEND = , /usr/src/lib/libc/sys/chflags.2:.Dv SF_ARCHIVED , SF_IMMUTABLE , SF_APPEND = , /usr/src/lib/libarchive/archive_entry.c:#ifdef SF_ARCHIVED /usr/src/lib/libarchive/archive_entry.c: { "noarch", L"noarch", SF_ARCHIVED, 0 }, /usr/src/lib/libarchive/archive_entry.c: { "noarchived", L"noarchived", SF_ARCHIVED, 0 }, /usr/src/sys/fs/msdosfs/msdosfs_vnops.c: vap->va_flags |=3D SF_ARCHIVED; /usr/src/sys/fs/msdosfs/msdosfs_vnops.c: if (vap->va_flags & ~SF_ARCHIVED) /usr/src/sys/fs/msdosfs/msdosfs_vnops.c: if (vap->va_flags & SF_ARCHIVED) /usr/src/sys/sys/stat.h:#define SF_ARCHIVED 0x00010000 /* file is archived = */ /usr/src/sys/sys/stat.h:#define SF_SETTABLE (SF_ARCHIVED | SF_IMMUTABLE | SF_APPEND | \ So it doesn't look like anything's utilizing the functionality, other than msdosfs, and all that really does is tweak the following attribute: #define ATTR_ARCHIVE 0x20 /* file is new or modified */ and vice versa. I vaguely remember archive file types in FAT32 from the Win95 days, but my memory is a bit hazy as to what the attribute actually does. Thanks, -Garrett --0050450172391002ee0492004b28 Content-Type: application/octet-stream; name="note-EPERM-SF_ARCHIVED-requirement.diff" Content-Disposition: attachment; filename="note-EPERM-SF_ARCHIVED-requirement.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gez5wqb21 SW5kZXg6IGxpYi9saWJjL3N5cy9jaGZsYWdzLjIKPT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gbGliL2xpYmMvc3lz L2NoZmxhZ3MuMgkocmV2aXNpb24gMjEzMzc3KQorKysgbGliL2xpYmMvc3lzL2NoZmxhZ3MuMgko d29ya2luZyBjb3B5KQpAQCAtMTQ4LDE0ICsxNDgsMTQgQEAKIHRoZSBlZmZlY3RpdmUgdXNlciBJ RCBpcyBub3QgdGhlIHN1cGVyLXVzZXIuCiAuSXQgQnEgRXIgRVBFUk0KIE9uZSBvZgotLkR2IFNG X0lNTVVUQUJMRSAsIFNGX0FQUEVORCAsCisuRHYgU0ZfQVJDSElWRUQgLCBTRl9JTU1VVEFCTEUg LCBTRl9BUFBFTkQgLAogb3IKIC5EdiBTRl9OT1VOTElOSwogaXMgc2V0IGFuZCB0aGUgdXNlciBp cyBlaXRoZXIgbm90IHRoZSBzdXBlci11c2VyIG9yCiBzZWN1cmVsZXZlbCBpcyBncmVhdGVyIHRo YW4gMC4KIC5JdCBCcSBFciBFUEVSTQogQSBub24tc3VwZXItdXNlciB0cmllcyB0byBzZXQgb25l IG9mCi0uRHYgU0ZfSU1NVVRBQkxFICwgU0ZfQVBQRU5EICwKKy5EdiBTRl9BUkNISVZFRCAsIFNG X0lNTVVUQUJMRSAsIFNGX0FQUEVORCAsCiBvcgogLkR2IFNGX05PVU5MSU5LIC4KIC5JdCBCcSBF ciBFUEVSTQo= --0050450172391002ee0492004b28 Content-Type: application/octet-stream; name="fail-chflags-with-bad-flags.diff" Content-Disposition: attachment; filename="fail-chflags-with-bad-flags.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gez6b9ry2 SW5kZXg6IHN5cy9rZXJuL3Zmc19zeXNjYWxscy5jCj09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9rZXJuL3Zm c19zeXNjYWxscy5jCShyZXZpc2lvbiAyMTMzNzcpCisrKyBzeXMva2Vybi92ZnNfc3lzY2FsbHMu Ywkod29ya2luZyBjb3B5KQpAQCAtMjcxMiw2ICsyNzEyLDggQEAKIAlpbnQgdmZzbG9ja2VkOwog CiAJQVVESVRfQVJHX0ZGTEFHUyh1YXAtPmZsYWdzKTsKKwlpZiAoKHVhcC0+ZmxhZ3MgJiAoU0Zf U0VUVEFCTEUgfCBVRl9TRVRUQUJMRSkpICE9IHVhcC0+ZmxhZ3MpCisJCXJldHVybiAoRU9QTk9U U1VQUCk7CiAJTkRJTklUKCZuZCwgTE9PS1VQLCBGT0xMT1cgfCBNUFNBRkUgfCBBVURJVFZOT0RF MSwgVUlPX1VTRVJTUEFDRSwKIAkgICAgdWFwLT5wYXRoLCB0ZCk7CiAJaWYgKChlcnJvciA9IG5h bWVpKCZuZCkpICE9IDApCkluZGV4OiBzeXMvc3lzL3N0YXQuaAo9PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMv c3lzL3N0YXQuaAkocmV2aXNpb24gMjEzMzc3KQorKysgc3lzL3N5cy9zdGF0LmgJKHdvcmtpbmcg Y29weSkKQEAgLTI2MSwyMiArMjYxLDI4IEBACiAgKgogICogU3VwZXItdXNlciBhbmQgb3duZXIg Y2hhbmdlYWJsZSBmbGFncy4KICAqLwotI2RlZmluZQlVRl9TRVRUQUJMRQkweDAwMDBmZmZmCS8q IG1hc2sgb2Ygb3duZXIgY2hhbmdlYWJsZSBmbGFncyAqLwogI2RlZmluZQlVRl9OT0RVTVAJMHgw MDAwMDAwMQkvKiBkbyBub3QgZHVtcCBmaWxlICovCiAjZGVmaW5lCVVGX0lNTVVUQUJMRQkweDAw MDAwMDAyCS8qIGZpbGUgbWF5IG5vdCBiZSBjaGFuZ2VkICovCiAjZGVmaW5lCVVGX0FQUEVORAkw eDAwMDAwMDA0CS8qIHdyaXRlcyB0byBmaWxlIG1heSBvbmx5IGFwcGVuZCAqLwogI2RlZmluZSBV Rl9PUEFRVUUJMHgwMDAwMDAwOAkvKiBkaXJlY3RvcnkgaXMgb3BhcXVlIHdydC4gdW5pb24gKi8K ICNkZWZpbmUgVUZfTk9VTkxJTksJMHgwMDAwMDAxMAkvKiBmaWxlIG1heSBub3QgYmUgcmVtb3Zl ZCBvciByZW5hbWVkICovCisKKwkJCQkJLyogbWFzayBvZiBzdXBlcnVzZXIgY2hhbmdlYWJsZSBm bGFncyAqLworI2RlZmluZQlVRl9TRVRUQUJMRQkoVUZfTk9EVU1QIHwgVUZfSU1NVVRBQkxFIHwg VUZfQVBQRU5EIHwgVUZfT1BBUVVFIHwgXAorCQkJIFVGX05PVU5MSU5LKQogLyoKICAqIFN1cGVy LXVzZXIgY2hhbmdlYWJsZSBmbGFncy4KICAqLwotI2RlZmluZQlTRl9TRVRUQUJMRQkweGZmZmYw MDAwCS8qIG1hc2sgb2Ygc3VwZXJ1c2VyIGNoYW5nZWFibGUgZmxhZ3MgKi8KICNkZWZpbmUJU0Zf QVJDSElWRUQJMHgwMDAxMDAwMAkvKiBmaWxlIGlzIGFyY2hpdmVkICovCiAjZGVmaW5lCVNGX0lN TVVUQUJMRQkweDAwMDIwMDAwCS8qIGZpbGUgbWF5IG5vdCBiZSBjaGFuZ2VkICovCiAjZGVmaW5l CVNGX0FQUEVORAkweDAwMDQwMDAwCS8qIHdyaXRlcyB0byBmaWxlIG1heSBvbmx5IGFwcGVuZCAq LwogI2RlZmluZQlTRl9OT1VOTElOSwkweDAwMTAwMDAwCS8qIGZpbGUgbWF5IG5vdCBiZSByZW1v dmVkIG9yIHJlbmFtZWQgKi8KICNkZWZpbmUJU0ZfU05BUFNIT1QJMHgwMDIwMDAwMAkvKiBzbmFw c2hvdCBpbm9kZSAqLwogCisJCQkJCS8qIG1hc2sgb2Ygc3VwZXJ1c2VyIGNoYW5nZWFibGUgZmxh Z3MgKi8KKyNkZWZpbmUJU0ZfU0VUVEFCTEUJKFNGX0FSQ0hJVkVEIHwgU0ZfSU1NVVRBQkxFIHwg U0ZfQVBQRU5EIHwgXAorCQkJIFNGX05PVU5MSU5LIHwgU0ZfU05BUFNIT1QpCisKICNpZmRlZiBf S0VSTkVMCiAvKgogICogU2hvcnRoYW5kIGFiYnJldmlhdGlvbnMgb2YgYWJvdmUuCg== --0050450172391002ee0492004b28-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 05:47:13 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9089B1065698 for ; Thu, 7 Oct 2010 05:47:13 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id 7485D8FC1A for ; Thu, 7 Oct 2010 05:47:13 +0000 (UTC) Received: from [173.241.24.124] (port=61505 helo=[10.0.0.109]) by postoffice.vicor.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.71) (envelope-from ) id 1P3jJs-0006W6-SA; Wed, 06 Oct 2010 22:47:11 -0700 Mime-Version: 1.0 (Apple Message framework v1081) From: Devin Teske In-Reply-To: <4CAD513F.3010903@DataIX.net> Date: Wed, 6 Oct 2010 22:47:07 -0700 Message-Id: References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <4CAD513F.3010903@DataIX.net> To: jhell X-Mailer: Apple Mail (2.1081) X-Scan-Signature: 73d140746fc6c9a2fb9e2efe27475922 X-Scan-Host: postoffice.vicor.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Brandon Gooch , freebsd-hackers@freebsd.org Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 05:47:13 -0000 On Oct 6, 2010, at 9:49 PM, jhell wrote: > On 10/06/2010 23:29, Devin Teske wrote: >=20 > I see no real advantage If you had to administer thousands of FreeBSD systems, you would. This script is obviously targeted to those for which are mentioned in = the header... "system administrators, power-users, and developers". > With that noted what advantage does this script bring compared to a > script like service(8) that is already in the base system? service(8) clearly doesn't do what this script does. The values that = associate to an executable rc.d script are but only a fraction of what = rc.conf(5) is used for. See `/etc/defaults/rc.conf' for all possible = values. > How much time does a end user spend enabling/disabling services for a > system? In our environment? A script like this empowers support personnel and = engineers with the ability to perform a generalized query on thousands = of machines simultaneously and collate the results back into a paging = application (such as less(1) or more(1)), finding possible = misconfigurations, problems, or other issues that arise by human error, = script error, or legacy options languishing from an upgrade process. = Further, with the ability to set options, it makes remediating problems = quite easy. > How much of a difference in time would this make to the end user as = per > say just having to echo a variable into an rc.conf? That's a sloppy way to manage production equipment/environments. > If this would be put in place into the base system where would it be = and > who would maintain it ? It would likely be placed in the same directory as sysctl(8) = (/sbin/sysrc). I would maintain it. > Is it feasible to expect an end user to read rc.conf(5) services(8) > while also consuming knowledge of etc/defaults/rc.conf & boot(8) as = well > sysctl.conf(5) and loader.conf(5). A friend of mine and I just installed 8.1-RELEASE-p1 onto his laptop. We = had to go through all those man-pages and more to get everything good = with the wpa_supplicant_* options. We didn't have to go to google, bing, = wolfram alpha, barnes & noble, or any place... all the answers for = setting up his laptop using 802.11g tied to a WAP w/ WPA2. But what does that have to do with the price of bread in China? This script helps system administrators, power-users, and developers. = It's designed to make the task of performing non-interactive edits to = rc.conf(5) not only easier, but cleaner. The files are left fully = in-tact by the script, which is the important part to note. It's not = going to sloppily keep tacking on new values for the same directive if = the script is called 1,000 times over. This enables such things as being = called from cron on a regular basis even. > Also you search for grep and awk in your script. No. I check for the presence of them with $PATH expansion. The have() function is a one-line function that remaps the `type' shell = built-in to have no output, yet still return the error status (or lack = thereof). If you read the sh(1) manual, you'll see that the `type' = built-in can be passed a shell keyword, a shell function, or failing = that, it will find the executable via $PATH expansion, and failing that = will return error status. The have() function that I have defined in my = script allows me to do this: if have grep; then # use grep else # use case with glob(7) pattern matching fi > If this were to be in > base, then utilities like this would not need to be searched for as = base > utilities are generally static to where they live in the file-system > already... /usr/bin/grep /usr/bin/awk for example. I do not think it > makes much sense for a base utility to search outside of its world for = a > executable especially for grep(1) and awk(1). Call these directly and > let the end user modify their PATH as to where the location of these > would actually be called from rather than adding extra complicity. The integral point of confusion is lack of knowledge that have() is = `type' which already performs $PATH expansion. I'm not probing for their = location, I'm testing for their existence so that I can use the name. = Think of it like a `try' statement. >> -> LEGAL DISCLAIMER <- >> This message contains confidential and proprietary information >> of the sender, and is intended only for the person(s) to whom it >> is addressed. Any use, distribution, copying or disclosure by any >> other person is strictly prohibited. If you have received this >> message in error, please notify the e-mail sender immediately, >> and delete the original message without making a copy. > Due to this disclaimer I am not sure that even the implicit BSD = license > can override this. Though there is a thin gray line here, it might be > suitable to remove this from future email unless you are specifically > addressing something to a specific person. Please. If you read it (as have I many times) it means nothing more than = that I own the IP rights to what I wrote and that if you aren't the = person that I sent this to (and you are whom I sent this to -- by-way of = the freebsd-hackers mailing list), then you should not read it, trash = it, and purge your trash. To my eyes, it didn't look all that legalese = to me, and thought it was pretty clear. >=20 > Regards, >=20 > --=20 >=20 > jhell,v -- Cheers, Devin Teske -> CONTACT INFORMATION <- Business Solutions Consultant II FIS - fisglobal.com 510-735-5650 Mobile 510-621-2038 Office 510-621-2020 Office Fax 909-477-4578 Home/Fax devin.teske@fisglobal.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> END TRANSMISSION <- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 07:23:20 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1ECD61065672 for ; Thu, 7 Oct 2010 07:23:20 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id CE9508FC0C for ; Thu, 7 Oct 2010 07:23:19 +0000 (UTC) Received: by mail-iw0-f182.google.com with SMTP id 8so28664iwn.13 for ; Thu, 07 Oct 2010 00:23:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :organization:user-agent:mime-version:to:cc:subject:references :in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; bh=FINEmm1hgUy3hhcXjSuqd1pEZOMTW4FDixkgS+Yk+d4=; b=tkTeK5Ic8WJ8QkDlhXGrvS7GiFOEej1M32o5YGUHGrPsfc4bQincGRB0m1juf3Gfbh 2sXcr3xYK1PE8Vlj8maWna99/9JZFjzhhNy4Ef1rMiTPkvu3Lb26GyrTlQVrzaftLLsu LLEWz2GIAU9Pf7cPl8rp5DfoVFEEkgNuJaqp4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:organization:user-agent:mime-version:to :cc:subject:references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=eFTnloe/pDrYdhUsfTJZrA7pZ2mwF/jAndTMm778aMeRHmifSEQwd0k9txzQTH1Yhf /MUk31PmCnTpscFwKD+U8PxXlgRN19I9fK4iFxfUTtzFugjy+uJgK0bt6Y3XILWjAM3Y 7qfhgK/i7pHeW8RNhKx2FeEIujy3bjkX6crHY= Received: by 10.231.157.207 with SMTP id c15mr401695ibx.143.1286436199246; Thu, 07 Oct 2010 00:23:19 -0700 (PDT) Received: from centel.dataix.local ([99.181.144.115]) by mx.google.com with ESMTPS id in12sm1896518ibb.15.2010.10.07.00.23.17 (version=SSLv3 cipher=RC4-MD5); Thu, 07 Oct 2010 00:23:18 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4CAD7563.1070706@DataIX.net> Date: Thu, 07 Oct 2010 03:23:15 -0400 From: jhell Organization: http://www.DataIX.net User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.9) Gecko/20100917 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Devin Teske References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <4CAD513F.3010903@DataIX.net> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Brandon Gooch , freebsd-hackers@freebsd.org Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 07:23:20 -0000 On 10/07/2010 01:47, Devin Teske wrote: > > On Oct 6, 2010, at 9:49 PM, jhell wrote: > >> On 10/06/2010 23:29, Devin Teske wrote: >> >> I see no real advantage > > If you had to administer thousands of FreeBSD systems, you would. > Normally I have found large scale (1000's) of Unix like systems benefiting from the use of cfengine that is available via ports(7) and the use of other such software as Nagios, MUNIN for example that allow you to achieve quick deployment and monitoring of production systems. Not that this is a real comfortable solution to this either but the values contained throughout FreeBSD systems are also available via bsnmpd(1). > This script is obviously targeted to those for which are mentioned in > the header... "system administrators, power-users, and developers". > > > >> With that noted what advantage does this script bring compared to >> a script like service(8) that is already in the base system? > > service(8) clearly doesn't do what this script does. The values that > associate to an executable rc.d script are but only a fraction of > what rc.conf(5) is used for. See `/etc/defaults/rc.conf' for all > possible values. > I am well aware of its use and the values it contains for a majority of the FreeBSD releases, dating back to 4.2 ;) Before that time it was mostly Linux. > >> How much time does a end user spend enabling/disabling services for >> a system? > > In our environment? A script like this empowers support personnel and > engineers with the ability to perform a generalized query on > thousands of machines simultaneously and collate the results back > into a paging application (such as less(1) or more(1)), finding > possible misconfigurations, problems, or other issues that arise by > human error, script error, or legacy options languishing from an > upgrade process. Further, with the ability to set options, it makes > remediating problems quite easy. > Alright, understandable. Large scale environments don't really fit into the PAGER area but I am following you. > >> How much of a difference in time would this make to the end user as >> per say just having to echo a variable into an rc.conf? > > That's a sloppy way to manage production equipment/environments. > Remember, there is a very few examples of your script to go on for usage and really not much real example usage scenarios that at least I can relate to by giving the script a quick look-over without getting really deep into it (yet). This is why the probing questions are coming up and that will at least help me if not others build a scenario at which your script can be looked at in possibly a more correct manner rather than passing judgment. > >> If this would be put in place into the base system where would it >> be and who would maintain it ? > > It would likely be placed in the same directory as sysctl(8) > (/sbin/sysrc). > > I would maintain it. > > >> Is it feasible to expect an end user to read rc.conf(5) >> services(8) while also consuming knowledge of etc/defaults/rc.conf >> & boot(8) as well sysctl.conf(5) and loader.conf(5). > > A friend of mine and I just installed 8.1-RELEASE-p1 onto his laptop. > We had to go through all those man-pages and more to get everything > good with the wpa_supplicant_* options. We didn't have to go to > google, bing, wolfram alpha, barnes & noble, or any place... all the > answers for setting up his laptop using 802.11g tied to a WAP w/ > WPA2. > > But what does that have to do with the price of bread in China? > Well unless those bread factories machines are powered by FreeBSD, I hope not much. ;) > This script helps system administrators, power-users, and developers. > It's designed to make the task of performing non-interactive edits to > rc.conf(5) not only easier, but cleaner. The files are left fully > in-tact by the script, which is the important part to note. It's not > going to sloppily keep tacking on new values for the same directive > if the script is called 1,000 times over. This enables such things as > being called from cron on a regular basis even. > > >> Also you search for grep and awk in your script. > > No. I check for the presence of them with $PATH expansion. > > The have() function is a one-line function that remaps the `type' > shell built-in to have no output, yet still return the error status > (or lack thereof). If you read the sh(1) manual, you'll see that the > `type' built-in can be passed a shell keyword, a shell function, or > failing that, it will find the executable via $PATH expansion, and > failing that will return error status. The have() function that I > have defined in my script allows me to do this: > > if have grep; then # use grep else # use case with glob(7) pattern > matching fi > > >> If this were to be in base, then utilities like this would not need >> to be searched for as base utilities are generally static to where >> they live in the file-system already... /usr/bin/grep /usr/bin/awk >> for example. I do not think it makes much sense for a base utility >> to search outside of its world for a executable especially for >> grep(1) and awk(1). Call these directly and let the end user modify >> their PATH as to where the location of these would actually be >> called from rather than adding extra complicity. > > The integral point of confusion is lack of knowledge that have() is > `type' which already performs $PATH expansion. I'm not probing for > their location, I'm testing for their existence so that I can use the > name. Think of it like a `try' statement. > Alright thank you for your explanation. I do not normally see this usage and this just sort of stood out at me and I did not want to assume what you were trying to accomplish, without asking. > >>> -> LEGAL DISCLAIMER <- This message contains confidential and >>> proprietary information of the sender, and is intended only for >>> the person(s) to whom it is addressed. Any use, distribution, >>> copying or disclosure by any other person is strictly >>> prohibited. If you have received this message in error, please >>> notify the e-mail sender immediately, and delete the original >>> message without making a copy. > >> Due to this disclaimer I am not sure that even the implicit BSD >> license can override this. Though there is a thin gray line here, >> it might be suitable to remove this from future email unless you >> are specifically addressing something to a specific person. > > Please. If you read it (as have I many times) it means nothing more > than that I own the IP rights to what I wrote and that if you aren't > the person that I sent this to (and you are whom I sent this to -- > by-way of the freebsd-hackers mailing list), then you should not read > it, trash it, and purge your trash. To my eyes, it didn't look all > that legalese to me, and thought it was pretty clear. > Ok Regards, -- jhell,v From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 08:15:03 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11D04106566B for ; Thu, 7 Oct 2010 08:15:03 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from out-0.mx.aerioconnect.net (outm.internet-mail-service.net [216.240.47.236]) by mx1.freebsd.org (Postfix) with ESMTP id C35778FC13 for ; Thu, 7 Oct 2010 08:15:02 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o977rOEq001088; Thu, 7 Oct 2010 00:53:25 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id D68122D6017; Thu, 7 Oct 2010 00:53:22 -0700 (PDT) Message-ID: <4CAD7CA1.4050202@freebsd.org> Date: Thu, 07 Oct 2010 00:54:09 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.9) Gecko/20100915 Thunderbird/3.1.4 MIME-Version: 1.0 To: jhell References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <4CAD513F.3010903@DataIX.net> <4CAD7563.1070706@DataIX.net> In-Reply-To: <4CAD7563.1070706@DataIX.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Brandon Gooch , freebsd-hackers@freebsd.org, Devin Teske Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 08:15:03 -0000 On 10/7/10 12:23 AM, jhell wrote: > > Alright thank you for your explanation. I do not normally see this usage > and this just sort of stood out at me and I did not want to assume what > you were trying to accomplish, without asking. three useage cases come to mind immediately. 1/ use within other scripts.. instead of the dozens of homegrown solutions people have written for puting something into /etc/rc.conf one can use this. 2/ what is the value of X on machines a,b,c foreach machine in a b c do ssh $machine sysrc X done you may well say "you could have used grep" bu tgrep doesn't give the default value vie the hierachy of .rc files. 2A is of course to correc teh values found to be wrong with (2) 3/ on a really small system, without an editor this may do a cleaner job than the usual "grep -v X /etc/rc.conf >/tmp/x;echo X >> /tmp/x; mv /tmp/x /etc/rc.conf" From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 10:36:01 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 2CF94106566B; Thu, 7 Oct 2010 10:36:01 +0000 (UTC) Date: Thu, 7 Oct 2010 10:36:01 +0000 From: Alexander Best To: Garrett Cooper Message-ID: <20101007103601.GA17089@freebsd.org> References: <20101005235054.GA45827@freebsd.org> <20101006173522.GA92402@freebsd.org> <20101006193827.GA13528@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Cc: freebsd-hackers@freebsd.org, Sergey Kandaurov Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 10:36:01 -0000 On Wed Oct 6 10, Garrett Cooper wrote: > On Wed, Oct 6, 2010 at 4:03 PM, Garrett Cooper wrote: > > On Wed, Oct 6, 2010 at 3:01 PM, Sergey Kandaurov wrote: > >> On 6 October 2010 23:38, Alexander Best wrote: > >>> On Wed Oct  6 10, Garrett Cooper wrote: > >>>> On Wed, Oct 6, 2010 at 10:35 AM, Alexander Best wrote: > >>>> > On Wed Oct  6 10, Garrett Cooper wrote: > >>>> >> On Tue, Oct 5, 2010 at 4:50 PM, Alexander Best wrote: > >>>> >> > hi there, > >>>> >> > > >>>> >> > i think the following example shows the problem better than a long explanation: > >>>> >> > > >>>> >> > `touch ftest && chflags arch ftest && chflags -vv 0 ftest`. > >>>> >> >  ^^non-root     ^^root                ^^non-root > >>>> >> > > >>>> >> > chflags claims to have cleared the 'arch' flag (which should be impossible as > >>>> >> > non-root user), but indeed has done nothing. > >>>> >> > > >>>> >> > i've tried the same with 'sappnd' and that works as can be expected. > >>>> >> > > >>>> >> > The issue was confirmed to exist in HEAD (me), stable/8 (pgollucc1, jpaetzel) > >>>> >> > and stable/7 (nox). > >>>> >> > On stable/6 it does NOT exist (jpaetzel). chflags properly fails with EPERM. > >>>> >> > >>>> >>     Fails for me when I call the syscall directly, as I would expect, > >>>> >> and passes when I'm superuser: > >>>> >> > >>>> >> $ ./test_chflags > >>>> >> (uid, euid) = (1000, 1000) > >>>> >> test_chflags: chflags: Operation not permitted > >>>> >> test_chflags: lchflags: Operation not permitted > >>>> >> $ sudo ./test_chflags > >>>> >> (uid, euid) = (0, 0) > >>>> >> > >>>> >>     According to my basic inspection in strtofflags > >>>> >> (.../lib/libc/gen/strtofflags.c), it works as well. > >>>> >>     And last but not least, executing the commands directly on the CLI work: > >>>> >> > >>>> >> $ tmpfile=`mktemp /tmp/chflags.XXXXXX` > >>>> >> $ chflags arch $tmpfile > >>>> >> chflags: /tmp/chflags.nQm1IL: Operation not permitted > >>>> >> $ rm $tmpfile > >>>> >> $ tmpfile=`mktemp /tmp/chflags.XXXXXX` > >>>> >> $ sudo chflags arch $tmpfile > >>>> >> $ sudo chflags noarch $tmpfile > >>>> >> $ rm $tmpfile > >>>> > > >>>> > thanks for your test app and helping out with this problem. i'm not sure > >>>> > however you understood the problem. probably i didn't explain it right: > >>>> > > >>>> > $ sudo rm -d /tmp/chflags.XXXXXX > >>>> > $ tmpfile=`mktemp /tmp/chflags.XXXXXX` > >>>> > $ sudo chflags arch $tmpfile > >>>> > $ chflags noarch $tmpfile > >>>> > > >>>> > is what's causing the problem. the last chflags call should fail, but it > >>>> > doesn't. > >>>> > >>>> Sorry... my CLI based example was stupid. I meant: > >>>> > >>>> $ tmpfile=`mktemp /tmp/chflags.XXXXXX` > >>>> $ chflags arch $tmpfile > >>>> chflags: /tmp/chflags.V2NpXR: Operation not permitted > >>>> $ chflags noarch $tmpfile > >>>> $ rm $tmpfile > >>>> > >>>> Currently chflags(2) states: > >>>> > >>>>      The SF_IMMUTABLE, SF_APPEND, SF_NOUNLINK, and SF_ARCHIVED flags may only > >>>>      be set or unset by the super-user.  Attempts to set these flags by non- > >>>>      super-users are rejected, >>> attempts by non-superusers to clear > >>>> flags that > >>>>      are already unset are silently ignored. <<<  These flags may be set at any > >>>>      time, but normally may only be unset when the system is in single-user > >>>>      mode.  (See init(8) for details.) > >>>> > >>>> So this behavior is already well documented :). The EPERM section > >>>> should really note SF_ARCHIVED though (whoever added the flag forgot > >>>> to add that particular item to the ERRORS section). > >>> > >>> that's perfectly alright. clearing an unset flag shouldn't cause any error to > >>> be returned. however in my example arch *does* get set and still trying to > >>> unset it as normal user doesn't return an error. > >>> > >> > >> It's even more interesting. > >> > >> As far as I could parse the code: > >> - UFS has no special handling for SF_ARCHIVED (I found only it for msdosfs) > > > >    _very_ interesting: > > > > [/sys]$ grep -r SF_ARCHIVED kern/ fs/ ufs/ | grep -v svn > > fs/msdosfs/msdosfs_vnops.c:             vap->va_flags |= SF_ARCHIVED; > > fs/msdosfs/msdosfs_vnops.c:             if (vap->va_flags & ~SF_ARCHIVED) > > fs/msdosfs/msdosfs_vnops.c:             if (vap->va_flags & SF_ARCHIVED) > > > >    The commit that introduced this change probably wasn't doing the > > right thing: http://svn.freebsd.org/viewvc/base/head/sys/fs/msdosfs/msdosfs_vnops.c?revision=5241&view=markup > > ; cp(1) probably should have been fixed in lieu of `fixing' msdosfs. > > > >> - ufs_setattr() does not handle unsetting SF_ARCHIVED, > >>  so all what it does is simply return zero. > > > >     [EOPNOTSUPP]       The underlying file system does not support file > >                        flags. > > > >    So I would expect for invalid flags to return EOPNOTSUPP. > > > > ... > > > > $ ~/test_chflags_negative > > test_chflags_negative: should not get here > > $ sudo ~/test_chflags_negative > > test_chflags_negative: should not get here > > > > *facepalm* > > > >    I think the problem in part is here (sys/stat.h): > > > >  * > >  * Super-user and owner changeable flags. > >  */ > > #define UF_SETTABLE     0x0000ffff      /* mask of owner changeable flags */ > > #define UF_NODUMP       0x00000001      /* do not dump file */ > > #define UF_IMMUTABLE    0x00000002      /* file may not be changed */ > > #define UF_APPEND       0x00000004      /* writes to file may only append */ > > #define UF_OPAQUE       0x00000008      /* directory is opaque wrt. union */ > > #define UF_NOUNLINK     0x00000010      /* file may not be removed or renamed */ > > /* > >  * Super-user changeable flags. > >  */ > > #define SF_SETTABLE     0xffff0000      /* mask of superuser changeable flags */ > > #define SF_ARCHIVED     0x00010000      /* file is archived */ > > #define SF_IMMUTABLE    0x00020000      /* file may not be changed */ > > #define SF_APPEND       0x00040000      /* writes to file may only append */ > > #define SF_NOUNLINK     0x00100000      /* file may not be removed or renamed */ > > #define SF_SNAPSHOT     0x00200000      /* snapshot inode */ > > > >    Note the *_SETTABLE macros, and the fact that they allow for more > > functionality than what's currently slotted with the one-hot encoded > > flags currently available. > >    SF_ARCHIVED is not present in the other BSDs or Mac OSX either (I > > did some hunting for a python bug related to chflags a few weeks > > ago)... and I'm not even sure what this functionality really buys us > > because it's not well described (but I'd be happy to get an > > explanation/history lesson). > > > >> - /bin/chflags doesn't check the actual flags value from inode after > >> calling chflags() syscall, and blindly assumes all is well, if chflags() > >> returns with zero, > > > >    Yeah... but ideally tests should be written for this stuff and > > exercised on all filesystems and exercised whenever code in this > > particular path is changed, because that would potentially turn into a > > noticeable performance hit [depending on how it's implemented in > > chflags(1)]. And lo and behold it already does exist under > > .../tools/regression/fstest/tests/chflags . I'll audit this once I get > > back home... > > For starters, the tests were moved to .../tools/regression/pjdfstest . > This fixes the manpage and the negative flags testcase at least. I > ran the pjdfstest on a UFS2 partition on my test machine and tmpfs, > and it passed chflags with flying colors. msdosfs unfortunately isn't > supported yet, but I did some manual testing and everything seemed ok. > I also need to check and see whether or not pjdfstest is doing the > right job with negative testcases. > I didn't have a ext2/3 or zfs pool to test with, so if someone > could poke around with those filesystems it would be much appreciated > :). > And finally, here are all of the references in the sourcebase to > SF_ARCHIVED: > > # /usr/local/bin/svnversion > 213377M > # grep -r SF_ARCHIVED /usr/src/ | grep -v svn > grep: /usr/src/tools/regression/pjdfstest/pjdfstest_5aaec5b222b60945b16daa0e8d61313d/pjdfstest_b4353ca81458e0bfc9ec5be8ff741eb2/usr/src/tools/regression/priv/priv_vfs_chflags.c: flags > |= SF_ARCHIVED; > /usr/src/tools/regression/priv/priv_vfs_chflags.c: flags |= SF_ARCHIVED; > /usr/src/tools/regression/priv/priv_vfs_chflags.c: flags |= SF_ARCHIVED; > /usr/src/tools/regression/pjdfstest/tests/chflags/00.t: allflags="UF_NODUMP,UF_IMMUTABLE,UF_APPEND,UF_NOUNLINK,UF_OPAQUE,SF_ARCHIVED,SF_IMMUTABLE,SF_APPEND,SF_NOUNLINK" > /usr/src/tools/regression/pjdfstest/tests/chflags/00.t: systemflags="SF_ARCHIVED,SF_IMMUTABLE,SF_APPEND,SF_NOUNLINK" > Binary file /usr/src/tools/regression/pjdfstest/pjdfstest matches > /usr/src/tools/regression/pjdfstest/pjdfstest.c:#ifdef SF_ARCHIVED > /usr/src/tools/regression/pjdfstest/pjdfstest.c: { SF_ARCHIVED, "SF_ARCHIVED" }, > : Operation not supported > grep: warning: /usr/src/sys/modules/tmpfs/@: recursive directory loop > /usr/src/lib/libc/gen/strtofflags.c: { "noarch", SF_ARCHIVED, 0 }, > /usr/src/lib/libc/gen/strtofflags.c: { "noarchived", SF_ARCHIVED, 0 }, > /usr/src/lib/libc/sys/chflags.2:.It Dv SF_ARCHIVED > /usr/src/lib/libc/sys/chflags.2:.Dv SF_ARCHIVED > /usr/src/lib/libc/sys/chflags.2:.Dv SF_ARCHIVED , SF_IMMUTABLE , SF_APPEND , > /usr/src/lib/libc/sys/chflags.2:.Dv SF_ARCHIVED , SF_IMMUTABLE , SF_APPEND , > /usr/src/lib/libarchive/archive_entry.c:#ifdef SF_ARCHIVED > /usr/src/lib/libarchive/archive_entry.c: { > "noarch", L"noarch", SF_ARCHIVED, 0 }, > /usr/src/lib/libarchive/archive_entry.c: { > "noarchived", L"noarchived", SF_ARCHIVED, 0 }, > /usr/src/sys/fs/msdosfs/msdosfs_vnops.c: vap->va_flags |= SF_ARCHIVED; > /usr/src/sys/fs/msdosfs/msdosfs_vnops.c: if (vap->va_flags & ~SF_ARCHIVED) > /usr/src/sys/fs/msdosfs/msdosfs_vnops.c: if (vap->va_flags & SF_ARCHIVED) > /usr/src/sys/sys/stat.h:#define SF_ARCHIVED 0x00010000 /* file is archived */ > /usr/src/sys/sys/stat.h:#define SF_SETTABLE (SF_ARCHIVED | > SF_IMMUTABLE | SF_APPEND | \ > > So it doesn't look like anything's utilizing the functionality, > other than msdosfs, and all that really does is tweak the following > attribute: > > #define ATTR_ARCHIVE 0x20 /* file is new or modified */ > > and vice versa. I vaguely remember archive file types in FAT32 > from the Win95 days, but my memory is a bit hazy as to what the > attribute actually does. occasionally i get errors during file copies from msdosfs to ufs2. windows seems to use the arch flag quite extensively actually. i think formating a new drive automatically markes it as 'archivable' and all new files get their flag set. so when copying files from fat32 to ufs i very often get an EPERM error, because as a non-root user that flag cannot be preserved. doesn't matter however since the copying suceeds. good thing the chflags operation takes place after copying the file and not beforehand (i.e. touch, chflags, copy). also: what about sorting the flags (in the manual page e.g.). should they be sorted alphabetically or by bitfields? also: is SF_SNAPSHOT really changeable by the super user? chflags(2) says it's not. cheers. alex > Thanks, > -Garrett -- a13x From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 12:37:46 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8DE0106564A for ; Thu, 7 Oct 2010 12:37:46 +0000 (UTC) (envelope-from rand@meridian-enviro.com) Received: from zimbra.meridian-enviro.com (zimbra.meridian-enviro.com [12.192.92.32]) by mx1.freebsd.org (Postfix) with ESMTP id A7E1E8FC0A for ; Thu, 7 Oct 2010 12:37:46 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra.meridian-enviro.com (Postfix) with ESMTP id 8587630701C9 for ; Thu, 7 Oct 2010 07:19:41 -0500 (CDT) X-Virus-Scanned: amavisd-new at meridian-enviro.com Received: from zimbra.meridian-enviro.com ([127.0.0.1]) by localhost (zimbra.meridian-enviro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RrZD3excC78K for ; Thu, 7 Oct 2010 07:19:41 -0500 (CDT) Received: from delta.meridian-enviro.com (delta.meridian-enviro.com [10.10.10.43]) by zimbra.meridian-enviro.com (Postfix) with ESMTP id 1179030701C7 for ; Thu, 7 Oct 2010 07:19:41 -0500 (CDT) Date: Thu, 07 Oct 2010 07:19:41 -0500 Message-ID: <87fwwici02.wl%rand@meridian-enviro.com> To: freebsd-hackers@freebsd.org X-Draft-From: ("nnml+rand:freebsd.hackers" 67823) References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <4CAD513F.3010903@DataIX.net> <4CAD7563.1070706@DataIX.net> From: Douglas K. Rand User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.7 Emacs/22.3 (i386-portbld-freebsd8.0) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 12:37:46 -0000 I think that this script might also fill a void with using Puppet as a configuration tool. Currently Puppet, as its default behaviour, uses files in /etc/rc.conf.d to set variables. I've found this approach really lacking because you cannot do things like: nfs_server_enable="YES" via a /etc/rc.conf.d file because multiple things (like lockd, mountd, nfsd, nfsserver, and statd) all need nfs_server_enable set, as one example. This script would provide an easy method for puppet to "edit" lines in /etc/rc.conf. I have a feature request though: How about a switch like "-c config-file" to change which file sysrc_set adds the line to. We use both /etc/rc.conf and /etc/rc.conf.local on some systems, and it would be nice to be able to specify that this particular knob should be placed in /etc/rc.conf.local. I'm unsure what should happen if -c is specified and the knob you want to change already exists in another rc.conf file: Should it be moved to the new location, or changed where it is? (Maybe that'd be yet another option to sysrc.) From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 13:42:43 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5506E106567A for ; Thu, 7 Oct 2010 13:42:43 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) by mx1.freebsd.org (Postfix) with ESMTP id EDA248FC1C for ; Thu, 7 Oct 2010 13:42:42 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.4/8.14.4) with ESMTP id o97DggUD003826; Thu, 7 Oct 2010 07:42:42 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.4/8.14.4/Submit) with ESMTP id o97Dggum003823; Thu, 7 Oct 2010 07:42:42 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Thu, 7 Oct 2010 07:42:42 -0600 (MDT) From: Warren Block To: "Douglas K. Rand" In-Reply-To: <87fwwici02.wl%rand@meridian-enviro.com> Message-ID: References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <4CAD513F.3010903@DataIX.net> <4CAD7563.1070706@DataIX.net> <87fwwici02.wl%rand@meridian-enviro.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (wonkity.com [127.0.0.1]); Thu, 07 Oct 2010 07:42:42 -0600 (MDT) Cc: freebsd-hackers@freebsd.org Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 13:42:43 -0000 On Thu, 7 Oct 2010, Douglas K. Rand wrote: > I think that this script might also fill a void with using Puppet as a > configuration tool. Currently Puppet, as its default behaviour, uses > files in /etc/rc.conf.d to set variables. I've found this approach > really lacking because you cannot do things like: > > nfs_server_enable="YES" > > via a /etc/rc.conf.d file because multiple things (like lockd, mountd, > nfsd, nfsserver, and statd) all need nfs_server_enable set, as one > example. This script would provide an easy method for puppet to "edit" > lines in /etc/rc.conf. Consider also the docs that tell the user to echo 'something_enable="YES"' >> /etc/rc.conf which can produce duplicate and possibly differing entries. Or non-working entries if there was no ending \n present, or even a broken rc.conf that breaks startup. From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 14:26:45 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E1A71065697 for ; Thu, 7 Oct 2010 14:26:45 +0000 (UTC) (envelope-from rand@meridian-enviro.com) Received: from zimbra.meridian-enviro.com (zimbra.meridian-enviro.com [12.192.92.32]) by mx1.freebsd.org (Postfix) with ESMTP id 1C2E38FC2B for ; Thu, 7 Oct 2010 14:26:44 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by zimbra.meridian-enviro.com (Postfix) with ESMTP id 79CCA30701C9; Thu, 7 Oct 2010 09:26:44 -0500 (CDT) X-Virus-Scanned: amavisd-new at meridian-enviro.com Received: from zimbra.meridian-enviro.com ([127.0.0.1]) by localhost (zimbra.meridian-enviro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0ysW4IFOPulo; Thu, 7 Oct 2010 09:26:44 -0500 (CDT) Received: from delta.meridian-enviro.com (delta.meridian-enviro.com [10.10.10.43]) by zimbra.meridian-enviro.com (Postfix) with ESMTP id 114E930701C7; Thu, 7 Oct 2010 09:26:44 -0500 (CDT) Date: Thu, 07 Oct 2010 09:26:44 -0500 Message-ID: <87aamqcc4b.wl%rand@meridian-enviro.com> From: Douglas K. Rand To: Warren Block In-Reply-To: References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <4CAD513F.3010903@DataIX.net> <4CAD7563.1070706@DataIX.net> <87fwwici02.wl%rand@meridian-enviro.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.7 Emacs/22.3 (i386-portbld-freebsd8.0) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: freebsd-hackers@freebsd.org Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 14:26:45 -0000 Warren> Consider also the docs that tell the user to Warren> echo 'something_enable="YES"' >> /etc/rc.conf Warren> which can produce duplicate and possibly differing entries. Yup! Which is exactly why I like the idea of this tool. From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 17:47:55 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0EB2210656A4 for ; Thu, 7 Oct 2010 17:47:55 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 5C5F38FC19 for ; Thu, 7 Oct 2010 17:47:53 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA18557; Thu, 07 Oct 2010 20:40:50 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4CAE0621.3070809@freebsd.org> Date: Thu, 07 Oct 2010 20:40:49 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: hackers@freebsd.org, freebsd-current@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: panic_cpu should be volatile X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 17:47:55 -0000 panic_cpu variable in kern_shutdown.c should be volatile otherwise it's cached in a register in the innermost while-loop in this code (observed on amd64 with base gcc and -O2): if (panic_cpu != PCPU_GET(cpuid)) while (atomic_cmpset_int(&panic_cpu, NOCPU, PCPU_GET(cpuid)) == 0) while (panic_cpu != NOCPU) ; /* nothing */ The patch is here: http://people.freebsd.org/~avg/panic_cpu.diff I also took a liberty to move the variable into the scope of panic() functions as it doesn't seem to be useful outside of it. But this is not necessary, of course. Big thanks to mdf@ for the hint and to kib@ and kan@ for memory model expertise. P.S. The assembly: .loc 1 544 0 movl panic_cpu(%rip), %eax .LVL134: .p2align 4,,7 .L210: cmpl $255, %eax jne .L210 jmp .L225 -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 17:47:56 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91FF11065693 for ; Thu, 7 Oct 2010 17:47:56 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C6E238FC1C for ; Thu, 7 Oct 2010 17:47:55 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA18456; Thu, 07 Oct 2010 20:33:11 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4CAE0457.8040900@freebsd.org> Date: Thu, 07 Oct 2010 20:33:11 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: zfs-devel@freebsd.org, hackers@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: zfs: ru_inblock/ru_oublock accounting X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 17:47:56 -0000 A simple, probably incomplete and perhaps incorrect patch for ru_inblock/ru_oublock accounting in zfs: http://people.freebsd.org/~avg/zfs-ru.diff Still quite better than nothing. P.S. This is about top -m io. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 17:53:49 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2581D1065695; Thu, 7 Oct 2010 17:53:49 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 383988FC25; Thu, 7 Oct 2010 17:53:47 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA18721; Thu, 07 Oct 2010 20:53:46 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4CAE092A.60905@freebsd.org> Date: Thu, 07 Oct 2010 20:53:46 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Matthew Fleming Subject: generic_stop_cpus: prevent parallel execution X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 17:53:49 -0000 Here is patch that applies the technique from panic() to generic_stop_cpus() to prevent its parallel execution on multiple CPUs: http://people.freebsd.org/~avg/generic_stop_cpus.diff In theory this could lead to two CPUs stopping each other and everyone else, and thus a total system halt. Also, in theory, we should have some smarter locking here, because two (or more CPUs) could be stopping unrelated sets of CPUs. But in practice, it seems, this function is only used to stop "all other" CPUs. Unless I overlooked other usages, that is. Additionally, I took this opportunity to make amd64-specific suspend_cpus() function use generic_stop_cpus() instead of rolling out essentially duplicate code. I couldn't see any reason no to consolidate, but perhaps I missed something. Big thanks to Matthew and his employer for the idea and example. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 17:57:12 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D66241065675 for ; Thu, 7 Oct 2010 17:57:12 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1633F8FC1D for ; Thu, 7 Oct 2010 17:57:11 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA18775 for ; Thu, 07 Oct 2010 20:57:10 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4CAE09F6.2050608@freebsd.org> Date: Thu, 07 Oct 2010 20:57:10 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4CA4C63F.4070503@icyb.net.ua> <4CAB43EF.8010807@freebsd.org> In-Reply-To: <4CAB43EF.8010807@freebsd.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: sysctl for querying kmem_map->size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 17:57:12 -0000 on 05/10/2010 18:27 Andriy Gapon said the following: > Here's an updated patch: > http://people.freebsd.org/~avg/sysctl-kmem_map_size2.diff > The new code wraps kmem_map->size into SYSCTL_PROC() to handle vm_size_t type > differences for different platforms. The idea is suggested by Ed Maste. So I am committing this. Thanks for no objections :) -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 19:05:02 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F783106566B; Thu, 7 Oct 2010 19:05:02 +0000 (UTC) (envelope-from jh@FreeBSD.org) Received: from gw01.mail.saunalahti.fi (gw01.mail.saunalahti.fi [195.197.172.115]) by mx1.freebsd.org (Postfix) with ESMTP id 190F28FC12; Thu, 7 Oct 2010 19:05:01 +0000 (UTC) Received: from a91-153-123-205.elisa-laajakaista.fi (a91-153-123-205.elisa-laajakaista.fi [91.153.123.205]) by gw01.mail.saunalahti.fi (Postfix) with SMTP id 65FC1151700; Thu, 7 Oct 2010 21:45:50 +0300 (EEST) Date: Thu, 7 Oct 2010 21:45:49 +0300 From: Jaakko Heinonen To: Alexander Best Message-ID: <20101007184549.GA76070@a91-153-123-205.elisa-laajakaista.fi> References: <20101005235054.GA45827@freebsd.org> <20101006173522.GA92402@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20101006173522.GA92402@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org, Garrett Cooper Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 19:05:02 -0000 On 2010-10-06, Alexander Best wrote: > $ sudo rm -d /tmp/chflags.XXXXXX > $ tmpfile=`mktemp /tmp/chflags.XXXXXX` > $ sudo chflags arch $tmpfile > $ chflags noarch $tmpfile > > is what's causing the problem. the last chflags call should fail, but it > doesn't. Here is a patch for UFS: %%% Index: sys/ufs/ufs/ufs_vnops.c =================================================================== --- sys/ufs/ufs/ufs_vnops.c (revision 213507) +++ sys/ufs/ufs/ufs_vnops.c (working copy) @@ -556,6 +556,9 @@ ufs_setattr(ap) & (SF_NOUNLINK | SF_IMMUTABLE | SF_APPEND) || (vap->va_flags & UF_SETTABLE) != vap->va_flags) return (EPERM); + if ((ip->i_flags & SF_SETTABLE) != + (vap->va_flags & SF_SETTABLE)) + return (EPERM); ip->i_flags &= SF_SETTABLE; ip->i_flags |= (vap->va_flags & UF_SETTABLE); DIP_SET(ip, i_flags, ip->i_flags); %%% The patch has a potential to break something if someone assumes that non-super-user can modify UF_SETTABLE flags with the SF_SETTABLE part set to zero. However with a quick peek this seems to be what NetBSD does. -- Jaakko From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 7 19:43:18 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7BF911065670; Thu, 7 Oct 2010 19:43:18 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0C3608FC13; Thu, 7 Oct 2010 19:43:18 +0000 (UTC) Received: by pvc21 with SMTP id 21so90376pvc.13 for ; Thu, 07 Oct 2010 12:43:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=I7z8iShKZvnK/9bFcF3jQb67E4po9BygX+1MwTXjiqU=; b=Uo5Nk2c15sPWQoPHEMDjSha/8rvjKUhgSxj3h/8LMlbbR7AKQqwwZasC04Q4Dwl9/Y n8qPEhszkxzdFBxO0jSZpzjp11SL55Qpefp9Zi1CPmxXZ1hmfcPFLYu0sHqoj95EmkiI YZoUWzSlAtc8ZAobKP2bPmEqOq1nGNBn7vDM0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=QQGIVb2+5SOvNtLqDFHNzwK0sw2NFToP7ODaTTzX31HLYWM5o7d6MGxeP1Pz5lpdZP 1BH55H31jK8/6USbdPXtj321k85YWSgxF1dqr//06ClFgEjxyWDWtR3BukfrXk6LJwHk uasMv0oIKvmgRofEXL2oxWklPdG0Ru3a23dsw= MIME-Version: 1.0 Received: by 10.142.141.10 with SMTP id o10mr1041368wfd.374.1286480597600; Thu, 07 Oct 2010 12:43:17 -0700 (PDT) Received: by 10.142.222.13 with HTTP; Thu, 7 Oct 2010 12:43:17 -0700 (PDT) In-Reply-To: <20101007184549.GA76070@a91-153-123-205.elisa-laajakaista.fi> References: <20101005235054.GA45827@freebsd.org> <20101006173522.GA92402@freebsd.org> <20101007184549.GA76070@a91-153-123-205.elisa-laajakaista.fi> Date: Thu, 7 Oct 2010 23:43:17 +0400 Message-ID: From: Sergey Kandaurov To: Jaakko Heinonen Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Alexander Best , Garrett Cooper , freebsd-hackers@freebsd.org Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 19:43:18 -0000 On 7 October 2010 22:45, Jaakko Heinonen wrote: > On 2010-10-06, Alexander Best wrote: >> $ sudo rm -d /tmp/chflags.XXXXXX >> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >> $ sudo chflags arch $tmpfile >> $ chflags noarch $tmpfile >> >> is what's causing the problem. the last chflags call should fail, but it >> doesn't. > > Here is a patch for UFS: > > %%% > Index: sys/ufs/ufs/ufs_vnops.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/ufs/ufs/ufs_vnops.c =A0 =A0 (revision 213507) > +++ sys/ufs/ufs/ufs_vnops.c =A0 =A0 (working copy) > @@ -556,6 +556,9 @@ ufs_setattr(ap) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0& (SF_NOUNLINK | S= F_IMMUTABLE | SF_APPEND) || > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(vap->va_flags & U= F_SETTABLE) !=3D vap->va_flags) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return (EP= ERM); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ((ip->i_flags & SF_SETTA= BLE) !=3D > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (vap->va_flags & SF= _SETTABLE)) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 return (EPE= RM); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->i_flags &=3D SF_SETTAB= LE; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->i_flags |=3D (vap->va_= flags & UF_SETTABLE); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0DIP_SET(ip, i_flags, ip->i= _flags); > %%% > > The patch has a potential to break something if someone assumes that > non-super-user can modify UF_SETTABLE flags with the SF_SETTABLE part > set to zero. However with a quick peek this seems to be what NetBSD > does. Just for reference: this comes from NetBSD PR kern/3491 and fixed before 1.3R. I just checked arch test, and it works as expected with the change. All chflags tests from fstest suite passed as well. --=20 wbr, pluknet From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 8 00:35:00 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61278106564A; Fri, 8 Oct 2010 00:35:00 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id BF1228FC17; Fri, 8 Oct 2010 00:34:59 +0000 (UTC) Received: by iwn8 with SMTP id 8so626526iwn.13 for ; Thu, 07 Oct 2010 17:34:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=7VZ8J/UHzYdZQMnkzkDhj2aD8ZFKNaN1HYb/QL4sAPs=; b=lDxqLkBaEhhu/93j35DNpdihjCdIpsZNOEaJC4YXtXaUuKAxC/q3sku0kBFA73c8Qq zv8rGtCdiuEj38MXkv+ogXx+AucDVMbAfo8d4dn2Ec2xhn6qJDXPYmX4gVx3l59OhFt6 ywxnnlmullnu108EFLhCnbLsuXPc3yAOFRvWA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=S5DWGQh64tpviE+YaXCjoy7pvMYG6JIu3fFKzb3OvGayj87IT5x47XsP4Ob0Ld0SMK KxTewuH9vfwggaVp3l4P0kf79JgrpBRCoNPK2y4ryM+k4ra34wTqfMXIWSxPWtKZKyCe +cJxKvY5Kc1HjPsxF/FbYmBho5Q9qpx4sfToc= MIME-Version: 1.0 Received: by 10.231.174.84 with SMTP id s20mr1685166ibz.94.1286498099275; Thu, 07 Oct 2010 17:34:59 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.184.3 with HTTP; Thu, 7 Oct 2010 17:34:59 -0700 (PDT) In-Reply-To: References: <20101005235054.GA45827@freebsd.org> <20101006173522.GA92402@freebsd.org> <20101007184549.GA76070@a91-153-123-205.elisa-laajakaista.fi> Date: Thu, 7 Oct 2010 17:34:59 -0700 X-Google-Sender-Auth: qcSHl39_5aoJvFGS6Dj888Pa0KU Message-ID: From: Garrett Cooper To: Sergey Kandaurov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Jaakko Heinonen , freebsd-hackers@freebsd.org, Alexander Best Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Oct 2010 00:35:00 -0000 On Thu, Oct 7, 2010 at 12:43 PM, Sergey Kandaurov wrote= : > On 7 October 2010 22:45, Jaakko Heinonen wrote: >> On 2010-10-06, Alexander Best wrote: >>> $ sudo rm -d /tmp/chflags.XXXXXX >>> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >>> $ sudo chflags arch $tmpfile >>> $ chflags noarch $tmpfile >>> >>> is what's causing the problem. the last chflags call should fail, but i= t >>> doesn't. >> >> Here is a patch for UFS: >> >> %%% >> Index: sys/ufs/ufs/ufs_vnops.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- sys/ufs/ufs/ufs_vnops.c =A0 =A0 (revision 213507) >> +++ sys/ufs/ufs/ufs_vnops.c =A0 =A0 (working copy) >> @@ -556,6 +556,9 @@ ufs_setattr(ap) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0& (SF_NOUNLINK | = SF_IMMUTABLE | SF_APPEND) || >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(vap->va_flags & = UF_SETTABLE) !=3D vap->va_flags) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return (E= PERM); >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ((ip->i_flags & SF_SETT= ABLE) !=3D >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (vap->va_flags & S= F_SETTABLE)) >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 return (EP= ERM); >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->i_flags &=3D SF_SETTA= BLE; >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->i_flags |=3D (vap->va= _flags & UF_SETTABLE); >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0DIP_SET(ip, i_flags, ip->= i_flags); >> %%% >> >> The patch has a potential to break something if someone assumes that >> non-super-user can modify UF_SETTABLE flags with the SF_SETTABLE part >> set to zero. However with a quick peek this seems to be what NetBSD >> does. > > Just for reference: > this comes from NetBSD PR kern/3491 and fixed before 1.3R. > I just checked arch test, and it works as expected with the change. > All chflags tests from fstest suite passed as well. This only potentially fixes UFS though -- not ext*, msdosfs, ntfs, ZFS, etc, which that it can still fail the requirement in the chflags(2) manpage that states: [EOPNOTSUPP] The underlying file system does not support file flags. Or should it be a different requirement, i.e. EINVAL for bogus values? Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 8 00:42:58 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 581AE1065673; Fri, 8 Oct 2010 00:42:58 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id D79578FC0C; Fri, 8 Oct 2010 00:42:57 +0000 (UTC) Received: by iwn8 with SMTP id 8so635427iwn.13 for ; Thu, 07 Oct 2010 17:42:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=GmPi5N6zrim5fybXg/tHjbBpS6fhAtkpQP9bRH4dbUM=; b=LCxqRJIAigt5sXmaN8SrPAdyi+7QxJSWAEUdSK9X+6H8eOpW5OyftNszIBHCnPTp25 bAiDlFMNgYMdt33pkhJlQJJLCEBzKuBWB3DI3/g4B2ceYqdBiRh5167Jvt+I84qMjaXz 3y/TFDsqh7JRBQHSXdSSs39l980rkqsTqywt8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=Hvs8zH5dOqE5gFN3t1jKKTXYBw92Nl81qrXp8/ytSy7RYAyvahna5ULm6SJRLL2vHG 4o7r82Zcko5jZNx5/6cSmC4J8ynPCTTbs4bAAfM2YM5z3vSKsAD00RJF9VsLMLiwEcBn 4aT1hmQkCVKPv9gNg2Epw86oT2tdWaWjBie/Y= MIME-Version: 1.0 Received: by 10.231.39.201 with SMTP id h9mr1728412ibe.27.1286498577514; Thu, 07 Oct 2010 17:42:57 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.184.3 with HTTP; Thu, 7 Oct 2010 17:42:57 -0700 (PDT) In-Reply-To: <20101007184549.GA76070@a91-153-123-205.elisa-laajakaista.fi> References: <20101005235054.GA45827@freebsd.org> <20101006173522.GA92402@freebsd.org> <20101007184549.GA76070@a91-153-123-205.elisa-laajakaista.fi> Date: Thu, 7 Oct 2010 17:42:57 -0700 X-Google-Sender-Auth: 5RIdEXqW46qVz3zfTSMEIWIV5J0 Message-ID: From: Garrett Cooper To: Jaakko Heinonen Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Alexander Best , freebsd-hackers@freebsd.org Subject: Re: issue with unsetting 'arch' flag X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Oct 2010 00:42:58 -0000 On Thu, Oct 7, 2010 at 11:45 AM, Jaakko Heinonen wrote: > On 2010-10-06, Alexander Best wrote: >> $ sudo rm -d /tmp/chflags.XXXXXX >> $ tmpfile=3D`mktemp /tmp/chflags.XXXXXX` >> $ sudo chflags arch $tmpfile >> $ chflags noarch $tmpfile >> >> is what's causing the problem. the last chflags call should fail, but it >> doesn't. > > Here is a patch for UFS: > > %%% > Index: sys/ufs/ufs/ufs_vnops.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/ufs/ufs/ufs_vnops.c =A0 =A0 (revision 213507) > +++ sys/ufs/ufs/ufs_vnops.c =A0 =A0 (working copy) > @@ -556,6 +556,9 @@ ufs_setattr(ap) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0& (SF_NOUNLINK | S= F_IMMUTABLE | SF_APPEND) || > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(vap->va_flags & U= F_SETTABLE) !=3D vap->va_flags) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return (EP= ERM); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if ((ip->i_flags & SF_SETTA= BLE) !=3D > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (vap->va_flags & SF= _SETTABLE)) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 return (EPE= RM); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->i_flags &=3D SF_SETTAB= LE; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->i_flags |=3D (vap->va_= flags & UF_SETTABLE); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0DIP_SET(ip, i_flags, ip->i= _flags); > %%% > > The patch has a potential to break something if someone assumes that > non-super-user can modify UF_SETTABLE flags with the SF_SETTABLE part > set to zero. However with a quick peek this seems to be what NetBSD > does. This seems ok in terms of the original issue that Alexander reported. Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 8 00:53:47 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B223106564A; Fri, 8 Oct 2010 00:53:47 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id DE4D28FC12; Fri, 8 Oct 2010 00:53:46 +0000 (UTC) Received: from [208.206.78.30] (port=38715 helo=dt.vicor.com) by postoffice.vicor.com with esmtpsa (SSLv3:RC4-MD5:128) (Exim 4.71) (envelope-from ) id 1P41DE-0003kV-DG; Thu, 07 Oct 2010 17:53:32 -0700 From: Devin Teske To: Stefan Esser In-Reply-To: <4CADE8E4.3060909@FreeBSD.org> References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <4CADE8E4.3060909@FreeBSD.org> Content-Type: text/plain Organization: Vicor, Inc Date: Thu, 07 Oct 2010 17:53:28 -0700 Message-Id: <1286499208.17025.91.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.0.2 (2.0.2-41.el4) Content-Transfer-Encoding: 7bit X-Scan-Signature: 99fa2bdd052573190db94e4de4bc9d6b X-Scan-Host: postoffice.vicor.com Cc: freebsd-hackers@freebsd.org, Devin Teske Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Oct 2010 00:53:47 -0000 On Thu, 2010-10-07 at 17:36 +0200, Stefan Esser wrote: > Am 07.10.2010 05:29, schrieb Devin Teske: > > > > On Oct 6, 2010, at 4:09 PM, Brandon Gooch wrote: > > > >> On Wed, Oct 6, 2010 at 3:45 PM, Devin Teske wrote: > >>> Hello fellow freebsd-hackers, > >>> > >>> Long-time hacker, first-time poster. > >>> > >>> I'd like to share a shell script that I wrote for FreeBSD system > >>> administration. > >>> > >> > >> It seems the list ate the attachment :( > > > > > > Here she is ^_^ Comments welcome. > > Hi Devin, > > I think your script is quite useful in specific situations, but getting > it into FreeBSD may prove harder than you might have expected. > > But it should be easy to get it into the ports tree, if it proves > useful it may later make it into a new release (and be merged back) ... I'm willing to see that through to fruition. Should be fun. > Since I'm travelling with no access to my FreeBSD systems, I can not > verify all details in my reply, but I hope it is still useful for you. > > Let me start with a few remarks: > > Your script is really useful for querying of rc.conf (et al.) settings, > less so for modifications to the files (IMHO). The reasons are many: > > 1) Changes to rc.conf do not take effect, immediately (different from > sysctl(1) settings). Funny that you would mention that. How this script came-to-be is that I first developed the internal functions (sysrc() and sysrc_set()) for a larger script called "host- setup" which I'm programming for our company to allow field-engineers to setup their system using /usr/bin/dialog (dialog(1)). This larger script uses the internal sysrc() function to query values before presenting them to the user. Conversely, when the user invokes the editor to change the value, the script does the following -- say, for example, in changing the hostname: 1. It uses sysrc_set() to change the value in the rc.conf (or rc.conf.local -- or any other file mentioned in $rc_conf_files within /etc/defaults/rc.conf) 2. It then re-reads the value from the system configuration files. 3. Calls hostname(1) with the new value to make it effective without a reboot. So, in reality, this script (sysrc(8)) could be used by other scripts/programs in much the same way I use it -- as a way to munge the rc.conf(5) files *in addition to using standard utilities such as route (8), hostname(1), ifconfig(8) and many others. I agree that teaching this tool how to make changes effective without a reboot is neither warranted nor desired. Since each directive that can be munged by this utility indeed pertains to a different subsystem (some modify the behavior of /etc/rc, some modify the behavior or /etc/rc.sendmail, some /etc/rc.firewall, some /etc/rc. some modify the behaviour of /etc/rc.d/*), it's beyond the scope of this utility to know what to do with the given setting once changed to make it effective without a reboot. However, I will say that in a large-scale deployment, simply having an ability to do the following is still beneficial: for host in host1 host2 host3 host4 host5 ...; do ssh $host /bin/sh -c "'sysrc ntpdate_flags="..." && reboot'" done or, if you really know your shell-foo: for host in host1 host2 host3 host4 host5 ...; do ssh $host /bin/sh -c "'sysrc ntpdate_flags="..." && service ntpdate stop && service ntpdate start'" done > 2) Changes to rc.conf are persistent and need not be applied on each > reboot (similar to changes to sysctl.conf, but different from > sysctl(1)) I'm not sure what you mean by "changes ... are persistent", but yes, I do agree that the average lay person is convinced that a reboot is required to make any change to the rc.conf files effective (however, you and I perhaps know better). It should also be noted that sysctl.conf(5) does not require a reboot to re-apply the settings contained within. These days, one can run the following command to reload sysctl.conf(5): service sysctl reload Conversely, for a specific example of rc.conf(5) being read continuously after boot, we often have to restart apache, and we do-so like this: /usr/local/etc/rc.d/httpd restart The httpd script in /usr/local/etc/rc.d utilizes all the usual suspects in rc(8) rcorder(8) and `/etc/rc.subr'. In fact, just about every script in either /etc/rc.d or /usr/local/etc/rc.d will re-read /etc/rc.conf when run. No reboot required, just munge the file and run your rc(8) script either directly or via service(8). > 3) While there is a specified order of evaluation when reading from > rc.conf files, it is not obvious that writing to the last file in > $rc_conf_files that defined some parameter (or the first one, if > the parameter is not currently defined) is the right thing to do. > (E.g. if there is a rc.conf and rc.conf.local, it may well be the > latter that should receive new parameter definitions.) I've have previous incarnations of these utilities that DID go through each and every file listed in rc_conf_files within /etc/defaults/rc.conf and clean-up ALL instances of the variable, however having been bitten by that, I think that it's better to replace only the last occurrence (the one that takes effect when source_rc_confs is called). We have at certain sites set up SUP (the Software Update Protocol -- the precursor to cvsup and csup in many ways) and configured cron-jobs to SUP certain files off of a central server. One of those files is /etc/rc.conf So what we would do, is to have all the configuration settings that apply to a given site put into /etc/rc.conf and all the settings that pertain to the local machine into /etc/rc.conf.local What this achieves is that -- since /etc/rc.conf is automatically shared among all machines in a given LAN -- we can plunk-down settings that we feel should apply to all machines in the site (like ntpdate_flags -- pointing ntpdate to a given server, allowing all machines in the site to sync up to a single time server), meanwhile, if a single machine needs to sync up with a different time-server, we can have ntpdate_flags in the local machines /etc/rc.conf.local override the value provided in /etc/rc.conf If we had sysrc munge ALL files rather than the last one, then we'd be breaking the setup scheme where /etc/rc.conf is to not be modified by- hand. > > 4) I seem to remember that there was support for a rc.conf.d directory, > but I'm not sure and cannot easily check this, right now. Yes, /etc/rc.subr in the load_rc_config() function will source a file according to the rc(8) script name (passed in as the first argument of load_rc_config()). Unfortunately, I think that the lack of context in the case of sysrc makes it impractical to go searching through all /etc/rc.conf.d/* files for some configuration value. sysrc isn't passed a name of a service the way that load_rc_config is. Though I agree that the rc.conf.d scripts shouldn't be neglected. For example, if /etc/rc.conf/httpd help the following text: httpd_enable="YES" And a user wanted to either query or change this setting, currently the sysrc script will say: $ sysrc httpd_enable sysrc: unknown variable 'httpd_enable' Because /etc/rc.conf.d isn't checked (as should be expected because we're not given a context of a service name). Though, perhaps via some "-f file" option to sysrc, we can bridge the gap... $ sysrc -f /etc/rc.conf.d httpd_enable httpd_enable: YES Or, prehaps: $ sysrc -n httpd httpd_enable httpd_enable: YES (envisioning "-n name" to cause the script to source /etc/rc.conf.d/name after sourcing the gammut of $rc_conf_files via source_rc_confs() in /etc/defaults/rc.conf). What do you think? I like both ideas. > Some suggestions in random order: > > a) Allow the user to specify files to operate on (with the defaults > set as in your current script). This would allow to edit sysctl.conf > (which has the same syntax as rc.conf) with your script, among other > things (e.g. to specify the rc.conf file to write to). I like that idea. I'm also thinking perhaps of adding a chroot(8) mechanism which would be useful for working in jails, however that may not be necessary since we now have jexec(8) these days. One could simply do: jexec JID sysrc hostname=foo jexec JID /bin/sh -c 'hostname "$( sysrc hostname )"' The first command performs the sysrc command within the jail, causing the rc.conf(5) files to be munged. The second command makes the change effective. > b) Allow the user to dump multiple parameters selected by a pattern or > all parameters (support for a wildcard and "-a" option to sysctl). > I understand that patterns and the shell variable expansion options > that can be used in parameter names (e.g. 'hostname%%.*' given as > example in your script) may conflict, but it would still be very > useful, for example to print all "ifconfig_*" values). I really thought long and hard about this one. I started out wanting it, but then changed my mind. In truth, because I was mirroring sysctl(8), I thought it would be rather pleasant to have a "-a" flag that dumped all the non-default values. This is very doable. I just had to sleep on it awhile to determine what I wanted to do (show values for each file? or show overall end-result values perloined by sourcing all of the files in a row, dumping only the effective values of each/every variable, thus producing a comprehensive list of directives that the system will see upon boot-up (in-turn providing the system administrator with a quick snapshot of how the system will boot -- both valuable for problem trouble shooting and also for double-checking your work before you reboot). What do you think? > c) Let user print comments for parameters (possibly selected by pattern) > as present in the defaults/rc.conf file. This should for example > allow to print the descriptions for "*ip6*" to help identify the > parameter to modify. (I think the "-d" option of sysctl dumps the > descriptions instead of or together with the values selected.) I really really like that idea! This brings us even closer to the operation of sysctl(8). This alone could make this script double it's current value (by giving us all a faster way to research possible directives that can be thrown into the rc_conf_files). I envision people using -d to quickly find out of a value is valid and to get a quick description of what it does and also show the default value. Simply displaying the default value is useful to even the most seasoned veterans because it answers the question: "do I need to customize the default value?" as quickly as possible. Of course, `-d' is currently used for remediating external dependencies out of the script (passing `-d' will cause the script to dump a list of internal dependencies upon exit-for-any-reason, producing a table of barewords that are used within the script and their associative meanings -- such as "shell keyword", "shell built-in", "shell function", "tracked alias' to external executables" etc.). > d) Add an option to report the file(s) where parameters are defined. > In fact, there are three cases of interest: 1) all files that set > the parameter, 2) the file from which the value will be used (the > last one from the list returned by 1) I guess), and 3) the file > that would be modified if the value was to be written. I really wanted to option too, but didn't want to delay the release of this script to the public domain any further. I'm envisioning a `-v' parameter for verbosity as well as an inheritable environment setting SYSRC_VERBOSE to do the same -- enable the logging of which file was ultimately queried for the answer or which file was ultimately munged in a set-request (via `name=value' syntax). > For a script imported as a port, there is no need for "have" and > "depend", since dependencies will be resolved when installing the > port (and you only use base functionality, anyway). I understand that > there helper functions make maintenance and porting to other systems > easier, but I'm quite sure they will not be accepted if the script is > ever to be imported into FreeBSD itself. I agree fully, 100%. The have(), depend(), and show_deps() functions in conjunction with the _depend, SYSRC_SHOW_DEPS, and other related variables have no home if this script makes it into the FreeBSD distribution. However, what these scripts provide right now is the following: a. I find it very useful both to myself and to less-experienced shell developers to have a call to depend() listing all dependencies for the following block of code prior to the actual execution of said block BECAUSE it helps readers know what is required should they want to rip out that block for their own use in their own code in their own shell script. For example, many times when I write shell scripts that accomplish complex tasks using low-level shell features, there is often people that desire to re-use a handy shell function but often forget to grab some other dependent function defined further up or farther below. The depend() syntax helps remind folks what is required for each block of code. b. Help determine whether a given environment is suitable for the script by first and foremost providing human-readable errors that give the user a succinct and disambiguous error message -- as we all know that sometimes a simple error of "grepfoo: command not found" is ultimately helpful, but when you have a multi-hundred line shell script it's handy to know who was trying to use `grepfoo' (the first argument of the depend function gives the user that clue by issuing instead error statements like "Missing dependency 'grepfoo' required by mygrep" -- developer knows that the `mygrep' shell function either needs to use have() -- aka type -- to work around cases where grepfoo may not be a valid command). This is invaluable when programming for embedded environments such as the crunchgen(1)'d mfsroot boot ``floppy image'' generated by the release(7) process documented in the FreeBSD Release Engineering process detailed at http://www.freebsd.org/doc/en_US.ISO8859-1/articles/releng/release- build.html ). c. These functions actually allow this script to run on foreign operating systems. For example, one can run this script on Linux. It will likely *ALWAYS* say "unknown variable '...'" since /etc/defaults/rc.conf doesn't exist on any OS except BSD derivatives of a specific flavor. Naturally... all of the above-mentioned benefits are nullified if/when sysrc gets slurped into the the base distribution. I have no qualms with judiciously removing all the depend(), have() and show_deps() stuff if it seems like this script has a shot at making it into the base distribution. Though, ultimately, I envisioned the sysrc() and sysrc_set() functions being slurped into /etc/rc.subr where they can be used by rc(8) scripts that naturally source /etc/rc.subr to get access to beauties such as load_rc_config(). After-which the sysrc(8) script would be reprogrammed to (in a FreeBSD fashion) source /etc/rc.subr and then use the functions as defined in there. > In either case, the comments relevant for usage of the script (e.g. > those below "Command Usage", "usage" and "sysrc $setting") should be > extracted into a man page. The rest of the commands and the portability > support would probably removed after an import into FreeBSD (keeping > them in the "vendor import" files, but none of the versions that are > extracted from the repository and used in FreeBSD). I do agree that the comments are ... excessive =) My desire was to get out the initial version of the script to the public domain (in 1.0-version fashion) in it's primordial state. I left all that stuff in there (the copious comments, the helper functions -- have (), depend(), and show_deps() -- etc.) to be a beacon of shining glory to budding shell-scripters. Though, in contrast to the agreed immediate removal of the helper functions, it would indeed be a shame to carte-blanche remove the copious documentation embedded within. You are right that the inline documentation really belongs in a *roff- formatted man-page, but I hadn't considered "vendor import" files as another possible destination. > I could perform the import of "sysrc" as a port, but I guess there > will be quite some discussion during the next few days and there > might be others interested in being maintainer of such port or there > might even be consensus to import it into the source tree ... I'm grateful for the offer. I've not yet been blessed (or should I say -- cursed) with the direct commit privileges, but am working on it (by first submitting things -- I'm told by many friends whom _do_ have commit privs that the process is essentially thusly stated, "find someone willing to commit code on your behalf and when they grow tired of managing your code, they'll curse you with direct commit privileges" ^_^, do you remember telling me that Julian? lol). > Best regards, STefan > > PS: This reply is not sent to the list, but you are free to reply > to the list and quote my message. I think there's enough conversation-worthy notes here that it's worth CC'ing the list back-in. Thanks for giving me the heads up that it's ok (I would have asked first if you hadn't). P.S. I think it's safe to say I chose the right mailing-list to post this stuff to (I scoured the list and read all the charters before chosing which list to burden with this script). So far so good ^_^ -- Cheers, Devin Teske -> CONTACT INFORMATION <- Business Solutions Consultant II FIS - fisglobal.com 510-735-5650 Mobile 510-621-2038 Office 510-621-2020 Office Fax 909-477-4578 Home/Fax devin.teske@fisglobal.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> FUN STUFF <- -----BEGIN GEEK CODE BLOCK----- Version 3.1 GAT/CS d(+) s: a- C++(++++) UB++++$ P++(++++) L++(++++) !E--- W++ N? o? K- w O M+ V- PS+ PE Y+ PGP- t(+) 5? X+(++) R>++ tv(+) b+(++) DI+(++) D(+) G+>++ e>+ h r>++ y+ ------END GEEK CODE BLOCK------ http://www.geekcode.com/ -> END TRANSMISSION <- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 8 08:23:31 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0F811065672; Fri, 8 Oct 2010 08:23:31 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BA0FA8FC2B; Fri, 8 Oct 2010 08:23:30 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA01056; Fri, 08 Oct 2010 11:23:29 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P48Ej-000AFi-8D; Fri, 08 Oct 2010 11:23:29 +0300 Message-ID: <4CAED500.3090001@freebsd.org> Date: Fri, 08 Oct 2010 11:23:28 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: hackers@freebsd.org References: <4CAE0457.8040900@freebsd.org> In-Reply-To: <4CAE0457.8040900@freebsd.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: Re: zfs: ru_inblock/ru_oublock accounting X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Oct 2010 08:23:31 -0000 on 07/10/2010 20:33 Andriy Gapon said the following: > > A simple, probably incomplete and perhaps incorrect patch for > ru_inblock/ru_oublock accounting in zfs: > http://people.freebsd.org/~avg/zfs-ru.diff I've updated the patch at the same location. Thanks to "swell.k" for pointing out that the changes are applicable in kernel only. > Still quite better than nothing. > > P.S. > This is about top -m io. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 8 09:11:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B848A106566B; Fri, 8 Oct 2010 09:11:31 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 6A4578FC0A; Fri, 8 Oct 2010 09:11:29 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA01714; Fri, 08 Oct 2010 12:11:28 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P48zA-000AID-HT; Fri, 08 Oct 2010 12:11:28 +0300 Resent-From: Andriy Gapon Resent-To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-hackers@freebsd.org, freebsd-ports@freebsd.org Resent-Date: Fri, 8 Oct 2010 12:11:28 +0300 Resent-Message-Id: <4CAEE040.8020401@freebsd.org> Resent-User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 Message-ID: <4CAEDF48.1030602@freebsd.org> Date: Fri, 08 Oct 2010 12:07:20 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-scsi@freebsd.org References: <4BCDEBF6.3030609@icyb.net.ua> <20100423184412.GA5016@nargothrond.kdm.org> <4BD1FC74.3090504@freebsd.org> <4CA30B24.8040707@freebsd.org> In-Reply-To: <4CA30B24.8040707@freebsd.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-UID: 18468 X-Keywords: Cc: Subject: [HEADSUP] changes to cam_get_device() and cam_open_device() X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Oct 2010 09:11:31 -0000 As there was no objections, I am going to commit changes to cam_get_device() that remove the following features: - ignoring 'r' and 'n' at the start of device name - ignoring whitespace at end of device name - parsing and ignoring slice and partition names in a device name cam(3) manual page is going to be updated to reflect the changes. This would simplify the code and disambiguate its behavior. Non-rewound and character disk/SCSI devices has not been supported for quite a while now. Support for parsing partition/slice names is incomplete (e.g. GPT scheme is not supported) and of questionable usefulness. Full diff is here: http://people.freebsd.org/~avg/cam.diff If you know of any functionality or application that would be broken by this change please stop me ASAP. Also, if you have any other reason to object to the change please speak up before the change is committed. Thank you. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 8 14:32:25 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 681B41065694 for ; Fri, 8 Oct 2010 14:32:25 +0000 (UTC) (envelope-from danger@FreeBSD.org) Received: from services.syscare.sk (services.syscare.sk [188.40.39.36]) by mx1.freebsd.org (Postfix) with ESMTP id 1E5CE8FC1A for ; Fri, 8 Oct 2010 14:32:24 +0000 (UTC) Received: from services.syscare.sk (services [188.40.39.36]) by services.syscare.sk (Postfix) with ESMTP id 61C3D4C97D for ; Fri, 8 Oct 2010 16:12:46 +0200 (CEST) X-Virus-Scanned: amavisd-new at rulez.sk Received: from services.syscare.sk ([188.40.39.36]) by services.syscare.sk (services.rulez.sk [188.40.39.36]) (amavisd-new, port 10024) with ESMTP id hx8gFdYb9aHY for ; Fri, 8 Oct 2010 16:12:44 +0200 (CEST) Received: from danger-mbp.local (188-167-67-67.dynamic.chello.sk [188.167.67.67]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: danger@rulez.sk) by services.syscare.sk (Postfix) with ESMTPSA id 53F254C968 for ; Fri, 8 Oct 2010 16:12:44 +0200 (CEST) Message-ID: <4CAF26DB.3060706@FreeBSD.org> Date: Fri, 08 Oct 2010 16:12:43 +0200 From: Daniel Gerzo Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.12pre) Gecko/20101003 Lanikai/3.1.5pre MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <4CAD513F.3010903@DataIX.net> <4CAD7563.1070706@DataIX.net> <87fwwici02.wl%rand@meridian-enviro.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Oct 2010 14:32:25 -0000 On 7.10.2010 15:42, Warren Block wrote: > Consider also the docs that tell the user to > echo 'something_enable="YES"' >> /etc/rc.conf > > which can produce duplicate and possibly differing entries. Or > non-working entries if there was no ending \n present, or even a broken like forgetting to add two '>' and not having a backup :) > rc.conf that breaks startup. -- S pozdravom / Best regards Daniel Gerzo From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 8 19:08:45 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FD631065775; Fri, 8 Oct 2010 19:08:45 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id A339D8FC15; Fri, 8 Oct 2010 19:08:44 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 48C9246B32; Fri, 8 Oct 2010 15:08:44 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3FFBC8A04F; Fri, 8 Oct 2010 15:08:43 -0400 (EDT) From: John Baldwin To: freebsd-current@freebsd.org Date: Fri, 8 Oct 2010 15:01:18 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20100819; KDE/4.4.5; amd64; ; ) References: <4CAE0621.3070809@freebsd.org> In-Reply-To: <4CAE0621.3070809@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201010081501.18870.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 08 Oct 2010 15:08:43 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: hackers@freebsd.org, Andriy Gapon Subject: Re: panic_cpu should be volatile X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Oct 2010 19:08:45 -0000 On Thursday, October 07, 2010 1:40:49 pm Andriy Gapon wrote: > > panic_cpu variable in kern_shutdown.c should be volatile otherwise it's cached in > a register in the innermost while-loop in this code (observed on amd64 with base > gcc and -O2): > if (panic_cpu != PCPU_GET(cpuid)) > while (atomic_cmpset_int(&panic_cpu, NOCPU, > PCPU_GET(cpuid)) == 0) > while (panic_cpu != NOCPU) > ; /* nothing */ > > The patch is here: > http://people.freebsd.org/~avg/panic_cpu.diff > > I also took a liberty to move the variable into the scope of panic() functions as > it doesn't seem to be useful outside of it. But this is not necessary, of course. Looks fine to me. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 8 19:08:48 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6AEE610657DE; Fri, 8 Oct 2010 19:08:48 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3CDF28FC18; Fri, 8 Oct 2010 19:08:48 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id E19C446B1A; Fri, 8 Oct 2010 15:08:47 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id EA0528A04E; Fri, 8 Oct 2010 15:08:46 -0400 (EDT) From: John Baldwin To: Garrett Cooper Date: Fri, 8 Oct 2010 15:06:25 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20100819; KDE/4.4.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201010081506.26087.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 08 Oct 2010 15:08:46 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: freebsd-hackers@freebsd.org, sbruno@freebsd.org, scottl@freebsd.org Subject: Re: Fix mfiutil compile with -DDEBUG X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Oct 2010 19:08:48 -0000 On Sunday, October 03, 2010 10:33:17 pm Garrett Cooper wrote: > make -DDEBUG is broken in mfiutil: > > $ make -DDEBUG > cc -O2 -pipe -fno-strict-aliasing -pipe -O2 -march=nocona > -fno-builtin-strftime -DDEBUG -Wall -Wcast-align -Woverflow > -Wsign-compare -Wunused -std=gnu99 -fstack-protector -Wsystem-headers > -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter > -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wno-uninitialized -Wno-pointer-sign -c > /usr/src/usr.sbin/mfiutil/mfi_config.c > /usr/src/usr.sbin/mfiutil/mfi_config.c: In function 'dump_config': > /usr/src/usr.sbin/mfiutil/mfi_config.c:1027: error: 'union mfi_pd_ref' > has no member named 'device_id' > /usr/src/usr.sbin/mfiutil/mfi_config.c:1083: error: 'union mfi_pd_ref' > has no member named 'device_id' > *** Error code 1 > > Stop in /usr/src/usr.sbin/mfiutil. > $ > > device_id is a field in the v field in the mfi_pd_ref union > (/sys/dev/mfi/mfireg.h): > > union mfi_pd_ref { > struct { > uint16_t device_id; > uint16_t seq_num; > } v; > uint32_t ref; > } __packed; Yes, there were different versions of these definitions in mfireg.h at one point. Your patch is fine. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 8 19:08:49 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C661106583B; Fri, 8 Oct 2010 19:08:49 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 06B2D8FC1C; Fri, 8 Oct 2010 19:08:49 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id B32F946B89; Fri, 8 Oct 2010 15:08:48 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id E15178A051; Fri, 8 Oct 2010 15:08:47 -0400 (EDT) From: John Baldwin To: Garrett Cooper Date: Fri, 8 Oct 2010 15:07:05 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20100819; KDE/4.4.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201010081507.06168.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 08 Oct 2010 15:08:47 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: freebsd-hackers@freebsd.org, sbruno@freebsd.org, scottl@freebsd.org Subject: Re: Make mfiutil(8) more robust X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Oct 2010 19:08:49 -0000 On Sunday, October 03, 2010 11:32:01 pm Garrett Cooper wrote: > On Sun, Oct 3, 2010 at 8:30 PM, Garrett Cooper wrote: > > As discussed offlist with some of the Yahoo! FreeBSD folks, > > mfiutil catches errors, but doesn't communicate it back up to the > > executing process. Examples follow... > > Before: I think these are both fine. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 9 08:28:18 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55AEE106566B for ; Sat, 9 Oct 2010 08:28:18 +0000 (UTC) (envelope-from jeremie@le-hen.org) Received: from smtp5-g21.free.fr (smtp5-g21.free.fr [212.27.42.5]) by mx1.freebsd.org (Postfix) with ESMTP id 8D1258FC13 for ; Sat, 9 Oct 2010 08:28:15 +0000 (UTC) Received: from endor.tataz.chchile.org (unknown [82.233.239.98]) by smtp5-g21.free.fr (Postfix) with ESMTP id 47B7CD4810C; Sat, 9 Oct 2010 10:28:09 +0200 (CEST) Received: from felucia.tataz.chchile.org (felucia.tataz.chchile.org [192.168.1.9]) by endor.tataz.chchile.org (Postfix) with ESMTP id 2ABCF33D42; Sat, 9 Oct 2010 08:28:08 +0000 (UTC) Received: by felucia.tataz.chchile.org (Postfix, from userid 1000) id 1993CA12C9; Sat, 9 Oct 2010 08:28:08 +0000 (UTC) Date: Sat, 9 Oct 2010 10:28:08 +0200 From: Jeremie Le Hen To: Bruce Cran Message-ID: <20101009082807.GA25457@felucia.tataz.chchile.org> References: <20100804141858.00003610@unknown> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20100804141858.00003610@unknown> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@freebsd.org, Samuel =?iso-8859-1?Q?Mart=EDn?= Moro Subject: Re: behaviour changes in mdconfig? or something related? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Oct 2010 08:28:18 -0000 Hi, On Wed, Aug 04, 2010 at 02:18:58PM +0100, Bruce Cran wrote: > On Wed, 4 Aug 2010 14:40:27 +0200 > Samuel Martín Moro wrote: > > > Since 8.1 (8.0?), after calling bsdlabel, I still have /dev/${dev}a, > > but/dev/${dev}c doesn't show up anymore. > > The 'c' partition is no longer created on FreeBSD 8 - you should > use /dev/${dev} instead of /dev/${dev}c : > http://www.freebsddiary.org/upgrade8.php . It would be nice to have devd(8) create a symlink from ${dev}c to ${dev} in /dev. Regards, -- Jeremie Le Hen Humans are born free and equal. But some are more equal than others. Coluche From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 9 17:03:47 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 168171065673 for ; Sat, 9 Oct 2010 17:03:47 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 052768FC1C for ; Sat, 9 Oct 2010 17:03:45 +0000 (UTC) Received: by iwn8 with SMTP id 8so2576834iwn.13 for ; Sat, 09 Oct 2010 10:03:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=W76H0OErhdJGnVDXBvEF98TBZQX3cRopfhqk4zPWa70=; b=mV86AiTndkM1CxEW4hCe9hbrqODuPqQWRv9sGxI1EQM65p+8ezYO+cayHGMYBLca/T XrH78a7XXhggNXRAkeSy4m5/Rv7HaLwL1J1cpicroe/mekU3VE7Dohv3kxGhiyILUDxA hmw+2NGmlaUBKkNvG493v5EYJv2OhL2OzuZ7Q= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=D8cd4r2NR5WOxZI7sMYiAgHILg9L4cRRPaXw3C5EaBblwpl8AdGQIdTcEAR8Gmhl+Z zaVotUW23HE632jkGXdi9RanIkO5964OcyXA2OYcVb7NnFyx8eW8zCwrkpYKqD6jShJi +TdRb/MqZIIoyc5WjIUNj+KmrIlohw+3bzcnE= MIME-Version: 1.0 Received: by 10.42.211.131 with SMTP id go3mr755294icb.393.1286643824723; Sat, 09 Oct 2010 10:03:44 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.184.3 with HTTP; Sat, 9 Oct 2010 10:03:44 -0700 (PDT) In-Reply-To: <4CAD7CA1.4050202@freebsd.org> References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <4CAD513F.3010903@DataIX.net> <4CAD7563.1070706@DataIX.net> <4CAD7CA1.4050202@freebsd.org> Date: Sat, 9 Oct 2010 10:03:44 -0700 X-Google-Sender-Auth: 9KEHVzlvVHr7acvm4ayhCNDpyVk Message-ID: From: Garrett Cooper To: Julian Elischer Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Brandon Gooch , Devin Teske , freebsd-hackers@freebsd.org Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Oct 2010 17:03:47 -0000 On Thu, Oct 7, 2010 at 12:54 AM, Julian Elischer wrote= : > =A0On 10/7/10 12:23 AM, jhell wrote: >> >> Alright thank you for your explanation. I do not normally see this usage >> and this just sort of stood out at me and I did not want to assume what >> you were trying to accomplish, without asking. > > three useage cases come to mind immediately. > > 1/ use within other scripts.. > =A0instead of the dozens of homegrown solutions people have written for p= uting > something > into /etc/rc.conf =A0one can use this. > > 2/ what is the value of X on machines a,b,c > =A0foreach machine in a b c > =A0do > =A0 =A0 =A0ssh $machine sysrc X > =A0done > > =A0 =A0you may well say "you could have used grep" bu tgrep doesn't give = the > default value vie the > =A0 hierachy of .rc files. > 2A is of course to correc teh values found to be wrong with (2) > > 3/ on a really small system, without an editor this may do a cleaner job > than the usual > =A0"grep -v X /etc/rc.conf >/tmp/x;echo X >> /tmp/x; mv /tmp/x /etc/rc.co= nf" I was going to say... 3A On a system where you're logged in via singleuser, sometimes terminal settings don't work correctly with editors (these days it's mostly because /usr isn't available so it can't load ncurses apps, some libs, termcap, etc). That would be a lifesaver in this case. But then I realized that this command probably would live in /usr/sbin and would probably need other apps in /usr/bin // /usr/sbin to run this command :). Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 9 17:37:03 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D28161065670 for ; Sat, 9 Oct 2010 17:37:03 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id B2E378FC08 for ; Sat, 9 Oct 2010 17:37:03 +0000 (UTC) Received: from [71.202.142.31] (port=61821 helo=[192.168.1.149]) by postoffice.vicor.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.71) (envelope-from ) id 1P4dLj-0004Uj-6l; Sat, 09 Oct 2010 10:36:49 -0700 Mime-Version: 1.0 (Apple Message framework v1081) From: Devin Teske In-Reply-To: Date: Sat, 9 Oct 2010 10:36:46 -0700 Message-Id: <0206CB74-6E83-4762-B26D-F95E024C90F0@vicor.com> References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <4CAD513F.3010903@DataIX.net> <4CAD7563.1070706@DataIX.net> <4CAD7CA1.4050202@freebsd.org> To: Garrett Cooper X-Mailer: Apple Mail (2.1081) X-Scan-Signature: ef683717e6f5a42ea3965c5696ed7059 X-Scan-Host: postoffice.vicor.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Brandon Gooch , freebsd-hackers@freebsd.org Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Oct 2010 17:37:04 -0000 On Oct 9, 2010, at 10:03 AM, Garrett Cooper wrote: > On Thu, Oct 7, 2010 at 12:54 AM, Julian Elischer = wrote: >> On 10/7/10 12:23 AM, jhell wrote: >>>=20 >>> Alright thank you for your explanation. I do not normally see this = usage >>> and this just sort of stood out at me and I did not want to assume = what >>> you were trying to accomplish, without asking. >>=20 >> three useage cases come to mind immediately. >>=20 >> 1/ use within other scripts.. >> instead of the dozens of homegrown solutions people have written for = puting >> something >> into /etc/rc.conf one can use this. >>=20 >> 2/ what is the value of X on machines a,b,c >> foreach machine in a b c >> do >> ssh $machine sysrc X >> done >>=20 >> you may well say "you could have used grep" bu tgrep doesn't give = the >> default value vie the >> hierachy of .rc files. >> 2A is of course to correc teh values found to be wrong with (2) >>=20 >> 3/ on a really small system, without an editor this may do a cleaner = job >> than the usual >> "grep -v X /etc/rc.conf >/tmp/x;echo X >> /tmp/x; mv /tmp/x = /etc/rc.conf" >=20 > I was going to say... >=20 > 3A On a system where you're logged in via singleuser, sometimes > terminal settings don't work correctly with editors (these days it's > mostly because /usr isn't available so it can't load ncurses apps, > some libs, termcap, etc). That would be a lifesaver in this case. I hadn't realized that. That's stupendous! Though I do believe that `/rescue/vi' exists these days as a = statically-linked binary (built using crunchgen(1) via = `rescue/rescue/Makefile' in the FreeBSD source tree). > But then I realized that this command probably would live in > /usr/sbin and would probably need other apps in /usr/bin // /usr/sbin > to run this command :). I envisioned it living in `/sbin' and it's actually free of dependencies = to external executables. If you execute it with the "-d" option ('d' for = 'dependencies'), you can see that it uses only shell internals. The only = thing this script truly depends on is `/bin/sh'. So it sounds like it = would fit the ability of working within single-user mode quite well = (remember, I developed it for embedded systems -- where things like = grep/sed/awk may even be missing). > Thanks! > -Garrett > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" -- Cheers, Devin Teske -> CONTACT INFORMATION <- Business Solutions Consultant II FIS - fisglobal.com 510-735-5650 Mobile 510-621-2038 Office 510-621-2020 Office Fax 909-477-4578 Home/Fax devin.teske@fisglobal.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> FUN STUFF <- -----BEGIN GEEK CODE BLOCK----- Version 3.1 GAT/CS d(+) s: a- C++(++++) UB++++$ P++(++++) L++(++++) !E--- W++ N? o? = K- w O M+ V- PS+ PE Y+ PGP- t(+) 5? X+(++) R>++ tv(+) b+(++) DI+(++) D(+) G+>++ = e>+ h r>++ y+=20 ------END GEEK CODE BLOCK------ http://www.geekcode.com/ -> END TRANSMISSION <- From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 9 18:02:02 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8A3F1065673 for ; Sat, 9 Oct 2010 18:02:02 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 863D18FC12 for ; Sat, 9 Oct 2010 18:02:02 +0000 (UTC) Received: by iwn8 with SMTP id 8so2631393iwn.13 for ; Sat, 09 Oct 2010 11:02:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=jJiQFbjRCz3Uu7lE1CeponxLmseX3+hZV0MlU6A43fI=; b=c+CAO1yQzfZZvhh/B3XyeMqIA/am2elnn78NXmuKx/0zjpinyn/Puboje+g5O7yToM ggpbGdQcuEwH/VC8x8NnqbPgmproRdzNDqXSBo1ZyKfyN+YUUgSaMg4DFDG8031r79d3 CpFsMTmQMLO0Imqkf3vy0ytLpydgUAGs96WLU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=GWf7hW9+v4DgiqqBsCPFLhnhmlFvA4gNjiuTEEAh9LQiq//BxE3fStv3S+wiAH5DbO riI5EK2hOvwecXo1TZ5ko59Egx93RH9TGAJ0UY7NdHvYqAOyICRf7isRP41BGLGqpb8T ueusSIzHgbQj3wnaBQ8BZT8xOK4J8qgWxnLs4= MIME-Version: 1.0 Received: by 10.42.215.134 with SMTP id he6mr777960icb.527.1286647321731; Sat, 09 Oct 2010 11:02:01 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.184.3 with HTTP; Sat, 9 Oct 2010 11:02:01 -0700 (PDT) In-Reply-To: <0206CB74-6E83-4762-B26D-F95E024C90F0@vicor.com> References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <4CAD513F.3010903@DataIX.net> <4CAD7563.1070706@DataIX.net> <4CAD7CA1.4050202@freebsd.org> <0206CB74-6E83-4762-B26D-F95E024C90F0@vicor.com> Date: Sat, 9 Oct 2010 11:02:01 -0700 X-Google-Sender-Auth: yYkpLGhQRHLKAb1qQmrc_ktaJv8 Message-ID: From: Garrett Cooper To: Devin Teske Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Brandon Gooch , freebsd-hackers@freebsd.org Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Oct 2010 18:02:02 -0000 On Sat, Oct 9, 2010 at 10:36 AM, Devin Teske wrote: > > On Oct 9, 2010, at 10:03 AM, Garrett Cooper wrote: > >> On Thu, Oct 7, 2010 at 12:54 AM, Julian Elischer wr= ote: >>> =A0On 10/7/10 12:23 AM, jhell wrote: >>>> >>>> Alright thank you for your explanation. I do not normally see this usa= ge >>>> and this just sort of stood out at me and I did not want to assume wha= t >>>> you were trying to accomplish, without asking. >>> >>> three useage cases come to mind immediately. >>> >>> 1/ use within other scripts.. >>> =A0instead of the dozens of homegrown solutions people have written for= puting >>> something >>> into /etc/rc.conf =A0one can use this. >>> >>> 2/ what is the value of X on machines a,b,c >>> =A0foreach machine in a b c >>> =A0do >>> =A0 =A0 =A0ssh $machine sysrc X >>> =A0done >>> >>> =A0 =A0you may well say "you could have used grep" bu tgrep doesn't giv= e the >>> default value vie the >>> =A0 hierachy of .rc files. >>> 2A is of course to correc teh values found to be wrong with (2) >>> >>> 3/ on a really small system, without an editor this may do a cleaner jo= b >>> than the usual >>> =A0"grep -v X /etc/rc.conf >/tmp/x;echo X >> /tmp/x; mv /tmp/x /etc/rc.= conf" >> >> =A0 =A0I was going to say... >> >> 3A On a system where you're logged in via singleuser, sometimes >> terminal settings don't work correctly with editors (these days it's >> mostly because /usr isn't available so it can't load ncurses apps, >> some libs, termcap, etc). That would be a lifesaver in this case. > > I hadn't realized that. That's stupendous! > > Though I do believe that `/rescue/vi' exists these days as a statically-l= inked binary (built using crunchgen(1) via `rescue/rescue/Makefile' in the = FreeBSD source tree). Yeah, but unfortunately if some filesystems aren't mounted [ /usr, /var, etc ] back in the 5.x days (that's when I got into FreeBSD) [until recently after ed@'s work on syscons and libteken?], most utilities like vi and ee are braindead when it comes to printing characters on the screen. I've screwed up and forgot to put down nfs shares as noauto a few times, and also tinkered around with ata <-> ahci within the past couple months, so until I learned enough sed to comment out lines in fstab, booting up my system didn't work out too well :/. >> =A0 =A0But then I realized that this command probably would live in >> /usr/sbin and would probably need other apps in /usr/bin // /usr/sbin >> to run this command :). > > I envisioned it living in `/sbin' and it's actually free of dependencies = to external executables. If you execute it with the "-d" option ('d' for 'd= ependencies'), you can see that it uses only shell internals. The only thin= g this script truly depends on is `/bin/sh'. So it sounds like it would fit= the ability of working within single-user mode quite well (remember, I dev= eloped it for embedded systems -- where things like grep/sed/awk may even b= e missing). Guess it goes to show that I should review the code :). Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 9 20:25:56 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43555106566C for ; Sat, 9 Oct 2010 20:25:56 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0453A8FC08 for ; Sat, 9 Oct 2010 20:25:55 +0000 (UTC) Received: by iwn8 with SMTP id 8so2743882iwn.13 for ; Sat, 09 Oct 2010 13:25:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=ozuMPYi+JgNGP5Ei5GjbIdqXCJ2NxG8b3OpSAFbgg4g=; b=mYIjgEtzIuiJKm1CWGA0YhAIf8Gvnsfe9L/jpzhwPTqK6Wk24fb0eHzMbVTrTJCCKs iL5/FcHem5cjU6O4MWXg66ROBB/E82/lnXhlH1lB/wBvPPzUTUj5dDDEk1TSKva421xH F4CbXMS8X7eh+cBV1td4dZGzaTAecGR7NAVQc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=WbgseswH/GkcqJzaT77ietPVXiiTrdMdQQ8kpxRDd3w7YExO09Pv0iMwdr/mQ8nks6 ooH3aoX/AiJXgH/QaRDoDAyDF4lWXm9ojsBUp4r1uHJt1eIg+82QbJxdPNr533BXdsUm qRwnBlEdxQSPHymciS+8+cLSz44TNIgDURny8= MIME-Version: 1.0 Received: by 10.42.46.142 with SMTP id k14mr851500icf.310.1286655955058; Sat, 09 Oct 2010 13:25:55 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.184.3 with HTTP; Sat, 9 Oct 2010 13:25:55 -0700 (PDT) In-Reply-To: <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> Date: Sat, 9 Oct 2010 13:25:55 -0700 X-Google-Sender-Auth: v3iSmydLVvNt1EtrmLkTwSGyL_c Message-ID: From: Garrett Cooper To: Devin Teske Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Brandon Gooch , freebsd-hackers@freebsd.org Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Oct 2010 20:25:56 -0000 On Wed, Oct 6, 2010 at 8:29 PM, Devin Teske wrote: > > On Oct 6, 2010, at 4:09 PM, Brandon Gooch wrote: > >> On Wed, Oct 6, 2010 at 3:45 PM, Devin Teske wrote: >>> Hello fellow freebsd-hackers, >>> >>> Long-time hacker, first-time poster. >>> >>> I'd like to share a shell script that I wrote for FreeBSD system >>> administration. >>> >> >> It seems the list ate the attachment :( > > > Here she is ^_^ Comments welcome. Hah. More nuclear reactor than bikeshed :D! > #!/bin/sh > # -*- tab-width: =A04 -*- ;; Emacs > # vi: set tabstop=3D4 =A0 =A0 :: Vi/ViM > # > # Default setting whether to dump a list of internal dependencies upon ex= it > # > : ${SYSRC_SHOW_DEPS:=3D0} > > ############################################################ GLOBALS > > # Global exit status variables > : ${SUCCESS:=3D0} > : ${FAILURE:=3D1} Should this really be set to something other than 0 or 1 by the end-user's environment? This would simplify a lot of return/exit calls... > # > # Program name > # > progname=3D"${0##*/}" > > # > # Options > # > SHOW_EQUALS=3D > SHOW_NAME=3D1 > > # Reserved for internal use > _depend=3D When documenting arguments passed to functions, I usually do something like= : # 1 - a var # 2 - another var # # ... etc because it's easier to follow for me at least. Various spots in the codebase have differing styles though (and it would be better to follow the style in /etc/rc.subr, et all for consistency, because this tool is a consumer of those APIs). > ############################################################ FUNCTION > > # fprintf $fd $fmt [ $opts ... ] > # > # Like printf, except allows you to print to a specific file-descriptor. = Useful > # for printing to stderr (fd=3D2) or some other known file-descriptor. > # > : dependency checks performed after depend-function declaration > : function ; fprintf ( ) # $fd $fmt [ $opts ... ] Dumb question. Does declaring `: dependency checks performed after depend-function declaration' and `: function' buy you anything other than readability via comments with syntax checking? > { > =A0 =A0 =A0 =A0local fd=3D$1 > =A0 =A0 =A0 =A0[ $# -gt 1 ] || return ${FAILURE-1} While working at IronPort, Doug (my tech lead) has convinced me that constructs like: if [ $# -le 1 ] then return ${FAILURE-1} fi Are a little more consistent and easier to follow than: [ $# -gt 1 ] || return ${FAILURE-1} Because some folks have a tendency to chain shell expressions, i.e. expr1 || expr2 && expr3 Instead of: if expr1 || expr2 then expr3 fi or... if ! expr1 then expr2 fi if [ $? -eq 0 ] then expr3 fi I've caught myself chaining 3 expressions together, and I broke that down into a simpler (but more longhand format), but I've caught people chaining 4+ expressions together, which just becomes unmanageable to follow (and sometimes bugs creep in because of operator ordering and expression evaluation and subshells, etc, but that's another topic for another time :)..). > =A0 =A0 =A0 =A0shift 1 > =A0 =A0 =A0 =A0printf "$@" >&$fd > } > > # eprintf $fmt [ $opts ... ] > # > # Print a message to stderr (fd=3D2). > # > : dependency checks performed after depend-function declaration > : function ; eprintf ( ) # $fmt [ $opts ... ] > { > =A0 =A0 =A0 =A0fprintf 2 "$@" > } > > # show_deps > # > # Print the current list of dependencies. > # > : dependency checks performed after depend-function declaration > : function ; show_deps ( ) # > { > =A0 =A0 =A0 =A0if [ "$SYSRC_SHOW_DEPS" =3D "1" ]; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0eprintf "Running internal dependency list:= \n" > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0local d > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0for d in $_depend; do > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0eprintf "\t%-15ss%s\n" "$d= " "$( type "$d" )" The command(1) -v builtin is more portable than the type(1) builtin for command existence lookups (it just doesn't tell you what the particular item is that you're dealing with like type(1) does). I just learned that it also handles other builtin lexicon like if, for, while, then, do, done, etc on FreeBSD at least; POSIX also declares that it needs to support that though, so I think it's a safe assumption to state that command -v will provide you with what you need. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0done > =A0 =A0 =A0 =A0fi > } > > # die [ $err_msg ... ] > # > # Optionally print a message to stderr before exiting with failure status= . > # > : dependency checks performed after depend-function declaration > : function ; die ( ) # [ $err_msg ... ] > { > =A0 =A0 =A0 =A0local fmt=3D"$1" > =A0 =A0 =A0 =A0[ $# -gt 0 ] && shift 1 > =A0 =A0 =A0 =A0[ =A0"$fmt" =A0] && eprintf "$fmt\n" "$@" "x$fmt" !=3D x ? It seems like it could be simplified to: if [ $# -gt 0 ] then local fmt=3D$1 shift 1 eprintf "$fmt\n" "$@" fi > =A0 =A0 =A0 =A0show_deps > =A0 =A0 =A0 =A0exit ${FAILURE-1} > } > > # have $anything > # > # Used for dependency calculations. Arguments are passed to the `type' bu= ilt-in > # to determine if a given command is available (either as a shell built-i= n or > # as an external binary). The return status is true if the given argument= is > # for an existing built-in or executable, otherwise false. > # > # This is a convenient method for building dependency lists and it aids i= n the > # readability of a script. For example, > # > # =A0 =A0 =A0 Example 1: have sed || die "sed is missing" > # =A0 =A0 =A0 Example 2: if have awk; then > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 # We have awk... > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0else > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 # We DON'T have awk... > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0fi > # =A0 =A0 =A0 Example 3: have reboot && reboot > # > : dependency checks performed after depend-function declaration > : function ; have ( ) # $anything > { > =A0 =A0 =A0 =A0type "$@" > /dev/null 2>&1 > } > > # depend $name [ $dependency ... ] > # > # Add a dependency. Die with error if dependency is not met. > # > : dependency checks performed after depend-function declaration > : function ; depend ( ) # $name [ $dependency ... ] > { > =A0 =A0 =A0 =A0local by=3D"$1" arg > =A0 =A0 =A0 =A0shift 1 > > =A0 =A0 =A0 =A0for arg in "$@"; do > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0local d Wouldn't it be better to declare this outside of the loop (I'm not sure how optimal it is to place it inside the loop)? > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0for d in $_depend ""; do > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0[ "$d" =3D "$arg" ] && bre= ak > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0done > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if [ ! "$d" ]; then Could you make this ` "x$d" =3D x ' instead? > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0have "$arg" || die \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"%s: Missi= ng dependency '%s' required by %s" \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"${prognam= e:-$0}" "$arg" "$by" The $0 substitution is unnecessary based on how you set progname above: $ foo=3Dyadda $ echo ${foo##*/} yadda $ foo=3Dyadda/badda/bing/bang $ echo ${foo##*/} bang > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0_depend=3D"$_depend${_depe= nd:+ }$arg" > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0fi > =A0 =A0 =A0 =A0done > } > > # > # Perform dependency calculations for above rudimentary functions. > # NOTE: Beyond this point, use the depend-function BEFORE dependency-use > # > depend fprintf =A0 'local' '[' 'return' 'shift' 'printf' > depend eprintf =A0 'fprintf' > depend show_deps 'if' '[' 'then' 'eprintf' 'local' 'for' 'do' 'done' 'fi' > depend die =A0 =A0 =A0 'local' '[' 'shift' 'eprintf' 'show_deps' 'exit' > depend have =A0 =A0 =A0'local' 'type' 'return' > depend depend =A0 =A0'local' 'shift' 'for' 'do' '[' 'break' 'done' 'if' '= then' \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 'have' 'die' 'fi' I'd say that you have bigger fish to try if your shell lacks the needed lexicon to parse built-ins like for, do, local, etc :)... > # usage > # > # Prints a short syntax statement and exits. > # > depend usage 'local' 'eprintf' 'die' > : function ; usage ( ) # > { > =A0 =A0 =A0 =A0local optfmt=3D"\t%-12s%s\n" > =A0 =A0 =A0 =A0local envfmt=3D"\t%-22s%s\n" > > =A0 =A0 =A0 =A0eprintf "Usage: %s [OPTIONS] name[=3Dvalue] ...\n" "${prog= name:-$0}" > > =A0 =A0 =A0 =A0eprintf "OPTIONS:\n" > =A0 =A0 =A0 =A0eprintf "$optfmt" "-h --help" \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"Print this message to stderr and exit." > =A0 =A0 =A0 =A0eprintf "$optfmt" "-d" \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"Print list of internal dependencies befor= e exit." > =A0 =A0 =A0 =A0eprintf "$optfmt" "-e" \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"Print query results as \`var=3Dvalue' (us= eful for producing" > =A0 =A0 =A0 =A0eprintf "$optfmt" "" \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"output to be fed back in). Ignored if -n = is specified." > =A0 =A0 =A0 =A0eprintf "$optfmt" "-n" \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"Show only variable values, not their name= s." > =A0 =A0 =A0 =A0eprintf "\n" > > =A0 =A0 =A0 =A0eprintf "ENVIRONMENT:\n" > =A0 =A0 =A0 =A0eprintf "$envfmt" "SYSRC_SHOW_DEPS" \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"Dump list of dependencies. Must be zero o= r one" > =A0 =A0 =A0 =A0eprintf "$envfmt" "" \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"(default: \`0')" > =A0 =A0 =A0 =A0eprintf "$envfmt" "RC_DEFAULTS" \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"Location of \`/etc/defaults/rc.conf' file= ." > > =A0 =A0 =A0 =A0die > } > > # sysrc $setting > # > # Get a system configuration setting from the collection of system- > # configuration files (in order: /etc/defaults/rc.conf /etc/rc.conf > # and /etc/rc.conf). > # > # Examples: > # > # =A0 =A0 =A0 sysrc sshd_enable > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns YES or NO > # =A0 =A0 =A0 sysrc defaultrouter > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns IP address of default router (if co= nfigured) > # =A0 =A0 =A0 sysrc 'hostname%%.*' > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns $hostname up to (but not including)= first `.' > # =A0 =A0 =A0 sysrc 'network_interfaces%%[$IFS]*' > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns first word of $network_interfaces > # =A0 =A0 =A0 sysrc 'ntpdate_flags##*[$IFS]' > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns last word of $ntpdate_flags (time s= erver address) > # =A0 =A0 =A0 sysrc usbd_flags-"default" > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns $usbd_flags or "default" if unset > # =A0 =A0 =A0 sysrc usbd_flags:-"default" > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns $usbd_flags or "default" if unset o= r NULL > # =A0 =A0 =A0 sysrc cloned_interfaces+"alternate" > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns "alternate" if $cloned_interfaces i= s set > # =A0 =A0 =A0 sysrc cloned_interfaces:+"alternate" > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns "alternate" if $cloned_interfaces i= s set and non-NULL > # =A0 =A0 =A0 sysrc '#kern_securelevel' > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns length in characters of $kern_secur= elevel > # =A0 =A0 =A0 sysrc 'hostname?' > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns NULL and error status 2 if $hostnam= e is unset (or if > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 set, returns the value of $hostname with no= error status) > # =A0 =A0 =A0 sysrc 'hostname:?' > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 returns NULL and error status 2 if $hostnam= e is unset or NULL > # =A0 =A0 =A0 =A0 =A0 =A0 =A0 (or if set and non-NULL, returns value with= out error status) > # I would probably just point someone to a shell manual, as available options and behavior may change, and behavior shouldn't (but potentially could) vary between versions of FreeBSD. > depend sysrc 'local' '[' 'return' '.' 'have' 'eval' 'echo' > : function ; sysrc ( ) # $varname > { > =A0 =A0 =A0 =A0: ${RC_DEFAULTS:=3D"/etc/defaults/rc.conf"} > > =A0 =A0 =A0 =A0local defaults=3D"$RC_DEFAULTS" > =A0 =A0 =A0 =A0local varname=3D"$1" > > =A0 =A0 =A0 =A0# Check arguments > =A0 =A0 =A0 =A0[ -r "$defaults" ] || return > =A0 =A0 =A0 =A0[ "$varname" ] || return > > =A0 =A0 =A0 =A0( # Execute within sub-shell to protect parent environment > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0[ -f "$defaults" -a -r "$defaults" ] && . = "$defaults" > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0have source_rc_confs && source_rc_confs > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0eval echo '"${'"$varname"'}"' 2> /dev/null > =A0 =A0 =A0 =A0) > } > > # ... | lrev > # lrev $file ... > # > # Reverse lines of input. Unlike rev(1) which reverses the ordering of > # characters on a single line, this function instead reverses the line > # sequencing. > # > # For example, the following input: > # > # =A0 =A0 =A0 Line 1 > # =A0 =A0 =A0 Line 2 > # =A0 =A0 =A0 Line 3 > # > # Becomes reversed in the following manner: > # > # =A0 =A0 =A0 Line 3 > # =A0 =A0 =A0 Line 2 > # =A0 =A0 =A0 Line 1 > # > depend lrev 'local' 'if' '[' 'then' 'while' 'do' 'shift' 'done' 'else' 'r= ead' \ > =A0 =A0 =A0 =A0 =A0 =A0'fi' 'echo' > : function ; lrev ( ) # $file ... > { > =A0 =A0 =A0 =A0local stdin_rev=3D > =A0 =A0 =A0 =A0if [ $# -gt 0 ]; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0# Reverse lines from files passed as posit= ional arguments. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0while [ $# -gt 0 ]; do > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0local file=3D"$1" > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0[ -f "$file" ] && lrev < "= $file" > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0shift 1 > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0done > =A0 =A0 =A0 =A0else > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0# Reverse lines from standard input > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0while read -r LINE; do > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0stdin_rev=3D"$LINE > $stdin_rev" > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0done > =A0 =A0 =A0 =A0fi > > =A0 =A0 =A0 =A0echo -n "$stdin_rev" > } > > # sysrc_set $setting $new_value > # > # Change a setting in the system configuration files (edits the files in-= place > # to change the value in the last assignment to the variable). If the var= iable > # does not appear in the source file, it is appended to the end of the pr= imary > # system configuration file `/etc/rc.conf'. > # > depend sysrc_set 'local' 'sysrc' '[' 'return' 'for' 'do' 'done' 'if' 'hav= e' \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 'then' 'else' 'while' 'read' 'case' 'esac= ' 'fi' 'break' \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 'eprintf' 'echo' 'lrev' > : function ; sysrc_set ( ) # $varname $new_value > { > =A0 =A0 =A0 =A0local rc_conf_files=3D"$( sysrc rc_conf_files )" > =A0 =A0 =A0 =A0local varname=3D"$1" new_value=3D"$2" IIRC I've run into issues doing something similar to this in the past, so I broke up the local declarations on 2+ lines. > =A0 =A0 =A0 =A0local file conf_files=3D > > =A0 =A0 =A0 =A0# Check arguments > =A0 =A0 =A0 =A0[ "$rc_conf_files" ] || return ${FAILURE-1} > =A0 =A0 =A0 =A0[ "$varname" ] || return ${FAILURE-1} > Why not just do... if [ "x$rc_conf_files" =3D x -o "x$varname" =3D x ] then return ${FAILURE-1} fi ...? > =A0 =A0 =A0 =A0# Reverse the order of files in rc_conf_files > =A0 =A0 =A0 =A0for file in $rc_conf_files; do > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0conf_files=3D"$file${conf_files:+ }$conf_f= iles" > =A0 =A0 =A0 =A0done > > =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0# Determine which file we are to operate on. If no files m= atch, we'll > =A0 =A0 =A0 =A0# simply append to the last file in the list (`/etc/rc.con= f'). > =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0local found=3D > =A0 =A0 =A0 =A0local regex=3D"^[[:space:]]*$varname=3D" > =A0 =A0 =A0 =A0for file in $conf_files; do > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0#if have grep; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if false; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0grep -q "$regex" $file && = found=3D1 Probably want to redirect stderr for the grep output to /dev/null, or test for the file's existence first, because rc_conf_files doesn't check for whether or not the file exists which would result in noise from your script: $ . /etc/defaults/rc.conf $ echo $rc_conf_files /etc/rc.conf /etc/rc.conf.local $ grep -q foo /etc/rc.local grep: /etc/rc.local: No such file or directory > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0else > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0while read LINE; do \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0case "$LIN= E" in \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0$varname= =3D*) found=3D1;; \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0esac; \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0done < $file > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0fi > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0[ "$found" ] && break > =A0 =A0 =A0 =A0done > > =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0# Perform sanity checks. > =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0if [ ! -w $file ]; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0eprintf "\n%s: cannot create %s: permissio= n denied\n" \ Being pedantic, I would capitalize the P in permission to match EACCES's output string. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"${progname:-$0}" "$file" > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return ${FAILURE-1} > =A0 =A0 =A0 =A0fi > > =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0# If not found, append new value to last file and return. > =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0if [ ! "$found" ]; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0echo "$varname=3D\"$new_value\"" >> $file > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return ${SUCCESS-0} > =A0 =A0 =A0 =A0fi > > =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0# Operate on the matching file, replacing only the last oc= currence. > =A0 =A0 =A0 =A0# > =A0 =A0 =A0 =A0local new_contents=3D"`lrev $file 2> /dev/null | \ > =A0 =A0 =A0 =A0( found=3D > =A0 =A0 =A0 =A0 =A0while read -r LINE; do > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if [ ! "$found" ]; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0#if have grep; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if false; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0match=3D"$= ( echo "$LINE" | grep "$regex" )" > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0else > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0case "$LIN= E" in > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0$varname= =3D*) match=3D1;; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 *) match=3D;; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0esac > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0fi > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if [ "$match" ]; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0LINE=3D"$v= arname"'=3D"'"$new_value"'"' > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0found=3D1 > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0fi > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0fi > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0echo "$LINE" > =A0 =A0 =A0 =A0 =A0done > =A0 =A0 =A0 =A0) | lrev`" > > =A0 =A0 =A0 =A0[ "$new_contents" ] \ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&& echo "$new_contents" > $file What if this write fails, or worse, 2+ people were modifying the file using different means at the same time? You could potentially lose/corrupt your data and your system is potentially hosed, is it not? Why not write the contents out to a [sort of?] temporary file (even $progname.$$ would suffice probably, but that would have potential security implications so mktemp(1) might be the way to go), then move the temporary file to $file? You might also want to use lockf to lock the file. > } > > ############################################################ MAIN SOURCE > > # > # Perform sanity checks > # > depend main '[' 'usage' > [ $# -gt 0 ] || usage > > # > # Process command-line options > # > depend main 'while' '[' 'do' 'case' 'usage' 'eprintf' \ > =A0 =A0 =A0 =A0 =A0 =A0'break' 'esac' 'shift' 'done' > while [ $# -gt 0 ]; do Why not just use the getopts shell built-in and shift $(( $OPTIND - 1 )) at the end? > =A0 =A0 =A0 =A0case "$1" in > =A0 =A0 =A0 =A0-h|--help) usage;; > =A0 =A0 =A0 =A0-d) SYSRC_SHOW_DEPS=3D1;; > =A0 =A0 =A0 =A0-e) SHOW_EQUALS=3D1;; > =A0 =A0 =A0 =A0-n) SHOW_NAME=3D;; > =A0 =A0 =A0 =A0-*) eprintf "%s: unrecognized option \`$1'\n" "${progname:= -$0}" > =A0 =A0 =A0 =A0 =A0 =A0usage;; > =A0 =A0 =A0 =A0 *) # Since it is impossible (in many shells, including bo= urne, c, > =A0 =A0 =A0 =A0 =A0 =A0# tennex-c, and bourne-again) to name a variable b= eginning with a > =A0 =A0 =A0 =A0 =A0 =A0# dash/hyphen [-], we will terminate the option-li= st at the first > =A0 =A0 =A0 =A0 =A0 =A0# item that doesn't begin with a dash. > =A0 =A0 =A0 =A0 =A0 =A0break;; > =A0 =A0 =A0 =A0esac > =A0 =A0 =A0 =A0shift 1 > done > [ "$SHOW_NAME" ] || SHOW_EQUALS=3D > > # > # Process command-line arguments > # > depend main '[' 'while' 'do' 'case' 'echo' 'sysrc' 'if' 'sysrc_set' 'then= ' \ > =A0 =A0 =A0 =A0 =A0 =A0'fi' 'esac' 'shift' 'done' > SEP=3D': ' > [ "$SHOW_EQUALS" ] && SEP=3D'=3D"' > while [ $# -gt 0 ]; do > =A0 =A0 =A0 =A0NAME=3D"${1%%=3D*}" > =A0 =A0 =A0 =A0case "$1" in > =A0 =A0 =A0 =A0*=3D*) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0echo -n "${SHOW_NAME:+$NAME$SEP}$( > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sysrc "$1" )${SHOW_EQUALS= :+\"}" > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if sysrc_set "$NAME" "${1#*=3D}"; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0echo " -> $( sysrc "$NAME"= )" > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0fi What happens if this set fails :)? It would be confusing to end users if you print out the value (and they expected it to be set), but it failed for some particular reason. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0;; > =A0 =A0 =A0 =A0*) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if ! IGNORED=3D"$( sysrc "$NAME?" )"; then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0echo "${progname:-$0}: unk= nown variable '$NAME'" > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0else > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0echo "${SHOW_NAME:+$NAME$S= EP}$( > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sysrc "$1" )${= SHOW_EQUALS:+\"}" Not sure if it's a gmail screwup or not, but is there supposed to be a newline between `$(' and `sysrc' ? And now some more important questions: 1. What if I do: sysrc PS1 :) (hint: variables inherited from the shell really shouldn't end up in the output / be queried)? 2. Could you add an analog for sysctl -a and sysctl -n ? 3. There are some more complicated scenarios that unfortunately this might not pass when setting variables (concerns that come to mind deal with user-set $rc_conf_files where values could be spread out amongst different rc.conf's, and where more complicated shell syntax would become a slippery slope for this utility, because one of the lesser used features within rc.conf is that it's nothing more than sourceable bourne shell script :)...). I would definitely test the following scenarios: #/etc/rc.conf-1: foo=3Dbaz #/etc/rc.conf-2: foo=3Dbar #/etc/rc.conf-3: foo=3D"$foo zanzibar" Scenario A: #/etc/rc.conf: rc_conf_files=3D"/etc/rc.conf-1 /etc/rc.conf-2" The value of foo should be set to bar; ideally the value of foo in /etc/rc.conf-2 should be set to a new value by the end user. Scenario B: #/etc/rc.conf: rc_conf_files=3D"/etc/rc.conf-2 /etc/rc.conf-1" The value of foo should be set to baz; ideally the value of foo in /etc/rc.conf-1 should be set to a new value by the end user. Scenario C: #/etc/rc.conf: rc_conf_files=3D"/etc/rc.conf-1 /etc/rc.conf-2 /etc/rc.conf-3" The value of foo should be set to `bar zanzibar'; ideally the value of foo in /etc/rc.conf-3 should be set to a new value by the end user (but that will affect the expected output potentially). Scenario D: #/etc/rc.conf: rc_conf_files=3D"/etc/rc.conf-2 /etc/rc.conf-1 /etc/rc.conf-3" The value of foo should be set to `baz zanzibar'; ideally the value of foo in /etc/rc.conf-3 should be set to a new value by the end user (but that will affect the expected output potentially). I'll probably think up some more scenarios later that should be tested... the easy way out is to state that the tool does a best effort at overwriting the last evaluated value. Overall, awesome looking tool and I'll be happy to test it Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 9 22:40:08 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B334410656EA; Sat, 9 Oct 2010 22:40:08 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id 8E31C8FC0A; Sat, 9 Oct 2010 22:40:08 +0000 (UTC) Received: from [71.202.142.31] (port=62596 helo=[192.168.1.149]) by postoffice.vicor.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.71) (envelope-from ) id 1P4i4u-0007C9-Ob; Sat, 09 Oct 2010 15:39:53 -0700 Mime-Version: 1.0 (Apple Message framework v1081) From: Devin Teske In-Reply-To: Date: Sat, 9 Oct 2010 15:39:44 -0700 Message-Id: <238E0B24-AA12-4684-9651-84DA665BE893@vicor.com> References: <1286397912.27308.40.camel@localhost.localdomain> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> To: Garrett Cooper X-Mailer: Apple Mail (2.1081) X-Scan-Signature: 8a06b7f1643b99a097dc4c6e49e2c27d X-Scan-Host: postoffice.vicor.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Brandon Gooch , freebsd-hackers@freebsd.org, Devin Teske Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Oct 2010 22:40:08 -0000 On Oct 9, 2010, at 1:25 PM, Garrett Cooper wrote: > On Wed, Oct 6, 2010 at 8:29 PM, Devin Teske wrote: >>=20 >> On Oct 6, 2010, at 4:09 PM, Brandon Gooch wrote: >>=20 >>> On Wed, Oct 6, 2010 at 3:45 PM, Devin Teske = wrote: >>>> Hello fellow freebsd-hackers, >>>>=20 >>>> Long-time hacker, first-time poster. >>>>=20 >>>> I'd like to share a shell script that I wrote for FreeBSD system >>>> administration. >>>>=20 >>>=20 >>> It seems the list ate the attachment :( >>=20 >>=20 >> Here she is ^_^ Comments welcome. >=20 > Hah. More nuclear reactor than bikeshed :D! ^_^ You're about to find out more... >=20 >> #!/bin/sh >> # -*- tab-width: 4 -*- ;; Emacs >> # vi: set tabstop=3D4 :: Vi/ViM >=20 >> # >> # Default setting whether to dump a list of internal dependencies = upon exit >> # >> : ${SYSRC_SHOW_DEPS:=3D0} >>=20 >> ############################################################ GLOBALS >>=20 >> # Global exit status variables >> : ${SUCCESS:=3D0} >> : ${FAILURE:=3D1} >=20 > Should this really be set to something other than 0 or 1 by the > end-user's environment? This would simplify a lot of return/exit > calls... A scenario that I envision that almost never arises, but... Say someone wanted to call my script but wanted to mask it to always = return with success (why? I dunno... it's conceivable though). Example: (this should be considered ugly -- because it is) FAILURE=3D0 && sysrc foo && reboot Efficacy: The `reboot' rvalue of '&&' will always execute because FAILURE. I don't really know why I got into the practice of writing scripts this = way... most likely a foregone conclusion that seemed like a good idea at = one time but never really amounted to anything substantive (in fact, it = should perhaps be considered heinous). I agree... a productionized version in the base distribution should lack = such oddities. The script should do: SUCCESS=3D0 FAILURE=3D1 and be done with it. Though, I've been sometimes known to follow the habits of C-programming = and instead do: EXIT_SUCCESS=3D0 EXIT_FAILURE=3D1 (real macros defined by system includes; though in C-land they aren't = 0/1 but rather 0/-1 IIRC) I just found it redundant to say: exit $EXIT_SUCCESS and shorter/more-succinct to say: exit $SUCCESS >> # >> # Program name >> # >> progname=3D"${0##*/}" >>=20 >> # >> # Options >> # >> SHOW_EQUALS=3D >> SHOW_NAME=3D1 >>=20 >> # Reserved for internal use >> _depend=3D >=20 > When documenting arguments passed to functions, I usually do something = like: >=20 > # 1 - a var > # 2 - another var > # > # ... etc >=20 > because it's easier to follow for me at least. >=20 > Various spots in the codebase have differing styles though (and it > would be better to follow the style in /etc/rc.subr, et all for > consistency, because this tool is a consumer of those APIs). I borrow my argument-documentation style from 15+ years of perl = programming. I think it's all just personal preference. Personally, I = like to jam it all one line specifically so that I can do a quick mark, = then "?function.*name" to jump up to the definition-line, "yy" (for = yank-yank; copies current line into buffer), then jump back to my mark, = "p" for paste, then replace the variables with what I intend to pass in = for the particular call. Using vi for years teaches interesting styles -- packing a list of = keywords onto a single line to grab/paste elsewhere are just one of = those little things you learn. >> ############################################################ FUNCTION >>=20 >> # fprintf $fd $fmt [ $opts ... ] >> # >> # Like printf, except allows you to print to a specific = file-descriptor. Useful >> # for printing to stderr (fd=3D2) or some other known = file-descriptor. >> # >> : dependency checks performed after depend-function declaration >> : function ; fprintf ( ) # $fd $fmt [ $opts ... ] >=20 > Dumb question. Does declaring `: dependency checks performed after > depend-function declaration' and `: function' buy you anything other > than readability via comments with syntax checking? The first ": dependency checks ..." is just a note to myself. I used ":" = syntax to make it stand-out differently than the "#" syntax. Not to = mention that when I go through my scripts (well, the ones that are = intended for functioning within an embedded environment at least) I = expect to see a call to "depend()" before a) each/every function and b) = each/every large contiguous block of code (or at least the blocks that = look like they are good candidates for re-use in other scripts). The second usage (": function") aids in finding the function declaration = among the usages. See, in Perl, I can simply search for "sub" preceding = the function name. In C, I tend to split the return type from the = function name and ensure that the function name always starts in = column-1 so I can search for "^funcname" to go to the declaration = opposed to the usages/references. In BASH, `function' is a valid keyword = and you can say "function funcname ( ) BLOCK" but unfortunately in good = ol' bourne shell, "function" is not an understood keyword, ... but = really liking this keyword, I decided to make use of it in bourne shell = by-way-of simply making it a non-executed-expression (preceded it with = ":" and terminated it with ";"). >> { >> local fd=3D$1 >> [ $# -gt 1 ] || return ${FAILURE-1} >=20 > While working at IronPort, Doug (my tech lead) has convinced me that > constructs like: >=20 > if [ $# -le 1 ] > then > return ${FAILURE-1} > fi Never did understand why folks insisted on splitting the if/then syntax = (or while/do or for/do etc.) into multiple lines. I've always found that = putting the semi-colon in there made it easier to read. > Are a little more consistent and easier to follow than: >=20 > [ $# -gt 1 ] || return ${FAILURE-1} >=20 > Because some folks have a tendency to chain shell expressions, i.e. I agree with you that any-more than one is excessive. I've often tried to emulate the C-expression "bool ? if-true : else" = using: ( bool && if-true ) || else but it's just not clean-looking. I still like the simple-elegance of "expr || if-false" and "expr && = if-true" ... but-again, only perhaps since my first-love is Perl (of = which I've programmed 15+ years), and statements like that are rampant = in Perl perhaps because the ol' Perl cookbooks of historical right = advocate their usage in such a manner. >=20 > expr1 || expr2 && expr3 >=20 > Instead of: >=20 > if expr1 || expr2 > then > expr3 > fi >=20 > or... >=20 > if ! expr1 > then > expr2 > fi > if [ $? -eq 0 ] > then > expr3 > fi >=20 > I've caught myself chaining 3 expressions together, and I broke that > down into a simpler (but more longhand format), but I've caught people > chaining 4+ expressions together, which just becomes unmanageable to > follow (and sometimes bugs creep in because of operator ordering and > expression evaluation and subshells, etc, but that's another topic for > another time :)..). Yeah, 3+ is gross in my opinion (agreed). >=20 >> shift 1 >> printf "$@" >&$fd >> } >>=20 >> # eprintf $fmt [ $opts ... ] >> # >> # Print a message to stderr (fd=3D2). >> # >> : dependency checks performed after depend-function declaration >> : function ; eprintf ( ) # $fmt [ $opts ... ] >> { >> fprintf 2 "$@" >> } >>=20 >> # show_deps >> # >> # Print the current list of dependencies. >> # >> : dependency checks performed after depend-function declaration >> : function ; show_deps ( ) # >> { >> if [ "$SYSRC_SHOW_DEPS" =3D "1" ]; then >> eprintf "Running internal dependency list:\n" >>=20 >> local d >> for d in $_depend; do >> eprintf "\t%-15ss%s\n" "$d" "$( type "$d" )" >=20 > The command(1) -v builtin is more portable than the type(1) builtin > for command existence lookups (it just doesn't tell you what the > particular item is that you're dealing with like type(1) does). >=20 Ah, coolness. command(1) is new to me just now ^_^ > I just learned that it also handles other builtin lexicon like if, > for, while, then, do, done, etc on FreeBSD at least; POSIX also > declares that it needs to support that though, so I think it's a safe > assumption to state that command -v will provide you with what you > need. I originally had been programming in tests for '!' and 'in', but in = POSIX bourne-shell, they aren't defined (though understood) in the = keyword table (so type(1) balks in bourne-shell while csh and bash do = respond to '!' and 'in' queries). Since you've pointed out command(1)... I now have a way of checking '!'. = Though unfortunately, "command -v", like type(1), also does not like = "in" (in bourne-shell at least). >=20 >> done >> fi >> } >>=20 >> # die [ $err_msg ... ] >> # >> # Optionally print a message to stderr before exiting with failure = status. >> # >> : dependency checks performed after depend-function declaration >> : function ; die ( ) # [ $err_msg ... ] >> { >> local fmt=3D"$1" >> [ $# -gt 0 ] && shift 1 >> [ "$fmt" ] && eprintf "$fmt\n" "$@" >=20 > "x$fmt" !=3D x ? It seems like it could be simplified to: >=20 > if [ $# -gt 0 ] > then > local fmt=3D$1 > shift 1 > eprintf "$fmt\n" "$@" > fi I never understood why people don't trust the tools they are using... `[' is very very similar (if not identical) to test(1) [ "..." ] is the same thing as [ -n "..." ] or test -n "..." [ ! "..." ] is the same things as [ -z "..." ] or test -z "..." I'll never understand why people have to throw an extra letter in there = and then compare it to that letter. If the variable expands to nothing, go ahead and let it. I've traced = every possible expansion of variables when used in the following manner: [ "$VAR" ] ... and it never fails. If $VAR is anything but null, the entire expression = will evaluate to true. Again... coming from 15+ years of perl has made my eyes read the = following block of code: if [ "$the_network_is_enabled" ]; then aloud in my head as "if the network is enabled, then ..." (not too far = of a stretch)... which has a sort of quintessential humanized logic to = it, don't you think? Now, contrast that with this block: if [ "x$the_network_is_enabled" =3D x ]; then (one might verbalize that in their head as "if x plus `the network is = enabled' is equal to x, then" ... which is more clear?) Yet, if I don't leave out the implied "-n" or "-z", is it more = acceptable? For instance... if [ -n "$the_network_is_enabled" ]; then But that would require the reader (performing intonation in their heads = as they read the code) to innately _know_ that "-n" is "this is = non-null" (where "this" is the rvalue to the keyword). >=20 >> show_deps >> exit ${FAILURE-1} >> } >>=20 >> # have $anything >> # >> # Used for dependency calculations. Arguments are passed to the = `type' built-in >> # to determine if a given command is available (either as a shell = built-in or >> # as an external binary). The return status is true if the given = argument is >> # for an existing built-in or executable, otherwise false. >> # >> # This is a convenient method for building dependency lists and it = aids in the >> # readability of a script. For example, >> # >> # Example 1: have sed || die "sed is missing" >> # Example 2: if have awk; then >> # # We have awk... >> # else >> # # We DON'T have awk... >> # fi >> # Example 3: have reboot && reboot >> # >> : dependency checks performed after depend-function declaration >> : function ; have ( ) # $anything >> { >> type "$@" > /dev/null 2>&1 >> } >>=20 >> # depend $name [ $dependency ... ] >> # >> # Add a dependency. Die with error if dependency is not met. >> # >> : dependency checks performed after depend-function declaration >> : function ; depend ( ) # $name [ $dependency ... ] >> { >> local by=3D"$1" arg >> shift 1 >>=20 >> for arg in "$@"; do >> local d >=20 > Wouldn't it be better to declare this outside of the loop (I'm not > sure how optimal it is to place it inside the loop)? I'm assuming you mean the "local d" statement. There's no restriction = that says you have to put your variable declarations at the beginning of = a block (like in C -- even if only within a superficial block { in the = middle of nowhere } ... like that). Meanwhile, in Perl, it's quite a difference to scope it to the loop = rather than the block. So, it all depends on whichever _looks_ nicer to = you ^_^ >> for d in $_depend ""; do >> [ "$d" =3D "$arg" ] && break >> done >> if [ ! "$d" ]; then >=20 > Could you make this ` "x$d" =3D x ' instead? =3D( I made the switch to using [ "..." ] (implied "-n") and [ ! "..." ] = (implied "-z") long ago because they intonate in my head so-darned well = ("!" becoming "NOT" of course). >> have "$arg" || die \ >> "%s: Missing dependency '%s' required = by %s" \ >> "${progname:-$0}" "$arg" "$by" >=20 > The $0 substitution is unnecessary based on how you set progname = above: >=20 > $ foo=3Dyadda > $ echo ${foo##*/} > yadda > $ foo=3Dyadda/badda/bing/bang > $ echo ${foo##*/} > bang Ah, another oddity of my programming style. I often experienced people ripping whole blocks or whole functions out = of my scripts and re-using them in their own scripts... So I adopted this coding practice where... whenever I anticipated people = doing this (usually I only anticipate people ripping whole functions), I = wanted the blocks of code to still be semi-functional. So what you're seeing is that everytime I rely on the global "progname" = within a re-usable code construct (a function for example), I would use = special parameter-expansion syntaxes that allow a fall-back default = value that was sensible ($0 in this case). So outside of functions within the script, you'll see: $progname -- the global is used explicitly without fallback (because people = ripping out a block in the main source should be smart enough to know to = check the globals section at the top) meanwhile, in a function: ${progname:-$0} So that if they ripped said-function into their own code and neglected = to define progname, the fallback default would be $0 which is expanded = by the shell always to be the first word (words being separated by any = character of $IFS) of the invocation line. >=20 >> _depend=3D"$_depend${_depend:+ }$arg" >> fi >> done >> } >>=20 >> # >> # Perform dependency calculations for above rudimentary functions. >> # NOTE: Beyond this point, use the depend-function BEFORE = dependency-use >> # >> depend fprintf 'local' '[' 'return' 'shift' 'printf' >> depend eprintf 'fprintf' >> depend show_deps 'if' '[' 'then' 'eprintf' 'local' 'for' 'do' 'done' = 'fi' >> depend die 'local' '[' 'shift' 'eprintf' 'show_deps' 'exit' >> depend have 'local' 'type' 'return' >> depend depend 'local' 'shift' 'for' 'do' '[' 'break' 'done' 'if' = 'then' \ >> 'have' 'die' 'fi' >=20 > I'd say that you have bigger fish to try if your shell lacks the > needed lexicon to parse built-ins like for, do, local, etc :)... Too true... I was being ULTRA pedantic in my embedded-environment testing. ^_^ Taking measures to test with different shells even... sh, bash, csh, = pdksh, zsh, etc. etc. etc. (glad to report that the script is ultra = portable) >> # usage >> # >> # Prints a short syntax statement and exits. >> # >> depend usage 'local' 'eprintf' 'die' >> : function ; usage ( ) # >> { >> local optfmt=3D"\t%-12s%s\n" >> local envfmt=3D"\t%-22s%s\n" >>=20 >> eprintf "Usage: %s [OPTIONS] name[=3Dvalue] ...\n" = "${progname:-$0}" >>=20 >> eprintf "OPTIONS:\n" >> eprintf "$optfmt" "-h --help" \ >> "Print this message to stderr and exit." >> eprintf "$optfmt" "-d" \ >> "Print list of internal dependencies before exit." >> eprintf "$optfmt" "-e" \ >> "Print query results as \`var=3Dvalue' (useful for = producing" >> eprintf "$optfmt" "" \ >> "output to be fed back in). Ignored if -n is = specified." >> eprintf "$optfmt" "-n" \ >> "Show only variable values, not their names." >> eprintf "\n" >>=20 >> eprintf "ENVIRONMENT:\n" >> eprintf "$envfmt" "SYSRC_SHOW_DEPS" \ >> "Dump list of dependencies. Must be zero or one" >> eprintf "$envfmt" "" \ >> "(default: \`0')" >> eprintf "$envfmt" "RC_DEFAULTS" \ >> "Location of \`/etc/defaults/rc.conf' file." >>=20 >> die >> } >>=20 >> # sysrc $setting >> # >> # Get a system configuration setting from the collection of system- >> # configuration files (in order: /etc/defaults/rc.conf /etc/rc.conf >> # and /etc/rc.conf). >> # >> # Examples: >> # >> # sysrc sshd_enable >> # returns YES or NO >> # sysrc defaultrouter >> # returns IP address of default router (if configured) >> # sysrc 'hostname%%.*' >> # returns $hostname up to (but not including) first `.' >> # sysrc 'network_interfaces%%[$IFS]*' >> # returns first word of $network_interfaces >> # sysrc 'ntpdate_flags##*[$IFS]' >> # returns last word of $ntpdate_flags (time server = address) >> # sysrc usbd_flags-"default" >> # returns $usbd_flags or "default" if unset >> # sysrc usbd_flags:-"default" >> # returns $usbd_flags or "default" if unset or NULL >> # sysrc cloned_interfaces+"alternate" >> # returns "alternate" if $cloned_interfaces is set >> # sysrc cloned_interfaces:+"alternate" >> # returns "alternate" if $cloned_interfaces is set and = non-NULL >> # sysrc '#kern_securelevel' >> # returns length in characters of $kern_securelevel >> # sysrc 'hostname?' >> # returns NULL and error status 2 if $hostname is unset = (or if >> # set, returns the value of $hostname with no error = status) >> # sysrc 'hostname:?' >> # returns NULL and error status 2 if $hostname is unset = or NULL >> # (or if set and non-NULL, returns value without error = status) >> # >=20 > I would probably just point someone to a shell manual, as available > options and behavior may change, and behavior shouldn't (but > potentially could) vary between versions of FreeBSD. I just checked "man 1 sh" on FreeBSD-8.1, and it did have copious = documentation on special expansion syntaxes. (beautiful!)... so you're = right, we could just point them at a sh(1) man-page. I somehow had it ingrained in my mind that the sh(1) man-page was = lacking while the bash(1) info-tex pages were the only places to find = documentation on the special expansion syntaxes. I'm glad to see they = are fully documented in FreeBSD these days (even back to 4.11 which I = checked just now). >> depend sysrc 'local' '[' 'return' '.' 'have' 'eval' 'echo' >> : function ; sysrc ( ) # $varname >> { >> : ${RC_DEFAULTS:=3D"/etc/defaults/rc.conf"} >>=20 >> local defaults=3D"$RC_DEFAULTS" >> local varname=3D"$1" >>=20 >> # Check arguments >> [ -r "$defaults" ] || return >> [ "$varname" ] || return >>=20 >> ( # Execute within sub-shell to protect parent environment >> [ -f "$defaults" -a -r "$defaults" ] && . "$defaults" >> have source_rc_confs && source_rc_confs >> eval echo '"${'"$varname"'}"' 2> /dev/null >> ) >> } >>=20 >> # ... | lrev >> # lrev $file ... >> # >> # Reverse lines of input. Unlike rev(1) which reverses the ordering = of >> # characters on a single line, this function instead reverses the = line >> # sequencing. >> # >> # For example, the following input: >> # >> # Line 1 >> # Line 2 >> # Line 3 >> # >> # Becomes reversed in the following manner: >> # >> # Line 3 >> # Line 2 >> # Line 1 >> # >> depend lrev 'local' 'if' '[' 'then' 'while' 'do' 'shift' 'done' = 'else' 'read' \ >> 'fi' 'echo' >> : function ; lrev ( ) # $file ... >> { >> local stdin_rev=3D >> if [ $# -gt 0 ]; then >> # >> # Reverse lines from files passed as positional = arguments. >> # >> while [ $# -gt 0 ]; do >> local file=3D"$1" >> [ -f "$file" ] && lrev < "$file" >> shift 1 >> done >> else >> # >> # Reverse lines from standard input >> # >> while read -r LINE; do >> stdin_rev=3D"$LINE >> $stdin_rev" >> done >> fi >>=20 >> echo -n "$stdin_rev" >> } >>=20 >> # sysrc_set $setting $new_value >> # >> # Change a setting in the system configuration files (edits the files = in-place >> # to change the value in the last assignment to the variable). If the = variable >> # does not appear in the source file, it is appended to the end of = the primary >> # system configuration file `/etc/rc.conf'. >> # >> depend sysrc_set 'local' 'sysrc' '[' 'return' 'for' 'do' 'done' 'if' = 'have' \ >> 'then' 'else' 'while' 'read' 'case' 'esac' 'fi' = 'break' \ >> 'eprintf' 'echo' 'lrev' >> : function ; sysrc_set ( ) # $varname $new_value >> { >> local rc_conf_files=3D"$( sysrc rc_conf_files )" >> local varname=3D"$1" new_value=3D"$2" >=20 > IIRC I've run into issues doing something similar to this in the past, > so I broke up the local declarations on 2+ lines. I find that the issue is only when you do something funky where you need = to know the return status after the assignment. `local' will always = return with success, so if you need to test the error status after an = assignment with local, you'll never get it. In those cases, it's best to = use local just to define the variable and then assign in another step to = which you can get the return error status of the command executed = within. For example: local foo=3D"$( some command )" if [ $? -ne 0 ]; then ... will never fire because local always returns true. Meanwhile,... local foo foo=3D"$( some command )" if [ $? -ne 0 ]; then ... will work as expected (if "some command" returns error status, then the = if-block will fire). >> local file conf_files=3D >>=20 >> # Check arguments >> [ "$rc_conf_files" ] || return ${FAILURE-1} >> [ "$varname" ] || return ${FAILURE-1} >>=20 >=20 > Why not just do... >=20 > if [ "x$rc_conf_files" =3D x -o "x$varname" =3D x ] > then > return ${FAILURE-1} > fi I think you'll find (quite pleasantly) that if you intonate the lines... "rc_conf_files [is non-null] OR return failure" "varname [is non-null] OR return failure" Sounds a lot better/cleaner than the intonation of the suggested = replacement: "if x plus rc_conf_files expands to something that is not equal = to x OR x plus the expansion of varname is not x then return failure" Not to mention that if the checking of additional arguments is required, = a single new line of similar appearance is added... whereas if you = wanted to expand the suggested replacement to handle another argument, = you'd have to add another "-o" case to the "[ ... ]" block which causes = the line to be pushed further to the right, requiring something like one = of the two following solutions: if [ "x$rc_conf_files" =3D x -o "x$varname" =3D x -o "x$third" =3D= x ] then ... or (slightly better) if [ "x$rc_conf_files" =3D x -o \ "x$varname" =3D x -o \ "x$third" =3D x ] then ... But then again... you're lacking something very importantant in both of = those that you don't get with the original syntax ([ "$blah" ] || return = ...)... clean diff outputs! and clean CVS differentials... and clean = RCS... Let's say that the sanity checks need to be expanded to test yet-another = variable. In the original syntax, the diff would be one line: + [ "$third" ] || return ${FAILURE-1} Otherwise, the diff is uglier (in my humble opinion): - if [ "x$rc_conf_files" =3D x -o "x$varname" =3D x ] + if [ "x$rc_conf_files" =3D x -o "x$varname" =3D x -o "x$third" =3D= x ] Make sense? I think looking at CVS diffs where only a single line is added to check = a new variable is much cleaner than a code-block which must be erased = and rewritten everytime the test is expanded. >=20 > ...? >=20 >> # Reverse the order of files in rc_conf_files >> for file in $rc_conf_files; do >> conf_files=3D"$file${conf_files:+ }$conf_files" >> done >>=20 >> # >> # Determine which file we are to operate on. If no files = match, we'll >> # simply append to the last file in the list (`/etc/rc.conf'). >> # >> local found=3D >> local regex=3D"^[[:space:]]*$varname=3D" >> for file in $conf_files; do >> #if have grep; then >> if false; then >> grep -q "$regex" $file && found=3D1 >=20 > Probably want to redirect stderr for the grep output to /dev/null, or > test for the file's existence first, because rc_conf_files doesn't > check for whether or not the file exists which would result in noise > from your script: >=20 > $ . /etc/defaults/rc.conf > $ echo $rc_conf_files > /etc/rc.conf /etc/rc.conf.local > $ grep -q foo /etc/rc.local > grep: /etc/rc.local: No such file or directory Good catch! I missed that ^_^ >=20 >> else >> while read LINE; do \ >> case "$LINE" in \ >> $varname=3D*) found=3D1;; \ >> esac; \ >> done < $file >> fi >> [ "$found" ] && break >> done >>=20 >> # >> # Perform sanity checks. >> # >> if [ ! -w $file ]; then >> eprintf "\n%s: cannot create %s: permission denied\n" = \ >=20 > Being pedantic, I would capitalize the P in permission to match > EACCES's output string. But, I actually copied the error verbatim from what the shell produces = if you actually try the command. So... if you remove the check (if [ ! -w $file ] ... ... ...) and try = the script as non-root, you'll get exactly that error message (with = lower-case 'p' on 'permission denied'). It wouldn't make sense for my script to use upper-case 'P' unless the = bourne-shell is patched to do the same. I'm simply fundamentally producing the same error message as the shell = safe for one difference... I try to detect the error before running into = it simply so I can throw a spurious newline before the error... causing = the output to more accurately mimick what sysctl(8) produces in the same = exact case (the case where a non-root user with insufficient privileges = tries to modify an MIB). Give it a shot... $ sysctl security.jail.set_hostname_allowed=3D1 security.jail.set_hostname_allowed: 1 sysctl: security.jail.set_hostname_allowed: Operation not permitted If I don't test for lack of write permissions first, and throw the error = out with a preceding new-line, the result would be: $ sysrc foo=3Dbar foo: barsysrc: cannot create /etc/rc.conf: permission denied Rather than: $sysrc foo=3Dbar foo: bar sysrc: cannot create /etc/rc.conf: permission denied >=20 >> "${progname:-$0}" "$file" >> return ${FAILURE-1} >> fi >>=20 >> # >> # If not found, append new value to last file and return. >> # >> if [ ! "$found" ]; then >> echo "$varname=3D\"$new_value\"" >> $file >> return ${SUCCESS-0} >> fi >>=20 >> # >> # Operate on the matching file, replacing only the last = occurrence. >> # >> local new_contents=3D"`lrev $file 2> /dev/null | \ >> ( found=3D >> while read -r LINE; do >> if [ ! "$found" ]; then >> #if have grep; then >> if false; then >> match=3D"$( echo "$LINE" | grep = "$regex" )" >> else >> case "$LINE" in >> $varname=3D*) match=3D1;; >> *) match=3D;; >> esac >> fi >> if [ "$match" ]; then >> LINE=3D"$varname"'=3D"'"$new_value"'"' >> found=3D1 >> fi >> fi >> echo "$LINE" >> done >> ) | lrev`" >>=20 >> [ "$new_contents" ] \ >> && echo "$new_contents" > $file >=20 > What if this write fails, or worse, 2+ people were modifying the file > using different means at the same time? You could potentially > lose/corrupt your data and your system is potentially hosed, is it > not? Why not write the contents out to a [sort of?] temporary file > (even $progname.$$ would suffice probably, but that would have > potential security implications so mktemp(1) might be the way to go), > then move the temporary file to $file? You might also want to use > lockf to lock the file. I'll investigate lockf, however I think it's one of those things that = you just live with (for example... what happens if two people issue a = sysctl(8) call at the exact same time ... whoever gets there last sets = the effective value). You'll notice that I do all my work in memory... If the buffer is empty, I don't write out the buffer. Much in the way that if an in-line sed (with -i for example) will also = check the memory contents before writing out the changes. Since error-checking is performed, there's no difference between doing = this on a temporary file (essentially the memory buffer is the temporary = file -- safe for wierd scenarios where memory fails you -- but then you = have bigger problems than possibly wiping out your rc.conf file -- like = perhaps scribbling on the disk in new and wonderful ways during memory = corruption). Also, since the calculations are done in memory and the read-in is = decidedly different than the write-out (read: not performed as a single = command), if two scripts operated simultaneously, here's what would = happen: script A reads rc.conf(5) script B does the same script A operates on in-memory buffer script B does the same script A writes out new rc.conf from modified memory buffer script B does the same whomever does the last write will have their contents preserved. The = unlucky first-writer will have his contents overwritten. I do not believe the kernel will allow the two writes to intertwine even = if firing at the exact same precise moment. I do believe that one will = block until the other finishes (we could verify this by looking at = perhaps the bourne-shell's '>' redirect operator to see if it flock's = the file during the redirect, which it may, or perhaps such things are = at lower levels). >=20 >> } >>=20 >> ############################################################ MAIN = SOURCE >>=20 >> # >> # Perform sanity checks >> # >> depend main '[' 'usage' >> [ $# -gt 0 ] || usage >>=20 >> # >> # Process command-line options >> # >> depend main 'while' '[' 'do' 'case' 'usage' 'eprintf' \ >> 'break' 'esac' 'shift' 'done' >> while [ $# -gt 0 ]; do >=20 > Why not just use the getopts shell built-in and shift $(( $OPTIND - 1 > )) at the end? ^_^ Well, I see getopt is an external dependency (bad) while getopts appears = to be a builtin. I'll have a looksie and see, but I find the case statement to be very = readable as it is. >=20 >> case "$1" in >> -h|--help) usage;; >> -d) SYSRC_SHOW_DEPS=3D1;; >> -e) SHOW_EQUALS=3D1;; >> -n) SHOW_NAME=3D;; >> -*) eprintf "%s: unrecognized option \`$1'\n" = "${progname:-$0}" >> usage;; >> *) # Since it is impossible (in many shells, including = bourne, c, >> # tennex-c, and bourne-again) to name a variable beginning = with a >> # dash/hyphen [-], we will terminate the option-list at = the first >> # item that doesn't begin with a dash. >> break;; >> esac >> shift 1 >> done >> [ "$SHOW_NAME" ] || SHOW_EQUALS=3D >>=20 >> # >> # Process command-line arguments >> # >> depend main '[' 'while' 'do' 'case' 'echo' 'sysrc' 'if' 'sysrc_set' = 'then' \ >> 'fi' 'esac' 'shift' 'done' >> SEP=3D': ' >> [ "$SHOW_EQUALS" ] && SEP=3D'=3D"' >> while [ $# -gt 0 ]; do >> NAME=3D"${1%%=3D*}" >> case "$1" in >> *=3D*) >> echo -n "${SHOW_NAME:+$NAME$SEP}$( >> sysrc "$1" )${SHOW_EQUALS:+\"}" >> if sysrc_set "$NAME" "${1#*=3D}"; then >> echo " -> $( sysrc "$NAME" )" >> fi >=20 > What happens if this set fails :)? It would be confusing to end users > if you print out the value (and they expected it to be set), but it > failed for some particular reason. No more confusing than sysctl(8) which does the same thing as I did (I = was in-fact mimicking sysctl(8) in this behavior). >=20 >> ;; >> *) >> if ! IGNORED=3D"$( sysrc "$NAME?" )"; then >> echo "${progname:-$0}: unknown variable = '$NAME'" >> else >> echo "${SHOW_NAME:+$NAME$SEP}$( >> sysrc "$1" )${SHOW_EQUALS:+\"}" >=20 > Not sure if it's a gmail screwup or not, but is there supposed to > be a newline between `$(' and `sysrc' ? Not a screw-up.... Since what appears between $( ... ) (back-ticks too `...`) is read using = readline(3), any leading whitespace is ignored. I'm using this technique to split the line because it was too long to be = accommodated-fully within an 80-character wide terminal window with = tab-width set to 8 (what nearly everybody defaults to these days). > And now some more important questions: >=20 > 1. What if I do: sysrc PS1 :) (hint: variables inherited from the > shell really shouldn't end up in the output / be queried)? Great question... hadn't thought of that. I could perhaps use a set(1) flag to clear the environment variables = prior to calling source_rc_confs. That seems to be a prudent thing to do = (or if not via set(1) built-in, via preening the list of current = variables and using unset built-in to kill them off in a for/in/do = loop). > 2. Could you add an analog for sysctl -a and sysctl -n ? The `-n' is already covered (see usage). I do agree `-a' is both warranted and highly useful (provides system = administrator a snapshot of what /etc/rc sees at boot after performing a = source_rc_confs -- great for either trouble-shooting boot problems or = taint-checking everything before a reboot). > 3. There are some more complicated scenarios that unfortunately > this might not pass when setting variables (concerns that come to mind > deal with user-set $rc_conf_files where values could be spread out > amongst different rc.conf's, and where more complicated shell syntax > would become a slippery slope for this utility, because one of the > lesser used features within rc.conf is that it's nothing more than > sourceable bourne shell script :)...). I would definitely test the > following scenarios: >=20 > #/etc/rc.conf-1: > foo=3Dbaz >=20 > #/etc/rc.conf-2: > foo=3Dbar >=20 > #/etc/rc.conf-3: > foo=3D"$foo zanzibar" >=20 > Scenario A: >=20 > #/etc/rc.conf: > rc_conf_files=3D"/etc/rc.conf-1 /etc/rc.conf-2" >=20 > The value of foo should be set to bar; ideally the value of foo in > /etc/rc.conf-2 should be set to a new value by the end user. >=20 > Scenario B: >=20 > #/etc/rc.conf: > rc_conf_files=3D"/etc/rc.conf-2 /etc/rc.conf-1" >=20 > The value of foo should be set to baz; ideally the value of foo in > /etc/rc.conf-1 should be set to a new value by the end user. >=20 > Scenario C: >=20 > #/etc/rc.conf: > rc_conf_files=3D"/etc/rc.conf-1 /etc/rc.conf-2 /etc/rc.conf-3" >=20 > The value of foo should be set to `bar zanzibar'; ideally the > value of foo in /etc/rc.conf-3 should be set to a new value by the end > user (but that will affect the expected output potentially). >=20 > Scenario D: >=20 > #/etc/rc.conf: > rc_conf_files=3D"/etc/rc.conf-2 /etc/rc.conf-1 /etc/rc.conf-3" >=20 > The value of foo should be set to `baz zanzibar'; ideally the > value of foo in /etc/rc.conf-3 should be set to a new value by the end > user (but that will affect the expected output potentially). >=20 > I'll probably think up some more scenarios later that should be > tested... the easy way out is to state that the tool does a best > effort at overwriting the last evaluated value. > Overall, awesome looking tool and I'll be happy to test it > Thanks! > -Garrett Well now.... If you really want to support ALL those possibilities... I _did_ have a = more complex routine which caught them all (each and every one), but it = wasn't quite as clean ^_^ If you really want me to break out the nuclear reactor, I'll work it = back in from one of the predecessors of this script which was 1,000+ = lines of code. However, I found that the need to catch such esoteric conditions was = far-out-weighed by the need to simplify the script and make a cleaner = approach. Yes, the rc.conf(5) scripts (whether we're talking about /etc/rc.conf, = /etc/rc.conf.local, or ones that are appended by the end-user) can be = quite complex beasts... And we could see things like this... foo=3Dbar; bar=3Dbaz; baz=3D123 And the script would not be able to find the correct instance that needs = to be replaced to get "bar" to be some new value. My nuclear-physics-type script could handle those instances (using sed = to reach into the line and replace only the baz portion and retain the = existing foo and baz declarations. What would you prefer though? Something that is cleaner, more readable, = easier to digest, more efficient, and has fewer dependencies, or one = that is more robust but may require a degree to digest? -- Cheers, Devin Teske -> CONTACT INFORMATION <- Business Solutions Consultant II FIS - fisglobal.com 510-735-5650 Mobile 510-621-2038 Office 510-621-2020 Office Fax 909-477-4578 Home/Fax devin.teske@fisglobal.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> END TRANSMISSION <- From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 9 23:46:44 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3732106564A for ; Sat, 9 Oct 2010 23:46:44 +0000 (UTC) (envelope-from eknath.iyer@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7AB9E8FC1B for ; Sat, 9 Oct 2010 23:46:44 +0000 (UTC) Received: by fxm12 with SMTP id 12so202803fxm.13 for ; Sat, 09 Oct 2010 16:46:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=9OT8juvT3PyZ0yMwFXyByiI2xR5X1dKkg7Oeks8SIFA=; b=CY0qx4hUjF/92cOEi8vMOFiTbCTbGnY4ow5AtxzE91sthEP4RTlLmtGQ82vM9UuL79 jsUKHPZny3aQlR7LwR8GofXOdXhCR6W/lSenOAuH5yY8LLHEHU/T3locARjgzGD+FuMN 6mzgJULc3H2m3tLBSrMbCUN0zSYnE0p+NS1kU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=Sv/JNFjgod4ROCsANt/TtRw0djPbrZAjaIYEbOQvDYuS67kcuC7gPBTZ4ZVblxuBUT GkTAbJZiuw91RxKEUCPOgtvM+dIE41CFa7hV8P8f3eMx7BAbla1NCUWEpbRlJIjxnYMC ZW+vVdAPy/7dtG1Ag4h9PZ81Fl2aFzTJ/apsY= MIME-Version: 1.0 Received: by 10.239.187.193 with SMTP id m1mr256489hbh.157.1286668003449; Sat, 09 Oct 2010 16:46:43 -0700 (PDT) Received: by 10.239.155.133 with HTTP; Sat, 9 Oct 2010 16:46:43 -0700 (PDT) Date: Sat, 9 Oct 2010 19:46:43 -0400 Message-ID: From: Eknath Venkataramani To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Scheduler Question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Oct 2010 23:46:45 -0000 D&I of the FreeBSD Operating System says it's gonna refer to the BSD default scheduler, the 'time share scheduler' does this mean sched_4BSD.c(In the introduction section of Chapter 4) handles only time-share process? If so, then how (or where) are the kernel processes/real time process scheduled? -- Eknath Venkataramani