From owner-freebsd-current@FreeBSD.ORG Tue Nov 18 17:07:10 2003 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EE3DA16A4E9 for ; Tue, 18 Nov 2003 17:07:10 -0800 (PST) Received: from smtp.mho.com (smtp.mho.net [64.58.4.6]) by mx1.FreeBSD.org (Postfix) with SMTP id AF6C643FBF for ; Tue, 18 Nov 2003 17:07:09 -0800 (PST) (envelope-from scottl@freebsd.org) Received: (qmail 9127 invoked by uid 1002); 19 Nov 2003 01:07:08 -0000 Received: from unknown (HELO ?10.4.1.5?) (64.58.1.252) by smtp.mho.net with SMTP; 19 Nov 2003 01:07:08 -0000 Date: Tue, 18 Nov 2003 18:09:31 -0700 (MST) From: Scott Long X-X-Sender: scottl@pooker.samsco.home To: dyson@iquest.net In-Reply-To: <200311190021.hAJ0Lj5e000832@dyson.jdyson.com> Message-ID: <20031118175434.A35215@pooker.samsco.home> References: <200311190021.hAJ0Lj5e000832@dyson.jdyson.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: current@freebsd.org cc: "M. Warner Losh" Subject: Re: Unfortunate dynamic linking for everything X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Nov 2003 01:07:11 -0000 On Tue, 18 Nov 2003 dyson@iquest.net wrote: > Scott Long said: > > On Tue, 18 Nov 2003 dyson@iquest.net wrote: > > > M. Warner Losh said: > > > > In message: <200311181307.hAID7uHa032514@dyson.jdyson.com> > > > > dyson@iquest.net writes: > > > > : It really doesn't make sense to arbitrarily cut-off a > > > > : discussion especially when a decision might be incorrect. > > > > > > > > I'd say that good technical discussion about why this is wrong would > > > > be good. However, emotional ones should be left behind. Except for > > > > John's message, most of the earlier messages have been more emotional > > > > than technical. > > > > > > > > John, do you have any good set of benchmarks that people can run to > > > > illustrate your point? > > > > > > > I'll see if I can find some of my old benchmarks (from memory, > > > not disk -- lots of stuff has rotted away.). I had hoped > > > to avoid doing much in the way of proof. Pointing to the > > > much more complex process map along with the implication > > > for significantly higher overhead :-). (The VM code generally > > > gets slower with more complex process maps and more memory used. > > > For smallish programs -- including those with a few shared libs, > > > changing the VM code won't help much, because the cost is built > > > in to the complexity of the process image.) > > > > I'm glad that real arguments are finally starting to surface =-) > > > > Our rationale for encouraging Gordon is as follows: > > > > 1. 4.x upgrade path: As we approach 5-STABLE, a lot of users might want > > to upgrade from 4-STABLE. Historically in 4.x, the / partition has > > been very modest in size. One just simply cannot cram the bloat that > > has grown in 5.x into a 4.x partition scheme. Of course there is the > > venerable 'dump - clean install - restore' scheme, but we were looking > > for something a little more user-friendly. > > > /bin and /sbin don't need to contain everything :-). Adding a full > featured /rescue should also help to counterbalance this (but maybe > the upgrade would be selective?) There are reasons for /usr/bin > and /usr/sbin (along with /usr/local, /usr/share.) > > Disk space is CHEAP and grandfathering 20MB hard drives (or the > yr2000 equivalent of 4GB hard drives) is nice -- but repartitioning > them with more reasonable root allocations seems like a good > idea anyway :-). 50MB root was silly when 2-4GB disks were common. > > Continuing to pay for a poorly made partitioning decision in the past > seems to be unworthy (in my opinion.) /boot has grown quite large too and threatens to be unbounded in size as times go on. Shaving off the 30-40MB of size in /bin and /sbin can help alleviate this, even on system formatted in 5.x partition sized. This argument wasn't the most compelling one by itself, but it played a part in the decision so I included it here. > > > > > 2. NSS support out-of-the-box: Again, this is a user-experience issue > > in that forcing NSS users to recompile world was seen as a potential > > roadblock to them. > > > The cool thing about properly implemented shared libs is that not everything > has to be shared. Even the kernel supports runtime loading/addition > of modules, and the NSS sounds like a good candidate for a library > feature. Burdening everything with the sparse .data associated with > libc seems to be a waste (but, perhaps the performance that was carefully > crafted into the codebase is no longer important?) Really -- it is > possible that performance isnt' important anymore, and fully accepting > the ongoing loss of performance for features (without careful tradeoffs) > might be an appropriate strategy. (In the case of the NSS stuff, it > shouldn't require everything to be built dynamic.) Unfortunately, our dynamic linker prevents dlopen() and friends from working in a static binary. There was some talk about having NSS lookups happen through a proxy process, but our implementation just plain wasn't done that way, and no one has stepped forward with alternate code. NSS modules can affect a surprisingly large number of utilities, including the shells. Selectively choosing which utilities to link static vs. shared would have resulted in quite a patchy result that probably would not have had a significant benefit. > > > > > 3. Binary security updates: there is a lot of interest in providing a > > binary update mechanism for doing security updates. Having a dynamic > > root means that vulnerable libraries can be updated without having to > > update all of the static binaries that might use them. > > > The additional hole of exploiting the system through the shared libs > is a negative tradeoff. Exploits in libraries happen though. The LD_LIBRARY_PATH attack is an old one that most Unixes are hopefully hardened against. > > > > > 4. I've probably forgotten the other factors... > > > > As for performance, we felt that the typical use of FreeBSD did not fall > > into the category of doing forkbomb performance tests. > > > Nope - even shell execution times (e.g. long shell scripts) will see a > difference. Prejudicial comments like 'fork-bombs' and > 'microbenchmarks' aren't really fair. Proper benchmarking is > time consuming and analysis is tricky. System loading conditions > are difficult to control (for experiments.) I apologize for the poor choice of words, I certainly did not mean to insult you or anyone else. > > Note that alot of the lost performance is hidden by the VM code (e.g. one > of the purposes for prezeroing on SMP box -- before SMP was fine grained.) > The cost of sparse data and needless inheritance of modified pages > has performance hits that are incredibly difficult to accurately (and > honestly) measure. Anyone can create 'preordained' benchmark results. > > This is one reason why I am loathe to get into the FreeBSD performance > game again -- it is much more difficult than a few micro-benchmarks :-) > that simply measure a fork exec time. > > John > > > I agree that reaching for improved performance is a beneficial goal, and that tradeoffs of performance vs. features must be carefully considered and balanced. With FreeBSD's popularity as a server platform, things like integrated NSS were too long in coming and were very compelling reasons to go this route. So it looks like the performance vs. features see-saw has tipped from one side to the other. Backing out dynamic root will not help rebalance it though. Again, I hope that this will inspire some graduate student somewhere to experiment with optimizing the VM system in this direction. Scott