From owner-freebsd-current@FreeBSD.ORG Tue Nov 18 20:47:45 2003 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC5F616A4CE; Tue, 18 Nov 2003 20:47:45 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 333A543FE5; Tue, 18 Nov 2003 20:47:44 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9p2/8.12.9) with ESMTP id hAJ4jYMg002782; Tue, 18 Nov 2003 23:45:34 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)hAJ4jYXb002779; Tue, 18 Nov 2003 23:45:34 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Tue, 18 Nov 2003 23:45:34 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: David Schultz In-Reply-To: <20031119040135.GA63031@VARK.homeunix.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: dyson@iquest.net cc: current@FreeBSD.ORG cc: "M. Warner Losh" Subject: Re: Unfortunate dynamic linking for everything X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Nov 2003 04:47:46 -0000 On Tue, 18 Nov 2003, David Schultz wrote: > On Tue, Nov 18, 2003, Robert Watson wrote: > > (2) Shells again, because they will be fork()d and exec()d frequently > > during heavily scripted activities, such as system boot, periodic > > events, large make jobs, etc. And presumably the only shell of > > interest is sh, although some of the supporting non-builtin binaries > > may also be of interest. > > This is the only performance argument that truly seems compelling, > particularly since fork()/exec() performance for dynamic binaries in > FreeBSD really sucks as compared to Solaris and AIX (and presumably > other systems that have gone dynamic long ago). I think this is mostly > an aspect of the dynamic linker and the lack of prebinding rather than > performance problems in the VM system. Some of Matthew Dodd's exploration of prebinding seems to indicate it's a big win for large applications with many libraries (XWindows, KDE, OpenOffice, whathaveyou), but that the base cost of loading relocation caches overwhelms the benefits of prebinding. More refinement could correct this, perhaps. Part of the cost of excessive cost of prebinding right now is that we almost always map libraries to different addresses for different programs, which results in a very large relocation cache. A quick sample on my notebook reveals libc mapped at no less than 28 different base addresses. On systems like Mac OS X, use of a "shared region" permits not only use of prebinding, but assumptions of common load addresses for common libraries. In addition, the "shared region" approach allows the reuse of page table and VM meta-data across many processes. Leaving aside special-purpose optimizations like that, using some simple heuristics, we could probably easily reduce the number of load addressed dramatically, making prebinding a far more feasible approach, and something we should explore thoroughly. My feeling is we should all go away for a day or two, and run our favorite macro-benchmark on our favorite sensitive dynamic linking-sensitive application. I.e., buildworld, system boot, periodic, or whatever. Attempt to focus on the actual performance loss, both the cause, and potential solutions. Returning to a static /bin/sh would be one potential solution. John Dyson's suggestion of "mixed mode" linking, in which we statically link known required parts of binaries, but permit dynamic linking of "plug-in" functionality is another quite reasonable solution, as long as we take care of the potential issue of "plug-ins" relying on symbols from a library that is statically linked to the application, but not in the set of symbols depended on by the application. Remaining with dynamic linking is another possible "solution" if macrobenchmarks don't reveal an interesting hit. If I had to guess, I think it probably comes down to comparing the costs of repeated fork/exec of /bin/sh, and that the "mixed mode" solution may be the easiest way to address a possible performance problem while retaining the dynamic linking benefits. But I want to be sure we address a real-world performance problem, not an anticipated one. For big applications like KDE, Open Office, et al, prebinding probably is the right approach, and hopefully we have plenty of time before 5.3 to refine Matthew's work and see if we can't get things up to speed. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories