From owner-freebsd-performance@FreeBSD.ORG Wed Mar 26 20:45:00 2008 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 564E51065670 for ; Wed, 26 Mar 2008 20:45:00 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from bunrab.catwhisker.org (adsl-63-193-123-122.dsl.snfc21.pacbell.net [63.193.123.122]) by mx1.freebsd.org (Postfix) with ESMTP id 19FE08FC25 for ; Wed, 26 Mar 2008 20:45:00 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from bunrab.catwhisker.org (localhost [127.0.0.1]) by bunrab.catwhisker.org (8.13.3/8.13.3) with ESMTP id m2QKVwUH006395 for ; Wed, 26 Mar 2008 13:31:58 -0700 (PDT) (envelope-from david@bunrab.catwhisker.org) Received: (from david@localhost) by bunrab.catwhisker.org (8.13.3/8.13.1/Submit) id m2QKVwtB006394 for performance@freebsd.org; Wed, 26 Mar 2008 13:31:58 -0700 (PDT) (envelope-from david) Date: Wed, 26 Mar 2008 13:31:58 -0700 From: David Wolfskill To: performance@freebsd.org Message-ID: <20080326203158.GA6302@bunrab.catwhisker.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="8t9RHnE3ZwKMSgU+" Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-Mailman-Approved-At: Wed, 26 Mar 2008 20:50:06 +0000 Cc: Subject: Are sysctl(8) values useful for measuring system resource consumption? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2008 20:45:00 -0000 --8t9RHnE3ZwKMSgU+ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable At ${work}, one of my projects is to help obtain information regarding the "developer experience," what resources are thus consumed, and figure out ways to mitigate the pain -- the objective, of course, being to help the developers be more productive within a FreeBSD environment. A couple of the perceived "pain points" are the time it takes to perform a CVS checkout and the time it takes to perform a build of the system (ours, at work -- not FreeBSD itself). As a step toward obtaining the information, I've cobbled up a Perl script that essentailly acts as a bit of "scaffolding" around time(1); the script sets things up to invoke time(1) with the "-l" flag (so we get the rusage structure information) and use "-o" to direct the output of time(1) to a file in /tmp, which the script then reads. The script then spits out a bunch of information as a single record in a CSV (Comma-Separated Variable) file (as that's the format my colleague wanted): start- and stop-timestamps, the hostname where the processes ran, the current working directory, real- and effective UIDs & GIDs; the exit code for the invoked command, the output from time(1), and (finally) the invoked command itself. (I then use a different script to read the CSV and update an RRD, then use rrdcgi(1) to generate graphs.) This has proved to be interesting, and quite possibly useful, but it merely provides a view as to the resources used by the processes being invoked from within the "scaffolding." I believe we would be well-served by also collecting information as to the resources being consumed by the system as a whole, as well -- for instance, if there's a lot of other activity on the machine in question, it might be nice to know that (and it might be even better if we had a way to characterize the rest of the workload as a whole). It would be handy if I could arrange to run vmstat(8), iostat(8), or netstat(1) in such a way that I got counters for the values immediately prior to starting the command being tested, then got a similar set of counters just after the test command completed, so I could store the "counter differences" some place handy. But that doesn't seem too readily feasible at this time. I had been trying to think of a decent way to get the overall system information for precisely the interval that I'm running the test, and then the thought occurred to me that perhaps I could invoke sysctl(8) with a suitable set of arguments both before & after invoking the process being tested; perhaps that would be a reasonable way to get information of the desired quality. I do not expect to necessarily be able to install random ports on the development machines, so there's a significant benefit to using an approach that doesn't require doing that. (I can use scripts that I write, as they are being invoked by the "test" user, from that test user's environment.) So this is a reality check: does that approach make sense? If not, what shortcomings does it have with respect to other alternatives? Please recall that the intent is to be able to place the rusage data from time(1) in a relevant context. I'm reasonably open to suggestions & alternatives. Thanks! Peace, david --=20 David H. Wolfskill david@catwhisker.org I submit that "conspiracy" would be an appropriate collective noun for cats. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --8t9RHnE3ZwKMSgU+ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (FreeBSD) iEYEARECAAYFAkfqsr0ACgkQmprOCmdXAD2nugCcDWrN/826zWt3PiBNai2wnCdu p5wAniGqHHJAJIyCt0G7pOg/2G7UzUho =BwZ1 -----END PGP SIGNATURE----- --8t9RHnE3ZwKMSgU+--