From owner-freebsd-arch@FreeBSD.ORG Sun Nov 23 13:22:33 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBE141065670 for ; Sun, 23 Nov 2008 13:22:33 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.188]) by mx1.freebsd.org (Postfix) with ESMTP id 4F1678FC14 for ; Sun, 23 Nov 2008 13:22:33 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id h3so818792nfh.33 for ; Sun, 23 Nov 2008 05:22:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender :to:subject:cc:mime-version:content-type:content-transfer-encoding :content-disposition:x-google-sender-auth; bh=GUA/6fgoHqfCV20GQYQEl5wguGg66gd+Y21IlcGFl/8=; b=Gjv6kjVr3uMPJcY+98g0ECmiue8NM2mlBGGOXaRjP2BvGzlZQBaHBsOsqwGcPnIwXT a/B+Hvos5g/5ymUu5duo3ZOFL7FdrGhMHl3ggUgHzxCA4eJmpUC2GnQ+7M6zucYlKZnG Qy+5/6z9wMCfGPn+4XH3QYvuZTS85JvJhnRfc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:mime-version:content-type :content-transfer-encoding:content-disposition:x-google-sender-auth; b=Mm06V8k0UYkHpTWRsH7c1oSAk6Kjc/IGO7PvqdvZtdgc4Dst/8YQIsBIXhMzNm3EWl RPDVc2Gmya2OyRPwF+wssjA5rkqJOzpLzWDDP5sWvUUIdvgYSJp6byMlRV3N9ROHOmfa mYASwk60jrQ1HUljf/cgAHEUbuankrGTMkspo= Received: by 10.86.50.6 with SMTP id x6mr1641852fgx.71.1227445342411; Sun, 23 Nov 2008 05:02:22 -0800 (PST) Received: by 10.86.30.17 with HTTP; Sun, 23 Nov 2008 05:02:22 -0800 (PST) Message-ID: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> Date: Sun, 23 Nov 2008 14:02:22 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "FreeBSD Arch" , freebsd-performance@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Google-Sender-Auth: a49a944b50fabd0c Cc: Joseph Koshy Subject: [PATCH] pmcannotate tool X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2008 13:22:33 -0000 pmcannotate is a tool that prints out sources of a tool (in C or assembly) with inlined profiling informations retrieved by a prior pmcstat analysis. If compared with things like callgraph generation, it prints out profiling on a per-instance basis and this can be useful to find, for example, badly handled caches, too high latency instructions, etc. The tool usage is pretty simple: pmcannotate [-a] [-h] [-k path] [-l level] samples.out binaryobj where samples.out is a pmcstat raw output and binaryobj is the binary object that has been profiled and is accessible for (ELF) symbols retrieving. The options are better described in manpages but briefly: - a: performs analysis on the assembly rather than the C source - h: usage and informations - k: specify a path for the kernel in order to locate correct objects for it - l: specify a lower boundary (in total percentage time) after which functions will be displayed nomore. A typical usage of pmcannotate can be some way of kernel annotation. For example, you can follow the steps below: 1) Generate a pmc raw output of system samples: # pmcstat -S ipm-unhalted-core-cycles -O samples.out 2) Copy the samples in the kernel building dir and cd there # cp samples.out /usr/src/sys/i386/compile/GENERIC/ ; cd /usr/src/sys/i386/compile/GENERIC/ 3) Run pmcannotate # pmcannotate -k . samples.out kernel.debug > kernel.ann In the example above please note that kernel.debug has to be used in order to produce a C annotated source. This happens because in order to get the binary sources we rely on the "objdump -S" command which wants binary compiled with debugging options. If not debugging options are present assembly analynsis is still possible, but no C-backed one will be available. objdump is not the only one tool on which pmcannotare rely. Infact, in order to have it working, pmcstat needs to be present too because we need to retrieve, from the pmcstat raw output, informations about the sampled PCs (in particular the name of the function they live within, its start and ending addresses). As long as currently pmcstat doesn't return those informations, a new option has been added to the tool (-m) which can extract (from a raw pmcstat output) all pc sampled, name of the functions and symbol bundaries they live within. Also please note that pmcannotate suffers of 2 limitations. Firstly, relying on objdump to dump the C source, with heavy optimization levels and lots of inlines the code gets difficult to read. Secondly, in particular on x86 but I guess it is not the only one case, the sample is always attributed to the instruction directly following the one that was interrupted. So in a C source view some samples may be attributed to the line below the one you're interested in. It's also important to keep in mind that if a line is a jump target or the start of a function the sample really belongs elsewhere. The patch can be found here: http://www.freebsd.org/~attilio/pmcannotate.diff/ where pmcannotate/ dir contains the code and needs to go under /usr/src/usr.sbin/ and the patch has diffs against pmcstat and Makefile. This work has been developed on the behalf of Nokia with important feedbacks and directions from Jeff Roberson. Testing and feedbacks (before it hits the tree) are welcome. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Sun Nov 23 14:11:37 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A61A1065679 for ; Sun, 23 Nov 2008 14:11:37 +0000 (UTC) (envelope-from joseph.koshy@gmail.com) Received: from gv-out-0910.google.com (gv-out-0910.google.com [216.239.58.185]) by mx1.freebsd.org (Postfix) with ESMTP id 3423E8FC21 for ; Sun, 23 Nov 2008 14:11:36 +0000 (UTC) (envelope-from joseph.koshy@gmail.com) Received: by gv-out-0910.google.com with SMTP id n8so287221gve.39 for ; Sun, 23 Nov 2008 06:11:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=5IdbVNNiGGrSFSsWNGygTuCD6GV+CDYtxSU8hcDy9/E=; b=Zdu5w7Io3sezjtOsht+Ax6EJ5RoTyBthuNy28vh4ZRjVW6hCj3PvUqqa+IJ6UtRZui SAWMPnavY1tbG9duXKrsSXYvLDgYNTJvyK65ILThi9A4YswkjNysAmwS+NLZbIZKGBF+ Vh4UvAmMo/CdULvkbu+gVUgZd2JbsLPbts5QQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=rPBTuf/nfNcEtZrfYc8BVmR2tprGyGX1iiOMWFYXxj0Rt72pPLyct3E+ULEgeeQGwt IPOM1HIaVkcHZZkWNpBa9aXCaEj+xDvRyJAS5p1VDSwi4H4n6vhCbDVNvOO60Ipcrah5 aOzAoDubF/O4gJ6QTV9Bg0Hx3tHDjGkkY6z2g= Received: by 10.86.100.19 with SMTP id x19mr1651409fgb.29.1227447898349; Sun, 23 Nov 2008 05:44:58 -0800 (PST) Received: by 10.86.96.13 with HTTP; Sun, 23 Nov 2008 05:44:58 -0800 (PST) Message-ID: <84dead720811230544l625a565w158ccc1833b73799@mail.gmail.com> Date: Sun, 23 Nov 2008 19:14:58 +0530 From: "Joseph Koshy" To: "Attilio Rao" In-Reply-To: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> Cc: FreeBSD Arch , freebsd-performance@freebsd.org Subject: Re: [PATCH] pmcannotate tool X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2008 14:11:37 -0000 > pmcannotate is a tool that prints out sources of a tool (in C or > assembly) with inlined profiling informations retrieved by a prior > pmcstat analysis. [snip] > This work has been developed on the behalf of Nokia with important > feedbacks and directions from Jeff Roberson. Are you "rookie" on IRC, who used to catch me with tons of questions whenever I logged on? > Testing and feedbacks (before it hits the tree) are welcome. The pmcstat changes seem fine. Koshy From owner-freebsd-arch@FreeBSD.ORG Sun Nov 23 17:39:16 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D642C106564A for ; Sun, 23 Nov 2008 17:39:16 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from agogare.doit.wisc.edu (agogare.doit.wisc.edu [144.92.197.211]) by mx1.freebsd.org (Postfix) with ESMTP id AECD08FC0A for ; Sun, 23 Nov 2008 17:39:16 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=ISO-8859-1; format=flowed Received: from avs-daemon.smtpauth2.wiscmail.wisc.edu by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 32bit)) id <0KAS00J02OXFPM00@smtpauth2.wiscmail.wisc.edu> for freebsd-arch@freebsd.org; Sun, 23 Nov 2008 10:39:15 -0600 (CST) Received: from trantor.tachypleus.net (adsl-99-154-3-101.dsl.mdsnwi.sbcglobal.net [99.154.3.101]) by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 32bit)) with ESMTPSA id <0KAS00IJLOXEFW00@smtpauth2.wiscmail.wisc.edu> for freebsd-arch@freebsd.org; Sun, 23 Nov 2008 10:39:15 -0600 (CST) Date: Sun, 23 Nov 2008 10:40:27 -0600 From: Nathan Whitehorn To: freebsd-arch@freebsd.org Message-id: <4929877B.6060307@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=99.154.3.101 X-Spam-PmxInfo: Server=avs-13, Version=5.4.2.344556, Antispam-Engine: 2.6.0.325393, Antispam-Data: 2008.11.23.162221, SenderIP=99.154.3.101 User-Agent: Thunderbird 2.0.0.17 (X11/20080928) Subject: Enumerable I2C busses X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2008 17:39:16 -0000 On Apple's PowerPC systems, the firmware device tree helpfully enumerates the system's I2C busses. Marco Trillo has recently written a driver for one of the system's I2C controllers in order to support the attached audio codecs, and I'm trying to figure out the best way to import it. The current I2C bus mechanism does not support the bus adding its own children and instead relies on hints or other out-of-band information for device attachment. It would be nice to do something like what the firmware-assisted PCI bus drivers do (ofw_pci, for instance): hijack child enumeration from the MI layer and attach information from the firmware. However, since all current I2C drivers' probe() routines return 0, I can't simply add the firmware devices, because as soon as the probe() methods of the existing drivers are called, they will take over all the devices on the bus. What is the best way to handle this? -Nathan From owner-freebsd-arch@FreeBSD.ORG Sun Nov 23 18:34:49 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C507A1065673; Sun, 23 Nov 2008 18:34:49 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 886438FC0A; Sun, 23 Nov 2008 18:34:49 +0000 (UTC) (envelope-from des@des.no) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 9A58D6D43F; Sun, 23 Nov 2008 18:18:25 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 67C428449F; Sun, 23 Nov 2008 19:18:25 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Nathan Whitehorn References: <4929877B.6060307@freebsd.org> Date: Sun, 23 Nov 2008 19:18:25 +0100 In-Reply-To: <4929877B.6060307@freebsd.org> (Nathan Whitehorn's message of "Sun, 23 Nov 2008 10:40:27 -0600") Message-ID: <86myfq9uha.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: Enumerable I2C busses X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2008 18:34:49 -0000 Nathan Whitehorn writes: > The current I2C bus mechanism does not support the bus adding its own > children [...] That's because the I2C protocol does not support device enumeration or identification. You have to know in advance what kind of devices are attached and at what address. Even worse, it is not uncommon for similar but not entirely compatible devices to use the same I2C address (for instance, every I2C-capable RTC chip uses the same address, even though they have different feature sets) DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Sun Nov 23 19:28:01 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3118E1065670 for ; Sun, 23 Nov 2008 19:28:01 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout021.mac.com (asmtpout021.mac.com [17.148.16.96]) by mx1.freebsd.org (Postfix) with ESMTP id 1DCE78FC14 for ; Sun, 23 Nov 2008 19:28:01 +0000 (UTC) (envelope-from xcllnt@mac.com) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Received: from [192.168.1.95] (209-128-86-226.BAYAREA.NET [209.128.86.226]) by asmtp021.mac.com (Sun Java(tm) System Messaging Server 6.3-7.03 (built Aug 7 2008; 32bit)) with ESMTPSA id <0KAS00424TYN2I00@asmtp021.mac.com>; Sun, 23 Nov 2008 10:28:01 -0800 (PST) Message-id: From: Marcel Moolenaar To: Nathan Whitehorn In-reply-to: <4929877B.6060307@freebsd.org> Date: Sun, 23 Nov 2008 10:27:59 -0800 References: <4929877B.6060307@freebsd.org> X-Mailer: Apple Mail (2.929.2) Cc: freebsd-arch@freebsd.org Subject: Re: Enumerable I2C busses X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2008 19:28:01 -0000 On Nov 23, 2008, at 8:40 AM, Nathan Whitehorn wrote: > On Apple's PowerPC systems, the firmware device tree helpfully > enumerates the system's I2C busses. Marco Trillo has recently > written a driver for one of the system's I2C controllers in order to > support the attached audio codecs, and I'm trying to figure out the > best way to import it. > > The current I2C bus mechanism does not support the bus adding its > own children and instead relies on hints or other out-of-band > information for device attachment. It would be nice to do something > like what the firmware-assisted PCI bus drivers do (ofw_pci, for > instance): hijack child enumeration from the MI layer and attach > information from the firmware. > > However, since all current I2C drivers' probe() routines return 0, I > can't simply add the firmware devices, because as soon as the > probe() methods of the existing drivers are called, they will take > over all the devices on the bus. > > What is the best way to handle this? I think the best approach is to have the probe() method actually test for something. This is the standard thing to do when the bus does not know a priori which driver to instantiate. I believe that the bus should never create a child of a specific devclass, unless instructed by the user (read: pre-binding directives). For ocpbus(4) (see sys/powerpc/mpc85xx/ocpbus.c and sys/powerpc/include/ocpbus.h), we created simple defines to identify the hardware and use that in the probe() method to bind the driver to the right hardware, as in: In uart(4), for example we have: ... error = BUS_READ_IVAR(parent, dev, OCPBUS_IVAR_DEVTYPE, &devtype); if (error) return (error); if (devtype != OCPBUS_DEVTYPE_UART) return (ENXIO); ... I'm not saying to copy it, but it does demonstrate that you can trivially implement something that eliminates the assumption that the bus instantiates children of a particular devclass and thus that the probe() method can always return 0. What we do need is a generic way (such as the OF device tree) to describe the hardware, its resource needs and how it's all wired together (think interrupt routing). -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Sun Nov 23 20:11:41 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99834106564A; Sun, 23 Nov 2008 20:11:41 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id 3E0298FC12; Sun, 23 Nov 2008 20:11:41 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (pD9E2D989.dip.t-dialin.net [217.226.217.137]) by redbull.bpaserver.net (Postfix) with ESMTP id 094882E0B5; Sun, 23 Nov 2008 20:56:08 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id E93A514823B; Sun, 23 Nov 2008 20:56:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1227470164; bh=hNCOUucfa+mQlnF2QCKnj5y9NjjX5ZaVy cNLhkaor4E=; h=Message-ID:Date:From:To:Cc:Subject:References: In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=uriUDcRA/aekMoWVLXs7RJoFpqguWplrb86NXd2kS5BC95Rjwxi9NqQrQ2HdKmDAS wDZ70A71IVrDzsVe72GLNbf7thQATkKbbHVvApodoaeOcepy5Hnj6Bgt7JoQoX8kiI3 pih1y56x1CXa1SH4jTnd7xA0kAawGVAkZkEb3Syab+LIRsR5GGhop3K4ITkmMzpxAg4 GosMoHM9n1D3rv1CENnCaEpb7E4Q+RfFou64gGPjrgeQlsmv0Lt9r6Gigc9JepdAjTg ft0L1LKQc0v4kmJzfcD40BfFFqK1Iy0iIWcZ5NAfXp84yHfowaA/b/ebflLkKBtDMUF 03bFYZ/dA== Received: (from www@localhost) by webmail.leidinger.net (8.14.2/8.13.8/Submit) id mANJu3aG070936; Sun, 23 Nov 2008 20:56:03 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from Luna.Leidinger.net (Luna.Leidinger.net [192.168.2.100]) by webmail.leidinger.net (Horde Framework) with HTTP; Sun, 23 Nov 2008 20:56:03 +0100 Message-ID: <20081123205603.17752y578er4bcqo@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Sun, 23 Nov 2008 20:56:03 +0100 From: Alexander Leidinger To: Attilio Rao References: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> In-Reply-To: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.3) / FreeBSD-8.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-MailScanner-ID: 094882E0B5.2388B X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, ORDB-RBL, SpamAssassin (not cached, score=-14.9, required 6, BAYES_00 -15.00, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: FreeBSD Arch , freebsd-performance@freebsd.org, Joseph Koshy Subject: Re: [PATCH] pmcannotate tool X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2008 20:11:41 -0000 Quoting Attilio Rao (from Sun, 23 Nov 2008 =20 14:02:22 +0100): > pmcannotate is a tool that prints out sources of a tool (in C or > assembly) with inlined profiling informations retrieved by a prior > pmcstat analysis. > If compared with things like callgraph generation, it prints out > profiling on a per-instance basis and this can be useful to find, for > example, badly handled caches, too high latency instructions, etc. Can this also be used to do some code coverage analysis? What I'm =20 interested in is to enable something, run some tests in userland, =20 disable this something, and then run a tool which tells me which parts =20 of specific functions where run or not. At first I hoped I can use dtrace for this... I had a dtrace training =20 and seen the userland probes in action, where you can trace every ASM =20 instruction, but unfortunately you can not do this with kernel probes. =20 I tried with fbt and syscall on a Solaris 10 machine. I haven't tested =20 with FreeBSD-dtrace yet, but I doubt it is more advanced in this =20 regard than the Solaris dtrace. So I'm still searching. Bye, Alexander. --=20 We should keep the Panama Canal. After all, we stole it fair and square. =09=09-- S. I. Hayakawa http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-arch@FreeBSD.ORG Sun Nov 23 20:30:06 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 821181065673 for ; Sun, 23 Nov 2008 20:30:06 +0000 (UTC) (envelope-from raj@semihalf.com) Received: from semihalf.com (semihalf.com [206.130.101.55]) by mx1.freebsd.org (Postfix) with ESMTP id 571F78FC0C for ; Sun, 23 Nov 2008 20:30:06 +0000 (UTC) (envelope-from raj@semihalf.com) Received: from mail.semihalf.com (mail.semihalf.com [83.15.139.206]) by semihalf.com (8.13.1/8.13.1) with ESMTP id mANKJ8W6014478; Sun, 23 Nov 2008 13:19:09 -0700 Received: from apn-77-112-41-180.gprs.plus.pl (apn-77-112-41-180.gprs.plus.pl [77.112.41.180]) by mail.semihalf.com (Postfix) with ESMTP id C4F9514C66; Sun, 23 Nov 2008 21:34:49 +0100 (CET) Message-Id: From: =?ISO-8859-2?Q?Rafa=B3_Jaworowski?= To: =?UTF-8?Q?Dag-Erling_Sm=C3=B8rgrav?= In-Reply-To: <86myfq9uha.fsf@ds4.des.no> Content-Type: text/plain; charset=UTF-8; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v929.2) Date: Sun, 23 Nov 2008 21:18:54 +0100 References: <4929877B.6060307@freebsd.org> <86myfq9uha.fsf@ds4.des.no> X-Mailer: Apple Mail (2.929.2) Cc: Nathan Whitehorn , freebsd-arch@freebsd.org Subject: Re: Enumerable I2C busses X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2008 20:30:06 -0000 On 2008-11-23, at 19:18, Dag-Erling Sm=C3=B8rgrav wrote: > Nathan Whitehorn writes: >> The current I2C bus mechanism does not support the bus adding its own >> children [...] > > That's because the I2C protocol does not support device enumeration or > identification. You have to know in advance what kind of devices are > attached and at what address. Even worse, it is not uncommon for > similar but not entirely compatible devices to use the same I2C =20 > address > (for instance, every I2C-capable RTC chip uses the same address, even > though they have different feature sets) Well, hard-coded addresses and conflicting assignments between vendors =20= do not technically prevent from scanning the bus; actually, our =20 current iicbus code can do bus scaning when compiled with a diag =20 define. The problem however is some slave devices are not well-=20 behaved, and they don't like to be read/written to other than in very =20= specific scenario: if polled during bus scan strange effects occur =20 e.g. they disappear from the bus, or do not react to consecutive =20 requests etc. Nathan, not sure if this helps you, but I have a nice i2c diagnostic =20 tool, which among other features lets the user scan the I2C bus for =20 present slave devices. This is done from userland, so doing similar =20 thing in-kernel wouldn't be a problem. I was planning to post this for =20= review this coming week, so you can have a look. Rafal= From owner-freebsd-arch@FreeBSD.ORG Sun Nov 23 20:49:55 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EEDA1065672 for ; Sun, 23 Nov 2008 20:49:55 +0000 (UTC) (envelope-from raykinsella78@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.27]) by mx1.freebsd.org (Postfix) with ESMTP id B8BD18FC08 for ; Sun, 23 Nov 2008 20:49:54 +0000 (UTC) (envelope-from raykinsella78@gmail.com) Received: by qw-out-2122.google.com with SMTP id 9so353594qwb.7 for ; Sun, 23 Nov 2008 12:49:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type:references; bh=i4ublRCwiw9D9OQd633fea8vGQGtOGSMvifdDSeMCNE=; b=XeMyeKmttouQzTePepSivJNzgUu/tjcxfdbEAwf9OG73Ysp6rKwYwWZEUfa5swn0Nl 54Hb0tEB4MISnNJOGsE2elI8hIq49NN8eaF/Aaw7JgYDA/JKMuWsEqin31u9Su7mctGp TK5tQHKndgA9rxIPVQfS38jnUz358BC7pSjZA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:references; b=ZbZZco9WWhNxt5e5QzrfcuMmWuOVSxMjQNq7Sl5LEsqecYTRZ7kkvGTpixu23c3Kc1 leyc0570ScWpc+/1yGDEVxrsL/X5crx/0AW2lCd3sv0VGQRwX2hhKVnRD2NeSozKDty6 iXBuWlsk4vEZjs+cip3M5V3brdTVKqbZKWC9g= Received: by 10.214.215.9 with SMTP id n9mr1587652qag.100.1227471989265; Sun, 23 Nov 2008 12:26:29 -0800 (PST) Received: by 10.214.181.14 with HTTP; Sun, 23 Nov 2008 12:26:29 -0800 (PST) Message-ID: <584ec6bb0811231226x7d1b53ccgc5e28bfc297009a2@mail.gmail.com> Date: Sun, 23 Nov 2008 20:26:29 +0000 From: "Ray Kinsella" To: "Attilio Rao" In-Reply-To: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> MIME-Version: 1.0 References: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FreeBSD Arch , freebsd-performance@freebsd.org, Joseph Koshy Subject: Re: [PATCH] pmcannotate tool X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2008 20:49:55 -0000 I know I am going to really show my FreeBSD ignorance here, but this is a patch of FreeBSD 8.0 Current isn't it ? Thanks Ray Kinsella On Sun, Nov 23, 2008 at 1:02 PM, Attilio Rao wrote: > pmcannotate is a tool that prints out sources of a tool (in C or > assembly) with inlined profiling informations retrieved by a prior > pmcstat analysis. > If compared with things like callgraph generation, it prints out > profiling on a per-instance basis and this can be useful to find, for > example, badly handled caches, too high latency instructions, etc. > > The tool usage is pretty simple: > pmcannotate [-a] [-h] [-k path] [-l level] samples.out binaryobj > > where samples.out is a pmcstat raw output and binaryobj is the binary > object that has been profiled and is accessible for (ELF) symbols > retrieving. > The options are better described in manpages but briefly: > - a: performs analysis on the assembly rather than the C source > - h: usage and informations > - k: specify a path for the kernel in order to locate correct objects for > it > - l: specify a lower boundary (in total percentage time) after which > functions will be displayed nomore. > > A typical usage of pmcannotate can be some way of kernel annotation. > For example, you can follow the steps below: > 1) Generate a pmc raw output of system samples: > # pmcstat -S ipm-unhalted-core-cycles -O samples.out > 2) Copy the samples in the kernel building dir and cd there > # cp samples.out /usr/src/sys/i386/compile/GENERIC/ ; cd > /usr/src/sys/i386/compile/GENERIC/ > 3) Run pmcannotate > # pmcannotate -k . samples.out kernel.debug > kernel.ann > > In the example above please note that kernel.debug has to be used in > order to produce a C annotated source. This happens because in order > to get the binary sources we rely on the "objdump -S" command which > wants binary compiled with debugging options. > If not debugging options are present assembly analynsis is still > possible, but no C-backed one will be available. > objdump is not the only one tool on which pmcannotare rely. Infact, in > order to have it working, pmcstat needs to be present too because we > need to retrieve, from the pmcstat raw output, informations about the > sampled PCs (in particular the name of the function they live within, > its start and ending addresses). As long as currently pmcstat doesn't > return those informations, a new option has been added to the tool > (-m) which can extract (from a raw pmcstat output) all pc sampled, > name of the functions and symbol bundaries they live within. > > Also please note that pmcannotate suffers of 2 limitations. > Firstly, relying on objdump to dump the C source, with heavy > optimization levels and lots of inlines the code gets difficult to > read. Secondly, in particular on x86 but I guess it is not the only > one case, the sample is always attributed to the instruction directly > following the one that was interrupted. So in a C source view some > samples may be attributed to the line below the one you're interested > in. It's also important to keep in mind that if a line is a jump > target or the start of a function the sample really belongs elsewhere. > > The patch can be found here: > http://www.freebsd.org/~attilio/pmcannotate.diff/ > > where pmcannotate/ dir contains the code and needs to go under > /usr/src/usr.sbin/ and the patch has diffs against pmcstat and > Makefile. > > This work has been developed on the behalf of Nokia with important > feedbacks and directions from Jeff Roberson. > > Testing and feedbacks (before it hits the tree) are welcome. > > Thanks, > Attilio > > > -- > Peace can only be achieved by understanding - A. Einstein > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to " > freebsd-performance-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Sun Nov 23 21:09:37 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2AF91065674 for ; Sun, 23 Nov 2008 21:09:37 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from adsum.doit.wisc.edu (adsum.doit.wisc.edu [144.92.197.210]) by mx1.freebsd.org (Postfix) with ESMTP id B3A3A8FC08 for ; Sun, 23 Nov 2008 21:09:37 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) MIME-version: 1.0 Content-transfer-encoding: 8BIT Content-type: text/plain; charset=UTF-8; format=flowed Received: from avs-daemon.smtpauth1.wiscmail.wisc.edu by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 32bit)) id <0KAT00E001G1WB00@smtpauth1.wiscmail.wisc.edu> for freebsd-arch@freebsd.org; Sun, 23 Nov 2008 15:09:37 -0600 (CST) Received: from trantor.tachypleus.net (adsl-99-154-3-101.dsl.mdsnwi.sbcglobal.net [99.154.3.101]) by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 32bit)) with ESMTPSA id <0KAT00DGD1FZVJ00@smtpauth1.wiscmail.wisc.edu>; Sun, 23 Nov 2008 15:09:36 -0600 (CST) Date: Sun, 23 Nov 2008 15:10:48 -0600 From: Nathan Whitehorn In-reply-to: To: =?UTF-8?B?UmFmYcWCIEphd29yb3dza2k=?= Message-id: <4929C6D8.7090305@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=99.154.3.101 X-Spam-PmxInfo: Server=avs-9, Version=5.4.1.325704, Antispam-Engine: 2.6.0.325393, Antispam-Data: 2008.11.23.205819, SenderIP=99.154.3.101 References: <4929877B.6060307@freebsd.org> <86myfq9uha.fsf@ds4.des.no> User-Agent: Thunderbird 2.0.0.17 (X11/20080928) Cc: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= , freebsd-arch@freebsd.org Subject: Re: Enumerable I2C busses X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2008 21:09:38 -0000 Rafał Jaworowski wrote: > > On 2008-11-23, at 19:18, Dag-Erling Smørgrav wrote: > >> Nathan Whitehorn writes: >>> The current I2C bus mechanism does not support the bus adding its own >>> children [...] >> >> That's because the I2C protocol does not support device enumeration or >> identification. You have to know in advance what kind of devices are >> attached and at what address. Even worse, it is not uncommon for >> similar but not entirely compatible devices to use the same I2C address >> (for instance, every I2C-capable RTC chip uses the same address, even >> though they have different feature sets) > > Well, hard-coded addresses and conflicting assignments between vendors > do not technically prevent from scanning the bus; actually, our current > iicbus code can do bus scaning when compiled with a diag define. The > problem however is some slave devices are not well-behaved, and they > don't like to be read/written to other than in very specific scenario: > if polled during bus scan strange effects occur e.g. they disappear from > the bus, or do not react to consecutive requests etc. All of this is true, but perhaps my question was badly worded. What I am trying to figure out is how to shove information from an out-of-band source (Open Firmware, in this case) into newbus without disrupting existing code. In that way, my question is not I2C specific -- we run into the same issue with the Open Firmware nexus node and pseudo-devices like cryptosoft that attach themselves. What I want to do is to have the I2C bus add the children that the firmware says it has. What the firmware cannot tell in advance, however, is which FreeBSD driver is responsible for those devices, and so the I2C bus driver can't know that without a translation table that I would prefer not to hack in to the bus driver. It seems reasonable to allow devices to use a real probe routine to look at the firmware's name and compatible properties, like we allow on other Open Firmware busses. The trouble is that existing drivers don't do this, because they expect to be attached with hints, so they will attach to all devices. I'm trying to figure out how to avoid this. My basic question comes down to whether there is a good way in newbus to handle busses that may be wholly or partially enumerated by firmware or some other method, and may also have devices that can only attach themselves if told to by hints. > Nathan, not sure if this helps you, but I have a nice i2c diagnostic > tool, which among other features lets the user scan the I2C bus for > present slave devices. This is done from userland, so doing similar > thing in-kernel wouldn't be a problem. I was planning to post this for > review this coming week, so you can have a look. It's not directly useful, no, but that's a very useful tool that will be handy to have. -Nathan From owner-freebsd-arch@FreeBSD.ORG Sun Nov 23 23:46:32 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52D46106568F for ; Sun, 23 Nov 2008 23:46:32 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id CF6798FC19 for ; Sun, 23 Nov 2008 23:46:31 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by fg-out-1718.google.com with SMTP id l26so1352724fgb.35 for ; Sun, 23 Nov 2008 15:46:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender :to:subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references :x-google-sender-auth; bh=5NSlstsAQS8jG+Qlq/nus2UrYQnzkgJzvrGs8nDnxCc=; b=gwbKDXLvD4Qkauvlsx3DFaCSKcRXZB6Mf7ij4iHdFCCBpRWqxCYMQvcLcA2Bug7KsU qCh5foJWH06jxQrJj8lno+Nj5ANuLDHpSZRGVgQSlyV35Afgsk4UButjXKM/ucSf3zac TqcZ8fEBcsLpMYNrpIvoZgYuIjl0d1lCP0KnE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references:x-google-sender-auth; b=WqDf0Phr5wlDonYjUpIcYnqdLzMk5uxkw4TBIuScajZuWZAyM0/9/rfuNHRBvGtl8m mQuP2snkjtUlVmDup930pz4J+ZVp90HqQtV4gBEfuGvPP6T1G9QyArET+Z/Fx7roOMif yjTOUhKz+SkjHrNAqEZ2S0F3HgdiStQe5yagA= Received: by 10.86.86.12 with SMTP id j12mr1852531fgb.64.1227483989893; Sun, 23 Nov 2008 15:46:29 -0800 (PST) Received: by 10.86.30.17 with HTTP; Sun, 23 Nov 2008 15:46:29 -0800 (PST) Message-ID: <3bbf2fe10811231546r44bd2aafqa3d714a4955f52ad@mail.gmail.com> Date: Mon, 24 Nov 2008 00:46:29 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Alexander Leidinger" In-Reply-To: <20081123205603.17752y578er4bcqo@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> <20081123205603.17752y578er4bcqo@webmail.leidinger.net> X-Google-Sender-Auth: 65f3d96ee3cd32b8 Cc: FreeBSD Arch , freebsd-performance@freebsd.org, Joseph Koshy Subject: Re: [PATCH] pmcannotate tool X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2008 23:46:32 -0000 2008/11/23, Alexander Leidinger : > Quoting Attilio Rao (from Sun, 23 Nov 2008 14:02:22 > +0100): > > > > pmcannotate is a tool that prints out sources of a tool (in C or > > assembly) with inlined profiling informations retrieved by a prior > > pmcstat analysis. > > If compared with things like callgraph generation, it prints out > > profiling on a per-instance basis and this can be useful to find, for > > example, badly handled caches, too high latency instructions, etc. > > > > Can this also be used to do some code coverage analysis? What I'm > interested in is to enable something, run some tests in userland, disable > this something, and then run a tool which tells me which parts of specific > functions where run or not. Yes, this is exactly what it does. You can see traces for any sampled PC and so get a profiling anslysis on a per-instance basis. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Sun Nov 23 23:47:01 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7978710656D8 for ; Sun, 23 Nov 2008 23:47:01 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.159]) by mx1.freebsd.org (Postfix) with ESMTP id 0221B8FC1C for ; Sun, 23 Nov 2008 23:47:00 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by fg-out-1718.google.com with SMTP id l26so1352726fgb.35 for ; Sun, 23 Nov 2008 15:47:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender :to:subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references :x-google-sender-auth; bh=9+Y+pnnEsKs/k0JGU4GpnkbmJqduyRszlodSbIES6fE=; b=c8EQsAq92njUfyarWcGNHZXgQJj+9p0tZB6EmB/CWrHxWhTF4UCkAvETbJyIFMPl6m N8okT+NXu6gw+O1t57rPNWnINFtKTQWIGdnf5MtkmmiurtwOafi5BeLByml8kAhwtaGb vG1yQFUi/mYr6JQg+roy57h4rNC0yorHqbDus= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references:x-google-sender-auth; b=MsUqSMFOUlU8aBmSFtiqssPzTuivY8VP6V+LQOqbT5kEfYMNrdZBgSEqvWs5UqH9Y+ xEqlJDrlfkLsAtXfTzH1sfa/EQB0TgSIJlezXwWhuo+FSI6ropEub8Cm6HyHWNmCAJQi xKa4Vj3Ry/cw+P3MuevGHVSaghkKoVFm7P7FA= Received: by 10.86.68.2 with SMTP id q2mr1872828fga.3.1227484020670; Sun, 23 Nov 2008 15:47:00 -0800 (PST) Received: by 10.86.30.17 with HTTP; Sun, 23 Nov 2008 15:47:00 -0800 (PST) Message-ID: <3bbf2fe10811231547w651dea65h211b82ae4dcef005@mail.gmail.com> Date: Mon, 24 Nov 2008 00:47:00 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Ray Kinsella" In-Reply-To: <584ec6bb0811231226x7d1b53ccgc5e28bfc297009a2@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> <584ec6bb0811231226x7d1b53ccgc5e28bfc297009a2@mail.gmail.com> X-Google-Sender-Auth: 85bb0ae8afebdc61 Cc: FreeBSD Arch , freebsd-performance@freebsd.org, Joseph Koshy Subject: Re: [PATCH] pmcannotate tool X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Nov 2008 23:47:01 -0000 2008/11/23, Ray Kinsella : > I know I am going to really show my FreeBSD ignorance here, but this is a > patch of FreeBSD 8.0 Current isn't it ? Yes, it is for 8.0. Is it giving you problems? Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 00:02:44 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B07B61065674; Mon, 24 Nov 2008 00:02:44 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 604168FC0A; Mon, 24 Nov 2008 00:02:44 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.2/8.14.1) with ESMTP id mAO025MM017664; Sun, 23 Nov 2008 17:02:05 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Sun, 23 Nov 2008 17:03:34 -0700 (MST) Message-Id: <20081123.170334.660270635.imp@bsdimp.com> To: xcllnt@mac.com From: "M. Warner Losh" In-Reply-To: References: <4929877B.6060307@freebsd.org> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: nwhitehorn@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Enumerable I2C busses X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 00:02:44 -0000 In message: Marcel Moolenaar writes: : : On Nov 23, 2008, at 8:40 AM, Nathan Whitehorn wrote: : : > On Apple's PowerPC systems, the firmware device tree helpfully : > enumerates the system's I2C busses. Marco Trillo has recently : > written a driver for one of the system's I2C controllers in order to : > support the attached audio codecs, and I'm trying to figure out the : > best way to import it. : > : > The current I2C bus mechanism does not support the bus adding its : > own children and instead relies on hints or other out-of-band : > information for device attachment. It would be nice to do something : > like what the firmware-assisted PCI bus drivers do (ofw_pci, for : > instance): hijack child enumeration from the MI layer and attach : > information from the firmware. The current i2c could easily override the hints things that I added there for platforms that didn't support a nice OF tree or the like. : > However, since all current I2C drivers' probe() routines return 0, I : > can't simply add the firmware devices, because as soon as the : > probe() methods of the existing drivers are called, they will take : > over all the devices on the bus. : > : > What is the best way to handle this? : : I think the best approach is to have the probe() method : actually test for something. This is the standard thing : to do when the bus does not know a priori which driver : to instantiate. I believe that the bus should never : create a child of a specific devclass, unless instructed : by the user (read: pre-binding directives). : : For ocpbus(4) (see sys/powerpc/mpc85xx/ocpbus.c and : sys/powerpc/include/ocpbus.h), we created simple defines : to identify the hardware and use that in the probe() : method to bind the driver to the right hardware, as in: : : In uart(4), for example we have: : : ... : error = BUS_READ_IVAR(parent, dev, OCPBUS_IVAR_DEVTYPE, : &devtype); : if (error) : return (error); : if (devtype != OCPBUS_DEVTYPE_UART) : return (ENXIO); : ... : : I'm not saying to copy it, but it does demonstrate that : you can trivially implement something that eliminates : the assumption that the bus instantiates children of : a particular devclass and thus that the probe() method : can always return 0.: I'm not saying copy it either. I don't like it for a variety of reasons... I don't like the fake devtypes, for one. It solves one problem, but introduces an number of others that have been talked about here. : What we do need is a generic way (such as the OF device : tree) to describe the hardware, its resource needs and : how it's all wired together (think interrupt routing). I think is really the right way to go. Linux currently has a flattened device tree that has information about what the device is, and what it is compatible with. The probe routines then match on the compat field to see if they should attach or not. This is what should be done for the Mac PPC i2c information in the OF tree. For the moment, we likely need a subclass of i2c to do this properly, but in the future, I'd love to move to using something like this to replace hints. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 00:08:47 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F88E106564A; Mon, 24 Nov 2008 00:08:47 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 13E578FC14; Mon, 24 Nov 2008 00:08:46 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.2/8.14.1) with ESMTP id mAO07S0g017717; Sun, 23 Nov 2008 17:07:28 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Sun, 23 Nov 2008 17:08:54 -0700 (MST) Message-Id: <20081123.170854.-626772149.imp@bsdimp.com> To: nwhitehorn@freebsd.org From: "M. Warner Losh" In-Reply-To: <4929C6D8.7090305@freebsd.org> References: <86myfq9uha.fsf@ds4.des.no> <4929C6D8.7090305@freebsd.org> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=utf-8 Content-Transfer-Encoding: base64 Cc: raj@semihalf.com, freebsd-arch@freebsd.org, des@des.no Subject: Re: Enumerable I2C busses X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 00:08:47 -0000 SW4gbWVzc2FnZTogPDQ5MjlDNkQ4LjcwOTAzMDVAZnJlZWJzZC5vcmc+DQogICAgICAgICAgICBO YXRoYW4gV2hpdGVob3JuIDxud2hpdGVob3JuQGZyZWVic2Qub3JnPiB3cml0ZXM6DQo6IFJhZmHF giBKYXdvcm93c2tpIHdyb3RlOg0KOiA+IA0KOiA+IE9uIDIwMDgtMTEtMjMsIGF0IDE5OjE4LCBE YWctRXJsaW5nIFNtw7hyZ3JhdiB3cm90ZToNCjogPiANCjogPj4gTmF0aGFuIFdoaXRlaG9ybiA8 bndoaXRlaG9ybkBmcmVlYnNkLm9yZz4gd3JpdGVzOg0KOiA+Pj4gVGhlIGN1cnJlbnQgSTJDIGJ1 cyBtZWNoYW5pc20gZG9lcyBub3Qgc3VwcG9ydCB0aGUgYnVzIGFkZGluZyBpdHMgb3duDQo6ID4+ PiBjaGlsZHJlbiBbLi4uXQ0KOiA+Pg0KOiA+PiBUaGF0J3MgYmVjYXVzZSB0aGUgSTJDIHByb3Rv Y29sIGRvZXMgbm90IHN1cHBvcnQgZGV2aWNlIGVudW1lcmF0aW9uIG9yDQo6ID4+IGlkZW50aWZp Y2F0aW9uLiAgWW91IGhhdmUgdG8ga25vdyBpbiBhZHZhbmNlIHdoYXQga2luZCBvZiBkZXZpY2Vz IGFyZQ0KOiA+PiBhdHRhY2hlZCBhbmQgYXQgd2hhdCBhZGRyZXNzLiAgRXZlbiB3b3JzZSwgaXQg aXMgbm90IHVuY29tbW9uIGZvcg0KOiA+PiBzaW1pbGFyIGJ1dCBub3QgZW50aXJlbHkgY29tcGF0 aWJsZSBkZXZpY2VzIHRvIHVzZSB0aGUgc2FtZSBJMkMgYWRkcmVzcw0KOiA+PiAoZm9yIGluc3Rh bmNlLCBldmVyeSBJMkMtY2FwYWJsZSBSVEMgY2hpcCB1c2VzIHRoZSBzYW1lIGFkZHJlc3MsIGV2 ZW4NCjogPj4gdGhvdWdoIHRoZXkgaGF2ZSBkaWZmZXJlbnQgZmVhdHVyZSBzZXRzKQ0KOiA+IA0K OiA+IFdlbGwsIGhhcmQtY29kZWQgYWRkcmVzc2VzIGFuZCBjb25mbGljdGluZyBhc3NpZ25tZW50 cyBiZXR3ZWVuIHZlbmRvcnMgDQo6ID4gZG8gbm90IHRlY2huaWNhbGx5IHByZXZlbnQgZnJvbSBz Y2FubmluZyB0aGUgYnVzOyBhY3R1YWxseSwgb3VyIGN1cnJlbnQgDQo6ID4gaWljYnVzIGNvZGUg Y2FuIGRvIGJ1cyBzY2FuaW5nIHdoZW4gY29tcGlsZWQgd2l0aCBhIGRpYWcgZGVmaW5lLiBUaGUg DQo6ID4gcHJvYmxlbSBob3dldmVyIGlzIHNvbWUgc2xhdmUgZGV2aWNlcyBhcmUgbm90IHdlbGwt YmVoYXZlZCwgYW5kIHRoZXkgDQo6ID4gZG9uJ3QgbGlrZSB0byBiZSByZWFkL3dyaXR0ZW4gdG8g b3RoZXIgdGhhbiBpbiB2ZXJ5IHNwZWNpZmljIHNjZW5hcmlvOiANCjogPiBpZiBwb2xsZWQgZHVy aW5nIGJ1cyBzY2FuIHN0cmFuZ2UgZWZmZWN0cyBvY2N1ciBlLmcuIHRoZXkgZGlzYXBwZWFyIGZy b20gDQo6ID4gdGhlIGJ1cywgb3IgZG8gbm90IHJlYWN0IHRvIGNvbnNlY3V0aXZlIHJlcXVlc3Rz IGV0Yy4NCjogDQo6IEFsbCBvZiB0aGlzIGlzIHRydWUsIGJ1dCBwZXJoYXBzIG15IHF1ZXN0aW9u IHdhcyBiYWRseSB3b3JkZWQuIFdoYXQgSSBhbSANCjogdHJ5aW5nIHRvIGZpZ3VyZSBvdXQgaXMg aG93IHRvIHNob3ZlIGluZm9ybWF0aW9uIGZyb20gYW4gb3V0LW9mLWJhbmQgDQo6IHNvdXJjZSAo T3BlbiBGaXJtd2FyZSwgaW4gdGhpcyBjYXNlKSBpbnRvIG5ld2J1cyB3aXRob3V0IGRpc3J1cHRp bmcgDQo6IGV4aXN0aW5nIGNvZGUuIEluIHRoYXQgd2F5LCBteSBxdWVzdGlvbiBpcyBub3QgSTJD IHNwZWNpZmljIC0tIHdlIHJ1biANCjogaW50byB0aGUgc2FtZSBpc3N1ZSB3aXRoIHRoZSBPcGVu IEZpcm13YXJlIG5leHVzIG5vZGUgYW5kIHBzZXVkby1kZXZpY2VzIA0KOiBsaWtlIGNyeXB0b3Nv ZnQgdGhhdCBhdHRhY2ggdGhlbXNlbHZlcy4NCg0KWW91IGFyZSBsb29raW5nIGF0IHRoZSBwcm9i bGVtIGluY29ycmVjdGx5LiAgSW4gbmV3YnVzLCBhIGNhc2UgbGlrZQ0KdGhpcyB0aGUgaTJjIGJ1 cyBzaG91bGQgYmUgYSBzdWJjbGFzcyAoc2F5IGkyY19vZikgdGhhdCBpcyBkZXJpdmVkDQpmcm9t IHRoZSBub3JtYWwgaTJjIGJ1cyBzdHVmZiwgYnV0IHJlcGxhY2VzIHRoZSBoaW50cyBpbnNlcnRp b24gb2YNCmRldmljZXMgd2l0aCBPRiBlbnVtZXJhdGlvbiBvZiBkZXZpY2VzLiAgVGhlIE9GIGhp Z2hlciBsZXZlbHMgd2lsbA0KYWxyZWFkeSBrbm93IHRvIGF0dGFjaCB0aGlzIGtpbmQgb2YgaTJj IGJ1cyB0byBjZXJ0YWluIGkyYw0KY29udHJvbGxlcnMsIG9yIGFsd2F5cyBvbiBjZXJ0YWluIHBs YXRmb3Jtcy4NCg0KOiBXaGF0IEkgd2FudCB0byBkbyBpcyB0byBoYXZlIHRoZSBJMkMgYnVzIGFk ZCB0aGUgY2hpbGRyZW4gdGhhdCB0aGUgDQo6IGZpcm13YXJlIHNheXMgaXQgaGFzLiBXaGF0IHRo ZSBmaXJtd2FyZSBjYW5ub3QgdGVsbCBpbiBhZHZhbmNlLCBob3dldmVyLCANCjogaXMgd2hpY2gg RnJlZUJTRCBkcml2ZXIgaXMgcmVzcG9uc2libGUgZm9yIHRob3NlIGRldmljZXMsIGFuZCBzbyB0 aGUgSTJDIA0KOiBidXMgZHJpdmVyIGNhbid0IGtub3cgdGhhdCB3aXRob3V0IGEgdHJhbnNsYXRp b24gdGFibGUgdGhhdCBJIHdvdWxkIA0KOiBwcmVmZXIgbm90IHRvIGhhY2sgaW4gdG8gdGhlIGJ1 cyBkcml2ZXIuDQoNClRoaXMgaXMgdGhlIGJpZ2dlciBwcm9ibGVtLiAgVG9kYXksIHdlIGFyZSBz dHVjayB3aXRoIGEgbGFtZSB0YWJsZQ0KdGhhdCB3aWxsIHRyYW5zbGF0ZSB0aGUgT0YgcHJvdmlk ZWQgcHJvcGVydGllcyBpbnRvIEZyZWVCU0QgZHJpdmVyDQpuYW1lcy4NCg0KOiBJdCBzZWVtcyBy ZWFzb25hYmxlIHRvIGFsbG93IGRldmljZXMgdG8gdXNlIGEgcmVhbCBwcm9iZSByb3V0aW5lIHRv IGxvb2sgDQo6IGF0IHRoZSBmaXJtd2FyZSdzIG5hbWUgYW5kIGNvbXBhdGlibGUgcHJvcGVydGll cywgbGlrZSB3ZSBhbGxvdyBvbiBvdGhlciANCjogT3BlbiBGaXJtd2FyZSBidXNzZXMuIFRoZSB0 cm91YmxlIGlzIHRoYXQgZXhpc3RpbmcgZHJpdmVycyBkb24ndCBkbyANCjogdGhpcywgYmVjYXVz ZSB0aGV5IGV4cGVjdCB0byBiZSBhdHRhY2hlZCB3aXRoIGhpbnRzLCBzbyB0aGV5IHdpbGwgYXR0 YWNoIA0KOiB0byBhbGwgZGV2aWNlcy4gSSdtIHRyeWluZyB0byBmaWd1cmUgb3V0IGhvdyB0byBh dm9pZCB0aGlzLg0KOiANCjogTXkgYmFzaWMgcXVlc3Rpb24gY29tZXMgZG93biB0byB3aGV0aGVy IHRoZXJlIGlzIGEgZ29vZCB3YXkgaW4gbmV3YnVzIHRvIA0KOiBoYW5kbGUgYnVzc2VzIHRoYXQg bWF5IGJlIHdob2xseSBvciBwYXJ0aWFsbHkgZW51bWVyYXRlZCBieSBmaXJtd2FyZSBvciANCjog c29tZSBvdGhlciBtZXRob2QsIGFuZCBtYXkgYWxzbyBoYXZlIGRldmljZXMgdGhhdCBjYW4gb25s eSBhdHRhY2ggDQo6IHRoZW1zZWx2ZXMgaWYgdG9sZCB0byBieSBoaW50cy4NCg0KWWVzLiAgVGhp cyBpcyBhIGJpdCBvZiBhIHByb2JsZW0uICBUaGUgcHJvYmxlbSBpcyB0aGF0IHRoZSBleGlzdGlu Zw0KaGludHMgbWVjaGFuaXNtIGNvbWJpbmVzIGRldmljZSB0cmVlIGFuZCBkcml2ZXIgdHJlZSBp bnRvIG9uZSwgYW5kIGluDQpzdWNoIGEgc2NlbmFyaW8sIHdlIHdpbmQgdXAgd2l0aCB0aGUgcHJv YmxlbSB0aGF0IHlvdSBoYXZlLg0KDQpPbmUgY291bGQgbWFrZSB0aGUgcHJvYmUgcm91dGluZXMg cmV0dXJuIEJVU19QUk9CRV9HRU5FUklDLCBhbmQgdGhhdA0Kd291bGQgaGVscCBzb21ld2hhdC4g IE9uZSBjb3VsZCBhbHNvIGhhdmUgdGhlIHByb2JlIHJvdXRpbmUgY2hlY2sgdG8NCnNlZSBpZiBh IHNwZWNpZmljIGRyaXZlciBpcyBhc3NpZ25lZCB0byB0aGUgZGV2aWNlIG9yIG5vdC4gIFRoYXQg d291bGQNCmFsc28gaGVscCwgYnV0IGRvZXMgbWVhbiBjaGFuZ2luZyBhbGwgdGhlIGkyYyBidXMg YXR0YWNoZWQgZHJpdmVycyBpbg0KdGhlIHRyZWUuDQoNCldhcm5lcg0K From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 00:17:15 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3849C106564A for ; Mon, 24 Nov 2008 00:17:15 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.170]) by mx1.freebsd.org (Postfix) with ESMTP id 11A828FC0C for ; Mon, 24 Nov 2008 00:17:14 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by wf-out-1314.google.com with SMTP id 24so2003734wfg.7 for ; Sun, 23 Nov 2008 16:17:14 -0800 (PST) Received: by 10.142.163.1 with SMTP id l1mr1367426wfe.108.1227484397873; Sun, 23 Nov 2008 15:53:17 -0800 (PST) Received: from ?10.0.1.199? (cpe-66-91-191-118.hawaii.res.rr.com [66.91.191.118]) by mx.google.com with ESMTPS id 20sm7179136wfi.47.2008.11.23.15.53.15 (version=SSLv3 cipher=RC4-MD5); Sun, 23 Nov 2008 15:53:16 -0800 (PST) Date: Sun, 23 Nov 2008 13:50:35 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Attilio Rao In-Reply-To: <3bbf2fe10811231546r44bd2aafqa3d714a4955f52ad@mail.gmail.com> Message-ID: <20081123135009.I971@desktop> References: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> <20081123205603.17752y578er4bcqo@webmail.leidinger.net> <3bbf2fe10811231546r44bd2aafqa3d714a4955f52ad@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Joseph Koshy , Alexander Leidinger , freebsd-performance@freebsd.org, FreeBSD Arch Subject: Re: [PATCH] pmcannotate tool X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 00:17:15 -0000 On Mon, 24 Nov 2008, Attilio Rao wrote: > 2008/11/23, Alexander Leidinger : >> Quoting Attilio Rao (from Sun, 23 Nov 2008 14:02:22 >> +0100): >> >> >>> pmcannotate is a tool that prints out sources of a tool (in C or >>> assembly) with inlined profiling informations retrieved by a prior >>> pmcstat analysis. >>> If compared with things like callgraph generation, it prints out >>> profiling on a per-instance basis and this can be useful to find, for >>> example, badly handled caches, too high latency instructions, etc. >>> >> >> Can this also be used to do some code coverage analysis? What I'm >> interested in is to enable something, run some tests in userland, disable >> this something, and then run a tool which tells me which parts of specific >> functions where run or not. > > Yes, this is exactly what it does. > You can see traces for any sampled PC and so get a profiling anslysis > on a per-instance basis. I would add that it is only sampled so you don't see every instruction executed. You can use gcov for that however. That's precisely what it's for. Thanks, Jeff > > Thanks, > Attilio > > > -- > Peace can only be achieved by understanding - A. Einstein > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 01:15:01 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 451021065672; Mon, 24 Nov 2008 01:15:01 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from outbound.icp-qv1-irony-out1.iinet.net.au (outbound.icp-qv1-irony-out1.iinet.net.au [203.59.1.108]) by mx1.freebsd.org (Postfix) with ESMTP id 677E18FC08; Mon, 24 Nov 2008 01:14:58 +0000 (UTC) (envelope-from lstewart@freebsd.org) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApoEAKyHKUl8qBSs/2dsb2JhbADNZYJ8 X-IronPort-AV: E=Sophos;i="4.33,655,1220198400"; d="scan'208";a="420066314" Received: from unknown (HELO lawrence1.loshell.room52.net) ([124.168.20.172]) by outbound.icp-qv1-irony-out1.iinet.net.au with ESMTP; 24 Nov 2008 09:45:00 +0900 Message-ID: <4929F90B.1040502@freebsd.org> Date: Mon, 24 Nov 2008 11:44:59 +1100 From: Lawrence Stewart User-Agent: Thunderbird 2.0.0.17 (X11/20081109) MIME-Version: 1.0 To: John Baldwin References: <492412E8.3060700@freebsd.org> <200811201502.23943.jhb@freebsd.org> <4925E30B.8010709@freebsd.org> <200811211348.41536.jhb@freebsd.org> In-Reply-To: <200811211348.41536.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: kthread_exit(9) unexpectedness X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 01:15:01 -0000 John Baldwin wrote: > On Thursday 20 November 2008 05:22:03 pm Lawrence Stewart wrote: >> John Baldwin wrote: >>> On Wednesday 19 November 2008 08:21:44 am Lawrence Stewart wrote: >>>> Hi all, >>>> >>>> I tracked down a deadlock in some of my code today to some weird >>>> behaviour in the kthread(9) KPI. The executive summary is that >>>> kthread_exit() thread termination notification using wakeup() behaves as >>>> expected intuitively in 8.x, but not in 7.x. >>> In 5.x/6.x/7.x kthreads are still processes and it has always been a > wakeup on >>> the proc pointer. kthread_create() in 7.x returns a proc pointer, not a >>> thread pointer for example. In 8.x kthreads are actual threads and >> Yep, but the processes have a *thread in them right? The API naming is >> obviously slightly misleading, but it essentially creates a new single >> threaded process prior to 8.x. > > Yes, but you have to go explicitly use FIRST_THREAD_IN_PROC(). Most of the > kernel modules I've written that use kthread's in < 8 do this: > > static struct proc *foo_thread; > > /* Called for MOD_LOAD. */ > static void > load(...) > { > > error = kthread_create(..., &foo_thread); > } > > static void > unload(...) > { > > /* set flag */ > msleep(foo_thread, ...); > } > > And never actually use the thread at all. However, if you write the code for > 8.x, now you _do_ get a kthread and sleep on the thread so it becomes: > > static struct thread *foo_thread; > > static void > load(...) > { > > error = kproc_kthread_add(..., proc0, &foo_thread); > } > > static void > unload(...) > { > > /* set flag */ > msleep(foo_thread, ...); > } > Sure, but to write the code in this way means you are exercising undocumented knowledge of the KPI. I suspect the average developer completely unfamiliar with the KPI would (and should!) use the man page to learn about the functionality it provides. With that basis in mind, it seems unreasonable to expect the developer to come to the conclusion that "...will initiate a call to wakeup(9) on the thread handle." refers to sleeping on the *proc passed in to kthread_create. Perhaps I'm not as switched on as the average developer, but when I read it I certainly did not understand that the KPI created processes and that the man page used the term thread to really mean a single threaded process. I also did no equate "thread handle" with the *proc returned by kthread_create. It seems to me that the kthread(9) man page is somewhat unclear with respect to what the KPI actually achieves, making references to thread related activities all over the place when in reality it's manipulating single threaded processes (which for all intensive purposes can be used like threads but are structurally different). I guess this is the cause for the underlying confusion. While we're on the topic, I'm also having trouble understanding the reasoning for renaming kthread_create to kthread_add, when in reality 8.x kthread_add is doing a *real* kthread creation and the originally named kthread_create seems to have been a bit of a misnomer (corrected in 8.x by moving the functionality into the kproc_* KPI). The changing of the **proc to a **thread argument is enough to ensure code from 7.x won't compile without a tweak on 8.x, so why the rename to add further confusion? With the same line of reasoning, kproc_kthread_add should probably be kproc_kthread_create? >>> kthread_add() and kproc_kthread_add() both return thread pointers. Hence > in >> Yup. >> >>> 8.x kthread_exit() is used for exiting kernel threads and wakes up the > thread >>> pointer, but in 7.x kthread_exit() is used for exiting kernel processes > and >>> wakes up the proc pointer. I think what is probably needed is to simply >> In the code, yes. Our documented behaviour as far as I can tell is >> different though, unless we equate a "thread handle" to "proc handle" >> prior to 8.x, which I don't think is the case - they are still different. > > It has always been the case in < 8 that you sleep on the proc handle (what > kthread_create() actually returns in < 8). And in fact, you even have to dig > around in the proc you get from kthread_create() to even find the thread > pointer as opposed to having the API hand it to you. > >>> document that arrangement as such. Note that the sleeping on proc pointer >> I agree that the arrangement needs to be better documented. The change >> in 8.x is subtle enough that reading the kthread man page in 7.x and 8.x >> doesn't immediately make it obvious what's going on. >> >>> has been the documented way to synchronize with kthread_exit() since 5.0. >>> >> Could you please point me at this documentation? I've missed it in my >> poking around thus far. > > It is probably only documented in numerous threads in the mail archives sadly, > but there have been several of them and there have been several fixes to get > this right (the randomdev thread and fdc(4) thread come to mind). If we had no man page, mail archives would be the next best thing. In this instance, we have a misleading man page and I think it would be more beneficial to align the documentation/implementation rather than leaving people confused. Apart from the discussion thus far, you haven't actually commented yet on my proposed single line change to kthread_exit() in 7.x to call wakeup on the *thread as well as the *proc. Do you have any specific thoughts on or objection to that idea? Cheers, Lawrence From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 06:39:56 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BC911065678 for ; Mon, 24 Nov 2008 06:39:56 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outG.internet-mail-service.net (outg.internet-mail-service.net [216.240.47.230]) by mx1.freebsd.org (Postfix) with ESMTP id 6ACFC8FC1E for ; Mon, 24 Nov 2008 06:39:56 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 251752364; Sun, 23 Nov 2008 22:25:10 -0800 (PST) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 5E9A12D600D; Sun, 23 Nov 2008 22:25:09 -0800 (PST) Message-ID: <492A48C8.9080302@elischer.org> Date: Sun, 23 Nov 2008 22:25:12 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.18 (Macintosh/20081105) MIME-Version: 1.0 To: Lawrence Stewart References: <492412E8.3060700@freebsd.org> <200811201502.23943.jhb@freebsd.org> <4925E30B.8010709@freebsd.org> <200811211348.41536.jhb@freebsd.org> <4929F90B.1040502@freebsd.org> In-Reply-To: <4929F90B.1040502@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: kthread_exit(9) unexpectedness X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 06:39:56 -0000 Lawrence Stewart wrote: > John Baldwin wrote: >> On Thursday 20 November 2008 05:22:03 pm Lawrence Stewart wrote: >>> John Baldwin wrote: >>>> On Wednesday 19 November 2008 08:21:44 am Lawrence Stewart wrote: >>>>> Hi all, >>>>> >>>>> I tracked down a deadlock in some of my code today to some weird >>>>> behaviour in the kthread(9) KPI. The executive summary is that >>>>> kthread_exit() thread termination notification using wakeup() >>>>> behaves as expected intuitively in 8.x, but not in 7.x. >>>> In 5.x/6.x/7.x kthreads are still processes and it has always been a >> wakeup on >>>> the proc pointer. kthread_create() in 7.x returns a proc pointer, >>>> not a thread pointer for example. In 8.x kthreads are actual >>>> threads and >>> Yep, but the processes have a *thread in them right? The API naming >>> is obviously slightly misleading, but it essentially creates a new >>> single threaded process prior to 8.x. >> >> Yes, but you have to go explicitly use FIRST_THREAD_IN_PROC(). Most >> of the kernel modules I've written that use kthread's in < 8 do this: >> >> static struct proc *foo_thread; >> >> /* Called for MOD_LOAD. */ >> static void >> load(...) >> { >> >> error = kthread_create(..., &foo_thread); >> } >> >> static void >> unload(...) >> { >> >> /* set flag */ >> msleep(foo_thread, ...); >> } >> >> And never actually use the thread at all. However, if you write the >> code for 8.x, now you _do_ get a kthread and sleep on the thread so it >> becomes: >> >> static struct thread *foo_thread; >> >> static void >> load(...) >> { >> >> error = kproc_kthread_add(..., proc0, &foo_thread); >> } >> >> static void >> unload(...) >> { >> >> /* set flag */ >> msleep(foo_thread, ...); >> } >> > > > Sure, but to write the code in this way means you are exercising > undocumented knowledge of the KPI. I suspect the average developer > completely unfamiliar with the KPI would (and should!) use the man page > to learn about the functionality it provides. > > With that basis in mind, it seems unreasonable to expect the developer > to come to the conclusion that "...will initiate a call to wakeup(9) on > the thread handle." refers to sleeping on the *proc passed in to > kthread_create. Perhaps I'm not as switched on as the average developer, > but when I read it I certainly did not understand that the KPI created > processes and that the man page used the term thread to really mean a > single threaded process. I also did no equate "thread handle" with the > *proc returned by kthread_create. > > It seems to me that the kthread(9) man page is somewhat unclear with > respect to what the KPI actually achieves, making references to thread > related activities all over the place when in reality it's manipulating > single threaded processes (which for all intensive purposes can be used > like threads but are structurally different). I guess this is the cause > for the underlying confusion. > > While we're on the topic, I'm also having trouble understanding the > reasoning for renaming kthread_create to kthread_add, when in reality > 8.x kthread_add is doing a *real* kthread creation and the originally > named kthread_create seems to have been a bit of a misnomer (corrected > in 8.x by moving the functionality into the kproc_* KPI). kthread_add was named tha tway to indicate that it is adding a thread to an existing process in addition to the thread already running the code.. I wanted anyone linking in a binary module to get a link failure, even if they manageed to sneak past the other safeguards. > > The changing of the **proc to a **thread argument is enough to ensure > code from 7.x won't compile without a tweak on 8.x, so why the rename to > add further confusion? With the same line of reasoning, > kproc_kthread_add should probably be kproc_kthread_create? kproc_kthread_add will add a thread to the process if it already exists, but if it doesn't it will make a new process. Hey, it's just how I think I guess. > >>>> kthread_add() and kproc_kthread_add() both return thread pointers. >>>> Hence >> in >>> Yup. >>> >>>> 8.x kthread_exit() is used for exiting kernel threads and wakes up the >> thread >>>> pointer, but in 7.x kthread_exit() is used for exiting kernel processes >> and >>>> wakes up the proc pointer. I think what is probably needed is to >>>> simply >>> In the code, yes. Our documented behaviour as far as I can tell is >>> different though, unless we equate a "thread handle" to "proc handle" >>> prior to 8.x, which I don't think is the case - they are still >>> different. >> >> It has always been the case in < 8 that you sleep on the proc handle >> (what kthread_create() actually returns in < 8). And in fact, you >> even have to dig around in the proc you get from kthread_create() to >> even find the thread pointer as opposed to having the API hand it to you. >> >>>> document that arrangement as such. Note that the sleeping on proc >>>> pointer >>> I agree that the arrangement needs to be better documented. The >>> change in 8.x is subtle enough that reading the kthread man page in >>> 7.x and 8.x doesn't immediately make it obvious what's going on. >>> >>>> has been the documented way to synchronize with kthread_exit() since >>>> 5.0. >>>> >>> Could you please point me at this documentation? I've missed it in my >>> poking around thus far. >> >> It is probably only documented in numerous threads in the mail >> archives sadly, but there have been several of them and there have >> been several fixes to get this right (the randomdev thread and fdc(4) >> thread come to mind). > > If we had no man page, mail archives would be the next best thing. In > this instance, we have a misleading man page and I think it would be > more beneficial to align the documentation/implementation rather than > leaving people confused. > > Apart from the discussion thus far, you haven't actually commented yet > on my proposed single line change to kthread_exit() in 7.x to call > wakeup on the *thread as well as the *proc. Do you have any specific > thoughts on or objection to that idea? I really haven't got this stuff in my head at the moment so I can't comment. > > Cheers, > Lawrence > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 07:39:30 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A1901065670; Mon, 24 Nov 2008 07:39:30 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id D1CC88FC16; Mon, 24 Nov 2008 07:39:29 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (pD9E2D05F.dip.t-dialin.net [217.226.208.95]) by redbull.bpaserver.net (Postfix) with ESMTP id 2CA572E0B4; Mon, 24 Nov 2008 08:39:25 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 529AD14D413; Mon, 24 Nov 2008 08:39:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1227512361; bh=fNcm2XHWMN/2yh3ubfXCLnPYnbFUCHtSm QyG0bbODKc=; h=Message-ID:Date:From:To:Cc:Subject:References: In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=JZp2/R+p5zFmCRHYFobkbcqRGVQ6e4wVOghrElAgyQ6EW5MWVkkURn5ClTta8u72H vYw3zqu0TBteK7W8ySSD/Yb2bT/L1XbnMRykTd4mxuT0Cn1YTyWrPvn93vduQhStg+k Fv9JIcRrT1TmWy4GnCjR0TK3WCRGBnYxeSvolKeRmQvs7SlvWNI0JDG/RpGMW2RFw57 8fbsY8+Pe88JJoZFxADHMO5TGF6QCFxFx7nJJPsuXrYAl7TJtEVwCHcGW6qfP/0bOTn A0Y7LBSM1NwzwwzZVdugYZQwPHsDYi+3Ptpb/53Ih/0Q67s2LpK4T3EDrIwOB/oXCE1 JsCnOzXEA== Received: (from www@localhost) by webmail.leidinger.net (8.14.2/8.13.8/Submit) id mAO7dLYc002882; Mon, 24 Nov 2008 08:39:21 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Mon, 24 Nov 2008 08:39:20 +0100 Message-ID: <20081124083920.16126d6j9o1q9mw4@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Mon, 24 Nov 2008 08:39:20 +0100 From: Alexander Leidinger To: Attilio Rao References: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> <20081123205603.17752y578er4bcqo@webmail.leidinger.net> <3bbf2fe10811231546r44bd2aafqa3d714a4955f52ad@mail.gmail.com> In-Reply-To: <3bbf2fe10811231546r44bd2aafqa3d714a4955f52ad@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.3) / FreeBSD-8.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-MailScanner-ID: 2CA572E0B4.C3668 X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, ORDB-RBL, SpamAssassin (not cached, score=-13.504, required 6, BAYES_00 -15.00, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, MIME_QP_LONG_LINE 1.40, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: FreeBSD Arch , freebsd-performance@freebsd.org, Joseph Koshy Subject: Re: [PATCH] pmcannotate tool X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 07:39:30 -0000 Quoting Attilio Rao (from Mon, 24 Nov 2008 =20 00:46:29 +0100): > 2008/11/23, Alexander Leidinger : >> Quoting Attilio Rao (from Sun, 23 Nov 2008 14:02:22 >> +0100): >> >> >> > pmcannotate is a tool that prints out sources of a tool (in C or >> > assembly) with inlined profiling informations retrieved by a prior >> > pmcstat analysis. >> > If compared with things like callgraph generation, it prints out >> > profiling on a per-instance basis and this can be useful to find, for >> > example, badly handled caches, too high latency instructions, etc. >> > >> >> Can this also be used to do some code coverage analysis? What I'm >> interested in is to enable something, run some tests in userland, disable >> this something, and then run a tool which tells me which parts of specifi= c >> functions where run or not. > > Yes, this is exactly what it does. > You can see traces for any sampled PC and so get a profiling anslysis > on a per-instance basis. Cool. Would be great if you could provide some example in the man page =20 or as a script which shows how to do this for kernel code. This would =20 immediately show us how good our regression tests are for their =20 specific areas of test. Bye, Alexander. --=20 In a family recipe you just discovered in an old book, the most vital measurement will be illegible. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 07:40:27 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9AFB1065672; Mon, 24 Nov 2008 07:40:26 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id 712F48FC22; Mon, 24 Nov 2008 07:40:26 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (pD9E2D05F.dip.t-dialin.net [217.226.208.95]) by redbull.bpaserver.net (Postfix) with ESMTP id EACAF2E271; Mon, 24 Nov 2008 08:40:18 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id B784F14D43D; Mon, 24 Nov 2008 08:40:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1227512415; bh=xSGpxrwwW2ZGSDvXcOTY5a9g0ZBmm3ROf XK7bQLfOaA=; h=Message-ID:Date:From:To:Cc:Subject:References: In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=ssPieHFxxvLLwh5K4cNsbiqx0+beYHS1OeoZgmLtAazYsdZzErPmI4enCSDydfDCa Mn0WHjU7VPvHtkHBPTEW0i5J6dKCv39yoHDTlnZeYwD6R9nHPfh9SsTnjIiAPyDl8xx o/kyHJ+0d/cMB1sNf99FMKxIv/NqdAv/XqwyFkS0jdf1M7z2DDvknAPlWa6IbYFSEw0 x0Gmz19EmwiuUuat9Ypr9bvnvAAnoIAvTFxE25A502Pds8UEaEBLd2g+0ai8gnWrKPE X6yaTcrfhlvL6D8Iv5YjSLmt2rE4hdfUu6qp3m2HXbPYy09iSctsJO257ZoMV4PNWW4 kS7367UEw== Received: (from www@localhost) by webmail.leidinger.net (8.14.2/8.13.8/Submit) id mAO7eFOJ003029; Mon, 24 Nov 2008 08:40:15 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Mon, 24 Nov 2008 08:40:15 +0100 Message-ID: <20081124084015.84153bmq6411va68@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Mon, 24 Nov 2008 08:40:15 +0100 From: Alexander Leidinger To: Jeff Roberson References: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> <20081123205603.17752y578er4bcqo@webmail.leidinger.net> <3bbf2fe10811231546r44bd2aafqa3d714a4955f52ad@mail.gmail.com> <20081123135009.I971@desktop> In-Reply-To: <20081123135009.I971@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.3) / FreeBSD-8.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-MailScanner-ID: EACAF2E271.EE169 X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, ORDB-RBL, SpamAssassin (not cached, score=-13.504, required 6, BAYES_00 -15.00, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, MIME_QP_LONG_LINE 1.40, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: Attilio Rao , FreeBSD Arch , freebsd-performance@freebsd.org, Joseph Koshy Subject: Re: [PATCH] pmcannotate tool X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 07:40:27 -0000 Quoting Jeff Roberson (from Sun, 23 Nov 2008 =20 13:50:35 -1000 (HST)): > > On Mon, 24 Nov 2008, Attilio Rao wrote: > >> 2008/11/23, Alexander Leidinger : >>> Quoting Attilio Rao (from Sun, 23 Nov 2008 14:02:2= 2 >>> +0100): >>> >>> >>>> pmcannotate is a tool that prints out sources of a tool (in C or >>>> assembly) with inlined profiling informations retrieved by a prior >>>> pmcstat analysis. >>>> If compared with things like callgraph generation, it prints out >>>> profiling on a per-instance basis and this can be useful to find, for >>>> example, badly handled caches, too high latency instructions, etc. >>>> >>> >>> Can this also be used to do some code coverage analysis? What I'm >>> interested in is to enable something, run some tests in userland, disabl= e >>> this something, and then run a tool which tells me which parts of specif= ic >>> functions where run or not. >> >> Yes, this is exactly what it does. >> You can see traces for any sampled PC and so get a profiling anslysis >> on a per-instance basis. > > I would add that it is only sampled so you don't see every =20 > instruction executed. You can use gcov for that however. That's =20 > precisely what it's for. How to use gcov for the kernel? Bye, Alexander. --=20 If only you knew she loved you, you could face the uncertainty of whether you love her. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 07:48:48 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 975DE1065672 for ; Mon, 24 Nov 2008 07:48:48 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.238]) by mx1.freebsd.org (Postfix) with ESMTP id 747898FC13 for ; Mon, 24 Nov 2008 07:48:48 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by rv-out-0506.google.com with SMTP id b25so1909477rvf.43 for ; Sun, 23 Nov 2008 23:48:48 -0800 (PST) Received: by 10.142.237.20 with SMTP id k20mr1526676wfh.218.1227512928103; Sun, 23 Nov 2008 23:48:48 -0800 (PST) Received: from ?10.0.1.199? (cpe-66-91-191-118.hawaii.res.rr.com [66.91.191.118]) by mx.google.com with ESMTPS id 22sm7707876wfi.58.2008.11.23.23.48.45 (version=SSLv3 cipher=RC4-MD5); Sun, 23 Nov 2008 23:48:46 -0800 (PST) Date: Sun, 23 Nov 2008 21:46:08 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: arch@freebsd.org Message-ID: <20081123213232.A971@desktop> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Subject: Limiting mbuf memory. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 07:48:48 -0000 I'm developing a patch for an alternate memory layout for mbuf clusters that relies on contigmalloc. Since this can fail, we'll still have to retain the capability of allocating traditional clusters. I'll report details on that later. I'm writing this email to address the issue of resource accounting in mbufs. Presently we use a set of limits on individual zones or sizes of mbufs. Standard mbufs, clusters, page size jumbos, 9k jumbos, and 16k jumbos. Each is administered sperately. I think this is getting a bit unwieldy. In the future, we may have even more sizes. This also introduces problems because I will have two cluster zones do they each get their own limit? I would like to consolidate this into a single limit on the number of pages in total allocated to networking. With perhaps some fractional reservation for standard mbufs and clusters to make sure they aren't overwhelmed by the larger buffers. This would be implemented by overriding the uma zone page allocator for each of the mbuf zones with one that counts pages for all. Should we reach the limit we'll block depending on the wait settings of the requestor. The limit and sleep will probably be protected by a global lock which won't be an issue because trips to the back end allocator are infrequent and protected by their own global lock anyhow. How do people feel about this? To be clear this would eliminate: nmbclusters, nmbjumbop, nmbjumbo9, nmbjumbo16 and related config settings and sysctls. They would be replaced by something like 'maxmbufbytes'. Presently we place no limit on small mbufs. I could go either way on this. It could be added to the limit or not. Thanks, Jeff From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 08:48:10 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EC13106567D for ; Mon, 24 Nov 2008 08:48:10 +0000 (UTC) (envelope-from raykinsella78@gmail.com) Received: from mail-qy0-f18.google.com (mail-qy0-f18.google.com [209.85.221.18]) by mx1.freebsd.org (Postfix) with ESMTP id AB0548FC1C for ; Mon, 24 Nov 2008 08:48:09 +0000 (UTC) (envelope-from raykinsella78@gmail.com) Received: by qyk11 with SMTP id 11so1662662qyk.19 for ; Mon, 24 Nov 2008 00:48:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type:references; bh=zgcFyX6E0gjEkJbC6JQieaZBFygoEgTifMuZI5nudSY=; b=WGiqvMvEtga1kEATfydidYJFGmDPxH5MuSHBp0lrvWljUYtKUsnh/0fYvzLWfbRwwk pqvx3FH2iQimo1qV1LT/sG1DjIRno2Xhj6na6fwq7C3WOrYs2ZTE7Ph+OtX9Vr/sWSEy /xdbAI1wGreSTfr4woMN+AvKB2N4Vrph5Xw9Q= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:references; b=r4VXThz4IgIDaoSaePhvRRkYHo7Uhu1BrbDncuQ0Xv9JC6G+UYzunn14QOWzhGxofq DkOumYwmuGh+YkjfzQZx2ha9PMZNpUIEpw6YWKaElLFJdHDAU78tp/l/+GRcDp9yPQg7 /HKUGSoTeM3cCqVpNLlTuFMN9zEIc86VqYXPE= Received: by 10.215.38.2 with SMTP id q2mr1774205qaj.7.1227516488720; Mon, 24 Nov 2008 00:48:08 -0800 (PST) Received: by 10.214.181.14 with HTTP; Mon, 24 Nov 2008 00:48:08 -0800 (PST) Message-ID: <584ec6bb0811240048x2304a759j2ac6fdea83a47773@mail.gmail.com> Date: Mon, 24 Nov 2008 08:48:08 +0000 From: "Ray Kinsella" To: "Attilio Rao" In-Reply-To: <3bbf2fe10811231547w651dea65h211b82ae4dcef005@mail.gmail.com> MIME-Version: 1.0 References: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> <584ec6bb0811231226x7d1b53ccgc5e28bfc297009a2@mail.gmail.com> <3bbf2fe10811231547w651dea65h211b82ae4dcef005@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FreeBSD Arch , freebsd-performance@freebsd.org, Joseph Koshy Subject: Re: [PATCH] pmcannotate tool X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 08:48:10 -0000 No, not at all. I am using 6.2 and 7.0 at the moment, I will build another disk with FreeBSD 8.0-CURRENT. Thanks Ray Kinsella On Sun, Nov 23, 2008 at 11:47 PM, Attilio Rao wrote: > 2008/11/23, Ray Kinsella : > > I know I am going to really show my FreeBSD ignorance here, but this is a > > patch of FreeBSD 8.0 Current isn't it ? > > Yes, it is for 8.0. > Is it giving you problems? > > Thanks, > Attilio > > > -- > Peace can only be achieved by understanding - A. Einstein > From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 08:52:38 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F3A43106564A; Mon, 24 Nov 2008 08:52:37 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id B23CE8FC0C; Mon, 24 Nov 2008 08:52:37 +0000 (UTC) (envelope-from des@des.no) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id F1D076D449; Mon, 24 Nov 2008 08:52:36 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id C36BD844A1; Mon, 24 Nov 2008 09:52:36 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: =?utf-8?Q?Rafa=C5=82?= Jaworowski References: <4929877B.6060307@freebsd.org> <86myfq9uha.fsf@ds4.des.no> Date: Mon, 24 Nov 2008 09:52:36 +0100 In-Reply-To: (=?utf-8?Q?=22Rafa=C5=82?= Jaworowski"'s message of "Sun, 23 Nov 2008 21:18:54 +0100") Message-ID: <86od054iaz.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Nathan Whitehorn , freebsd-arch@freebsd.org Subject: Re: Enumerable I2C busses X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 08:52:38 -0000 Rafa=C5=82 Jaworowski writes: > Well, hard-coded addresses and conflicting assignments between vendors > do not technically prevent from scanning the bus; actually, our > current iicbus code can do bus scaning when compiled with a diag > define. I haven't looked at how that is implemented, but - I2C version 3 describes device enumeration and identification, but AFAIK very few devices actually support it. This is really stupid, BTW. Philips could easily have required the first few bytes of the device address space to contain a vendor / device ID, along the lines of PCI and USB, from the start. I wonder: did it not occur to them, or did they intentionally leave it out to save a few microcents per chip? DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 09:07:59 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55CB21065673 for ; Mon, 24 Nov 2008 09:07:59 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 46B9B8FC1D for ; Mon, 24 Nov 2008 09:07:59 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id DD1791A3C3D; Mon, 24 Nov 2008 00:52:23 -0800 (PST) Date: Mon, 24 Nov 2008 00:52:23 -0800 From: Alfred Perlstein To: Jeff Roberson Message-ID: <20081124085223.GY28578@elvis.mu.org> References: <20081123213232.A971@desktop> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20081123213232.A971@desktop> User-Agent: Mutt/1.4.2.3i Cc: arch@freebsd.org Subject: Re: Limiting mbuf memory. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 09:07:59 -0000 * Jeff Roberson [081123 23:48] wrote: > I'm developing a patch for an alternate memory layout for mbuf clusters > that relies on contigmalloc. Since this can fail, we'll still have to > retain the capability of allocating traditional clusters. I'll report > details on that later. I'm writing this email to address the issue of > resource accounting in mbufs. > > Presently we use a set of limits on individual zones or sizes of mbufs. > Standard mbufs, clusters, page size jumbos, 9k jumbos, and 16k jumbos. > Each is administered sperately. I think this is getting a bit unwieldy. > In the future, we may have even more sizes. This also introduces problems > because I will have two cluster zones do they each get their own limit? > > I would like to consolidate this into a single limit on the number of > pages in total allocated to networking. With perhaps some fractional > reservation for standard mbufs and clusters to make sure they aren't > overwhelmed by the larger buffers. > > This would be implemented by overriding the uma zone page allocator for > each of the mbuf zones with one that counts pages for all. Should we > reach the limit we'll block depending on the wait settings of the > requestor. The limit and sleep will probably be protected by a global > lock which won't be an issue because trips to the back end allocator are > infrequent and protected by their own global lock anyhow. > > How do people feel about this? To be clear this would eliminate: > > nmbclusters, nmbjumbop, nmbjumbo9, nmbjumbo16 and related config settings > and sysctls. They would be replaced by something like 'maxmbufbytes'. > Presently we place no limit on small mbufs. I could go either way on > this. It could be added to the limit or not. This sounds good but please take into consideration the possibility of deadlock due to resource allocation to a single pool that can happen. It might make sense to keep the small and large mbuf limits separate or something like that. Might also make sense to retain the limits but set them all to "unlimited" (withing the global limit) unless configured otherwise for various custom set ups. I don't feel too strongly about this, just some points to consider. -- - Alfred Perlstein From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 10:57:31 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5DB7F1065749 for ; Mon, 24 Nov 2008 10:57:31 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.234]) by mx1.freebsd.org (Postfix) with ESMTP id 3E6C28FC0A for ; Mon, 24 Nov 2008 10:57:31 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by rv-out-0506.google.com with SMTP id b25so1971393rvf.43 for ; Mon, 24 Nov 2008 02:57:31 -0800 (PST) Received: by 10.142.139.19 with SMTP id m19mr1584560wfd.343.1227524250883; Mon, 24 Nov 2008 02:57:30 -0800 (PST) Received: from ?10.0.1.199? (cpe-66-91-191-118.hawaii.res.rr.com [66.91.191.118]) by mx.google.com with ESMTPS id 22sm7967265wfd.53.2008.11.24.02.57.28 (version=SSLv3 cipher=RC4-MD5); Mon, 24 Nov 2008 02:57:29 -0800 (PST) Date: Mon, 24 Nov 2008 00:54:51 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Alfred Perlstein In-Reply-To: <20081124085223.GY28578@elvis.mu.org> Message-ID: <20081124005404.H971@desktop> References: <20081123213232.A971@desktop> <20081124085223.GY28578@elvis.mu.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org Subject: Re: Limiting mbuf memory. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 10:57:31 -0000 On Mon, 24 Nov 2008, Alfred Perlstein wrote: > * Jeff Roberson [081123 23:48] wrote: >> I'm developing a patch for an alternate memory layout for mbuf clusters >> that relies on contigmalloc. Since this can fail, we'll still have to >> retain the capability of allocating traditional clusters. I'll report >> details on that later. I'm writing this email to address the issue of >> resource accounting in mbufs. >> >> Presently we use a set of limits on individual zones or sizes of mbufs. >> Standard mbufs, clusters, page size jumbos, 9k jumbos, and 16k jumbos. >> Each is administered sperately. I think this is getting a bit unwieldy. >> In the future, we may have even more sizes. This also introduces problems >> because I will have two cluster zones do they each get their own limit? >> >> I would like to consolidate this into a single limit on the number of >> pages in total allocated to networking. With perhaps some fractional >> reservation for standard mbufs and clusters to make sure they aren't >> overwhelmed by the larger buffers. >> >> This would be implemented by overriding the uma zone page allocator for >> each of the mbuf zones with one that counts pages for all. Should we >> reach the limit we'll block depending on the wait settings of the >> requestor. The limit and sleep will probably be protected by a global >> lock which won't be an issue because trips to the back end allocator are >> infrequent and protected by their own global lock anyhow. >> >> How do people feel about this? To be clear this would eliminate: >> >> nmbclusters, nmbjumbop, nmbjumbo9, nmbjumbo16 and related config settings >> and sysctls. They would be replaced by something like 'maxmbufbytes'. >> Presently we place no limit on small mbufs. I could go either way on >> this. It could be added to the limit or not. > > This sounds good but please take into consideration the possibility > of deadlock due to resource allocation to a single pool that can > happen. > > It might make sense to keep the small and large mbuf limits separate > or something like that. This is what I meant in the third paragraph. > > Might also make sense to retain the limits but set them all to > "unlimited" (withing the global limit) unless configured otherwise > for various custom set ups. I think this is a good idea. > > I don't feel too strongly about this, just some points to consider. I appreciate the feedback. Jeff > > -- > - Alfred Perlstein > From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 11:07:08 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96E241065677 for ; Mon, 24 Nov 2008 11:07:08 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 697A68FC1A for ; Mon, 24 Nov 2008 11:07:08 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id mAOB78N3019836 for ; Mon, 24 Nov 2008 11:07:08 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id mAOB77kj019831 for freebsd-arch@FreeBSD.org; Mon, 24 Nov 2008 11:07:07 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 24 Nov 2008 11:07:07 GMT Message-Id: <200811241107.mAOB77kj019831@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 11:07:08 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 15:30:31 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAF401065678 for ; Mon, 24 Nov 2008 15:30:31 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from agogare.doit.wisc.edu (agogare.doit.wisc.edu [144.92.197.211]) by mx1.freebsd.org (Postfix) with ESMTP id 9062F8FC18 for ; Mon, 24 Nov 2008 15:30:31 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) MIME-version: 1.0 Content-transfer-encoding: 8BIT Content-type: text/plain; charset=UTF-8; format=flowed Received: from avs-daemon.smtpauth2.wiscmail.wisc.edu by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 32bit)) id <0KAU00A0CGEVIB00@smtpauth2.wiscmail.wisc.edu> for freebsd-arch@freebsd.org; Mon, 24 Nov 2008 09:30:31 -0600 (CST) Received: from trantor.tachypleus.net (adsl-99-154-3-101.dsl.mdsnwi.sbcglobal.net [99.154.3.101]) by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 32bit)) with ESMTPSA id <0KAU00204GESFZ70@smtpauth2.wiscmail.wisc.edu>; Mon, 24 Nov 2008 09:30:29 -0600 (CST) Date: Mon, 24 Nov 2008 09:31:42 -0600 From: Nathan Whitehorn In-reply-to: <20081123.170854.-626772149.imp@bsdimp.com> To: "M. Warner Losh" Message-id: <492AC8DE.6050902@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=99.154.3.101 X-Spam-PmxInfo: Server=avs-12, Version=5.4.1.325704, Antispam-Engine: 2.6.0.325393, Antispam-Data: 2008.11.24.151645, SenderIP=99.154.3.101 References: <86myfq9uha.fsf@ds4.des.no> <4929C6D8.7090305@freebsd.org> <20081123.170854.-626772149.imp@bsdimp.com> User-Agent: Thunderbird 2.0.0.17 (X11/20080928) Cc: freebsd-arch@freebsd.org Subject: Re: Enumerable I2C busses X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 15:30:31 -0000 M. Warner Losh wrote: > In message: <4929C6D8.7090305@freebsd.org> > Nathan Whitehorn writes: > : Rafał Jaworowski wrote: > : > > : > On 2008-11-23, at 19:18, Dag-Erling Smørgrav wrote: > : > > : >> Nathan Whitehorn writes: > : >>> The current I2C bus mechanism does not support the bus adding its own > : >>> children [...] > : >> > : >> That's because the I2C protocol does not support device enumeration or > : >> identification. You have to know in advance what kind of devices are > : >> attached and at what address. Even worse, it is not uncommon for > : >> similar but not entirely compatible devices to use the same I2C address > : >> (for instance, every I2C-capable RTC chip uses the same address, even > : >> though they have different feature sets) > : > > : > Well, hard-coded addresses and conflicting assignments between vendors > : > do not technically prevent from scanning the bus; actually, our current > : > iicbus code can do bus scaning when compiled with a diag define. The > : > problem however is some slave devices are not well-behaved, and they > : > don't like to be read/written to other than in very specific scenario: > : > if polled during bus scan strange effects occur e.g. they disappear from > : > the bus, or do not react to consecutive requests etc. > : > : All of this is true, but perhaps my question was badly worded. What I am > : trying to figure out is how to shove information from an out-of-band > : source (Open Firmware, in this case) into newbus without disrupting > : existing code. In that way, my question is not I2C specific -- we run > : into the same issue with the Open Firmware nexus node and pseudo-devices > : like cryptosoft that attach themselves. > > You are looking at the problem incorrectly. In newbus, a case like > this the i2c bus should be a subclass (say i2c_of) that is derived > from the normal i2c bus stuff, but replaces the hints insertion of > devices with OF enumeration of devices. The OF higher levels will > already know to attach this kind of i2c bus to certain i2c > controllers, or always on certain platforms. Yes, this is exactly what I wanted to do, like how ofw_pci works. > : What I want to do is to have the I2C bus add the children that the > : firmware says it has. What the firmware cannot tell in advance, however, > : is which FreeBSD driver is responsible for those devices, and so the I2C > : bus driver can't know that without a translation table that I would > : prefer not to hack in to the bus driver. > > This is the bigger problem. Today, we are stuck with a lame table > that will translate the OF provided properties into FreeBSD driver > names. At the moment, I don't believe Apple uses any of the current very small number of I2C device drivers in tree. So I may skip the table for the time being, assuming the hack below is OK. In future, this may change, since G5 systems require software thermal control. But that will be the subject of another mail to this list... > : It seems reasonable to allow devices to use a real probe routine to look > : at the firmware's name and compatible properties, like we allow on other > : Open Firmware busses. The trouble is that existing drivers don't do > : this, because they expect to be attached with hints, so they will attach > : to all devices. I'm trying to figure out how to avoid this. > : > : My basic question comes down to whether there is a good way in newbus to > : handle busses that may be wholly or partially enumerated by firmware or > : some other method, and may also have devices that can only attach > : themselves if told to by hints. > > Yes. This is a bit of a problem. The problem is that the existing > hints mechanism combines device tree and driver tree into one, and in > such a scenario, we wind up with the problem that you have. > > One could make the probe routines return BUS_PROBE_GENERIC, and that > would help somewhat. One could also have the probe routine check to > see if a specific driver is assigned to the device or not. That would > also help, but does mean changing all the i2c bus attached drivers in > the tree. I think changing existing I2C drivers may be unavoidable. Would there be any objection to changing the MI iicbus drivers to return BUS_PROBE_NOWILDCARD in their probe routines? It seems to have been introduced (by you) to solve more or less exactly this problem. By my count, the relevant files are: dev/iicbus/ds133x.c dev/iicbus/icee.c dev/iicbus/ad7418.c dev/iicbus/iicsmb.c dev/iicbus/ds1672.c dev/iicbus/if_ic.c dev/iicbus/iic.c I would also like to change iicbus_probe to return -1000 like dev/pci/pci.c to allow it to be overridden by a subclass. Please let me know if this is a terrible idea or if I have forgotten any I2C device drivers. -Nathan From owner-freebsd-arch@FreeBSD.ORG Mon Nov 24 17:57:09 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 967851065672; Mon, 24 Nov 2008 17:57:09 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 3B3388FC14; Mon, 24 Nov 2008 17:57:09 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.2/8.14.1) with ESMTP id mAOHuaBR038425; Mon, 24 Nov 2008 10:56:36 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Mon, 24 Nov 2008 10:58:00 -0700 (MST) Message-Id: <20081124.105800.-267230932.imp@bsdimp.com> To: nwhitehorn@FreeBSD.org From: "M. Warner Losh" In-Reply-To: <492AC8DE.6050902@freebsd.org> References: <4929C6D8.7090305@freebsd.org> <20081123.170854.-626772149.imp@bsdimp.com> <492AC8DE.6050902@freebsd.org> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=utf-8 Content-Transfer-Encoding: base64 Cc: freebsd-arch@FreeBSD.org Subject: Re: Enumerable I2C busses X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Nov 2008 17:57:09 -0000 SW4gbWVzc2FnZTogPDQ5MkFDOERFLjYwNTA5MDJAZnJlZWJzZC5vcmc+DQogICAgICAgICAgICBO YXRoYW4gV2hpdGVob3JuIDxud2hpdGVob3JuQGZyZWVic2Qub3JnPiB3cml0ZXM6DQo6IE0uIFdh cm5lciBMb3NoIHdyb3RlOg0KOiA+IEluIG1lc3NhZ2U6IDw0OTI5QzZEOC43MDkwMzA1QGZyZWVi c2Qub3JnPg0KOiA+ICAgICAgICAgICAgIE5hdGhhbiBXaGl0ZWhvcm4gPG53aGl0ZWhvcm5AZnJl ZWJzZC5vcmc+IHdyaXRlczoNCjogPiA6IFJhZmHFgiBKYXdvcm93c2tpIHdyb3RlOg0KOiA+IDog PiANCjogPiA6ID4gT24gMjAwOC0xMS0yMywgYXQgMTk6MTgsIERhZy1FcmxpbmcgU23DuHJncmF2 IHdyb3RlOg0KOiA+IDogPiANCjogPiA6ID4+IE5hdGhhbiBXaGl0ZWhvcm4gPG53aGl0ZWhvcm5A ZnJlZWJzZC5vcmc+IHdyaXRlczoNCjogPiA6ID4+PiBUaGUgY3VycmVudCBJMkMgYnVzIG1lY2hh bmlzbSBkb2VzIG5vdCBzdXBwb3J0IHRoZSBidXMgYWRkaW5nIGl0cyBvd24NCjogPiA6ID4+PiBj aGlsZHJlbiBbLi4uXQ0KOiA+IDogPj4NCjogPiA6ID4+IFRoYXQncyBiZWNhdXNlIHRoZSBJMkMg cHJvdG9jb2wgZG9lcyBub3Qgc3VwcG9ydCBkZXZpY2UgZW51bWVyYXRpb24gb3INCjogPiA6ID4+ IGlkZW50aWZpY2F0aW9uLiAgWW91IGhhdmUgdG8ga25vdyBpbiBhZHZhbmNlIHdoYXQga2luZCBv ZiBkZXZpY2VzIGFyZQ0KOiA+IDogPj4gYXR0YWNoZWQgYW5kIGF0IHdoYXQgYWRkcmVzcy4gIEV2 ZW4gd29yc2UsIGl0IGlzIG5vdCB1bmNvbW1vbiBmb3INCjogPiA6ID4+IHNpbWlsYXIgYnV0IG5v dCBlbnRpcmVseSBjb21wYXRpYmxlIGRldmljZXMgdG8gdXNlIHRoZSBzYW1lIEkyQyBhZGRyZXNz DQo6ID4gOiA+PiAoZm9yIGluc3RhbmNlLCBldmVyeSBJMkMtY2FwYWJsZSBSVEMgY2hpcCB1c2Vz IHRoZSBzYW1lIGFkZHJlc3MsIGV2ZW4NCjogPiA6ID4+IHRob3VnaCB0aGV5IGhhdmUgZGlmZmVy ZW50IGZlYXR1cmUgc2V0cykNCjogPiA6ID4gDQo6ID4gOiA+IFdlbGwsIGhhcmQtY29kZWQgYWRk cmVzc2VzIGFuZCBjb25mbGljdGluZyBhc3NpZ25tZW50cyBiZXR3ZWVuIHZlbmRvcnMgDQo6ID4g OiA+IGRvIG5vdCB0ZWNobmljYWxseSBwcmV2ZW50IGZyb20gc2Nhbm5pbmcgdGhlIGJ1czsgYWN0 dWFsbHksIG91ciBjdXJyZW50IA0KOiA+IDogPiBpaWNidXMgY29kZSBjYW4gZG8gYnVzIHNjYW5p bmcgd2hlbiBjb21waWxlZCB3aXRoIGEgZGlhZyBkZWZpbmUuIFRoZSANCjogPiA6ID4gcHJvYmxl bSBob3dldmVyIGlzIHNvbWUgc2xhdmUgZGV2aWNlcyBhcmUgbm90IHdlbGwtYmVoYXZlZCwgYW5k IHRoZXkgDQo6ID4gOiA+IGRvbid0IGxpa2UgdG8gYmUgcmVhZC93cml0dGVuIHRvIG90aGVyIHRo YW4gaW4gdmVyeSBzcGVjaWZpYyBzY2VuYXJpbzogDQo6ID4gOiA+IGlmIHBvbGxlZCBkdXJpbmcg YnVzIHNjYW4gc3RyYW5nZSBlZmZlY3RzIG9jY3VyIGUuZy4gdGhleSBkaXNhcHBlYXIgZnJvbSAN CjogPiA6ID4gdGhlIGJ1cywgb3IgZG8gbm90IHJlYWN0IHRvIGNvbnNlY3V0aXZlIHJlcXVlc3Rz IGV0Yy4NCjogPiA6IA0KOiA+IDogQWxsIG9mIHRoaXMgaXMgdHJ1ZSwgYnV0IHBlcmhhcHMgbXkg cXVlc3Rpb24gd2FzIGJhZGx5IHdvcmRlZC4gV2hhdCBJIGFtIA0KOiA+IDogdHJ5aW5nIHRvIGZp Z3VyZSBvdXQgaXMgaG93IHRvIHNob3ZlIGluZm9ybWF0aW9uIGZyb20gYW4gb3V0LW9mLWJhbmQg DQo6ID4gOiBzb3VyY2UgKE9wZW4gRmlybXdhcmUsIGluIHRoaXMgY2FzZSkgaW50byBuZXdidXMg d2l0aG91dCBkaXNydXB0aW5nIA0KOiA+IDogZXhpc3RpbmcgY29kZS4gSW4gdGhhdCB3YXksIG15 IHF1ZXN0aW9uIGlzIG5vdCBJMkMgc3BlY2lmaWMgLS0gd2UgcnVuIA0KOiA+IDogaW50byB0aGUg c2FtZSBpc3N1ZSB3aXRoIHRoZSBPcGVuIEZpcm13YXJlIG5leHVzIG5vZGUgYW5kIHBzZXVkby1k ZXZpY2VzIA0KOiA+IDogbGlrZSBjcnlwdG9zb2Z0IHRoYXQgYXR0YWNoIHRoZW1zZWx2ZXMuDQo6 ID4gDQo6ID4gWW91IGFyZSBsb29raW5nIGF0IHRoZSBwcm9ibGVtIGluY29ycmVjdGx5LiAgSW4g bmV3YnVzLCBhIGNhc2UgbGlrZQ0KOiA+IHRoaXMgdGhlIGkyYyBidXMgc2hvdWxkIGJlIGEgc3Vi Y2xhc3MgKHNheSBpMmNfb2YpIHRoYXQgaXMgZGVyaXZlZA0KOiA+IGZyb20gdGhlIG5vcm1hbCBp MmMgYnVzIHN0dWZmLCBidXQgcmVwbGFjZXMgdGhlIGhpbnRzIGluc2VydGlvbiBvZg0KOiA+IGRl dmljZXMgd2l0aCBPRiBlbnVtZXJhdGlvbiBvZiBkZXZpY2VzLiAgVGhlIE9GIGhpZ2hlciBsZXZl bHMgd2lsbA0KOiA+IGFscmVhZHkga25vdyB0byBhdHRhY2ggdGhpcyBraW5kIG9mIGkyYyBidXMg dG8gY2VydGFpbiBpMmMNCjogPiBjb250cm9sbGVycywgb3IgYWx3YXlzIG9uIGNlcnRhaW4gcGxh dGZvcm1zLg0KOiANCjogWWVzLCB0aGlzIGlzIGV4YWN0bHkgd2hhdCBJIHdhbnRlZCB0byBkbywg bGlrZSBob3cgb2Z3X3BjaSB3b3Jrcy4NCjogDQo6ID4gOiBXaGF0IEkgd2FudCB0byBkbyBpcyB0 byBoYXZlIHRoZSBJMkMgYnVzIGFkZCB0aGUgY2hpbGRyZW4gdGhhdCB0aGUgDQo6ID4gOiBmaXJt d2FyZSBzYXlzIGl0IGhhcy4gV2hhdCB0aGUgZmlybXdhcmUgY2Fubm90IHRlbGwgaW4gYWR2YW5j ZSwgaG93ZXZlciwgDQo6ID4gOiBpcyB3aGljaCBGcmVlQlNEIGRyaXZlciBpcyByZXNwb25zaWJs ZSBmb3IgdGhvc2UgZGV2aWNlcywgYW5kIHNvIHRoZSBJMkMgDQo6ID4gOiBidXMgZHJpdmVyIGNh bid0IGtub3cgdGhhdCB3aXRob3V0IGEgdHJhbnNsYXRpb24gdGFibGUgdGhhdCBJIHdvdWxkIA0K OiA+IDogcHJlZmVyIG5vdCB0byBoYWNrIGluIHRvIHRoZSBidXMgZHJpdmVyLg0KOiA+IA0KOiA+ IFRoaXMgaXMgdGhlIGJpZ2dlciBwcm9ibGVtLiAgVG9kYXksIHdlIGFyZSBzdHVjayB3aXRoIGEg bGFtZSB0YWJsZQ0KOiA+IHRoYXQgd2lsbCB0cmFuc2xhdGUgdGhlIE9GIHByb3ZpZGVkIHByb3Bl cnRpZXMgaW50byBGcmVlQlNEIGRyaXZlcg0KOiA+IG5hbWVzLg0KOiANCjogQXQgdGhlIG1vbWVu dCwgSSBkb24ndCBiZWxpZXZlIEFwcGxlIHVzZXMgYW55IG9mIHRoZSBjdXJyZW50IHZlcnkgc21h bGwgDQo6IG51bWJlciBvZiBJMkMgZGV2aWNlIGRyaXZlcnMgaW4gdHJlZS4gU28gSSBtYXkgc2tp cCB0aGUgdGFibGUgZm9yIHRoZSANCjogdGltZSBiZWluZywgYXNzdW1pbmcgdGhlIGhhY2sgYmVs b3cgaXMgT0suIEluIGZ1dHVyZSwgdGhpcyBtYXkgY2hhbmdlLCANCjogc2luY2UgRzUgc3lzdGVt cyByZXF1aXJlIHNvZnR3YXJlIHRoZXJtYWwgY29udHJvbC4gQnV0IHRoYXQgd2lsbCBiZSB0aGUg DQo6IHN1YmplY3Qgb2YgYW5vdGhlciBtYWlsIHRvIHRoaXMgbGlzdC4uLg0KOiANCjogPiA6IEl0 IHNlZW1zIHJlYXNvbmFibGUgdG8gYWxsb3cgZGV2aWNlcyB0byB1c2UgYSByZWFsIHByb2JlIHJv dXRpbmUgdG8gbG9vayANCjogPiA6IGF0IHRoZSBmaXJtd2FyZSdzIG5hbWUgYW5kIGNvbXBhdGli bGUgcHJvcGVydGllcywgbGlrZSB3ZSBhbGxvdyBvbiBvdGhlciANCjogPiA6IE9wZW4gRmlybXdh cmUgYnVzc2VzLiBUaGUgdHJvdWJsZSBpcyB0aGF0IGV4aXN0aW5nIGRyaXZlcnMgZG9uJ3QgZG8g DQo6ID4gOiB0aGlzLCBiZWNhdXNlIHRoZXkgZXhwZWN0IHRvIGJlIGF0dGFjaGVkIHdpdGggaGlu dHMsIHNvIHRoZXkgd2lsbCBhdHRhY2ggDQo6ID4gOiB0byBhbGwgZGV2aWNlcy4gSSdtIHRyeWlu ZyB0byBmaWd1cmUgb3V0IGhvdyB0byBhdm9pZCB0aGlzLg0KOiA+IDogDQo6ID4gOiBNeSBiYXNp YyBxdWVzdGlvbiBjb21lcyBkb3duIHRvIHdoZXRoZXIgdGhlcmUgaXMgYSBnb29kIHdheSBpbiBu ZXdidXMgdG8gDQo6ID4gOiBoYW5kbGUgYnVzc2VzIHRoYXQgbWF5IGJlIHdob2xseSBvciBwYXJ0 aWFsbHkgZW51bWVyYXRlZCBieSBmaXJtd2FyZSBvciANCjogPiA6IHNvbWUgb3RoZXIgbWV0aG9k LCBhbmQgbWF5IGFsc28gaGF2ZSBkZXZpY2VzIHRoYXQgY2FuIG9ubHkgYXR0YWNoIA0KOiA+IDog dGhlbXNlbHZlcyBpZiB0b2xkIHRvIGJ5IGhpbnRzLg0KOiA+IA0KOiA+IFllcy4gIFRoaXMgaXMg YSBiaXQgb2YgYSBwcm9ibGVtLiAgVGhlIHByb2JsZW0gaXMgdGhhdCB0aGUgZXhpc3RpbmcNCjog PiBoaW50cyBtZWNoYW5pc20gY29tYmluZXMgZGV2aWNlIHRyZWUgYW5kIGRyaXZlciB0cmVlIGlu dG8gb25lLCBhbmQgaW4NCjogPiBzdWNoIGEgc2NlbmFyaW8sIHdlIHdpbmQgdXAgd2l0aCB0aGUg cHJvYmxlbSB0aGF0IHlvdSBoYXZlLg0KOiA+IA0KOiA+IE9uZSBjb3VsZCBtYWtlIHRoZSBwcm9i ZSByb3V0aW5lcyByZXR1cm4gQlVTX1BST0JFX0dFTkVSSUMsIGFuZCB0aGF0DQo6ID4gd291bGQg aGVscCBzb21ld2hhdC4gIE9uZSBjb3VsZCBhbHNvIGhhdmUgdGhlIHByb2JlIHJvdXRpbmUgY2hl Y2sgdG8NCjogPiBzZWUgaWYgYSBzcGVjaWZpYyBkcml2ZXIgaXMgYXNzaWduZWQgdG8gdGhlIGRl dmljZSBvciBub3QuICBUaGF0IHdvdWxkDQo6ID4gYWxzbyBoZWxwLCBidXQgZG9lcyBtZWFuIGNo YW5naW5nIGFsbCB0aGUgaTJjIGJ1cyBhdHRhY2hlZCBkcml2ZXJzIGluDQo6ID4gdGhlIHRyZWUu DQo6IA0KOiBJIHRoaW5rIGNoYW5naW5nIGV4aXN0aW5nIEkyQyBkcml2ZXJzIG1heSBiZSB1bmF2 b2lkYWJsZS4gV291bGQgdGhlcmUgYmUgDQo6IGFueSBvYmplY3Rpb24gdG8gY2hhbmdpbmcgdGhl IE1JIGlpY2J1cyBkcml2ZXJzIHRvIHJldHVybiANCjogQlVTX1BST0JFX05PV0lMRENBUkQgaW4g dGhlaXIgcHJvYmUgcm91dGluZXM/IEl0IHNlZW1zIHRvIGhhdmUgYmVlbiANCjogaW50cm9kdWNl ZCAoYnkgeW91KSB0byBzb2x2ZSBtb3JlIG9yIGxlc3MgZXhhY3RseSB0aGlzIHByb2JsZW0uIEJ5 IG15IA0KOiBjb3VudCwgdGhlIHJlbGV2YW50IGZpbGVzIGFyZToNCjogZGV2L2lpY2J1cy9kczEz M3guYw0KOiBkZXYvaWljYnVzL2ljZWUuYw0KOiBkZXYvaWljYnVzL2FkNzQxOC5jDQo6IGRldi9p aWNidXMvaWljc21iLmMNCjogZGV2L2lpY2J1cy9kczE2NzIuYw0KOiBkZXYvaWljYnVzL2lmX2lj LmMNCjogZGV2L2lpY2J1cy9paWMuYw0KOiANCjogSSB3b3VsZCBhbHNvIGxpa2UgdG8gY2hhbmdl IGlpY2J1c19wcm9iZSB0byByZXR1cm4gLTEwMDAgbGlrZSANCjogZGV2L3BjaS9wY2kuYyB0byBh bGxvdyBpdCB0byBiZSBvdmVycmlkZGVuIGJ5IGEgc3ViY2xhc3MuIFBsZWFzZSBsZXQgbWUgDQo6 IGtub3cgaWYgdGhpcyBpcyBhIHRlcnJpYmxlIGlkZWEgb3IgaWYgSSBoYXZlIGZvcmdvdHRlbiBh bnkgSTJDIGRldmljZSANCjogZHJpdmVycy4NCg0KU2hvcnQgdGVybSwgdGhpcyBpcyB0aGUgcmln aHQgZml4LiAgVGhlcmUgd2FzIGFuIG9iamVjdGlvbiwgSSB0aGluayBieQ0KTWFyY2VsLCB0byB0 aGlzIGFwcHJvYWNoLiAgSG93ZXZlciwgaGlzIG9iamVjdGlvbnMgd2VyZSBwYXJ0IG9mIGENCmxh cmdlciBzZXQgb2Ygb2JqZWN0aW9ucyBhbmQgSSB0aGluayB0aGF0IHdlJ3JlIHdvcmtpbmcgdG8g c29sdmUNCnRob3NlLg0KDQpXYXJuZXINCg== From owner-freebsd-arch@FreeBSD.ORG Tue Nov 25 00:09:13 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66E8B1065740 for ; Tue, 25 Nov 2008 00:09:13 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from proxy.meer.net (proxy.meer.net [64.13.141.13]) by mx1.freebsd.org (Postfix) with ESMTP id 3367F8FC0C for ; Tue, 25 Nov 2008 00:09:13 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from mail.meer.net (mail.meer.net [64.13.141.3]) by proxy.meer.net (8.14.2/8.14.2) with ESMTP id mAONm0m3004755; Mon, 24 Nov 2008 15:48:01 -0800 (PST) (envelope-from gnn@neville-neil.com) Received: from mail2.meer.net (mail2.meer.net [64.13.141.16]) by mail.meer.net (8.13.3/8.13.3/meer) with ESMTP id mAONk4Wd006211; Mon, 24 Nov 2008 15:46:04 -0800 (PST) (envelope-from gnn@neville-neil.com) Received: from minion.local.neville-neil.com (209.249.190.254.available.above.net [209.249.190.254] (may be forged)) (authenticated bits=0) by mail2.meer.net (8.14.1/8.14.1) with ESMTP id mAONk3PU009335; Mon, 24 Nov 2008 15:46:04 -0800 (PST) (envelope-from gnn@neville-neil.com) Date: Mon, 24 Nov 2008 18:46:02 -0500 Message-ID: From: gnn@freebsd.org To: Jeff Roberson In-Reply-To: <20081123213232.A971@desktop> References: <20081123213232.A971@desktop> User-Agent: Wanderlust/2.15.5 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.7 Emacs/22.3 (i386-apple-darwin9.5.0) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-Canit-CHI2: 0.50 X-Bayes-Prob: 0.5 (Score 0, tokens from: ) X-Spam-Score: 0.10 () [Tag at 5.00] COMBINED_FROM X-CanItPRO-Stream: default X-Canit-Stats-ID: 2387736 - 476b1cc662f4 X-Scanned-By: CanIt (www . roaringpenguin . com) on 64.13.141.13 Cc: arch@freebsd.org Subject: Re: Limiting mbuf memory. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2008 00:09:13 -0000 At Sun, 23 Nov 2008 21:46:08 -1000 (HST), Jeff Roberson wrote: > > I'm developing a patch for an alternate memory layout for mbuf clusters > that relies on contigmalloc. Since this can fail, we'll still have to > retain the capability of allocating traditional clusters. I'll report > details on that later. I'm writing this email to address the issue of > resource accounting in mbufs. > > Presently we use a set of limits on individual zones or sizes of mbufs. > Standard mbufs, clusters, page size jumbos, 9k jumbos, and 16k jumbos. > Each is administered sperately. I think this is getting a bit unwieldy. > In the future, we may have even more sizes. This also introduces problems > because I will have two cluster zones do they each get their own limit? > > I would like to consolidate this into a single limit on the number of > pages in total allocated to networking. With perhaps some fractional > reservation for standard mbufs and clusters to make sure they aren't > overwhelmed by the larger buffers. > > This would be implemented by overriding the uma zone page allocator for > each of the mbuf zones with one that counts pages for all. Should we > reach the limit we'll block depending on the wait settings of the > requestor. The limit and sleep will probably be protected by a global > lock which won't be an issue because trips to the back end allocator are > infrequent and protected by their own global lock anyhow. > > How do people feel about this? To be clear this would eliminate: > > nmbclusters, nmbjumbop, nmbjumbo9, nmbjumbo16 and related config settings > and sysctls. They would be replaced by something like 'maxmbufbytes'. > Presently we place no limit on small mbufs. I could go either way on > this. It could be added to the limit or not. > I think this is a good idea with the caveat that I prefer the idea in paragraph 3 about reserving a bit of head room so we don't deadlock. A very common bug in the past was to run out of mbufs when using a lot of small UDP packets. Best, George From owner-freebsd-arch@FreeBSD.ORG Tue Nov 25 02:06:11 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6E041065670 for ; Tue, 25 Nov 2008 02:06:11 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.177]) by mx1.freebsd.org (Postfix) with ESMTP id C59078FC13 for ; Tue, 25 Nov 2008 02:06:11 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by wa-out-1112.google.com with SMTP id m34so1172744wag.27 for ; Mon, 24 Nov 2008 18:06:11 -0800 (PST) Received: by 10.114.195.19 with SMTP id s19mr2357596waf.10.1227578771499; Mon, 24 Nov 2008 18:06:11 -0800 (PST) Received: from ?10.0.1.199? (cpe-66-91-191-118.hawaii.res.rr.com [66.91.191.118]) by mx.google.com with ESMTPS id n20sm4588315pof.10.2008.11.24.18.06.08 (version=SSLv3 cipher=RC4-MD5); Mon, 24 Nov 2008 18:06:10 -0800 (PST) Date: Mon, 24 Nov 2008 16:03:35 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: gnn@freebsd.org In-Reply-To: Message-ID: <20081124160316.I971@desktop> References: <20081123213232.A971@desktop> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org Subject: Re: Limiting mbuf memory. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2008 02:06:12 -0000 On Mon, 24 Nov 2008, gnn@freebsd.org wrote: > At Sun, 23 Nov 2008 21:46:08 -1000 (HST), > Jeff Roberson wrote: >> >> I'm developing a patch for an alternate memory layout for mbuf clusters >> that relies on contigmalloc. Since this can fail, we'll still have to >> retain the capability of allocating traditional clusters. I'll report >> details on that later. I'm writing this email to address the issue of >> resource accounting in mbufs. >> >> Presently we use a set of limits on individual zones or sizes of mbufs. >> Standard mbufs, clusters, page size jumbos, 9k jumbos, and 16k jumbos. >> Each is administered sperately. I think this is getting a bit unwieldy. >> In the future, we may have even more sizes. This also introduces problems >> because I will have two cluster zones do they each get their own limit? >> >> I would like to consolidate this into a single limit on the number of >> pages in total allocated to networking. With perhaps some fractional >> reservation for standard mbufs and clusters to make sure they aren't >> overwhelmed by the larger buffers. >> >> This would be implemented by overriding the uma zone page allocator for >> each of the mbuf zones with one that counts pages for all. Should we >> reach the limit we'll block depending on the wait settings of the >> requestor. The limit and sleep will probably be protected by a global >> lock which won't be an issue because trips to the back end allocator are >> infrequent and protected by their own global lock anyhow. >> >> How do people feel about this? To be clear this would eliminate: >> >> nmbclusters, nmbjumbop, nmbjumbo9, nmbjumbo16 and related config settings >> and sysctls. They would be replaced by something like 'maxmbufbytes'. >> Presently we place no limit on small mbufs. I could go either way on >> this. It could be added to the limit or not. >> > > I think this is a good idea with the caveat that I prefer the idea in > paragraph 3 about reserving a bit of head room so we don't deadlock. > A very common bug in the past was to run out of mbufs when using a lot > of small UDP packets. Ok, I believe the existing per-type limits will facilitate this. Thanks, Jeff > > Best, > George > From owner-freebsd-arch@FreeBSD.ORG Tue Nov 25 10:19:05 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 350631065696 for ; Tue, 25 Nov 2008 10:19:05 +0000 (UTC) (envelope-from PublicServicePartnershipLtd_769894@dotmailer.co.uk) Received: from mta22.mta.dotmailer.co.uk (mta22.mta.dotmailer.co.uk [80.87.10.22]) by mx1.freebsd.org (Postfix) with ESMTP id DE2628FC21 for ; Tue, 25 Nov 2008 10:19:04 +0000 (UTC) (envelope-from PublicServicePartnershipLtd_769894@dotmailer.co.uk) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=gen-key1; d=dotmailer.co.uk; h=From:To:Date:Subject:MIME-Version:Content-Type:List-Unsubscribe:Message-ID; i=PublicServicePartnershipLtd_769894@dotmailer.co.uk; bh=yOw6LFoen6uadm2CRvtcVP3yL6c=; b=iKsNzgS2kikPIe/9VFHsnPCMfB01UhQ8eytKcn0ys5dJKLdi4W8jj1Roz6UydpGkO53JfP+qQDIN LGUyXWyD8g== DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=gen-key1; d=dotmailer.co.uk; b=DzZ14bDh8pCgTO4sLqfeex6IdsZ0w2F0srFUVCANRHwhAfxEir5KegbdqrL9XmQdhM8DF9bqgms9 oX60Hebz0A==; Received: from CHEWBACCA (80.87.6.5) by mta22.mta.dotmailer.co.uk (PowerMTA(TM) v3.5r10) id h5f8ho0k80ki for ; Tue, 25 Nov 2008 09:51:31 +0000 (envelope-from ) From: "Mike Cross" To: "freebsd-arch@freebsd.org" Date: Tue, 25 Nov 2008 09:51:30 +0000 MIME-Version: 1.0 X-Mailer: dotMailer X-EMdotMailDroidID: 7 X-EMdotMailID: 443873275 Message-ID: Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Emailing to the Public Sector Made Easy X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2008 10:19:05 -0000 From owner-freebsd-arch@FreeBSD.ORG Tue Nov 25 16:40:54 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E39141065670; Tue, 25 Nov 2008 16:40:54 +0000 (UTC) (envelope-from lulf@freebsd.org) Received: from bene2.itea.ntnu.no (bene2.itea.ntnu.no [IPv6:2001:700:300:3::57]) by mx1.freebsd.org (Postfix) with ESMTP id 350B78FC0C; Tue, 25 Nov 2008 16:40:54 +0000 (UTC) (envelope-from lulf@freebsd.org) Received: from localhost (localhost [127.0.0.1]) by bene2.itea.ntnu.no (Postfix) with ESMTP id 5367090006; Tue, 25 Nov 2008 17:40:52 +0100 (CET) Received: from nobby (unknown [IPv6:2001:700:300:3::184]) by bene2.itea.ntnu.no (Postfix) with ESMTP id 8D6E790005; Tue, 25 Nov 2008 17:40:51 +0100 (CET) Date: Tue, 25 Nov 2008 16:40:40 +0100 From: Ulf Lilleengen To: freebsd-current@freebsd.org Message-ID: <20081125154040.GA12632@nobby.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HcAYCG3uE/tztfnV" Content-Disposition: inline User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: Debian amavisd-new at bene2.itea.ntnu.no Cc: freebsd-wip@freebsd.org, freebsd-arch@freebsd.org Subject: HEADSUP: CVS/Mirror mode for csup to be merged soon X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2008 16:40:55 -0000 --HcAYCG3uE/tztfnV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, After some feedback on previous patches and some adjustments, I think the CVSMode for csup project have come to a place where a wider testing audience is needed, and I would like to make this a call for review and a HEADSUP to allow willing reviewers and eventual protesters to give their opinion before merging this to HEAD. A few things about the current state of CVSMode: - Complete CVS mode (mirror mode) is supported, allowing the whole CVS repository to be fetched by csup. - rsync fetch supported if not explicitly not wanted by user or not support= ed by server. - Support using the status file to speed up detailing of files. This means = no bigger inpact on files that are up to date. For the state of the code itself, I have went over it a couple of times the last couple of days, fixing style issues and a few differences between cvsup and csup. One important thing to note is that the impact on the existing cs= up operation is _minimal_, so that the risk of introducing bugs to the normal csup operation is very small, and because of this I see no problems with committing the current version. If you find any issues, please e-mail me, a= nd I will look at it. So, for those of you wanting to test, please do so now. If people are okay with this, I would like to merge it by the end of the week/early next week. A patch can be found here: http://people.freebsd.org/~lulf/csup_cvsmode.diff or you can just do a checkout of projects/csup_cvsmode --=20 Ulf Lilleengen --HcAYCG3uE/tztfnV Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkksHHcACgkQCILg8nMIdCUtkQCfab66AfmNbh4r5CPK2MHXji3l PbYAn3fGb1WVfqGlca9ZUtpGpjPJ35hI =5m2O -----END PGP SIGNATURE----- --HcAYCG3uE/tztfnV-- From owner-freebsd-arch@FreeBSD.ORG Tue Nov 25 20:03:08 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 780E31065677; Tue, 25 Nov 2008 20:03:08 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) by mx1.freebsd.org (Postfix) with ESMTP id E9D278FC25; Tue, 25 Nov 2008 20:03:07 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.2/8.14.2) with ESMTP id mAPJq3H4060391; Tue, 25 Nov 2008 22:52:03 +0300 (MSK) (envelope-from marck@rinet.ru) Date: Tue, 25 Nov 2008 22:52:03 +0300 (MSK) From: Dmitry Morozovsky To: Ulf Lilleengen In-Reply-To: <20081125154040.GA12632@nobby.lan> Message-ID: References: <20081125154040.GA12632@nobby.lan> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0.1 (woozle.rinet.ru [0.0.0.0]); Tue, 25 Nov 2008 22:52:03 +0300 (MSK) Cc: freebsd-current@freebsd.org, freebsd-wip@freebsd.org, freebsd-arch@freebsd.org Subject: Re: HEADSUP: CVS/Mirror mode for csup to be merged soon X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2008 20:03:08 -0000 On Tue, 25 Nov 2008, Ulf Lilleengen wrote: UL> Hello, UL> UL> After some feedback on previous patches and some adjustments, I think the UL> CVSMode for csup project have come to a place where a wider testing audience UL> is needed, and I would like to make this a call for review and a HEADSUP to UL> allow willing reviewers and eventual protesters to give their opinion before UL> merging this to HEAD. A few things about the current state of CVSMode: UL> UL> - Complete CVS mode (mirror mode) is supported, allowing the whole CVS UL> repository to be fetched by csup. UL> - rsync fetch supported if not explicitly not wanted by user or not supported UL> by server. UL> - Support using the status file to speed up detailing of files. This means no UL> bigger inpact on files that are up to date. UL> UL> For the state of the code itself, I have went over it a couple of times the UL> last couple of days, fixing style issues and a few differences between cvsup UL> and csup. One important thing to note is that the impact on the existing csup UL> operation is _minimal_, so that the risk of introducing bugs to the normal UL> csup operation is very small, and because of this I see no problems with UL> committing the current version. If you find any issues, please e-mail me, and UL> I will look at it. UL> UL> So, for those of you wanting to test, please do so now. If people are okay UL> with this, I would like to merge it by the end of the week/early next week. UL> UL> A patch can be found here: http://people.freebsd.org/~lulf/csup_cvsmode.diff UL> or you can just do a checkout of projects/csup_cvsmode Just to make sure it does not get lost: After creating RELENG_7_1 branch csupping is broken with: Updating collection src-all/cvs Edit src/bin/chio/chio.c,v /home/ncvs/src/bin/chio/chio.c,v: Checksum mismatch -- will transfer entire file Edit src/contrib/bind9/CHANGES,v /home/ncvs/src/contrib/bind9/CHANGES,v: Checksum mismatch -- will transfer entire file Edit src/contrib/bind9/COPYRIGHT,v ... [a lot of them] .. and finally /home/ncvs/src/sys/i386/conf/NOTES,v: Checksum mismatch -- will transfer entire file Edit src/sys/i386/conf/PAE,v Error applying diff: -1 Updater failed: Protocol error Error is 'Detailer failed: Premature EOF from server' I use RELENG_7 csup with your previous patch version Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-arch@FreeBSD.ORG Tue Nov 25 21:05:05 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 400D31065672; Tue, 25 Nov 2008 21:05:05 +0000 (UTC) (envelope-from lulf@freebsd.org) Received: from bene2.itea.ntnu.no (bene2.itea.ntnu.no [IPv6:2001:700:300:3::57]) by mx1.freebsd.org (Postfix) with ESMTP id 220A88FC0C; Tue, 25 Nov 2008 21:05:04 +0000 (UTC) (envelope-from lulf@freebsd.org) Received: from localhost (localhost [127.0.0.1]) by bene2.itea.ntnu.no (Postfix) with ESMTP id AB17790002; Tue, 25 Nov 2008 22:05:02 +0100 (CET) Received: from nobby (unknown [IPv6:2001:700:300:3::184]) by bene2.itea.ntnu.no (Postfix) with ESMTP id 15F2990001; Tue, 25 Nov 2008 22:05:02 +0100 (CET) Date: Tue, 25 Nov 2008 21:04:51 +0100 From: Ulf Lilleengen To: Dmitry Morozovsky Message-ID: <20081125200451.GA3160@nobby.lan> References: <20081125154040.GA12632@nobby.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="17pEHd4RhPHOinZp" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: Debian amavisd-new at bene2.itea.ntnu.no Cc: freebsd-current@freebsd.org, freebsd-wip@freebsd.org, freebsd-arch@freebsd.org Subject: Re: HEADSUP: CVS/Mirror mode for csup to be merged soon X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2008 21:05:05 -0000 --17pEHd4RhPHOinZp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Nov 25, 2008 at 10:52:03PM +0300, Dmitry Morozovsky wrote: > On Tue, 25 Nov 2008, Ulf Lilleengen wrote: >=20 > UL> Hello, > UL>=20 > UL> After some feedback on previous patches and some adjustments, I think= the > UL> CVSMode for csup project have come to a place where a wider testing a= udience > UL> is needed, and I would like to make this a call for review and a HEAD= SUP to > UL> allow willing reviewers and eventual protesters to give their opinion= before > UL> merging this to HEAD. A few things about the current state of CVSMode: > UL>=20 > UL> - Complete CVS mode (mirror mode) is supported, allowing the whole CVS > UL> repository to be fetched by csup. > UL> - rsync fetch supported if not explicitly not wanted by user or not s= upported > UL> by server. > UL> - Support using the status file to speed up detailing of files. This = means no > UL> bigger inpact on files that are up to date. > UL>=20 > UL> For the state of the code itself, I have went over it a couple of tim= es the > UL> last couple of days, fixing style issues and a few differences betwee= n cvsup > UL> and csup. One important thing to note is that the impact on the exist= ing csup > UL> operation is _minimal_, so that the risk of introducing bugs to the n= ormal > UL> csup operation is very small, and because of this I see no problems w= ith > UL> committing the current version. If you find any issues, please e-mail= me, and > UL> I will look at it. > UL>=20 > UL> So, for those of you wanting to test, please do so now. If people are= okay > UL> with this, I would like to merge it by the end of the week/early next= week. > UL>=20 > UL> A patch can be found here: http://people.freebsd.org/~lulf/csup_cvsmo= de.diff > UL> or you can just do a checkout of projects/csup_cvsmode >=20 > Just to make sure it does not get lost: >=20 > After creating RELENG_7_1 branch csupping is broken with: >=20 >=20 > Updating collection src-all/cvs > Edit src/bin/chio/chio.c,v > /home/ncvs/src/bin/chio/chio.c,v: Checksum mismatch -- will transfer enti= re=20 > file > Edit src/contrib/bind9/CHANGES,v > /home/ncvs/src/contrib/bind9/CHANGES,v: Checksum mismatch -- will transfe= r=20 > entire file > Edit src/contrib/bind9/COPYRIGHT,v >=20 > ... [a lot of them] .. and finally >=20 > /home/ncvs/src/sys/i386/conf/NOTES,v: Checksum mismatch -- will transfer = entire=20 > file > Edit src/sys/i386/conf/PAE,v > Error applying diff: -1 > Updater failed: Protocol error > Error is 'Detailer failed: Premature EOF from server' >=20 > I use RELENG_7 csup with your previous patch version >=20 The previous patch will break, and the issue should be fixed in the latest version. --=20 Ulf Lilleengen --17pEHd4RhPHOinZp Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkksWmMACgkQCILg8nMIdCXtswCeMX0028FB8vy5a3qZ4UUF+t4h k9IAnjMQuc9mMT19uHoC/GxHwAJnXfee =b60s -----END PGP SIGNATURE----- --17pEHd4RhPHOinZp-- From owner-freebsd-arch@FreeBSD.ORG Tue Nov 25 23:00:52 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EBFE1065673; Tue, 25 Nov 2008 23:00:52 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from proxy.meer.net (proxy.meer.net [64.13.141.13]) by mx1.freebsd.org (Postfix) with ESMTP id 823438FC13; Tue, 25 Nov 2008 23:00:52 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from mail.meer.net (mail.meer.net [64.13.141.3]) by proxy.meer.net (8.14.2/8.14.2) with ESMTP id mAPN0ZeL080221; Tue, 25 Nov 2008 15:00:48 -0800 (PST) (envelope-from gnn@neville-neil.com) Received: from mail2.meer.net (mail2.meer.net [64.13.141.16]) by mail.meer.net (8.13.3/8.13.3/meer) with ESMTP id mAPMvuiH033877; Tue, 25 Nov 2008 14:57:56 -0800 (PST) (envelope-from gnn@neville-neil.com) Received: from minion.local.neville-neil.com (209.249.190.254.available.above.net [209.249.190.254] (may be forged)) (authenticated bits=0) by mail2.meer.net (8.14.1/8.14.1) with ESMTP id mAPMvtZU025484; Tue, 25 Nov 2008 14:57:55 -0800 (PST) (envelope-from gnn@neville-neil.com) Date: Tue, 25 Nov 2008 17:57:54 -0500 Message-ID: From: gnn@freebsd.org To: "Attilio Rao" In-Reply-To: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> References: <3bbf2fe10811230502t3cc52809i6ac91082f780b730@mail.gmail.com> User-Agent: Wanderlust/2.15.5 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.7 Emacs/22.3 (i386-apple-darwin9.5.0) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-Canit-CHI2: 0.50 X-Bayes-Prob: 0.5 (Score 0, tokens from: ) X-Spam-Score: 0.10 () [Tag at 5.00] COMBINED_FROM X-CanItPRO-Stream: default X-Canit-Stats-ID: 2401205 - 6363b7530a94 X-Scanned-By: CanIt (www . roaringpenguin . com) on 64.13.141.13 Cc: FreeBSD Arch , freebsd-performance@freebsd.org, Joseph Koshy Subject: Re: [PATCH] pmcannotate tool X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2008 23:00:52 -0000 At Sun, 23 Nov 2008 14:02:22 +0100, Attilio Rao wrote: > > pmcannotate is a tool that prints out sources of a tool (in C or > assembly) with inlined profiling informations retrieved by a prior > pmcstat analysis. > If compared with things like callgraph generation, it prints out > profiling on a per-instance basis and this can be useful to find, for > example, badly handled caches, too high latency instructions, etc. > > The tool usage is pretty simple: > pmcannotate [-a] [-h] [-k path] [-l level] samples.out binaryobj > > where samples.out is a pmcstat raw output and binaryobj is the binary > object that has been profiled and is accessible for (ELF) symbols > retrieving. > The options are better described in manpages but briefly: > - a: performs analysis on the assembly rather than the C source > - h: usage and informations > - k: specify a path for the kernel in order to locate correct objects for it > - l: specify a lower boundary (in total percentage time) after which > functions will be displayed nomore. > > A typical usage of pmcannotate can be some way of kernel annotation. > For example, you can follow the steps below: > 1) Generate a pmc raw output of system samples: > # pmcstat -S ipm-unhalted-core-cycles -O samples.out > 2) Copy the samples in the kernel building dir and cd there > # cp samples.out /usr/src/sys/i386/compile/GENERIC/ ; cd > /usr/src/sys/i386/compile/GENERIC/ > 3) Run pmcannotate > # pmcannotate -k . samples.out kernel.debug > kernel.ann > > In the example above please note that kernel.debug has to be used in > order to produce a C annotated source. This happens because in order > to get the binary sources we rely on the "objdump -S" command which > wants binary compiled with debugging options. > If not debugging options are present assembly analynsis is still > possible, but no C-backed one will be available. > objdump is not the only one tool on which pmcannotare rely. Infact, in > order to have it working, pmcstat needs to be present too because we > need to retrieve, from the pmcstat raw output, informations about the > sampled PCs (in particular the name of the function they live within, > its start and ending addresses). As long as currently pmcstat doesn't > return those informations, a new option has been added to the tool > (-m) which can extract (from a raw pmcstat output) all pc sampled, > name of the functions and symbol bundaries they live within. > > Also please note that pmcannotate suffers of 2 limitations. > Firstly, relying on objdump to dump the C source, with heavy > optimization levels and lots of inlines the code gets difficult to > read. Secondly, in particular on x86 but I guess it is not the only > one case, the sample is always attributed to the instruction directly > following the one that was interrupted. So in a C source view some > samples may be attributed to the line below the one you're interested > in. It's also important to keep in mind that if a line is a jump > target or the start of a function the sample really belongs elsewhere. > > The patch can be found here: > http://www.freebsd.org/~attilio/pmcannotate.diff/ > > where pmcannotate/ dir contains the code and needs to go under > /usr/src/usr.sbin/ and the patch has diffs against pmcstat and > Makefile. > > This work has been developed on the behalf of Nokia with important > feedbacks and directions from Jeff Roberson. > > Testing and feedbacks (before it hits the tree) are welcome. > Hi, First of all, this is excellent work. As soon as this and some other changes in PMC hit 7.x I'll be rolling this out to all the developers I work with. I've tested this on amd64 on HEAD, and with the changes we have talked about privately (%jx vs. %x) it works quite well. Secondly, I would like to request a feature. I would like to be able to get output in a more easily parsable format so I can write some Emacs code to highlight C code with the output. I'd like something along the lines of: path:function:line:percentage Keep up the good work! Later, George From owner-freebsd-arch@FreeBSD.ORG Sat Nov 29 04:28:45 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 74DD61065670 for ; Sat, 29 Nov 2008 04:28:45 +0000 (UTC) (envelope-from peter@wemm.org) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.226]) by mx1.freebsd.org (Postfix) with ESMTP id 4F17F8FC18 for ; Sat, 29 Nov 2008 04:28:45 +0000 (UTC) (envelope-from peter@wemm.org) Received: by rv-out-0506.google.com with SMTP id b25so1585057rvf.43 for ; Fri, 28 Nov 2008 20:28:45 -0800 (PST) Received: by 10.143.43.7 with SMTP id v7mr3581800wfj.192.1227931670691; Fri, 28 Nov 2008 20:07:50 -0800 (PST) Received: by 10.142.255.21 with HTTP; Fri, 28 Nov 2008 20:07:50 -0800 (PST) Message-ID: Date: Fri, 28 Nov 2008 20:07:50 -0800 From: "Peter Wemm" To: "Marcel Moolenaar" In-Reply-To: <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> Cc: FreeBSD Arch Subject: Re: RFC: making gpart default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Nov 2008 04:28:45 -0000 On Thu, Sep 25, 2008 at 9:59 AM, Marcel Moolenaar wrote: > All, > > I'd like to switch all architectures to gpart for the reasons given > below. All current partitioning schemes are supported by gpart and > work on all platforms. On top of that, ia64 and powerpc are using > gpart exclusively already. [..] > In short: gpart is the first step towards a unified set of > tools and interfaces and provides the basis for extending > file system related tools by allowing us to attach real > meaning to partition types. With the commit and undo feature, > gpart is ready for use by next generation installers that > allow us to use any partitioning scheme on any platforms. > > Thoughts? oh my god. I just tried to use gpart. This needs some SERIOUS help. First, the 'gpart create' man page doesn't say what "scheme" is. After guessing, I tried: overcee# gpart create -s gpt /dev/twed1 gpart: 22 scheme 'gpt' What does that mean? It turns out that I didn't have GEOM_PART_GPT compiled in. After continuing the guessing game: overcee# gpart create -s gpt /dev/twed1 gpart: 22 provider '/dev/twed1' That was useful. Out other tools generally allow /dev prefixes to be optional. overcee# gpart create -s gpt twed1 twed1 created Now what? Boot code.. there's no example of this either. I tried: overcee# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 twed1 gpart: /dev/twed1p1: Invalid argument I suppose that beats "22". This works though: overcee# gpart bootcode -b /boot/pmbr twed1 This doesn't: overcee# gpart bootcode -p /boot/gptboot -i 1 twed1 gpart: /dev/twed1p1: Invalid argument I haven't figured this out yet. I'm guessing this is because /boot/gptboot isn't a multiple of 512 bytes. The error message is obviously giving no help here. Let's try padding it: overcee# dd if=/boot/gptboot of=/tmp/gptboot conv=sync 14+1 records in 15+0 records out 7680 bytes transferred in 0.000098 secs (78375316 bytes/sec) overcee# gpart bootcode -p /tmp/gptboot -i 1 twed1 overcee# Yep, that worked. Now for a partition... overcee# gpart add -b 512 -s 512m -t freebsd-ufs twed1 gpart: 22 size '512m' Huh? "22"? overcee# gpart add -b 512 -s 1048576 -t freebsd-ufs twed1 twed1p2 added But at least I think I'm getting some progress: overcee# gpart show twed1 => 34 976771053 twed1 GPT (500.1GB) 34 478 1 freebsd-boot (244.7KB) 512 1048576 2 freebsd-ufs (536.9MB) 1049088 975721999 - free - (499.6GB) So I continue.. I figure gpart would pick the first free space: overcee# gpart add -s 4058062 -t freebsd-ufs twed1 gpart: Option 'b' not specified. Apparently not... overcee# gpart add -b 1049088 -s 4058062 -t freebsd-ufs twed1 twed1p3 added Now one has to do a gpart show ; add ; show ; add loop to get the start address. This is really, really raw and unfriendly stuff. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Sat Nov 29 04:41:58 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCC4B106564A; Sat, 29 Nov 2008 04:41:58 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 951538FC08; Sat, 29 Nov 2008 04:41:58 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.2/8.14.1) with ESMTP id mAT4dMVP035096; Fri, 28 Nov 2008 21:39:23 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Fri, 28 Nov 2008 21:39:25 -0700 (MST) Message-Id: <20081128.213925.-1597343424.imp@bsdimp.com> To: des@des.no From: "M. Warner Losh" In-Reply-To: <86od054iaz.fsf@ds4.des.no> References: <86myfq9uha.fsf@ds4.des.no> <86od054iaz.fsf@ds4.des.no> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=utf-8 Content-Transfer-Encoding: base64 Cc: raj@semihalf.com, nwhitehorn@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Enumerable I2C busses X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Nov 2008 04:41:59 -0000 SW4gbWVzc2FnZTogPDg2b2QwNTRpYXouZnNmQGRzNC5kZXMubm8+DQogICAgICAgICAgICBEYWct RXJsaW5nX1Ntw7hyZ3JhdiA8ZGVzQGRlcy5ubz4gd3JpdGVzOg0KOiBSYWZhxYIgSmF3b3Jvd3Nr aSA8cmFqQHNlbWloYWxmLmNvbT4gd3JpdGVzOg0KOiA+IFdlbGwsIGhhcmQtY29kZWQgYWRkcmVz c2VzIGFuZCBjb25mbGljdGluZyBhc3NpZ25tZW50cyBiZXR3ZWVuIHZlbmRvcnMNCjogPiBkbyBu b3QgdGVjaG5pY2FsbHkgcHJldmVudCBmcm9tIHNjYW5uaW5nIHRoZSBidXM7IGFjdHVhbGx5LCBv dXINCjogPiBjdXJyZW50IGlpY2J1cyBjb2RlIGNhbiBkbyBidXMgc2NhbmluZyB3aGVuIGNvbXBp bGVkIHdpdGggYSBkaWFnDQo6ID4gZGVmaW5lLg0KOiANCjogSSBoYXZlbid0IGxvb2tlZCBhdCBo b3cgdGhhdCBpcyBpbXBsZW1lbnRlZCwgYnV0IC0gSTJDIHZlcnNpb24gMw0KOiBkZXNjcmliZXMg ZGV2aWNlIGVudW1lcmF0aW9uIGFuZCBpZGVudGlmaWNhdGlvbiwgYnV0IEFGQUlLIHZlcnkgZmV3 DQo6IGRldmljZXMgYWN0dWFsbHkgc3VwcG9ydCBpdC4NCjogDQo6IFRoaXMgaXMgcmVhbGx5IHN0 dXBpZCwgQlRXLiAgUGhpbGlwcyBjb3VsZCBlYXNpbHkgaGF2ZSByZXF1aXJlZCB0aGUNCjogZmly c3QgZmV3IGJ5dGVzIG9mIHRoZSBkZXZpY2UgYWRkcmVzcyBzcGFjZSB0byBjb250YWluIGEgdmVu ZG9yIC8gZGV2aWNlDQo6IElELCBhbG9uZyB0aGUgbGluZXMgb2YgUENJIGFuZCBVU0IsIGZyb20g dGhlIHN0YXJ0LiAgSSB3b25kZXI6IGRpZCBpdA0KOiBub3Qgb2NjdXIgdG8gdGhlbSwgb3IgZGlk IHRoZXkgaW50ZW50aW9uYWxseSBsZWF2ZSBpdCBvdXQgdG8gc2F2ZSBhIGZldw0KOiBtaWNyb2Nl bnRzIHBlciBjaGlwPw0KDQpFRVBST01zIHR5cGljYWxseSBkb24ndCBoYXZlIHJlZ2lzdGVycyBh dCBhbGwuICBUaGV5IGFyZSBqdXN0IG1lbW9yeS4NCkZvcmNpbmcgdGhlbSB0byBoYXZlIHNldCBj b250ZW50IHdvdWxkIGJyZWFrIGFsbCBraW5kcyBvZg0KYXBwbGljYXRpb25zLg0KDQpXYXJuZXIN Cg0K From owner-freebsd-arch@FreeBSD.ORG Sat Nov 29 14:07:49 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48F5D1065672 for ; Sat, 29 Nov 2008 14:07:49 +0000 (UTC) (envelope-from rpaulo@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.175]) by mx1.freebsd.org (Postfix) with ESMTP id D00EA8FC14 for ; Sat, 29 Nov 2008 14:07:48 +0000 (UTC) (envelope-from rpaulo@gmail.com) Received: by ug-out-1314.google.com with SMTP id 30so2128260ugs.39 for ; Sat, 29 Nov 2008 06:07:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer:sender; bh=c0XVmBKLHYpINxumuXxIlKWYXs//7u/79KpKt2oGJpM=; b=fjp3FAW4ESmpQQd7XpQ1nniPt9Lplvjskmw+05TOCO07At5pN7W3rZ5y/+HQQA28Ah dLtt3ASToShq75ccFZK8LS1hPt88+9XbyZ1qmasdbM61yXVfbZ1nI46lT4IvkgVxryZr r0b3CSNkhXfqs2wZOrXQxWVmnMrCjXH3GpgsY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer:sender; b=N0ybpeRSG2EWYYqRP5dbbsx3aPZ9+6lumW4FuHoU3EyiZ8DuTAe944H4ZB/46KCjDt L8rjTrSLdA6N4DrYGga64ZVcPGC/O48zI6XVkZnQOHTJqgqbdL647neLqoVOqKb4cpq8 S/VqXRsRkW6tA9YcsU8mWMDL1krtBKRBSaOpI= Received: by 10.66.222.6 with SMTP id u6mr1721712ugg.19.1227965944448; Sat, 29 Nov 2008 05:39:04 -0800 (PST) Received: from epsilon.lan (bl6-145-143.dsl.telepac.pt [82.155.145.143]) by mx.google.com with ESMTPS id j34sm1883783ugc.53.2008.11.29.05.39.02 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 29 Nov 2008 05:39:03 -0800 (PST) Message-Id: <28C48AAC-CF27-4434-819C-A2688D6485CC@fnop.net> From: Rui Paulo To: Peter Wemm In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Date: Sat, 29 Nov 2008 13:38:45 +0000 References: <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> X-Mailer: Apple Mail (2.929.2) Sender: Rui Paulo Cc: FreeBSD Arch , Marcel Moolenaar Subject: Re: RFC: making gpart default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Nov 2008 14:07:49 -0000 On 29 Nov 2008, at 04:07, Peter Wemm wrote: > On Thu, Sep 25, 2008 at 9:59 AM, Marcel Moolenaar > wrote: >> All, >> >> I'd like to switch all architectures to gpart for the reasons given >> below. All current partitioning schemes are supported by gpart and >> work on all platforms. On top of that, ia64 and powerpc are using >> gpart exclusively already. > [..] >> In short: gpart is the first step towards a unified set of >> tools and interfaces and provides the basis for extending >> file system related tools by allowing us to attach real >> meaning to partition types. With the commit and undo feature, >> gpart is ready for use by next generation installers that >> allow us to use any partitioning scheme on any platforms. >> >> Thoughts? > > oh my god. I just tried to use gpart. This needs some SERIOUS help. I agree. > > > First, the 'gpart create' man page doesn't say what "scheme" is. Right. Also, the gpart man page should have much more examples. > After guessing, I tried: > > overcee# gpart create -s gpt /dev/twed1 > gpart: 22 scheme 'gpt' > > What does that mean? It turns out that I didn't have GEOM_PART_GPT > compiled in. Yes, these should probably go into sys/conf instead of GENERIC. > After continuing the guessing game: > > overcee# gpart create -s gpt /dev/twed1 > gpart: 22 provider '/dev/twed1' > > That was useful. Out other tools generally allow /dev prefixes to > be optional. In this case, I think '/dev/' should be stripped and then sent to GEOM. Or maybe I don't know what I'm talking about. > overcee# gpart create -s gpt twed1 > twed1 created > > Now what? Boot code.. there's no example of this either. I tried: > > overcee# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 twed1 > gpart: /dev/twed1p1: Invalid argument > > I suppose that beats "22". > > This works though: > overcee# gpart bootcode -b /boot/pmbr twed1 > > This doesn't: > overcee# gpart bootcode -p /boot/gptboot -i 1 twed1 > gpart: /dev/twed1p1: Invalid argument > I haven't figured this out yet. I'm guessing this is because > /boot/gptboot isn't a multiple of 512 bytes. The error message is > obviously giving no help here. > > Let's try padding it: > overcee# dd if=/boot/gptboot of=/tmp/gptboot conv=sync > 14+1 records in > 15+0 records out > 7680 bytes transferred in 0.000098 secs (78375316 bytes/sec) > overcee# gpart bootcode -p /tmp/gptboot -i 1 twed1 > overcee# > > Yep, that worked. Now for a partition... > > overcee# gpart add -b 512 -s 512m -t freebsd-ufs twed1 > gpart: 22 size '512m' > Huh? "22"? > > overcee# gpart add -b 512 -s 1048576 -t freebsd-ufs twed1 > twed1p2 added Size prefixes should be supported, yes. > But at least I think I'm getting some progress: > overcee# gpart show twed1 > => 34 976771053 twed1 GPT (500.1GB) > 34 478 1 freebsd-boot (244.7KB) > 512 1048576 2 freebsd-ufs (536.9MB) > 1049088 975721999 - free - (499.6GB) > > So I continue.. I figure gpart would pick the first free space: > overcee# gpart add -s 4058062 -t freebsd-ufs twed1 > gpart: Option 'b' not specified. gpt did this with no problems... :-/ > > > Apparently not... > overcee# gpart add -b 1049088 -s 4058062 -t freebsd-ufs twed1 > twed1p3 added > > Now one has to do a gpart show ; add ; show ; add loop to get the > start address. > > > This is really, really raw and unfriendly stuff. Yes. This looks like fdisk, something we really want to avoid. I'll see what I can do to help. Regards, -- Rui Paulo From owner-freebsd-arch@FreeBSD.ORG Sat Nov 29 14:29:52 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85ED61065675 for ; Sat, 29 Nov 2008 14:29:52 +0000 (UTC) (envelope-from joao.barros@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id 18E708FC13 for ; Sat, 29 Nov 2008 14:29:51 +0000 (UTC) (envelope-from joao.barros@gmail.com) Received: by fg-out-1718.google.com with SMTP id l26so1195111fgb.35 for ; Sat, 29 Nov 2008 06:29:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=6NDEV/cVpxASQyTU0UTaHOF6YOUPy5SvYKQDv4Cpb6c=; b=eyiY+SF7q8w0CbmqSc7Uj6HGDdGDwq7yIsa0zAa+fQH1S9sr0vXppiOGPdkSVHDWDb 6O323BU+5l8A1CoLicsCswwuM5tW6Suc5IGbsFXhHvO/H+qOQaV7+jsUIu62bfUcmyBe ph1UDq7LKnHbHjDXIG5/GLryH6E3UyQPKTBPM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=gTNujrgGiUUxV8DaDsRitKiSZopeKmcY4HBb180AvippG76wlq52Zl6QkXqA4vhu2o evIYJ6gCAf1lWYfM4rOs2cIMXujOtD4Y5uUs1azssJQ0oMFaSVV2tegmm8oF9z5VuiTD cY06Xu7bdbPHbucNGfRNlRxghdmr4NdtTZyCc= Received: by 10.181.149.19 with SMTP id b19mr3161704bko.82.1227967745976; Sat, 29 Nov 2008 06:09:05 -0800 (PST) Received: by 10.180.236.13 with HTTP; Sat, 29 Nov 2008 06:09:05 -0800 (PST) Message-ID: <70e8236f0811290609h2539ede7jc01778edac9c1d5@mail.gmail.com> Date: Sat, 29 Nov 2008 14:09:05 +0000 From: "Joao Barros" To: "Peter Wemm" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> Cc: FreeBSD Arch , Marcel Moolenaar Subject: Re: RFC: making gpart default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Nov 2008 14:29:52 -0000 On Sat, Nov 29, 2008 at 4:07 AM, Peter Wemm wrote: > On Thu, Sep 25, 2008 at 9:59 AM, Marcel Moolenaar wrote: >> All, >> >> I'd like to switch all architectures to gpart for the reasons given >> below. All current partitioning schemes are supported by gpart and >> work on all platforms. On top of that, ia64 and powerpc are using >> gpart exclusively already. > [..] >> In short: gpart is the first step towards a unified set of >> tools and interfaces and provides the basis for extending >> file system related tools by allowing us to attach real >> meaning to partition types. With the commit and undo feature, >> gpart is ready for use by next generation installers that >> allow us to use any partitioning scheme on any platforms. >> >> Thoughts? > > oh my god. I just tried to use gpart. This needs some SERIOUS help. > > First, the 'gpart create' man page doesn't say what "scheme" is. True. > After guessing, I tried: > > overcee# gpart create -s gpt /dev/twed1 > gpart: 22 scheme 'gpt' > > What does that mean? It turns out that I didn't have GEOM_PART_GPT compiled in. A recent CURRENT has it by default. > > After continuing the guessing game: > > overcee# gpart create -s gpt /dev/twed1 > gpart: 22 provider '/dev/twed1' > > That was useful. Out other tools generally allow /dev prefixes to be optional. > > overcee# gpart create -s gpt twed1 > twed1 created > > Now what? Boot code.. there's no example of this either. I tried: > > overcee# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 twed1 > gpart: /dev/twed1p1: Invalid argument > > I suppose that beats "22". > > This works though: > overcee# gpart bootcode -b /boot/pmbr twed1 > > This doesn't: > overcee# gpart bootcode -p /boot/gptboot -i 1 twed1 > gpart: /dev/twed1p1: Invalid argument > I haven't figured this out yet. I'm guessing this is because > /boot/gptboot isn't a multiple of 512 bytes. The error message is > obviously giving no help here. > > Let's try padding it: > overcee# dd if=/boot/gptboot of=/tmp/gptboot conv=sync > 14+1 records in > 15+0 records out > 7680 bytes transferred in 0.000098 secs (78375316 bytes/sec) > overcee# gpart bootcode -p /tmp/gptboot -i 1 twed1 > overcee# > > Yep, that worked. Now for a partition... Fixed on Nov 18: http://svn.freebsd.org/viewvc/base?view=revision&revision=185038 > > overcee# gpart add -b 512 -s 512m -t freebsd-ufs twed1 > gpart: 22 size '512m' > Huh? "22"? This would be nice... > > overcee# gpart add -b 512 -s 1048576 -t freebsd-ufs twed1 > twed1p2 added > > But at least I think I'm getting some progress: > overcee# gpart show twed1 > => 34 976771053 twed1 GPT (500.1GB) > 34 478 1 freebsd-boot (244.7KB) > 512 1048576 2 freebsd-ufs (536.9MB) > 1049088 975721999 - free - (499.6GB) > > So I continue.. I figure gpart would pick the first free space: > overcee# gpart add -s 4058062 -t freebsd-ufs twed1 > gpart: Option 'b' not specified. This would be nice too... > > Apparently not... > overcee# gpart add -b 1049088 -s 4058062 -t freebsd-ufs twed1 > twed1p3 added > > Now one has to do a gpart show ; add ; show ; add loop to get the start address. > Yes, it could be smarter and more helpfull. > > This is really, really raw and unfriendly stuff. -- Joao Barros From owner-freebsd-arch@FreeBSD.ORG Sat Nov 29 21:10:00 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 429FB1065672 for ; Sat, 29 Nov 2008 21:10:00 +0000 (UTC) (envelope-from peter@wemm.org) Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.168]) by mx1.freebsd.org (Postfix) with ESMTP id 2743B8FC14 for ; Sat, 29 Nov 2008 21:10:00 +0000 (UTC) (envelope-from peter@wemm.org) Received: by wf-out-1314.google.com with SMTP id 24so2002170wfg.7 for ; Sat, 29 Nov 2008 13:09:59 -0800 (PST) Received: by 10.142.125.9 with SMTP id x9mr3846514wfc.236.1227992999754; Sat, 29 Nov 2008 13:09:59 -0800 (PST) Received: by 10.142.255.21 with HTTP; Sat, 29 Nov 2008 13:09:59 -0800 (PST) Message-ID: Date: Sat, 29 Nov 2008 13:09:59 -0800 From: "Peter Wemm" To: "Joao Barros" In-Reply-To: <70e8236f0811290609h2539ede7jc01778edac9c1d5@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> <70e8236f0811290609h2539ede7jc01778edac9c1d5@mail.gmail.com> Cc: FreeBSD Arch , Marcel Moolenaar Subject: Re: RFC: making gpart default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Nov 2008 21:10:00 -0000 On Sat, Nov 29, 2008 at 6:09 AM, Joao Barros wrote: > On Sat, Nov 29, 2008 at 4:07 AM, Peter Wemm wrote: >> On Thu, Sep 25, 2008 at 9:59 AM, Marcel Moolenaar wrote: >>> All, >>> >>> I'd like to switch all architectures to gpart for the reasons given >>> below. All current partitioning schemes are supported by gpart and >>> work on all platforms. On top of that, ia64 and powerpc are using >>> gpart exclusively already. >> [..] >>> In short: gpart is the first step towards a unified set of >>> tools and interfaces and provides the basis for extending >>> file system related tools by allowing us to attach real >>> meaning to partition types. With the commit and undo feature, >>> gpart is ready for use by next generation installers that >>> allow us to use any partitioning scheme on any platforms. >>> >>> Thoughts? >> >> oh my god. I just tried to use gpart. This needs some SERIOUS help. >> >> First, the 'gpart create' man page doesn't say what "scheme" is. > > True. My gripe was that it just said "this is how you specify the scheme". What is a scheme? Is this where you type "guid" or "gpt" or "mbr"? Is it lowercase or uppercase? Is it a filename to the backing .so name? Is it the numerical index in some array inside the g_part kernel module? >> After guessing, I tried: >> >> overcee# gpart create -s gpt /dev/twed1 >> gpart: 22 scheme 'gpt' >> >> What does that mean? It turns out that I didn't have GEOM_PART_GPT compiled in. > > A recent CURRENT has it by default. Only in GENERIC. That didn't help my machine which has been running and upgraded all the way from freebsd-3.2. I used to have it turned on at one point. The point was that gpart was gave me no clues about diagnosing the problem. However, conf/DEFAULTS still has GEOM_BSD and GEOM_MBR in it. Recall that this thread is about having g_part take over mbr and bsd, which means switching tools. Having tools that give an error message of "22" isn't going to cut it. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Sat Nov 29 21:56:07 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D0CA106564A for ; Sat, 29 Nov 2008 21:56:07 +0000 (UTC) (envelope-from peter@wemm.org) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.237]) by mx1.freebsd.org (Postfix) with ESMTP id E91758FC19 for ; Sat, 29 Nov 2008 21:56:06 +0000 (UTC) (envelope-from peter@wemm.org) Received: by rv-out-0506.google.com with SMTP id b25so1806280rvf.43 for ; Sat, 29 Nov 2008 13:56:06 -0800 (PST) Received: by 10.142.156.2 with SMTP id d2mr3873179wfe.102.1227995766423; Sat, 29 Nov 2008 13:56:06 -0800 (PST) Received: by 10.142.255.21 with HTTP; Sat, 29 Nov 2008 13:56:06 -0800 (PST) Message-ID: Date: Sat, 29 Nov 2008 13:56:06 -0800 From: "Peter Wemm" To: "Marcel Moolenaar" In-Reply-To: <0F1745AA-611F-40B2-85F3-32FD78BC4B58@mac.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> <0F1745AA-611F-40B2-85F3-32FD78BC4B58@mac.com> Cc: FreeBSD Arch Subject: Re: RFC: making gpart default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Nov 2008 21:56:07 -0000 On Sat, Nov 29, 2008 at 1:32 PM, Marcel Moolenaar wrote: > On Nov 28, 2008, at 8:07 PM, Peter Wemm wrote: > >> First, the 'gpart create' man page doesn't say what "scheme" is. > > Yes, the manpage needs some work. > >> After guessing, I tried: >> >> overcee# gpart create -s gpt /dev/twed1 >> gpart: 22 scheme 'gpt' >> >> What does that mean? It turns out that I didn't have GEOM_PART_GPT >> compiled in. > > Oops, forgot about parsing the error string... > I just fixed it (rev 185454). > > The background: geom(8) simply prints the error that > the kernel creates. This then requires that the kernel > creates a user-friendly error message. This is not a > good idea, because it side-steps things like i18n > completely. > > So, for gpart I chose a different approach. The kernel > uses a certain form for the error messages, which is: > [ 'value'] > The intend is to have user-space interpret what is > meant and print something the user understands. This > I forgot to do. > > For now, the translation is straight-forward, but it > should not be too hard to improve upon it further. > > In general, geom(8) needs to do a bit more pre- and post- > processing. It mostly just converts the command line into > a gctl request and as such forces the user to do all the > work. I'll work on that... For the record, I like the idea of having a consistent control interface. I switched my machine to use GEOM_PART_MBR/BSD as well. A couple of thoughts.. * Can gpart enumerate the list of schemes that the kernel supports? If so, it could avoid the problem of having to interpret the kernel's reaction to avoidable errors. * The same goes for the list of 'geoms'. 'geom disk list' (among other things) can find the providers list. gpart could avoid passing bogus provider names into the kernel in the first place. * A couple of DWIM concessions would go along way. Humanized number suffixes, the ability to search for start addresses automatically, find next free partition index etc. I'd like this sort of thing to work: # gpart add -s 4g -t freebsd-ufs twed1 created partition 3 from 1049088 to 8388608 on twed1. # gpart add -t freebsd-ufs twed1 created partition 7 from 30409216 to 946361871 on twed1 * There should be some guidance or hints on laying out disks. For example, a gpart create -s gpt on a raid volume ends up with a start sector of 34 for the free space. There should be a documentation hint to round up start sectors to a power of 2 and/or block size on a raid. eg: if you have a raid with 64K stripes, then move the start sector from 34 to 128. Otherwise we end up with file systems issuing transactions that can split across multiple raid stripes. FWIW, I conveniently filled this hole with boot code. The last issue isn't specific to gpart. There was one device at work where the fdisk free space starts at sector 63. (31.5K). When creating 16K ufs blocks, the particular raid controller generated *two* operations for every single file system read/write we did. UFS helpfully did it all in what it thought were 16K transactions, (or clustered 64K transactions) but were actually unaligned thanks to the default mbr layout. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Sat Nov 29 22:32:32 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 553CC106564A for ; Sat, 29 Nov 2008 22:32:32 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout022.mac.com (asmtpout022.mac.com [17.148.16.97]) by mx1.freebsd.org (Postfix) with ESMTP id 48DC88FC13 for ; Sat, 29 Nov 2008 22:32:32 +0000 (UTC) (envelope-from xcllnt@mac.com) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Received: from [192.168.1.96] (209-128-86-226.BAYAREA.NET [209.128.86.226]) by asmtp022.mac.com (Sun Java(tm) System Messaging Server 6.3-7.03 (built Aug 7 2008; 32bit)) with ESMTPSA id <0KB400J3U6I72B40@asmtp022.mac.com> for arch@freebsd.org; Sat, 29 Nov 2008 13:32:32 -0800 (PST) Message-id: <0F1745AA-611F-40B2-85F3-32FD78BC4B58@mac.com> From: Marcel Moolenaar To: Peter Wemm In-reply-to: Date: Sat, 29 Nov 2008 13:32:31 -0800 References: <57809A37-B81C-4F50-A418-CD9303F06B72@mac.com> X-Mailer: Apple Mail (2.929.2) Cc: FreeBSD Arch Subject: Re: RFC: making gpart default X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Nov 2008 22:32:32 -0000 On Nov 28, 2008, at 8:07 PM, Peter Wemm wrote: > First, the 'gpart create' man page doesn't say what "scheme" is. Yes, the manpage needs some work. > After guessing, I tried: > > overcee# gpart create -s gpt /dev/twed1 > gpart: 22 scheme 'gpt' > > What does that mean? It turns out that I didn't have GEOM_PART_GPT > compiled in. Oops, forgot about parsing the error string... I just fixed it (rev 185454). The background: geom(8) simply prints the error that the kernel creates. This then requires that the kernel creates a user-friendly error message. This is not a good idea, because it side-steps things like i18n completely. So, for gpart I chose a different approach. The kernel uses a certain form for the error messages, which is: [ 'value'] The intend is to have user-space interpret what is meant and print something the user understands. This I forgot to do. For now, the translation is straight-forward, but it should not be too hard to improve upon it further. In general, geom(8) needs to do a bit more pre- and post- processing. It mostly just converts the command line into a gctl request and as such forces the user to do all the work. I'll work on that... -- Marcel Moolenaar xcllnt@mac.com