From owner-freebsd-arch@FreeBSD.ORG Mon Feb 8 11:06:51 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2D6210656AA for ; Mon, 8 Feb 2010 11:06:51 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 78CC28FC1B for ; Mon, 8 Feb 2010 11:06:51 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o18B6pUK087310 for ; Mon, 8 Feb 2010 11:06:51 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o18B6oah087308 for freebsd-arch@FreeBSD.org; Mon, 8 Feb 2010 11:06:50 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 8 Feb 2010 11:06:50 GMT Message-Id: <201002081106.o18B6oah087308@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Feb 2010 11:06:51 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Fri Feb 12 04:30:53 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE4CE106566C; Fri, 12 Feb 2010 04:30:52 +0000 (UTC) (envelope-from rajatjain@juniper.net) Received: from exprod7og121.obsmtp.com (exprod7og121.obsmtp.com [64.18.2.20]) by mx1.freebsd.org (Postfix) with ESMTP id 1DCB18FC08; Fri, 12 Feb 2010 04:30:51 +0000 (UTC) Received: from source ([66.129.224.36]) (using TLSv1) by exprod7ob121.postini.com ([64.18.6.12]) with SMTP ID DSNKS3TZe8WzYsKOhhr+zRsc0sdv7dwTQCFw@postini.com; Thu, 11 Feb 2010 20:30:52 PST Received: from emailbng1.jnpr.net (10.209.194.15) by P-EMHUB03-HQ.jnpr.net (172.24.192.37) with Microsoft SMTP Server id 8.1.393.1; Thu, 11 Feb 2010 20:29:22 -0800 Received: from emailbng3.jnpr.net ([10.209.194.27]) by emailbng1.jnpr.net with Microsoft SMTPSVC(6.0.3790.3959); Fri, 12 Feb 2010 09:59:19 +0530 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Fri, 12 Feb 2010 09:59:19 +0530 Message-ID: <8506939B503B404A84BBB12293FC45F606A74F94@emailbng3.jnpr.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Understanding PCI enumeration for hot-plug Thread-Index: Acqrm/FL8jWF2VBsS6yi1Nozfl/z7Q== From: Rajat Jain To: , X-OriginalArrivalTime: 12 Feb 2010 04:29:19.0868 (UTC) FILETIME=[F1CFABC0:01CAAB9B] Cc: freebsd-ia32@freebsd.org, bms@freebsd.org, freebsd-ppc@freebsd.org Subject: Understanding PCI enumeration for hot-plug X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Feb 2010 04:30:53 -0000 Hello, I need to support build hot-plug in a FreeBSD based system where I know in advance the devices (and the PCI hierarchy on them) that can be hot-plugged into the system.=20 I'm trying to understand the PCI enumeration / re-enumeration process in the context of hot-plug / unplug. I'm assuming the case where the firmware is really dumb and the kernel needs to manage / allocate all PCI resources.=20 I'm going through the PCI specs (and PCI-to-PCI bridge specs) and here is what I think needs to be done when ever PCI / PCIe devices are added / removed from the system. I would appreciate if some one could please confirm my understanding and point out if I am missing something: PCI-REOURCE / BUS-NUMBER MGMT =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D [IFF NECESSARY] PCI configuration space of all the bridges needs to be re-written, right from the immediate parent of the device being removed until the host bridge in order to ensure that: a) Each parent bridge has secondary and subordinate bus range set so as to include all the bus numbers below it. This is required to forward configuration transactions. b) Each parent bridge has IO base and IO limit set so as to include all the IO address space below it. c) Each parent bridge has Memory base and Memory limit set so as to include all the Memory address space below it. d) Each parent bridge has Pre-fetch Memory base and Memory limit set so as to include all the Pre-fetch Memory address space below it. Note-1: The reason a/b/c/d above are marked "IFF NECESSARY" is that we can avoid all the above work if we can pre-allocate the above resources for future devices, and set these parent bridges ranges accordingly.=20 IN other words, consider a system where we know in advance, the PCI device tree on the devices that can be hot-plugged into the system. Here we can set aside PCI bus numbers and IO / Mem / Prefetchable memory ranges for them in advance. And thus configure the parent bridges to already include those ranges. Thus later when the devices are added, none of the parent bridges will need to be re-programmed. Is my understanding correct? DEVICE DETECTION & INITIALIZATION =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D e) Upon detection of the device (By attempting to read its config space), the most important item is to program its BARs to appropriate address spaces as requested by the device. The BARs need to be programmed such that they are included in the appropriate Base / limit registers of all the bridges upstream. Correct? Again, if we've used the strategy specified in Note-1, we can simply use the range we've already aside for this device. MY QUESTIONS =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1) Is my above understanding correct? 2) Does anything else also needs to be done in order to make it work? 3) As I said I'm trying to make it fast and optimize for the case where I know the devices [thus the PCI tree] that can be plugged in. Will my strategy specified in Note-1 work? So all I need to configure is my newly detected devices / bridges and not the existing ones... Thanks in Advance, Rajat Jain From owner-freebsd-arch@FreeBSD.ORG Sat Feb 13 17:22:38 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 479001065676; Sat, 13 Feb 2010 17:22:38 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-iw0-f180.google.com (mail-iw0-f180.google.com [209.85.223.180]) by mx1.freebsd.org (Postfix) with ESMTP id 026B08FC0A; Sat, 13 Feb 2010 17:22:37 +0000 (UTC) Received: by iwn10 with SMTP id 10so512329iwn.13 for ; Sat, 13 Feb 2010 09:22:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=9YzCuYyL4sOOrXoPVYjvoZUrKeRsctOGSEU62cJbOEw=; b=NzTJHdkgm0iJs2AIQfDRCOEx8iw0dFGpoQYu7lCmhw7BbirQfSY+G8GAe3XR7FOXyB CabEHQ9JyD3Ug6dLi4TXdnnc7zKb1zxaOvUA05CEJpAu21ByBv7Ug7rmXLcDSFvmUYqq pyVk0hDmZEKrMKvr0CZYfPwSEldF85LbUDrK4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=cfzfP8syYx23Mo7h/KYniLIf+UP4nesQLwmT+OPZetbgkVPD+NqDfjuKOGmMlzORzV KOYcy8U1qr8aQHwrVa0Emt7IZqmqxYbc3L1rjg9GlIlrlbyXKlmO880YOVjogEsIDEsa CBozuJysngYGJl/RJitGZ1djMCOynAaWCCdMQ= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.231.154.213 with SMTP id p21mr3308702ibw.42.1266081756365; Sat, 13 Feb 2010 09:22:36 -0800 (PST) In-Reply-To: <4e6cba831001280238x6a86e9f8vf5b7858b4bb82178@mail.gmail.com> References: <4e6cba831001280238x6a86e9f8vf5b7858b4bb82178@mail.gmail.com> Date: Sat, 13 Feb 2010 18:22:36 +0100 X-Google-Sender-Auth: f81a0a35428aa50f Message-ID: <3bbf2fe11002130922n2ca90130x4da8d47185d04978@mail.gmail.com> From: Attilio Rao To: Giovanni Trematerra Content-Type: text/plain; charset=UTF-8 Cc: freebsd-arch@freebsd.org Subject: Re: kthread interface X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Feb 2010 17:22:38 -0000 2010/1/28 Giovanni Trematerra : > Hi all, > There is a race in kthread_exit when all the threads of a kernel > process exit at same time. I came up with a quick and dirty patch that > resolve the issue at least in my test case. > > http://www.trematerra.net/patches/kthread_exit.diff > > Nonetheless I see space for improvement into kthread interface. > At present, with kproc_kthread_add you could have a kernel process > without a main thread and that seems to me only a way to logical > grouping threads and pretty useless. > I propose to remove kproc_kthread_add and don't let kthread_exit call > kproc_exit on the last exiting thread but demand user to handle > process termination. > If you need kernel threads but no reason to have a kernel process with > a main thread that acts as a coordinator you can attach them to proc0 > by kthread_add passing NULL for (struct proc *) argument. The patch is right but for a different reason than what you expose. What really happens is that kproc_kthread_add() has mostly an evil semantic. In the common case you define a precise callback, but you are not going to know if the callback you are offering will end up being the first thread (thus 'reconduited' to kproc_create()) or just one of the others (thus added via kthread_add()). The problem is that in the former case the callback should close with a kproc_exit() call, while on the latter an ideal thread_exit() should happen. As long as the callback can't know in which case it is, the kthread_exit() (used in this context, but as I can see, not in all of them) needs to take care of this problem. This is only possible, however, if the callbacks also ensure (and so far I don't see any similar safety belt somewhere) that the threads adding is correctly drained when the threads are going to die. This is a race the primitive itself can't handle (aka: the count of threads within the kproc can't grow up when the first thread start dying) because it needs of external help. The normal proc/thread working model is quite different because the process is created with an explicit interface (fork1()) and it is closed by an explicit interface (eventually reused, but exit1() takes explicitly place in the interested paths) and the threads are added with a explicit logic (via thr interfaces). kproc_kthread_add() does complicate all this simple logic, of course. As a first band-aid your patch is ok, but what I really would like is that the whole semantic gets sanitized and gets a better logic (in other words: explicit ordering of the kproc/kthreads creation and destruction). Attilio -- Peace can only be achieved by understanding - A. Einstein