From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 12:21:25 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6CE17106566B for ; Sun, 2 Oct 2011 12:21:25 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 88D098FC1E for ; Sun, 2 Oct 2011 12:21:18 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 1CB324AC2D for ; Sun, 2 Oct 2011 16:21:16 +0400 (MSD) Date: Sun, 2 Oct 2011 16:21:09 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <358651269.20111002162109@serebryakov.spb.ru> To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Subject: Memory allocation in kernel -- what to use in which situation? What is the best for page-sized allocations? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 12:21:25 -0000 Hello, Freebsd-hackers. Here are several memory-allocation mechanisms in the kernel. The two I'm aware of is MALLOC_DEFINE()/malloc()/free() and uma_* (zone(9)). As far as I understand, malloc() is general-purpose, but it has fixed "transaction cost" (in term of memory consumption) for each block allocated, and is not very suitable for allocation of many small blocks, as lots of memory will be wasted for bookkeeping. zone(9) allocator, on other hand, have very low cost of each allocated block, but could allocate only pre-configured fixed-size blocks, and ideal for allocation tons of small objects (and provide API for reusing them, too!). Am I right? But what if I need to allocate a lot (say, 16K-32K) of page-sized blocks? Not in one chunk, for sure, but in lifetime of my kernel module. Which allocator should I use? It seems, the best one will be very low-level only-page-sized allocator. Is here any in kernel? --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 12:36:05 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4EBD1065673 for ; Sun, 2 Oct 2011 12:36:05 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id AF5DF8FC0A for ; Sun, 2 Oct 2011 12:36:04 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 482554AC1C for ; Sun, 2 Oct 2011 16:35:49 +0400 (MSD) Date: Sun, 2 Oct 2011 16:35:42 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <1076522387.20111002163542@serebryakov.spb.ru> To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Subject: Way to get current tick number in kernel? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 12:36:05 -0000 Hello, Freebsd-hackers. What should I use to measure short intervals of time between events in kernel? I don't need any "time" in means of, for example, time(3) API, but some monotonically and uniformly increasing counter with known frequency. As cheap as possible, without complex calculations :) --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 13:06:53 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD06E106564A; Sun, 2 Oct 2011 13:06:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 312B08FC0A; Sun, 2 Oct 2011 13:06:52 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p92D6J2N041710 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 2 Oct 2011 16:06:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id p92D6IdK069500; Sun, 2 Oct 2011 16:06:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id p92D6IxP069499; Sun, 2 Oct 2011 16:06:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 2 Oct 2011 16:06:18 +0300 From: Kostik Belousov To: Lev Serebryakov Message-ID: <20111002130618.GJ1511@deviant.kiev.zoral.com.ua> References: <358651269.20111002162109@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="FaO1PrAXXyeXdeCc" Content-Disposition: inline In-Reply-To: <358651269.20111002162109@serebryakov.spb.ru> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org Subject: Re: Memory allocation in kernel -- what to use in which situation? What is the best for page-sized allocations? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 13:06:53 -0000 --FaO1PrAXXyeXdeCc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Oct 02, 2011 at 04:21:09PM +0400, Lev Serebryakov wrote: > Hello, Freebsd-hackers. >=20 > Here are several memory-allocation mechanisms in the kernel. The two > I'm aware of is MALLOC_DEFINE()/malloc()/free() and uma_* (zone(9)). >=20 > As far as I understand, malloc() is general-purpose, but it has > fixed "transaction cost" (in term of memory consumption) for each > block allocated, and is not very suitable for allocation of many small > blocks, as lots of memory will be wasted for bookkeeping. >=20 > zone(9) allocator, on other hand, have very low cost of each > allocated block, but could allocate only pre-configured fixed-size > blocks, and ideal for allocation tons of small objects (and provide > API for reusing them, too!). >=20 > Am I right? >=20 > But what if I need to allocate a lot (say, 16K-32K) of page-sized > blocks? Not in one chunk, for sure, but in lifetime of my kernel > module. Which allocator should I use? It seems, the best one will be > very low-level only-page-sized allocator. Is here any in kernel? Very short answer is that you could use kmem_alloc() or kmem_malloc(). kmem_alloc() sleeps, while kmem_malloc() may be directed to not sleep. --FaO1PrAXXyeXdeCc Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk6IYckACgkQC3+MBN1Mb4h2mwCgnqjicdk2ZMFW8uYRWmrAc4zZ aY4An0nOQezMQzQvrwZBgfywonhuobWJ =hx5q -----END PGP SIGNATURE----- --FaO1PrAXXyeXdeCc-- From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 13:19:58 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 38A071065673; Sun, 2 Oct 2011 13:19:58 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id D3AB08FC0C; Sun, 2 Oct 2011 13:19:57 +0000 (UTC) Received: by vcbf13 with SMTP id f13so3165971vcb.13 for ; Sun, 02 Oct 2011 06:19:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=Yia7roXR//u6yMTPx8629QUDp1OIn81/i9vT5JMZkH4=; b=hsyfWCE8nk4PHT+qYV2grvS0INqXNE9f2zfSNLg93CSWEQ8OdAjy3Aw5kon5IvYA2w 8gdVA4QZc/5lpGxdzfCFoXFHH886QTC+Y5UH1UrDckaQz6Dvw5cvKKZSl3Q1uHkleqac o3C9AeymQkuy4X44H9CVwjwiGVWiPTa3FL62E= MIME-Version: 1.0 Received: by 10.52.98.199 with SMTP id ek7mr13467275vdb.433.1317560268123; Sun, 02 Oct 2011 05:57:48 -0700 (PDT) Received: by 10.52.179.228 with HTTP; Sun, 2 Oct 2011 05:57:48 -0700 (PDT) In-Reply-To: <358651269.20111002162109@serebryakov.spb.ru> References: <358651269.20111002162109@serebryakov.spb.ru> Date: Sun, 2 Oct 2011 14:57:48 +0200 Message-ID: From: Davide Italiano To: lev@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Memory allocation in kernel -- what to use in which situation? What is the best for page-sized allocations? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 13:19:58 -0000 2011/10/2 Lev Serebryakov : > Hello, Freebsd-hackers. > > =A0Here are several memory-allocation mechanisms in the kernel. The two > I'm aware of is MALLOC_DEFINE()/malloc()/free() and uma_* (zone(9)). > > =A0As far as I understand, malloc() is general-purpose, but it has > fixed "transaction cost" (in term of memory consumption) for each > block allocated, and is not very suitable for allocation of many small > blocks, as lots of memory will be wasted for bookkeeping. > > =A0zone(9) allocator, on other hand, have very low cost of each > allocated block, but could allocate only pre-configured fixed-size > blocks, and ideal for allocation tons of small objects (and provide > API for reusing them, too!). > > =A0Am I right? > > =A0 But what if I need to allocate a lot (say, 16K-32K) of page-sized > blocks? Not in one chunk, for sure, but in lifetime of my kernel > module. Which allocator should I use? It seems, the best one will be > very low-level only-page-sized allocator. Is here any in kernel? > > -- > // Black Lion AKA Lev Serebryakov > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > My 2cents: Everytime you request a certain amount of memory bigger than 4KB using kernel malloc(), it results in a direct call to uma_large_malloc(). Right now, uma_large_malloc() calls kmem_malloc() (i.e. the memory is requested to the VM directly). This kind of approach has two main drawbacks: 1) it heavily fragments the kernel heap 2) when free() is called on these multipage chunks, it in turn calls uma_large_free(), which immediately calls the VM system to unmap and free the chunk of memory. The unmapping requires a system-wide TLB shootdown, i.e. a global action by every processor in the system. I'm currently working supervised by alc@ to an intermediate layer that sits between UMA and the VM, which goal is satisfyinh efficiently requests > 4KB (so, the one you want considering you're asking for 16KB-32KB), but the work is in an early stage. Best, Davide From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 13:45:54 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D16C1065675; Sun, 2 Oct 2011 13:45:54 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id D2C548FC12; Sun, 2 Oct 2011 13:45:53 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 2294A4AC1C; Sun, 2 Oct 2011 17:45:52 +0400 (MSD) Date: Sun, 2 Oct 2011 17:45:45 +0400 From: Lev Serebryakov Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <1393358703.20111002174545@serebryakov.spb.ru> To: Davide Italiano In-Reply-To: References: <358651269.20111002162109@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, lev@freebsd.org Subject: Re: Memory allocation in kernel -- what to use in which situation? What is the best for page-sized allocations? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 13:45:54 -0000 Hello, Davide. You wrote 2 =EE=EA=F2=FF=E1=F0=FF 2011 =E3., 16:57:48: >> =A0 But what if I need to allocate a lot (say, 16K-32K) of page-sized >> blocks? Not in one chunk, for sure, but in lifetime of my kernel >> module. Which allocator should I use? It seems, the best one will be >> very low-level only-page-sized allocator. Is here any in kernel? > My 2cents: > Everytime you request a certain amount of memory bigger than 4KB using > kernel malloc(), it results in a direct call to uma_large_malloc(). > Right now, uma_large_malloc() calls kmem_malloc() (i.e. the memory is > requested to the VM directly). > This kind of approach has two main drawbacks: > 1) it heavily fragments the kernel heap > 2) when free() is called on these multipage chunks, it in turn calls > uma_large_free(), which immediately calls the VM system to unmap and > free the chunk of memory. The unmapping requires a system-wide TLB > shootdown, i.e. a global action by every processor in the system. > I'm currently working supervised by alc@ to an intermediate layer that > sits between UMA and the VM, which goal is satisfyinh efficiently requests >> 4KB (so, the one you want considering you're asking for > 16KB-32KB), but the work is in an early stage. I was not very clear here. I'm saying about page-sized blocks, but many of them. 16K-32K is not a size in bytes, but count of page-sized blocks my code needs :) BTW, I/O is often require big buffers, up to MAXPHYS (128KiB for now), do you mean, that any allocation of such memory has considerable performance penalties, especially on multi-core and multi-CPU systems? --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 14:00:28 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2AA5610656B5; Sun, 2 Oct 2011 14:00:28 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id BF1748FC1D; Sun, 2 Oct 2011 14:00:27 +0000 (UTC) Received: by vws11 with SMTP id 11so3192881vws.13 for ; Sun, 02 Oct 2011 07:00:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=BI9TNsP56M68bxOorp32Bs2FqLcER0wbw7/Lx6NtZ5k=; b=hMQasSiPGMVZxDxibR8KYUeouFKxL84ScbqtVS1jvEo8YDSAle0HPNhN31I3Z7wrbv mpsUosmHCBbF9CbIPTbOG2HtOlFeMFsfPIgiZr+GHBPWW/ezNgEMzyiKV4NEL17gS/6y eaIvRRSnDS2Ss3qRsFc0mtHyWiZkFoUX8xXdM= MIME-Version: 1.0 Received: by 10.52.75.195 with SMTP id e3mr12655231vdw.299.1317564026836; Sun, 02 Oct 2011 07:00:26 -0700 (PDT) Received: by 10.52.179.228 with HTTP; Sun, 2 Oct 2011 07:00:26 -0700 (PDT) In-Reply-To: <1393358703.20111002174545@serebryakov.spb.ru> References: <358651269.20111002162109@serebryakov.spb.ru> <1393358703.20111002174545@serebryakov.spb.ru> Date: Sun, 2 Oct 2011 16:00:26 +0200 Message-ID: From: Davide Italiano To: lev@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Memory allocation in kernel -- what to use in which situation? What is the best for page-sized allocations? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 14:00:28 -0000 2011/10/2 Lev Serebryakov : > Hello, Davide. > You wrote 2 =D0=BE=D0=BA=D1=82=D1=8F=D0=B1=D1=80=D1=8F 2011 =D0=B3., 16:5= 7:48: > >>> =C2=A0 But what if I need to allocate a lot (say, 16K-32K) of page-size= d >>> blocks? Not in one chunk, for sure, but in lifetime of my kernel >>> module. Which allocator should I use? It seems, the best one will be >>> very low-level only-page-sized allocator. Is here any in kernel? > >> My 2cents: >> Everytime you request a certain amount of memory bigger than 4KB using >> kernel malloc(), it results in a direct call to uma_large_malloc(). >> Right now, uma_large_malloc() calls kmem_malloc() (i.e. the memory is >> requested to the VM directly). >> This kind of approach has two main drawbacks: >> 1) it heavily fragments the kernel heap >> 2) when free() is called on these multipage chunks, it in turn calls >> uma_large_free(), which immediately calls the VM system to unmap and >> free the chunk of memory. =C2=A0The unmapping requires a system-wide TLB >> shootdown, i.e. a global action by every processor in the system. > >> I'm currently working supervised by alc@ to an intermediate layer that >> sits between UMA and the VM, which goal is satisfyinh efficiently > requests >> 4KB (so, the one you want considering you're asking for >> 16KB-32KB), but the work is in an early stage. > =C2=A0I was not very clear here. I'm saying about page-sized blocks, but > =C2=A0many of them. 16K-32K is not a size in bytes, but count of page-siz= ed > =C2=A0blocks my code needs :) > ok. > =C2=A0BTW, I/O is often require big buffers, up to MAXPHYS (128KiB for > =C2=A0now), do you mean, that any allocation of such memory has > =C2=A0considerable performance penalties, especially on multi-core and > =C2=A0multi-CPU systems? > In fact, the main client of such kind of allocations is the ZFS filesystem (this is due to its mechanism of adaptative cache replacement, ARC). Afaik, at the time in which UMA was written, such kind of allocations you describe were so infrequent that no initial effort was made in order to optimize them. People tried to address this issue by having ZFS create a large number of UMA zones for large allocations of different sizes. Unfortunately, one of the side-effects of this approach was the growth of the fragmentation, so we're investigating about. > -- > // Black Lion AKA Lev Serebryakov > > From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 14:38:11 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCCFF1065670 for ; Sun, 2 Oct 2011 14:38:11 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 0F2338FC08 for ; Sun, 2 Oct 2011 14:38:08 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 455684AC2D; Sun, 2 Oct 2011 18:38:05 +0400 (MSD) Date: Sun, 2 Oct 2011 18:37:58 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <16010671866.20111002183758@serebryakov.spb.ru> To: Davide Italiano In-Reply-To: References: <358651269.20111002162109@serebryakov.spb.ru> <1393358703.20111002174545@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Memory allocation in kernel -- what to use in which situation? What is the best for page-sized allocations? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 14:38:11 -0000 Hello, Davide. You wrote 2 =D0=BE=D0=BA=D1=82=D1=8F=D0=B1=D1=80=D1=8F 2011 =D0=B3., 18:00:= 26: >> =C2=A0BTW, I/O is often require big buffers, up to MAXPHYS (128KiB for >> =C2=A0now), do you mean, that any allocation of such memory has >> =C2=A0considerable performance penalties, especially on multi-core and >> =C2=A0multi-CPU systems? >> > In fact, the main client of such kind of allocations is the ZFS > filesystem (this is due to its mechanism of adaptative cache > replacement, ARC). Afaik, at the time in which UMA was written, such > kind of allocations you describe were so infrequent that no initial > effort was made in order to optimize them. > People tried to address this issue by having ZFS create a large number > of UMA zones for large allocations of different sizes. Unfortunately, > one of the side-effects of this approach was the growth of the > fragmentation, so we're investigating about. What about these geom modules, which allocate buffers, because need to read more, than requested by upper layer? geom_cache and geom_raid3, for example? And "my" geom_raid5 -- I begin to understand, why original author of geom_raid5 (which need MAXPHYS-sized buffers regularry) wrote its own memory management layer... --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 15:08:39 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67359106566B; Sun, 2 Oct 2011 15:08:39 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 07EB58FC0C; Sun, 2 Oct 2011 15:08:38 +0000 (UTC) Received: by vcbf13 with SMTP id f13so3206197vcb.13 for ; Sun, 02 Oct 2011 08:08:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=uq2e0y8rkKK+Lf0Op19AaNwzR/e6qA6AsOBtUjn61yQ=; b=MVi9sf/NQ2ZLS7Gm5QVGpgqxgaT7d4UuXiKpAL9f4Nf2dj9+1fYkEm0jPxIpYTG5nD mwFkZWSK7Dbj93P4nPOpeGu0aV4T0GSaCuqlsqKrcJYAdMmH6TUt4FM74WvYqkn7uFO5 f4ytBDh3ztGIwWq3qlXYzqRbTJBl9Nm0kJxyI= MIME-Version: 1.0 Received: by 10.52.36.212 with SMTP id s20mr9833456vdj.31.1317568117571; Sun, 02 Oct 2011 08:08:37 -0700 (PDT) Received: by 10.52.179.228 with HTTP; Sun, 2 Oct 2011 08:08:37 -0700 (PDT) In-Reply-To: <16010671866.20111002183758@serebryakov.spb.ru> References: <358651269.20111002162109@serebryakov.spb.ru> <1393358703.20111002174545@serebryakov.spb.ru> <16010671866.20111002183758@serebryakov.spb.ru> Date: Sun, 2 Oct 2011 17:08:37 +0200 Message-ID: From: Davide Italiano To: lev@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Memory allocation in kernel -- what to use in which situation? What is the best for page-sized allocations? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 15:08:39 -0000 On Sun, Oct 2, 2011 at 4:37 PM, Lev Serebryakov wrote: > Hello, Davide. > You wrote 2 =D0=BE=D0=BA=D1=82=D1=8F=D0=B1=D1=80=D1=8F 2011 =D0=B3., 18:0= 0:26: > >>> =C2=A0BTW, I/O is often require big buffers, up to MAXPHYS (128KiB for >>> =C2=A0now), do you mean, that any allocation of such memory has >>> =C2=A0considerable performance penalties, especially on multi-core and >>> =C2=A0multi-CPU systems? >>> >> In fact, the main client of such kind of allocations is the ZFS >> filesystem (this is due to its mechanism of adaptative cache >> replacement, ARC). Afaik, at the time in which UMA was written, such >> kind of allocations you describe were so infrequent that no initial >> effort was made in order to optimize them. >> People tried to address this issue by having ZFS create a large number >> of UMA zones for large allocations of different sizes. Unfortunately, >> one of the side-effects of this approach was the growth of the >> fragmentation, so we're investigating about. > =C2=A0 What about these geom modules, which allocate buffers, because nee= d > =C2=A0to read more, than requested by upper layer? geom_cache and > =C2=A0geom_raid3, for example? I wasn't aware about that, thanks a lot for pointing me out. I'll surely look at them. > =C2=A0 And "my" geom_raid5 -- I begin to understand, why original author > =C2=A0of geom_raid5 (which need MAXPHYS-sized buffers regularry) wrote it= s > =C2=A0own memory management layer... > If you're interested in what we're doing, contact me or poke me on efnet.or= g. > -- > // Black Lion AKA Lev Serebryakov > > From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 15:32:31 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DFD91065672; Sun, 2 Oct 2011 15:32:31 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 53C0E8FC12; Sun, 2 Oct 2011 15:32:31 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 82D594AC1C; Sun, 2 Oct 2011 19:32:29 +0400 (MSD) Date: Sun, 2 Oct 2011 19:32:23 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <1258376930.20111002193223@serebryakov.spb.ru> To: current@freebsd.org, hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: Lawrence Stewart Subject: alq_open_flags() panics in _mtx_lock_flags() X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 15:32:31 -0000 Hello, FreeBSD. I'm trying to create logging queue with alq kernel API. I call alq_open_flags() like this in my module: error =3D alq_open_flags(&sc->sc_alq, sc->sc_vnode_name, curthread->td_ucred, ALQ_DEFAULT_CMODE, sizeof(struct g_log_entry), ALQ_ORDERED); and my system (10-CURRENT) panics with this stack trace (top frames are DDB-related, so I omit them): #5 0xc06101c3 in kdb_trap (type=3D12, code=3D0, tf=3D0xc4e29990) at /usr/home/lev/FreeBSD-head/sys/kern/subr_kdb.c:620 #6 0xc08290af in trap_fatal (frame=3D0xc4e29990, eva=3D136) at /usr/home/lev/FreeBSD-head/sys/i386/i386/trap.c:958 #7 0xc08292c0 in trap_pfault (frame=3D0xc4e29990, usermode=3D0, eva=3D136) at /usr/home/lev/FreeBSD-head/sys/i386/i386/trap.c:880 #8 0xc0829f46 in trap (frame=3D0xc4e29990) at /usr/home/lev/FreeBSD-head/sys/i386/i386/trap.c:555 #9 0xc0812e7c in calltrap () at /usr/home/lev/FreeBSD-head/sys/i386/i386/exception.s:168 #10 0xc05cafc0 in _mtx_lock_flags (m=3D0x78, opts=3D0,=20 file=3D0xc088c140 "/usr/home/lev/FreeBSD-head/sys/kern/vfs_subr.c",=20 line=3D2169) at /usr/home/lev/FreeBSD-head/sys/kern/kern_mutex.c:194 #11 0xc0672eb2 in vref (vp=3D0x0) at /usr/home/lev/FreeBSD-head/sys/kern/vfs_subr.c:2169 #12 0xc066932f in namei (ndp=3D0xc4e29b74) at /usr/home/lev/FreeBSD-head/sys/kern/vfs_lookup.c:264 #13 0xc0682900 in vn_open_cred (ndp=3D0xc4e29b74, flagp=3D0xc4e29be8, cmode= =3D384,=20 vn_open_flags=3D0, cred=3D0xc50fee80, fp=3D0x0) at /usr/home/lev/FreeBSD-head/sys/kern/vfs_vnops.c:137 #14 0xc5c42609 in alq_open_flags (alqp=3D0xc550dc08,=20 file=3D0xc5108d40 "/usr/ada4.log", cred=3D0xc50fee80, cmode=3D384, size= =3D28, flags=3D16) at /usr/home/lev/FreeBSD-head/sys/modules/alq/../../kern/kern_alq.c:451 It seems, that vref() get NULL instead of valid pointer to struct vnode. But I have no idea -- why?! Yes, I have no such file created, but man alq(9) says, that it will create file for me. And if I point to existed file, it panic anyway. What do I do wrong?! --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 18:21:29 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99133106566B for ; Sun, 2 Oct 2011 18:21:29 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 497F98FC0A for ; Sun, 2 Oct 2011 18:21:29 +0000 (UTC) Received: by qadz30 with SMTP id z30so1456458qad.13 for ; Sun, 02 Oct 2011 11:21:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=cUVI2uh949XcN3eGb6IVvmT5Kcz00K3TCz91S2ccecA=; b=ChshxPhbcJpiu+fwW0k/pVBq1aJFRQlVUxEz/HpKQwtHQVEQYbRQly15LzmOAzetwC /CI4QlntjlFEL91/l1mMoWqk0deBCUypOqVQTUvLQiIn7vk/Xx7XwKiDgscJDc3BHriG s0PoT6hNypEZjWc+e+YXD6dioCMedaSyfjTNU= MIME-Version: 1.0 Received: by 10.229.69.142 with SMTP id z14mr10529490qci.167.1317579688586; Sun, 02 Oct 2011 11:21:28 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.229.83.196 with HTTP; Sun, 2 Oct 2011 11:21:28 -0700 (PDT) In-Reply-To: <358651269.20111002162109@serebryakov.spb.ru> References: <358651269.20111002162109@serebryakov.spb.ru> Date: Sun, 2 Oct 2011 11:21:28 -0700 X-Google-Sender-Auth: GruFjWlqspxl3rV2p-nXSzPOCaY Message-ID: From: mdf@FreeBSD.org To: lev@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Memory allocation in kernel -- what to use in which situation? What is the best for page-sized allocations? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 18:21:29 -0000 2011/10/2 Lev Serebryakov : > Hello, Freebsd-hackers. > > =A0Here are several memory-allocation mechanisms in the kernel. The two > I'm aware of is MALLOC_DEFINE()/malloc()/free() and uma_* (zone(9)). > > =A0As far as I understand, malloc() is general-purpose, but it has > fixed "transaction cost" (in term of memory consumption) for each > block allocated, and is not very suitable for allocation of many small > blocks, as lots of memory will be wasted for bookkeeping. > > =A0zone(9) allocator, on other hand, have very low cost of each > allocated block, but could allocate only pre-configured fixed-size > blocks, and ideal for allocation tons of small objects (and provide > API for reusing them, too!). > > =A0Am I right? No one has quite answered this question, IMO, so here's my 2 cents. malloc(9) on smaller sizes (<=3D PAGE_SIZE) uses uma(9) under the covers. There are a set of uma zones for 16, 32, 64, 128, ... PAGE_SIZE bytes and malloc(9) looks up the malloc size in a small array to determine which uma zone to allocate from. So malloc(9) on small sizes doesn't have overhead of bookkeeping, but it does have overhead of rounding to the next highest malloc uma bucket. At $WORK we found, for example, that 48 bytes and 96 bytes were very common sizes and so I added uma zones there (and few other odd sies determined by using the malloc statistics option). > =A0 But what if I need to allocate a lot (say, 16K-32K) of page-sized > blocks? Not in one chunk, for sure, but in lifetime of my kernel > module. Which allocator should I use? It seems, the best one will be > very low-level only-page-sized allocator. Is here any in kernel? 4k allocations, as has been pointed out, get a single kernel page in both the virtual space and physical space. They (like all the large allocations) use a field in the vm_page for the physical page backing the virtual address to record info about the allocation. Any allocation PAGE_SIZE and larger will round up to the next multiple of pages and allocate whole pages. IMO the problems here are (1) as was pointed out, TLB shootdown on free(9), and (2) the current algorithm for finding space in a kmem_map is a linear search and doesn't track where there are fragmented chunks, so it's not terribly efficient when finding larger sies, and the PAGE_SIZE allocations will not fill in fragmented areas. Cheers, matthew From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 18:26:26 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4191A106564A for ; Sun, 2 Oct 2011 18:26:26 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id DCF0C8FC13 for ; Sun, 2 Oct 2011 18:26:25 +0000 (UTC) Received: by qyk4 with SMTP id 4so3286560qyk.13 for ; Sun, 02 Oct 2011 11:26:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=/+rBO+upofJnj9W4k9tD/WmZaCTRHhZvHc7b2SXfMfY=; b=VL1sOIBwcMcFPmHw+fL07FfR6pI5T2QE9fnUVA+V4n8uA6p0AU9Jm4SmgoQqI6xlYY 2oNV0bW5Ufl5ZwHDbGsM8XnUvtshsUel6LhDEV+LsgDRYkcHiqTuOqWdS7saIpvSOzep cWvgeY6IXtPQoaRSWnf31EF/Q30TTvsQvQzdw= MIME-Version: 1.0 Received: by 10.229.69.142 with SMTP id z14mr10531758qci.167.1317579984939; Sun, 02 Oct 2011 11:26:24 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.229.83.196 with HTTP; Sun, 2 Oct 2011 11:26:24 -0700 (PDT) In-Reply-To: <1076522387.20111002163542@serebryakov.spb.ru> References: <1076522387.20111002163542@serebryakov.spb.ru> Date: Sun, 2 Oct 2011 11:26:24 -0700 X-Google-Sender-Auth: 71AZvNyl6x5I5w40Gc-ogaV9Yys Message-ID: From: mdf@FreeBSD.org To: lev@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Way to get current tick number in kernel? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 18:26:26 -0000 2011/10/2 Lev Serebryakov : > Hello, Freebsd-hackers. > > =A0What should I use to measure short intervals of time between events > in kernel? I don't need any "time" in means of, for example, time(3) > API, but some monotonically and uniformly increasing counter with > known frequency. As cheap as possible, without complex calculations :) There are several global variables that may suffice. 'ticks' is the current tick value. 'time_second' is the current time in seconds, but this is adjusted when the system time is changed. 'time_uptime' is the current uptime in seconds and is the base upon which time_second is computed, depending on the system clock. Cheers, matthew From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 19:32:01 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A052C1065670; Sun, 2 Oct 2011 19:32:01 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 365B58FC18; Sun, 2 Oct 2011 19:32:00 +0000 (UTC) Received: by gyf2 with SMTP id 2so3598954gyf.13 for ; Sun, 02 Oct 2011 12:32:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=aHT4/vpOTY3msQbvI1HAJw/7Js1fgLv78BCO+8UlOpY=; b=TSsVeLlDCDCko/pHXsvz8nCOAwmK7U6Tt1VWlakgJ1Tu3ZJxuStHbFcoVqyb6d9WMP VID1mkaJZFrTUNxUw3ez1ipRJknxGKSNMZLTvwhXk8LtbbOE7/6Hj9TES31YJWEYP9k1 osj6cMchlpQL7INiEGF2EK3SrRG3cIMQll4NQ= MIME-Version: 1.0 Received: by 10.68.19.196 with SMTP id h4mr75595483pbe.39.1317582114935; Sun, 02 Oct 2011 12:01:54 -0700 (PDT) Received: by 10.142.166.3 with HTTP; Sun, 2 Oct 2011 12:01:54 -0700 (PDT) In-Reply-To: References: <358651269.20111002162109@serebryakov.spb.ru> Date: Sun, 2 Oct 2011 14:01:54 -0500 Message-ID: From: Alan Cox To: mdf@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, lev@freebsd.org Subject: Re: Memory allocation in kernel -- what to use in which situation? What is the best for page-sized allocations? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alc@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 19:32:01 -0000 On Sun, Oct 2, 2011 at 1:21 PM, wrote: > 2011/10/2 Lev Serebryakov : > > Hello, Freebsd-hackers. > > > > Here are several memory-allocation mechanisms in the kernel. The two > > I'm aware of is MALLOC_DEFINE()/malloc()/free() and uma_* (zone(9)). > > > > As far as I understand, malloc() is general-purpose, but it has > > fixed "transaction cost" (in term of memory consumption) for each > > block allocated, and is not very suitable for allocation of many small > > blocks, as lots of memory will be wasted for bookkeeping. > > > > zone(9) allocator, on other hand, have very low cost of each > > allocated block, but could allocate only pre-configured fixed-size > > blocks, and ideal for allocation tons of small objects (and provide > > API for reusing them, too!). > > > > Am I right? > > No one has quite answered this question, IMO, so here's my 2 cents. > > malloc(9) on smaller sizes (<= PAGE_SIZE) uses uma(9) under the > covers. There are a set of uma zones for 16, 32, 64, 128, ... > PAGE_SIZE bytes and malloc(9) looks up the malloc size in a small > array to determine which uma zone to allocate from. > > So malloc(9) on small sizes doesn't have overhead of bookkeeping, but > it does have overhead of rounding to the next highest malloc uma > bucket. At $WORK we found, for example, that 48 bytes and 96 bytes > were very common sizes and so I added uma zones there (and few other > odd sies determined by using the malloc statistics option). > > > But what if I need to allocate a lot (say, 16K-32K) of page-sized > > blocks? Not in one chunk, for sure, but in lifetime of my kernel > > module. Which allocator should I use? It seems, the best one will be > > very low-level only-page-sized allocator. Is here any in kernel? > > 4k allocations, as has been pointed out, get a single kernel page in > both the virtual space and physical space. They (like all the large > allocations) use a field in the vm_page for the physical page backing > the virtual address to record info about the allocation. > > Any allocation PAGE_SIZE and larger will round up to the next multiple > of pages and allocate whole pages. IMO the problems here are (1) as > was pointed out, TLB shootdown on free(9), and (2) the current > algorithm for finding space in a kmem_map is a linear search and > doesn't track where there are fragmented chunks, so it's not terribly > efficient when finding larger sies, and the PAGE_SIZE allocations will > not fill in fragmented areas. > > Regarding #2, no, it is not linear; it is an amortized logarithmic first fit. Every node in every vm map, including the kmem map, is augmented with free space information. This is used by the first fit traversal to skip entire subtrees that contain insufficient space. Regards, Alan From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 21:07:00 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 334B6106566C for ; Sun, 2 Oct 2011 21:07:00 +0000 (UTC) (envelope-from lichray@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id E41C08FC18 for ; Sun, 2 Oct 2011 21:06:59 +0000 (UTC) Received: by ywp17 with SMTP id 17so3613968ywp.13 for ; Sun, 02 Oct 2011 14:06:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=73EcQdRmyGcsIWBqF8bYCRhCjWioS2LV/N9vTFo8C9A=; b=ZL52gw5mmoTIFbJFvReqcycCKgA7mFItynP2Nkk9keAq/ABeV2jfX9bkVb9+n7vaRp H1pU9bj23TUZVhixhFlSLkcgb/WyYkkf3SbUJQd4w+o8Ailrws/8ZBk+YftGR+dRKmk8 6w3i2SOB0zhcO0AhQZs39TpJVj8LoUCR8dfFI= Received: by 10.236.143.101 with SMTP id k65mr23979530yhj.82.1317589618226; Sun, 02 Oct 2011 14:06:58 -0700 (PDT) Received: from compaq.yuetime (m8d0736d0.tmodns.net. [208.54.7.141]) by mx.google.com with ESMTPS id v4sm9921358yhk.3.2011.10.02.14.06.57 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 02 Oct 2011 14:06:57 -0700 (PDT) Date: Sun, 2 Oct 2011 16:06:49 -0500 From: Zhihao Yuan To: freebsd-hackers@freebsd.org Message-ID: <20111002210649.GA5774@compaq.yuetime> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="h31gzZEtNLTqOjlF" Content-Disposition: inline Received: by 10.231.36.69 with HTTP; Sun, 2 Oct 2011 13:44:59 -0700 (PDT) User-Agent: Mutt/1.5.21 (2010-09-15) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: [nvi-iconv]I need some help on testing/code review X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 21:07:00 -0000 --h31gzZEtNLTqOjlF Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Hi, I'm seeking some help on getting my GSoC project, nvi-iconv, into FreeBSD base system. This project adds the multibyte encoding supports to nvi. Here is it's FreeBSD Wiki page: http://wiki.freebsd.org/ZhihaoSoC2011 The first step is to get some more testing reports. To be more specific, I need some help on four untested CJK encodings, ja_JP.SJIS, ko_KR.CP949, zh_TW.Big5, zh_HK.Big5HKSCS, and other non-Unicode, non-CJK multibyte encodings (if they're available on FreeBSD). The patches can be downloaded at: SVN diff: https://github.com/downloads/lichray/nvi2/nvi2-freebsd-2011-08-26-svn.diff.gz Git diff: https://github.com/downloads/lichray/nvi2/nvi2-freebsd-2011-08-26.diff.gz To test the new nvi, you need a src tree >= 9.0b and a base system built WITH_ICONV=1. Patch the src tree with (a file needs to be moved with -E): patch -p0 -E < /path/to/nvi2-freebsd-2011-08-26.diff And cd to usr.bin/vi under the src tree, make WITH_ICONV=1 Now you can run the new nvi with ./nvi . If you want to replace the system vi with this one, make WITH_ICONV=1 install *NOTE* FreeBSD's libncursesw only recognizes a subset of the locales which are supported by libc. Which means, not all libc locales work. For example, LC_CTYPE=zh_CN.GBK won't work; use zh_CN.eucCN instead. And I need some help on code review. The code is based on nvi-1.79 and nvi-1.8x (which is used by NetBSD). A committer with some experience on libiconv is preferred. -- Zhihao Yuan, nickname lichray The best way to predict the future is to invent it. ___________________________________________________ 4BSD -- http://4bsd.biz/ --h31gzZEtNLTqOjlF Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Hi, I'm seeking some help on getting my GSoC project= , nvi-iconv into FreeBSD base system. This project adds the multi= byte encoding supports to nvi. Here is the FreeBSD?? The first step is to getting some more testing reports. -- Zhihao Yuan, nickname lichray The best way to p= redict the future is to invent it. _____________________________________= ______________ 4BSD -- [1]http://4bsd.biz/= References 1. 3D"http://4bsd.biz/" --h31gzZEtNLTqOjlF-- From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 2 21:08:43 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0704E1065672 for ; Sun, 2 Oct 2011 21:08:43 +0000 (UTC) (envelope-from lichray@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id B8D378FC14 for ; Sun, 2 Oct 2011 21:08:42 +0000 (UTC) Received: by gyf2 with SMTP id 2so3642074gyf.13 for ; Sun, 02 Oct 2011 14:08:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=avNVuv/9xLqmQR9qK1GXG/glpAENYsi1KjWOQya970o=; b=Cq5K/k+oaNNfPWb0cdSr7CtEXXwBSHnM4jQG2qT3coXbfz5b2COc24mmUkl5GjosSQ 4swcvqf4r828uzcOUmp5BvuXNKovVdlUu39JAG9YizztBplIDL3lksdcXl5YNaXDZkvK ebPlCKjLGogQmDBw4UW9FWV6PiLV0K3xncu1Q= Received: by 10.150.254.1 with SMTP id b1mr1050091ybi.122.1317589722081; Sun, 02 Oct 2011 14:08:42 -0700 (PDT) Received: from compaq.yuetime (m8d0736d0.tmodns.net. [208.54.7.141]) by mx.google.com with ESMTPS id o7sm31750210anp.18.2011.10.02.14.08.40 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 02 Oct 2011 14:08:41 -0700 (PDT) Date: Sun, 2 Oct 2011 16:08:36 -0500 From: Zhihao Yuan To: other Message-ID: <20111002210836.GA5861@compaq.yuetime> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="BXVAT5kNtrzKuDFl" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: [nvi-iconv]I need some help on testing/code review X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Oct 2011 21:08:43 -0000 --BXVAT5kNtrzKuDFl Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I'm seeking some help on getting my GSoC project, nvi-iconv, into FreeBSD= =20 base system. This project adds the multibyte encoding supports to nvi. Here is it's FreeBSD Wiki page: http://wiki.freebsd.org/ZhihaoSoC2011 The first step is to get some more testing reports. To be more specific, I need some help on four untested CJK encodings, ja_JP.SJIS, ko_KR.CP949, zh_TW.Big5, zh_HK.Big5HKSCS, and other non-Unicode, non-CJK multibyte encodings. The patches can be downloaded at: SVN diff: https://github.com/downloads/lichray/nvi2/nvi2-freebsd-2011-08-26-svn.diff.= gz Git diff: https://github.com/downloads/lichray/nvi2/nvi2-freebsd-2011-08-26.diff.gz To test the new nvi, you need a src tree >=3D 9.0b and a base system built WITH_ICONV=3D1. Patch the src tree with (a file needs to be moved with -E): patch -p0 -E < /path/to/nvi2-freebsd-2011-08-26.diff And cd to usr.bin/vi under the src tree, make WITH_ICONV=3D1 Now you can run the new nvi with ./nvi . If you want to replace the system vi with this one, make WITH_ICONV=3D1 install *NOTE* FreeBSD's libncursesw only recognizes a subset of the locales which are supported by libc. Which means, not all libc locales work. For example, LC_CTYPE=3Dzh_CN.GBK won't work; use zh_CN.eucCN instead. And I need some help on code review. The code is based on nvi-1.79 and nvi-1.8x (which is used by NetBSD). A committer with some experience on=20 libiconv is preferred. --=20 Zhihao Yuan, nickname lichray The best way to predict the future is to invent it. ___________________________________________________ 4BSD -- http://4bsd.biz/ --BXVAT5kNtrzKuDFl Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBAgAGBQJOiNLUAAoJEDM1qsGtFE5ZW2EIALq0a/L5rRLo8psZwhVPTFQd XIkXCbzLSSrwl7HWWdN/bId6QuJ5+UHF38D1u/pbymLD48AxcttSOgPd0smPymSk MNFQ4O5fifmtnDCOKYjGM01yhwVPMeK8FZ5td+sdL2N1rAUmc1FuUCCBHsBqBBBV 13XQH2Ttw10Pt9grrEyX+K1gQDBDie7AMdU7Hh6/5qU6BHPvWSLMxQaRsA+q8rEy 6wPVL2hD9UMvNCKXfENBwPjZItNksXGd+KEv8WIXUd+pMLPsNv8WaXfsT9L/oD71 bB7SlkcpUlKVZN7oAzsCn9cSPOrYX07LoTguHUgbJYTC39X9GgpB5UxB6U7gqzM= =d8uk -----END PGP SIGNATURE----- --BXVAT5kNtrzKuDFl-- From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 00:58:33 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 301E1106566B; Mon, 3 Oct 2011 00:58:33 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id EAF578FC14; Mon, 3 Oct 2011 00:58:32 +0000 (UTC) Received: from lstewart1.loshell.room52.net (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id AD99C7E824; Mon, 3 Oct 2011 11:43:09 +1100 (EST) Message-ID: <4E89051D.30908@freebsd.org> Date: Mon, 03 Oct 2011 11:43:09 +1100 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0.2) Gecko/20110914 Thunderbird/6.0.2 MIME-Version: 1.0 To: lev@FreeBSD.org References: <1258376930.20111002193223@serebryakov.spb.ru> In-Reply-To: <1258376930.20111002193223@serebryakov.spb.ru> Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on lauren.room52.net Cc: hackers@FreeBSD.org Subject: Re: alq_open_flags() panics in _mtx_lock_flags() X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 00:58:33 -0000 [trimmed current@ from CC] Hi Lev, On 10/03/11 02:32, Lev Serebryakov wrote: > Hello, FreeBSD. > > I'm trying to create logging queue with alq kernel API. I call > alq_open_flags() like this in my module: > > error = alq_open_flags(&sc->sc_alq, sc->sc_vnode_name, > curthread->td_ucred, ALQ_DEFAULT_CMODE, > sizeof(struct g_log_entry), ALQ_ORDERED); > > and my system (10-CURRENT) panics with this stack trace (top frames > are DDB-related, so I omit them): > > #5 0xc06101c3 in kdb_trap (type=12, code=0, tf=0xc4e29990) > at /usr/home/lev/FreeBSD-head/sys/kern/subr_kdb.c:620 > #6 0xc08290af in trap_fatal (frame=0xc4e29990, eva=136) > at /usr/home/lev/FreeBSD-head/sys/i386/i386/trap.c:958 > #7 0xc08292c0 in trap_pfault (frame=0xc4e29990, usermode=0, eva=136) > at /usr/home/lev/FreeBSD-head/sys/i386/i386/trap.c:880 > #8 0xc0829f46 in trap (frame=0xc4e29990) > at /usr/home/lev/FreeBSD-head/sys/i386/i386/trap.c:555 > #9 0xc0812e7c in calltrap () > at /usr/home/lev/FreeBSD-head/sys/i386/i386/exception.s:168 > #10 0xc05cafc0 in _mtx_lock_flags (m=0x78, opts=0, > file=0xc088c140 "/usr/home/lev/FreeBSD-head/sys/kern/vfs_subr.c", > line=2169) at /usr/home/lev/FreeBSD-head/sys/kern/kern_mutex.c:194 > #11 0xc0672eb2 in vref (vp=0x0) > at /usr/home/lev/FreeBSD-head/sys/kern/vfs_subr.c:2169 > #12 0xc066932f in namei (ndp=0xc4e29b74) > at /usr/home/lev/FreeBSD-head/sys/kern/vfs_lookup.c:264 > #13 0xc0682900 in vn_open_cred (ndp=0xc4e29b74, flagp=0xc4e29be8, cmode=384, > vn_open_flags=0, cred=0xc50fee80, fp=0x0) > at /usr/home/lev/FreeBSD-head/sys/kern/vfs_vnops.c:137 > #14 0xc5c42609 in alq_open_flags (alqp=0xc550dc08, > file=0xc5108d40 "/usr/ada4.log", cred=0xc50fee80, cmode=384, size=28, > flags=16) > at /usr/home/lev/FreeBSD-head/sys/modules/alq/../../kern/kern_alq.c:451 > > It seems, that vref() get NULL instead of valid pointer to struct > vnode. But I have no idea -- why?! Yes, I have no such file created, > but man alq(9) says, that it will create file for me. And if I point > to existed file, it panic anyway. > > What do I do wrong?! > I don't see anything obviously wrong with your call to alq_open_flags(), other than that I think you want to set your buffer size to be a multiple of "sizeof(struct g_log_entry)". The small queue size wouldn't cause the panic you're seeing though. It's hard to draw any real conclusions without seeing your code. Are you able to share it? Cheers, Lawrence From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 06:43:36 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75937106564A; Mon, 3 Oct 2011 06:43:36 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 123FC8FC0A; Mon, 3 Oct 2011 06:43:36 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 58DD74AC1C; Mon, 3 Oct 2011 10:43:34 +0400 (MSD) Date: Mon, 3 Oct 2011 10:43:27 +0400 From: Lev Serebryakov Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <535056585.20111003104327@serebryakov.spb.ru> To: Lawrence Stewart In-Reply-To: <4E89051D.30908@freebsd.org> References: <1258376930.20111002193223@serebryakov.spb.ru> <4E89051D.30908@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: hackers@FreeBSD.org Subject: Re: alq_open_flags() panics in _mtx_lock_flags() X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 06:43:36 -0000 Hello, Lawrence. You wrote 3 =EE=EA=F2=FF=E1=F0=FF 2011 =E3., 4:43:09: > I don't see anything obviously wrong with your call to alq_open_flags(), > other than that I think you want to set your buffer size to be a=20 > multiple of "sizeof(struct g_log_entry)". The small queue size wouldn't > cause the panic you're seeing though. > It's hard to draw any real conclusions without seeing your code. Are you > able to share it? As I wrote to current@, fs@ and geom@, it seems, that vnode API could not be called from geom threads (g_events), at least with topology lock held. So, it is not ALQ-specific problem, but some mis-interaction between GEOM and VFS levels. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 08:01:29 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6ACC106564A for ; Mon, 3 Oct 2011 08:01:29 +0000 (UTC) (envelope-from fabien.thomas@netasq.com) Received: from work.netasq.com (mars.netasq.com [91.212.116.3]) by mx1.freebsd.org (Postfix) with ESMTP id 5D87B8FC1A for ; Mon, 3 Oct 2011 08:01:28 +0000 (UTC) Received: from [10.2.1.1] (unknown [10.2.1.1]) by work.netasq.com (Postfix) with ESMTPSA id 0CDDE740016; Mon, 3 Oct 2011 10:00:29 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: Fabien Thomas In-Reply-To: <20110922212602.GS26743@acme.spoerlein.net> Date: Mon, 3 Oct 2011 10:01:27 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4E712D11.7040202@FreeBSD.org> <4E75B67E.1000802@FreeBSD.org> <20110922190535.GR26743@acme.spoerlein.net> <20110922212602.GS26743@acme.spoerlein.net> To: hackers@freebsd.org X-Mailer: Apple Mail (2.1084) Cc: =?iso-8859-1?Q?Ulrich_Sp=F6rlein?= , Arnaud Lacombe Subject: Re: my git development snapshot(s) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 08:01:30 -0000 On Sep 22, 2011, at 11:26 PM, Ulrich Sp=F6rlein wrote: > On Thu, 2011-09-22 at 15:52:43 -0400, Arnaud Lacombe wrote: >> Hi, >>=20 >> On Thu, Sep 22, 2011 at 3:05 PM, Ulrich Sp=F6rlein = wrote: >>> On Sun, 2011-09-18 at 12:14:38 +0300, Andriy Gapon wrote: >>>>=20 >>>> Just decided to follow the global trends and trying to throw all of = my >>>> local/private changes at you in hope that the "crowd-sourcing = magic" might >>>> somehow happen :-) This seems definitely easier than carefully = producing the >>>> patch files and keeping them up-to-date. >>>>=20 >>>> So, my newly cloned gitorious repository: >>>> https://gitorious.org/~avg/freebsd/avgbsd >>>>=20 >>>> And the first branch of interest: >>>> https://gitorious.org/~avg/freebsd/avgbsd/commits/devel-20110915 >>>=20 >>> I'll throw mine in as well: >>>=20 >>> https://github.com/uqs/freebsd-head/branches >>>=20 >> is that the same as tree as Fabien's, or a new snapshot ? >>=20 >> If not, couldn't we agree to have all the same tree in order to ease >> code sharing between all of them ? >>=20 >> I see there is already a https://github.com/freebsd/freebsd tree, >> which seem different than Fabien's tree on github. It really look = like >> The FreeBSD Project (https://github.com/freebsd/) is not able to >> provide consistency. >=20 > The freebsd-head tree is the same that everybody in the world can get = by > simply running git svn clone against the FreeBSD subversion server > (you'd need a lot of patience, though). >=20 > It's also available from git.freebsd.your.org and on code.google.com > (except that Google's git backend sucks ass). >=20 > Fabien's tree is cut off at an arbitrary date and if you happen to > choose a different date, well you cannot merge any branches with that > repository. No it was not cut at all it have been made at a time where github dont allow large repo for free. All the branches >=3D 4.11 was in and=20 with full history from rev 1. As Arnaud told me it seems at some point in time that at least stable_8 branch was broken. To be sure there is no other branches like this i've rebuilded the repo and done a git push --force on Gitorious. The effect of merge for people that use it is that git will not successfully merge the tree as the history is not the same (for stable_8 at least). To solve that i've done: git merge -Xtheirs upstream/svn_stable_8 on my reference branch (that dont contain modifications). After that, merging from this branch will be ok. As more and more people use git i hope that at some point in time we will have an official repo hosted on FreeBSD servers. Fabien From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 09:38:03 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07970106564A for ; Mon, 3 Oct 2011 09:38:03 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id C4DB78FC13 for ; Mon, 3 Oct 2011 09:38:02 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 6B8FF4AC1C for ; Mon, 3 Oct 2011 13:38:01 +0400 (MSD) Date: Mon, 3 Oct 2011 13:37:54 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <558829232.20111003133754@serebryakov.spb.ru> To: hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: Subject: Atomic increment and get? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 09:38:03 -0000 Hello, Hackers. Is here atomic increment and get (or add and get) operation in kernel? I cannot find one. Here is atomic_add_32(), but it doesn't return result. And here is no atomic_add_64() on 32 bit system :( --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 14:47:57 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 41BE8106564A for ; Mon, 3 Oct 2011 14:47:57 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 060478FC0A for ; Mon, 3 Oct 2011 14:47:56 +0000 (UTC) Received: by qadz30 with SMTP id z30so2053466qad.13 for ; Mon, 03 Oct 2011 07:47:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=QTwE4j5w5nPWHOHQQV5C20AIVBHCDsovjNjYX+4C0d8=; b=F9frXHUMzrqnYjTGC2OmxnR9SqU6gIfEXXrHC5is7dGFBxFI8W4xDK0fV0937Xfy6x +HHsO/BFSXI4O4gRT8iRHBFLfcLzcpsPGRfqmHvLMdzWlMWjLzaXFI+oUTaiPC3yGCSC /6aVDcvrTyOS3pDUQrnLwZ6uq6ZG4a2+Y1JAU= MIME-Version: 1.0 Received: by 10.224.87.82 with SMTP id v18mr8059qal.74.1317651552846; Mon, 03 Oct 2011 07:19:12 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.229.83.196 with HTTP; Mon, 3 Oct 2011 07:19:12 -0700 (PDT) In-Reply-To: <558829232.20111003133754@serebryakov.spb.ru> References: <558829232.20111003133754@serebryakov.spb.ru> Date: Mon, 3 Oct 2011 07:19:12 -0700 X-Google-Sender-Auth: a20qC71k5RSbe_Igf3gfekOqTPE Message-ID: From: mdf@FreeBSD.org To: lev@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: Atomic increment and get? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 14:47:57 -0000 2011/10/3 Lev Serebryakov : > =A0Is here atomic increment and get (or add and get) operation in > =A0kernel? I cannot find one. Here is atomic_add_32(), but it doesn't > =A0return result. And here is no atomic_add_64() on 32 bit system :( See atomic_fetchadd_int. Not all hardware has 64-bit atomic instructions available in 32-bit mode so it's not machine-independent. Generally, for this case (and potentially for sub-32-bit atomic operations [1]) a lock is used. [1] with masking and shifting 32-bit atomics can be used to set data in an 8 or 16 bit variable; see a recent commit to sys/vm. Cheers, matthew From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 14:59:49 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C56A9106566B; Mon, 3 Oct 2011 14:59:49 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 3AA4A8FC14; Mon, 3 Oct 2011 14:59:48 +0000 (UTC) Received: by wwe3 with SMTP id 3so5531237wwe.31 for ; Mon, 03 Oct 2011 07:59:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=zYgCbi5Bc0IFcYvS0BCbg2IAcVjUru/ebTJuhA5IYq8=; b=JkCefo/HIE8SF/Q8dZiUD1bD2angv/wKqYQ0F3vYbhx4AfbocrJTQOOQ4dtZHoZdXF LNYv08n2dVOIjBF8TSWkd0U6vUArOBVYq2zFucC/miX2XiDMJRbTMcrvPhzuuyXXjTOa DtjL7nMKjWW9Jwmb3ZE69xWUibkdL7UwOqh4g= MIME-Version: 1.0 Received: by 10.227.179.14 with SMTP id bo14mr3114152wbb.90.1317652518834; Mon, 03 Oct 2011 07:35:18 -0700 (PDT) Received: by 10.180.96.104 with HTTP; Mon, 3 Oct 2011 07:35:18 -0700 (PDT) In-Reply-To: <558829232.20111003133754@serebryakov.spb.ru> References: <558829232.20111003133754@serebryakov.spb.ru> Date: Mon, 3 Oct 2011 10:35:18 -0400 Message-ID: From: Ryan Stone To: lev@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: Atomic increment and get? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 14:59:49 -0000 2011/10/3 Lev Serebryakov : > Hello, Hackers. > > =A0Is here atomic increment and get (or add and get) operation in > =A0kernel? I cannot find one. Here is atomic_add_32(), but it doesn't > =A0return result. And here is no atomic_add_64() on 32 bit system :( man atomic gives all of the atomic primitives provided in the kernel. In this case you want atomic_fetchadd_X. From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 15:14:41 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9FFF51065672 for ; Mon, 3 Oct 2011 15:14:41 +0000 (UTC) (envelope-from nitw.satish@gmail.com) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) by mx1.freebsd.org (Postfix) with ESMTP id 7EF098FC0C for ; Mon, 3 Oct 2011 15:14:41 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1RAjvx-00043W-JN for freebsd-hackers@freebsd.org; Mon, 03 Oct 2011 07:55:57 -0700 Date: Mon, 3 Oct 2011 07:55:57 -0700 (PDT) From: satish kondapalli To: freebsd-hackers@freebsd.org Message-ID: <1317653757588-4865200.post@n5.nabble.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 03 Oct 2011 15:57:22 +0000 Subject: NUMA Support is there in FreeBSD. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 15:14:41 -0000 Hi, I am new to FreeBSD, I just want know whether FreeBSD supports NUMA. If FreeBSD supports NUMA what are the kernel API to allocate memory? is there any example driver or any driver which is using the NUMA API? please provide some inputs... Thanks Sateesh -- View this message in context: http://freebsd.1045724.n5.nabble.com/NUMA-Support-is-there-in-FreeBSD-tp4865200p4865200.html Sent from the freebsd-hackers mailing list archive at Nabble.com. From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 16:31:37 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DC781065672 for ; Mon, 3 Oct 2011 16:31:37 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 0ED898FC08 for ; Mon, 3 Oct 2011 16:31:36 +0000 (UTC) Received: by qyk10 with SMTP id 10so2483688qyk.13 for ; Mon, 03 Oct 2011 09:31:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=Z6F/gmq5yqHsQ/eFTzFzuezMJaEWPMHC8qK1uR3cuN4=; b=gP14QREbKiQReEnJi3jDa83jBD4faxJ5K8+iq2DORVq3qN6ZpyElNA2CmvtESpl5fm fSTWMiFZCcd/cr9l9jUKxAovl+oxa666snr4AaV1WEj4lxhM1xlxC4RFNx3EMn3CcLSS SrYyWel5iaFHGgY7NgJFkD2kiCMbofPEBIDkA= MIME-Version: 1.0 Received: by 10.229.231.149 with SMTP id jq21mr103209qcb.243.1317659496212; Mon, 03 Oct 2011 09:31:36 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.229.83.196 with HTTP; Mon, 3 Oct 2011 09:31:35 -0700 (PDT) In-Reply-To: <1317653757588-4865200.post@n5.nabble.com> References: <1317653757588-4865200.post@n5.nabble.com> Date: Mon, 3 Oct 2011 09:31:35 -0700 X-Google-Sender-Auth: sJfAt-IHII4jBflos18IOOtbsZE Message-ID: From: mdf@FreeBSD.org To: satish kondapalli Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org Subject: Re: NUMA Support is there in FreeBSD. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 16:31:37 -0000 On Mon, Oct 3, 2011 at 7:55 AM, satish kondapalli wrote: > I am new to FreeBSD, I just want know whether FreeBSD supports NUMA. > If FreeBSD supports NUMA what are the kernel API to allocate memory? > is there any example driver or any driver which is using the NUMA API? > > please provide some inputs... The kernel is NUMA-aware (at least for x86), and memory is allocated round-robin amongst the memory domains. There are not yet any KPIs for allocating memory in a specific NUMA domain, nor for binding specific threads / processes to get their memory local to a bound cpu instead of round robin. There have been several discussions but no one has taken the lead and proposed some KPIs yet. At $WORK the round-robin is sufficient to get consistent performance numbers and we have not yet started any experimentation with binding specific threads to either CPU or memory. Cheers, matthew From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 17:24:42 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7035106566C; Mon, 3 Oct 2011 17:24:42 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-ww0-f42.google.com (mail-ww0-f42.google.com [74.125.82.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0BC198FC14; Mon, 3 Oct 2011 17:24:41 +0000 (UTC) Received: by wwn22 with SMTP id 22so3709704wwn.1 for ; Mon, 03 Oct 2011 10:24:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=2VxS+ihx2QoIv7RG3rj/akOWPVC+SJkG6THL4b0zPw8=; b=JFdSQx0J3K9DtSoH0dAYXzdY9hVOYqjxGD3ipM+r7lUW7kEe+eIAjtGF5WHzVH72Y0 EC7Pu9jgx+jWIhm+aMhiWuqmjHm+RW+sGCNG5nFRUmq4fHP3u0oYG9DtDSSeNMwEdZHU pl2Amp+/p/lLzUQ2CboezDH1QYuob3MZEPDAo= MIME-Version: 1.0 Received: by 10.216.21.74 with SMTP id q52mr3620987weq.36.1317662680943; Mon, 03 Oct 2011 10:24:40 -0700 (PDT) Received: by 10.180.103.33 with HTTP; Mon, 3 Oct 2011 10:24:40 -0700 (PDT) In-Reply-To: References: <1317653757588-4865200.post@n5.nabble.com> Date: Mon, 3 Oct 2011 13:24:40 -0400 Message-ID: From: Arnaud Lacombe To: mdf@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, satish kondapalli Subject: Re: NUMA Support is there in FreeBSD. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 17:24:42 -0000 Hi, On Mon, Oct 3, 2011 at 12:31 PM, wrote: > On Mon, Oct 3, 2011 at 7:55 AM, satish kondapalli = wrote: >> I am new to FreeBSD, I just want know whether FreeBSD supports NUMA. >> If FreeBSD supports NUMA what are the kernel API to allocate memory? >> is there any example driver or any driver which is using the NUMA API? >> >> please provide some inputs... > > The kernel is NUMA-aware (at least for x86), > What "x86" ? i386 ? amd64 ? both ? > and memory is allocated > round-robin amongst the memory domains. =A0There are not yet any KPIs > for allocating memory in a specific NUMA domain, nor for binding > specific threads / processes to get their memory local to a bound cpu > instead of round robin. > I'm not sure to follow you. Say you have 2 memory domain attached to 2 different CPU package, each providing a memory domain, 4 physical core and eventually 8 virtual. Say you have a network adapter supporting 8 RX/TX queue, dispatching RX packet to 8 netisr. Ideally, you'd want those 8 queue/netisr to each have an affinity for a given CPU/memory domain, have the network adapter route flow evenly on those those 8 CPU. Now, if you allocated an mbuf from memory domain 1, and end up being processed by a CPU in domain 0, that likely to introduce performance penalty. Now, what about userland ? This is certainly an horribly big picture :/ - Arnaud > There have been several discussions but no one has taken the lead and > proposed some KPIs yet. =A0At $WORK the round-robin is sufficient to get > consistent performance numbers and we have not yet started any > experimentation with binding specific threads to either CPU or memory. > > Cheers, > matthew > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 17:34:31 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F1701065672 for ; Mon, 3 Oct 2011 17:34:31 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id DF6218FC0C for ; Mon, 3 Oct 2011 17:34:30 +0000 (UTC) Received: by qyk4 with SMTP id 4so4195346qyk.13 for ; Mon, 03 Oct 2011 10:34:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=x39dKFox3FwKpx8UwR3s7ByitAo5cwVZfvxPvo2LBDI=; b=EUsmGE6/FhKbS/7UT/kvjPJ7WFhbmEVfFr22OBwsHV+FB1BOJ/DRbvsHmqLdfITDmx l0OSP2OeyRYXAUvm63vYZxG7jfU7dXgFZMlz1eoOs375T3aVrUTL7jCscH52rHMQP4+k bmCS8MPceBeV1/2XBWY3j5vErYBXkIvWQVOXw= MIME-Version: 1.0 Received: by 10.229.68.200 with SMTP id w8mr186376qci.167.1317663270002; Mon, 03 Oct 2011 10:34:30 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.229.83.196 with HTTP; Mon, 3 Oct 2011 10:34:29 -0700 (PDT) In-Reply-To: References: <1317653757588-4865200.post@n5.nabble.com> Date: Mon, 3 Oct 2011 10:34:29 -0700 X-Google-Sender-Auth: Xmls_eE91vPzy7DVOS-Y3wjjKrk Message-ID: From: mdf@FreeBSD.org To: Arnaud Lacombe Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, satish kondapalli Subject: Re: NUMA Support is there in FreeBSD. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 17:34:31 -0000 On Mon, Oct 3, 2011 at 10:24 AM, Arnaud Lacombe wrote: > Hi, > > On Mon, Oct 3, 2011 at 12:31 PM, =A0 wrote: >> On Mon, Oct 3, 2011 at 7:55 AM, satish kondapalli wrote: >>> I am new to FreeBSD, I just want know whether FreeBSD supports NUMA. >>> If FreeBSD supports NUMA what are the kernel API to allocate memory? >>> is there any example driver or any driver which is using the NUMA API? >>> >>> please provide some inputs... >> >> The kernel is NUMA-aware (at least for x86), >> > What "x86" ? i386 ? amd64 ? both ? Both; see sys/x86/acpica/srat.c which parses the SRAT table. >> and memory is allocated >> round-robin amongst the memory domains. =A0There are not yet any KPIs >> for allocating memory in a specific NUMA domain, nor for binding >> specific threads / processes to get their memory local to a bound cpu >> instead of round robin. >> > I'm not sure to follow you. Say you have 2 memory domain attached to 2 > different CPU package, each providing a memory domain, 4 physical core > and eventually 8 virtual. Say you have a network adapter supporting 8 > RX/TX queue, dispatching RX packet to 8 netisr. Ideally, you'd want > those 8 queue/netisr to each have an affinity for a given CPU/memory > domain, have the network adapter route flow evenly on those those 8 > CPU. Now, if you allocated an mbuf from memory domain 1, and end up > being processed by a CPU in domain 0, that likely to introduce > performance penalty. Your statement isn't incorrect. What I'm saying is that there's no KPI for requesting bound memory because, while the netstat example is a fine one for where local memory is desired, the majority [1] of processing is not bound to a CPU and so round-robin allocations will produce uniform performance results -- that is, not the best possible, but not wildly fluctuating as scheduling decisions over different runs give different remote memory penalties. [1] for some definition of 'majority'. > Now, what about userland ? > > This is certainly an horribly big picture :/ Yes, and it's why I said just that there's no KPI. One reason there is no KPI is that there's a lot of fiddly bits to take into account. My experience at IBM on AIX was that NUMA is very easy to get wrong; specifically what one usually wants is for the OS to get the answer right (especially for userspace) without a lot of manual tuning; except for some specific applications like netstat queues or a machine doing HPC or mostly running e.g. an Oracle db server, there's too much happening for any one program to configure itself "right" for all the uses of that code. I remember a lot of customer reports of problems from overly aggressive local memory use. Most of the time no one complained when things had consistent performance, even if that wasn't quite as fast as possible. In fact, I may be wrong about the round-robin; I sent jhb@ a patch and I have no recollection anymore whether it's actually in CURRENT. It's been over a year since I thought about this much (BSDCan 2010 was the last time I remember). Cheers, matthew From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 19:54:52 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21C61106564A for ; Mon, 3 Oct 2011 19:54:52 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id AC26E8FC19 for ; Mon, 3 Oct 2011 19:54:51 +0000 (UTC) Received: by wyj26 with SMTP id 26so4797957wyj.13 for ; Mon, 03 Oct 2011 12:54:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=4DMqX6498EqpPLiwBYENJsadgIBI/Tn4qp28CKQ3WU8=; b=TAGLLBygKcp3basZQ8M8OacjGLaRds8WR/YT4Mk82tON/0IbnlZTFVDPGt4hMUqd4N P6J73vR3zuXbRn6Ciz7/upCpw8gxpunMQu81/1oL+sm33uhuHreY1sra+JY0EY8VGzLy nus5tIJVPVylMpOz203SxGSiM16ZNyt6Nzs88= MIME-Version: 1.0 Received: by 10.227.153.211 with SMTP id l19mr376662wbw.51.1317671690082; Mon, 03 Oct 2011 12:54:50 -0700 (PDT) Received: by 10.180.103.33 with HTTP; Mon, 3 Oct 2011 12:54:50 -0700 (PDT) In-Reply-To: References: Date: Mon, 3 Oct 2011 15:54:50 -0400 Message-ID: From: Arnaud Lacombe To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Hackers , Mark Johnston Subject: Re: Installation of kernel symbols file in a separate directory [Was: Re: Experiences with FreeBSD 9.0-BETA2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 19:54:52 -0000 Hi, On Wed, Sep 28, 2011 at 10:30 AM, Ryan Stone wrote: > You might be interested in this PR: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=153157&cat= > > This does the same thing for userland .symbols files, and teaches gdb > how to find them. > Actually, I've made my own implementation of this. It is available here: https://github.com/lacombar/freebsd/commit/e6cb6f2d20ffb6a060124f98c4f4cc660fe6f8bb However, it is a bit more intrusive, as highlighted in the following: https://github.com/lacombar/freebsd/commit/8850b8acf5008c6f9de019ebc3691b090c894605 - Arnaud From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 21:02:02 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B4241065670; Mon, 3 Oct 2011 21:02:02 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 94FCC8FC12; Mon, 3 Oct 2011 21:02:01 +0000 (UTC) Received: by wyj26 with SMTP id 26so4883669wyj.13 for ; Mon, 03 Oct 2011 14:02:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=i6MXMXSLLD2l2uRAIUixvjOhMVXhT41kAGWwc0Bk5vE=; b=mSs1aLzEgxk6nI3ondhR1+W+mPIct7DAUr6cO7BcwUAOkRChlPj+A0A2bRhrSsrAYH ELKQhkdEVx3tAythETA5CU2QUJZqNToW/Wc+sJXh152EMRVDxt69p+DCyWURe8q12045 m4eAPkmGa5XO/HZ4LmT4gCv5+2x7jQ2T3bI44= MIME-Version: 1.0 Received: by 10.227.153.211 with SMTP id l19mr443284wbw.51.1317675720264; Mon, 03 Oct 2011 14:02:00 -0700 (PDT) Received: by 10.180.103.33 with HTTP; Mon, 3 Oct 2011 14:02:00 -0700 (PDT) In-Reply-To: References: <1317653757588-4865200.post@n5.nabble.com> Date: Mon, 3 Oct 2011 17:02:00 -0400 Message-ID: From: Arnaud Lacombe To: mdf@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, satish kondapalli , jhb@FreeBSD.org Subject: Re: NUMA Support is there in FreeBSD. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 21:02:02 -0000 Hi, [Add jhb@ to the CC list] On Mon, Oct 3, 2011 at 1:34 PM, wrote: > On Mon, Oct 3, 2011 at 10:24 AM, Arnaud Lacombe wrot= e: >> Hi, >> >> On Mon, Oct 3, 2011 at 12:31 PM, =A0 wrote: >>> On Mon, Oct 3, 2011 at 7:55 AM, satish kondapalli wrote: >>>> I am new to FreeBSD, I just want know whether FreeBSD supports NUMA. >>>> If FreeBSD supports NUMA what are the kernel API to allocate memory? >>>> is there any example driver or any driver which is using the NUMA API? >>>> >>>> please provide some inputs... >>> >>> The kernel is NUMA-aware (at least for x86), >>> >> What "x86" ? i386 ? amd64 ? both ? > > Both; see sys/x86/acpica/srat.c which parses the SRAT table. > >>> and memory is allocated >>> round-robin amongst the memory domains. =A0There are not yet any KPIs >>> for allocating memory in a specific NUMA domain, nor for binding >>> specific threads / processes to get their memory local to a bound cpu >>> instead of round robin. >>> >> I'm not sure to follow you. Say you have 2 memory domain attached to 2 >> different CPU package, each providing a memory domain, 4 physical core >> and eventually 8 virtual. Say you have a network adapter supporting 8 >> RX/TX queue, dispatching RX packet to 8 netisr. Ideally, you'd want >> those 8 queue/netisr to each have an affinity for a given CPU/memory >> domain, have the network adapter route flow evenly on those those 8 >> CPU. Now, if you allocated an mbuf from memory domain 1, and end up >> being processed by a CPU in domain 0, that likely to introduce >> performance penalty. > > Your statement isn't incorrect. =A0What I'm saying is that there's no > KPI for requesting bound memory because, while the netstat example is > a fine one for where local memory is desired, the majority [1] of > processing is not bound to a CPU and so round-robin allocations will > produce uniform performance results -- that is, not the best possible, > but not wildly fluctuating as scheduling decisions over different runs > give different remote memory penalties. > > [1] for some definition of 'majority'. > >> Now, what about userland ? >> >> This is certainly an horribly big picture :/ > > Yes, and it's why I said just that there's no KPI. =A0One reason there > is no KPI is that there's a lot of fiddly bits to take into account. > > My experience at IBM on AIX was that NUMA is very easy to get wrong; > specifically what one usually wants is for the OS to get the answer > right (especially for userspace) without a lot of manual tuning; > except for some specific applications like netstat queues or a machine > doing HPC or mostly running e.g. an Oracle db server, there's too much > happening for any one program to configure itself "right" for all the > uses of that code. =A0I remember a lot of customer reports of problems > from overly aggressive local memory use. =A0Most of the time no one > complained when things had consistent performance, even if that wasn't > quite as fast as possible. > Is there any project in progress to get this addressed ? In the past year, I can only see 3 commit related to NUMA, one of them being concerning only ia64. Btw, I'd be interested to see how FreeBSD 9.0 and a recent Linux kernel behave on +2 CPU package machines. - Arnaud [0]: http://lwn.net/Articles/254445/ > In fact, I may be wrong about the round-robin; I sent jhb@ a patch and > I have no recollection anymore whether it's actually in CURRENT. =A0It's > been over a year since I thought about this much (BSDCan 2010 was the > last time I remember). > > Cheers, > matthew > From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 3 21:40:00 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B8EC61065672; Mon, 3 Oct 2011 21:40:00 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 4F0058FC12; Mon, 3 Oct 2011 21:40:00 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:906c:6af3:5301:18c6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 5376B4AC1C; Tue, 4 Oct 2011 01:39:58 +0400 (MSD) Date: Tue, 4 Oct 2011 01:39:50 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <176698417.20111004013950@serebryakov.spb.ru> To: mdf@FreeBSD.org In-Reply-To: References: <1317653757588-4865200.post@n5.nabble.com> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, satish kondapalli , Arnaud Lacombe Subject: Re: NUMA Support is there in FreeBSD. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2011 21:40:00 -0000 Hello, Mdf. You wrote 3 =EE=EA=F2=FF=E1=F0=FF 2011 =E3., 21:34:29: > Your statement isn't incorrect. What I'm saying is that there's no > KPI for requesting bound memory because, while the netstat example is > a fine one for where local memory is desired, the majority [1] of > processing is not bound to a CPU and so round-robin allocations will > produce uniform performance results -- that is, not the best possible, > but not wildly fluctuating as scheduling decisions over different runs > give different remote memory penalties. We have exactly the same config at ${WORK}, as Arnaud describes. And we need to process huge (4Gbit+ wire speed in small -- 100-1000 bytes -- packets) UDP traffic. Without fixed affinity of "netisr" threads our system drops some packets on the way between DMA-mapped network card buffers and kernel structures. One big difference: we use Solaris and it have all needed API, KPI and userland control utilities to tune system, both kernel-side and userland-side. Even Solaris, though, could no process such traffic "automagically". We didn't try FreeBSD, as our ops knows nothing about it (I'm only FreeBSD fan in team and I'm developer, not operations)... I wrote this as example, that for some tasks system NEEDS all these NUMA-specific knobs. BTW, NUMA-aware allocator in HotSpot (Sun's, errr, sorry, Oracle's Java VM), added between Java6 and Java7, increased performance for some workloads up to 300% on 72-way system (SunFire 15K), and gives about 3% performance drop on worst situations :) And it was allocator in virtual machine! But it would have been impossible without kernel API, so this changes works well only on HotSpot/Solaris ;-) Again, I wrote this all to show, that NUMA-awareness could be very useful on big iron. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 15:06:27 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CEAE7106564A for ; Tue, 4 Oct 2011 15:06:27 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 915348FC0A for ; Tue, 4 Oct 2011 15:06:27 +0000 (UTC) Received: by gyf2 with SMTP id 2so731762gyf.13 for ; Tue, 04 Oct 2011 08:06:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:from:to:subject:date:content-type :content-transfer-encoding:x-mailer; bh=mRgfx4VgOu8iKChmhMvm/DTHsL/EFEyKk5Uu4dVPhCM=; b=U37QsCWdzUYbJQYW6JblnnOBQTz1hVv3+6dlXlYLUp5LUKefIWG2z5hoBOu6zjjoOY XGc7T7mYPyM6qTfmKBdvTAmQwOIgNlNRsABw5Ag9ZNnNAI4UASnwHEfNw+KaSMAAwGmq H/kRG678nR3EfdRpIqLw/AU2nLzPS1DaUElcw= Received: by 10.223.6.77 with SMTP id 13mr1846846fay.135.1317740786561; Tue, 04 Oct 2011 08:06:26 -0700 (PDT) Received: from DEV ([82.193.208.173]) by mx.google.com with ESMTPS id a1sm25786883fab.4.2011.10.04.08.06.23 (version=SSLv3 cipher=OTHER); Tue, 04 Oct 2011 08:06:24 -0700 (PDT) Message-ID: <20111004.150623.171.1@DEV> From: rank1seeker@gmail.com To: hackers@freebsd.org Date: Tue, 04 Oct 2011 17:06:23 +0200 Content-Type: text/plain; charset="Windows-1250" Content-Transfer-Encoding: quoted-printable X-Mailer: POP Peeper (3.8.0.0) Cc: Subject: Add user to croosworld DESTDIR X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 15:06:27 -0000 When I wana add user into DESTDIR, I chroot into it.=0D=0AHowever, in this = case, DESTDIR has an arch of amd64, while running machine is = i386.=0D=0ASo that isn't an option as all execute attempts would = fail(Exec format error).=0D=0A=0D=0AI've tried with pw's '-V etcdir' flag = and it did created user in amd64 DESTDIR, BUT without home = dir!=0D=0A=0D=0AI've found = this:=0D=0Ahttp://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D2108497+0+archive/2001/freebsd-questions/20011028.freebsd-questions=0D=0A=0D=0AI = did however, managed to writte wrapper function, which created home and = mail layout, as would original FreeBSD installation.=0D=0ABUT, all = $DESTDIR/home/$USER dirs, are owned by 0:0 and I can't find a way to = chown it to DESTDIR user.=0D=0A=0D=0AThat LAST step is all I = need.=0D=0AAnyone has any idea?=0D=0A=0D=0A=0D=0ADomagoj Smol=E8i=E6 From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 15:37:24 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A8201065673 for ; Tue, 4 Oct 2011 15:37:24 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id E657E8FC08 for ; Tue, 4 Oct 2011 15:37:23 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 3C42F1FFC35 for ; Tue, 4 Oct 2011 15:20:24 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 1E22B844E9; Tue, 4 Oct 2011 17:20:24 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: hackers@freebsd.org Date: Tue, 04 Oct 2011 17:20:23 +0200 Message-ID: <86sjn84wco.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Subject: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 15:37:24 -0000 Does anyone actually use nscd? I ask because when I cleaned up a slew of aliasing bugs a couple of years ago, I believe I may have introduced a bug; I got exactly two complaints, and neither of the complainants could be bothered to try the workaround I suggested and report back. Although the code quality is atrocious, nscd is actually a pretty good idea. I suspect the reason why nobody uses it is that it's off by default and people simply don't know about it. Besides nuking it, which would be a shame, we have a range of options, from "just fixing the bug so those who want to use it can" in one end to "finding someone willing to clean it up and maintain it and enable it by default" in the other. (no, I'm not volunteering to maintain it) DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 16:13:37 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC361106564A for ; Tue, 4 Oct 2011 16:13:37 +0000 (UTC) (envelope-from talon@lpthe.jussieu.fr) Received: from shiva.jussieu.fr (shiva.jussieu.fr [134.157.0.129]) by mx1.freebsd.org (Postfix) with ESMTP id 794218FC08 for ; Tue, 4 Oct 2011 16:13:37 +0000 (UTC) Received: from parthe.lpthe.jussieu.fr (parthe.lpthe.jussieu.fr [134.157.10.1]) by shiva.jussieu.fr (8.14.4/jtpda-5.4) with ESMTP id p94G0i27022049 for ; Tue, 4 Oct 2011 18:00:44 +0200 (CEST) X-Ids: 164 Received: from niobe.lpthe.jussieu.fr (niobe.lpthe.jussieu.fr [134.157.10.41]) by parthe.lpthe.jussieu.fr (Postfix) with ESMTP id 35A4D1FEF1 for ; Tue, 4 Oct 2011 18:00:43 +0200 (CEST) Received: by niobe.lpthe.jussieu.fr (Postfix, from userid 2005) id 224474893; Tue, 4 Oct 2011 18:00:43 +0200 (CEST) Date: Tue, 4 Oct 2011 18:00:43 +0200 From: Michel Talon To: hackers@freebsd.org Message-ID: <20111004160043.GA16034@lpthe.jussieu.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Miltered: at jchkmail.jussieu.fr with ID 4E8B2DAC.001 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 4E8B2DAC.001/134.157.10.1/parthe.lpthe.jussieu.fr/parthe.lpthe.jussieu.fr/ X-Mailman-Approved-At: Tue, 04 Oct 2011 16:17:10 +0000 Cc: Subject: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 16:13:37 -0000 Des wrote: > Does anyone actually use nscd? I am using it since a lot of time. I have not experienced annoying bugs in all that time. The last time i have been hit is when installing some new softs which require adding some user and some group with pw. Of course this doesn't work well with caching these data, and i had completely forgotten i was using a cache. This is very perplexing. -- Michel TALON From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 16:25:03 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 790CE106564A for ; Tue, 4 Oct 2011 16:25:03 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from pikmeer.webweaving.org (unknown [IPv6:2001:1af8:4100:b040::20]) by mx1.freebsd.org (Postfix) with ESMTP id 149518FC0A for ; Tue, 4 Oct 2011 16:25:02 +0000 (UTC) Received: from [192.168.1.100] (188.28.49.60.threembb.co.uk [188.28.49.60]) (authenticated bits=0) by pikmeer.webweaving.org (8.14.4/8.14.4) with ESMTP id p94GNPQ1004266 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 4 Oct 2011 16:23:56 GMT (envelope-from dirkx@webweaving.org) Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=us-ascii From: Dirk-Willem van Gulik In-Reply-To: <20111004160043.GA16034@lpthe.jussieu.fr> Date: Tue, 4 Oct 2011 17:26:45 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20111004160043.GA16034@lpthe.jussieu.fr> To: Michel Talon X-Mailer: Apple Mail (2.1244.3) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.5 (pikmeer.webweaving.org [178.18.23.51]); Tue, 04 Oct 2011 16:23:56 +0000 (UTC) Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 16:25:03 -0000 On 4 Oct 2011, at 17:00, Michel Talon wrote: > Des wrote: >> Does anyone actually use nscd? >=20 > I am using it since a lot of time. I have not experienced annoying = bugs > in all that time. The last time i have been hit is when installing = some > new softs which require adding some user and some group with pw. Of > course this doesn't work well with caching these data, and i had > completely forgotten i was using a cache. This is very perplexing. Same here. It just works. And you forget about it. But always beware = that it caches - and that caching is not system wide - but per user. And = I've seen a few cases where I suspect it serialises or otherwise blocks = on barrages of DNS queries. But it is helpful - when the data needs to = come from ldap or whereever. Dw. From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 16:27:44 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1AC61065677 for ; Tue, 4 Oct 2011 16:27:44 +0000 (UTC) (envelope-from sean@coreitpro.com) Received: from masakari.coreitpro.com (masakari.coreitpro.com [38.98.245.188]) by mx1.freebsd.org (Postfix) with ESMTP id 521848FC0A for ; Tue, 4 Oct 2011 16:27:44 +0000 (UTC) Received: from Uller.local (c-76-124-116-189.hsd1.pa.comcast.net [76.124.116.189]) (authenticated bits=0) by masakari.coreitpro.com (8.14.4/8.14.4) with ESMTP id p94Fltl7066026 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Tue, 4 Oct 2011 11:47:55 -0400 (EDT) (envelope-from sean@coreitpro.com) Message-ID: <4E8B2AA6.90204@coreitpro.com> Date: Tue, 04 Oct 2011 11:47:50 -0400 From: "Sean M. Collins" Organization: Core IT Pro User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <86sjn84wco.fsf@ds4.des.no> In-Reply-To: <86sjn84wco.fsf@ds4.des.no> X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 16:27:44 -0000 I've never heard of the utility until you mentioned it. I'd nuke it, since really there are more popular alternatives like Redis and Memcached in the ports tree that most people will reach for first. -- Sean M. Collins From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 16:37:31 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11BC6106566C for ; Tue, 4 Oct 2011 16:37:31 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9A3D88FC08 for ; Tue, 4 Oct 2011 16:37:30 +0000 (UTC) Received: by wyj26 with SMTP id 26so991421wyj.13 for ; Tue, 04 Oct 2011 09:37:29 -0700 (PDT) Received: by 10.216.190.131 with SMTP id e3mr1742572wen.48.1317744404549; Tue, 04 Oct 2011 09:06:44 -0700 (PDT) Received: from dfleuriot-at-hi-media.com ([83.167.62.196]) by mx.google.com with ESMTPS id j18sm4737957wbo.6.2011.10.04.09.06.42 (version=SSLv3 cipher=OTHER); Tue, 04 Oct 2011 09:06:43 -0700 (PDT) Message-ID: <4E8B2F11.8050008@my.gd> Date: Tue, 04 Oct 2011 18:06:41 +0200 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <20111004.150623.171.1@DEV> In-Reply-To: <20111004.150623.171.1@DEV> Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Subject: Re: Add user to croosworld DESTDIR X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 16:37:31 -0000 On 10/4/11 5:06 PM, rank1seeker@gmail.com wrote: > When I wana add user into DESTDIR, I chroot into it. > However, in this case, DESTDIR has an arch of amd64, while running machine is i386. > So that isn't an option as all execute attempts would fail(Exec format error). > > I've tried with pw's '-V etcdir' flag and it did created user in amd64 DESTDIR, BUT without home dir! > > I've found this: > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=2108497+0+archive/2001/freebsd-questions/20011028.freebsd-questions > > I did however, managed to writte wrapper function, which created home and mail layout, as would original FreeBSD installation. > BUT, all $DESTDIR/home/$USER dirs, are owned by 0:0 and I can't find a way to chown it to DESTDIR user. > > That LAST step is all I need. > Anyone has any idea? > Grep the newly added user's UID/GID from $DESTDIR/etc/passwd , chown to these, should work even if they don't exist on your original system. From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 16:51:31 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DF2E2106564A for ; Tue, 4 Oct 2011 16:51:31 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id A03538FC18 for ; Tue, 4 Oct 2011 16:51:31 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id D9C171FFC35; Tue, 4 Oct 2011 16:51:29 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 12CAA845E2; Tue, 4 Oct 2011 18:51:28 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Trond =?utf-8?Q?Endrest=C3=B8l?= References: <86sjn84wco.fsf@ds4.des.no> Date: Tue, 04 Oct 2011 18:51:27 +0200 In-Reply-To: ("Trond =?utf-8?Q?Endrest=C3=B8l=22's?= message of "Tue, 4 Oct 2011 18:19:38 +0200 (CEST)") Message-ID: <86obxw4s4w.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 16:51:32 -0000 Trond Endrest=C3=B8l writes: > It's in daily use at Gj=C3=B8vik Technical College (Fagskolen i Gj=C3=B8v= ik),=20 > here in Norway. Both the mail and web servers authenticates our users=20 > by LDAP, and nscd certainly speeds up the lookups. OK. No trouble with clients dying of SIGPIPE? I could never reproduce the bug, but both users who reported problems used ldap, and I don't have an LDAP server to test against, so I thought it might be specific to LDAP. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 17:04:19 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14B8F106564A for ; Tue, 4 Oct 2011 17:04:19 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 4A0238FC12 for ; Tue, 4 Oct 2011 17:04:17 +0000 (UTC) Received: from julian-mac.elischer.org (home-nat.elischer.org [67.100.89.137]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id p94H2LXg096013 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 4 Oct 2011 10:02:22 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <4E8B3C17.7000902@freebsd.org> Date: Tue, 04 Oct 2011 10:02:15 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.23) Gecko/20110920 Thunderbird/3.1.15 MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> In-Reply-To: <86obxw4s4w.fsf@ds4.des.no> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: hackers@freebsd.org, =?UTF-8?B?VHJvbmQgRW5kcmVzdMO4bA==?= Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 17:04:19 -0000 On 10/4/11 9:51 AM, Dag-Erling Smørgrav wrote: > Trond Endrestøl writes: >> It's in daily use at Gjøvik Technical College (Fagskolen i Gjøvik), >> here in Norway. Both the mail and web servers authenticates our users >> by LDAP, and nscd certainly speeds up the lookups. > OK. No trouble with clients dying of SIGPIPE? I could never reproduce > the bug, but both users who reported problems used ldap, and I don't > have an LDAP server to test against, so I thought it might be specific > to LDAP. > > DES I had never heard of it until now but it looks as though I could have used it several times in the past. We should have people announce new features just like new committers. "Hi, my name is nscd, I cache data that is accessed through the nsswitch system.... etc." From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 17:16:34 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71C1D106564A for ; Tue, 4 Oct 2011 17:16:33 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2FC158FC0A for ; Tue, 4 Oct 2011 17:16:32 +0000 (UTC) Received: by yxk36 with SMTP id 36so904404yxk.13 for ; Tue, 04 Oct 2011 10:16:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=yCRK7gWoAgb/g/JNXMFFqyoOogXiEKmgLFI69kahrWk=; b=CftdcA5Y3mkbYn0gCrMjfLopvHuheZuUU9n+4b9UNlX74PBC8m5qSvUo25eb+uHlRU fPai/kxZUrIbu/kgzCooazKT4zi6EsujCQQYaXJaZQ150j+jl3jRE+vaCp+AhWyVh2Un 5PdhfQwge1ALFGXZiU6YUKn2H4ugQrlBn5DmY= MIME-Version: 1.0 Received: by 10.236.175.164 with SMTP id z24mr7980126yhl.114.1317748592464; Tue, 04 Oct 2011 10:16:32 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.236.103.33 with HTTP; Tue, 4 Oct 2011 10:16:32 -0700 (PDT) In-Reply-To: <86obxw4s4w.fsf@ds4.des.no> References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> Date: Tue, 4 Oct 2011 10:16:32 -0700 X-Google-Sender-Auth: OjhwZU8Axvd2gHj9GZ2TzB-mWTA Message-ID: From: Artem Belevich To: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org, =?ISO-8859-1?Q?Trond_Endrest=F8l?= Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 17:16:34 -0000 2011/10/4 Dag-Erling Sm=F8rgrav : > Trond Endrest=F8l writes: >> It's in daily use at Gj=F8vik Technical College (Fagskolen i Gj=F8vik), >> here in Norway. Both the mail and web servers authenticates our users >> by LDAP, and nscd certainly speeds up the lookups. > > OK. =A0No trouble with clients dying of SIGPIPE? =A0I could never reprodu= ce > the bug, but both users who reported problems used ldap, and I don't > have an LDAP server to test against, so I thought it might be specific > to LDAP. I do use nscd at work where we have fairly large NIS database. And I do have a way to reproduce the SIGPIPE problem. Populate ~30K entries in NIS passwd database, enable nscd and then run top. In my case top used to die with SIGPIPE pretty reliably. I've fixed the issue locally by setting SO_NOSIGPIPE on the socket in __open_cached_connection() in lib/libc/net/nscachedcli.c and I've been running with the fix for few months now. --Artem diff --git a/lib/libc/net/nscachedcli.c b/lib/libc/net/nscachedcli.c index 1323805..cd941db 100644 --- a/lib/libc/net/nscachedcli.c +++ b/lib/libc/net/nscachedcli.c @@ -196,6 +196,7 @@ __open_cached_connection(struct cached_connection_params const *params) struct sockaddr_un client_address; int client_address_len, client_socket; int res; + int on =3D 1; assert(params !=3D NULL); @@ -214,6 +215,8 @@ __open_cached_connection(struct cached_connection_params const *params) } _fcntl(client_socket, F_SETFL, O_NONBLOCK); + _setsockopt(client_socket, SOL_SOCKET, SO_NOSIGPIPE, (void *)&on, sizeof(on)); + retval =3D malloc(sizeof(struct cached_connection_)); assert(retval !=3D NULL); memset(retval, 0, sizeof(struct cached_connection_)); From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 17:39:13 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4CBA1065670 for ; Tue, 4 Oct 2011 17:39:13 +0000 (UTC) (envelope-from jos@catnook.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id A65FD8FC1A for ; Tue, 4 Oct 2011 17:39:13 +0000 (UTC) Received: by vws11 with SMTP id 11so853312vws.13 for ; Tue, 04 Oct 2011 10:39:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.89.36 with SMTP id bl4mr1583636vdb.10.1317748061141; Tue, 04 Oct 2011 10:07:41 -0700 (PDT) Received: by 10.52.188.35 with HTTP; Tue, 4 Oct 2011 10:07:41 -0700 (PDT) In-Reply-To: <4E8B2AA6.90204@coreitpro.com> References: <86sjn84wco.fsf@ds4.des.no> <4E8B2AA6.90204@coreitpro.com> Date: Tue, 4 Oct 2011 10:07:41 -0700 Message-ID: From: Jos Backus To: "Sean M. Collins" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 17:39:14 -0000 On Tue, Oct 4, 2011 at 8:47 AM, Sean M. Collins wrote: > I've never heard of the utility until you mentioned it. > > I'd nuke it, since really there are more popular alternatives like Redis > and Memcached in the ports tree that most people will reach for first. > > Fwiw, nscd serves a somewhat different purpose, at least on Linux. It caches name service switch backend lookups. I can't really imagine one replacing it with Redis or memcached; those tools would require a bunch of work to integrate them with the resolver code in libc/libresolv/whathaveyou. Jos -- > Sean M. Collins > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Jos Backus jos at catnook.com From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 17:32:56 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 076951065673 for ; Tue, 4 Oct 2011 17:32:56 +0000 (UTC) (envelope-from realbushman@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id B5DAE8FC17 for ; Tue, 4 Oct 2011 17:32:55 +0000 (UTC) Received: by qadz30 with SMTP id z30so740249qad.13 for ; Tue, 04 Oct 2011 10:32:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9/45fT419NzVxNLsqOLGimDfOrBS8Pf8On7R83zS2e8=; b=bsIQAdnggRFy8/bCMv/8LUxNkVrm3o1cj5AfRonL+aR8WhSu9BV9nrVFbLc59p/baS 3nceP8Wm8HcCgIHUzMl64AvWq+brJtlnmc0Mk2X4ZlhkL4CCY7CDwvMV7yPHABAucE3j jcXFQmBrR7wToiX1rw3o066+g5JjLwECP1ja0= MIME-Version: 1.0 Received: by 10.229.64.204 with SMTP id f12mr1153311qci.249.1317748141603; Tue, 04 Oct 2011 10:09:01 -0700 (PDT) Received: by 10.229.8.66 with HTTP; Tue, 4 Oct 2011 10:09:01 -0700 (PDT) In-Reply-To: <4E8B2AA6.90204@coreitpro.com> References: <86sjn84wco.fsf@ds4.des.no> <4E8B2AA6.90204@coreitpro.com> Date: Tue, 4 Oct 2011 19:09:01 +0200 Message-ID: From: Michael Bushkov To: "Sean M. Collins" Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Tue, 04 Oct 2011 17:52:20 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 17:32:56 -0000 Hi, Disclaimer: I've written the nscd utility, so I can be a bit biased. On Tue, Oct 4, 2011 at 5:47 PM, Sean M. Collins wrote: > I've never heard of the utility until you mentioned it. > > I'd nuke it, since really there are more popular alternatives like Redis > and Memcached in the ports tree that most people will reach for first. I think you're mixing 2 different things there. nscd is integrated with nsswitch and allows you to cache different kinds of system data (groups, users, etc). IIRC utilities like memcached and redis lack this integration and I don't know any way of hooking them into nsswitch. They're actually just caching backends and you need additional code to make them work with nsswitch. Cheers, Michael > > -- > Sean M. Collins > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 17:54:06 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C923A106564A; Tue, 4 Oct 2011 17:54:06 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 79FE58FC22; Tue, 4 Oct 2011 17:54:06 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id CBA241FFC39; Tue, 4 Oct 2011 17:54:05 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id AFE08845E2; Tue, 4 Oct 2011 19:54:05 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Artem Belevich References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> Date: Tue, 04 Oct 2011 19:54:05 +0200 In-Reply-To: (Artem Belevich's message of "Tue, 4 Oct 2011 10:16:32 -0700") Message-ID: <86fwj84p8i.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org, Trond =?utf-8?Q?Endrest=C3=B8l?= Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 17:54:06 -0000 Artem Belevich writes: > And I do have a way to reproduce the SIGPIPE problem. Populate ~30K > entries in NIS passwd database, enable nscd and then run top. In my > case top used to die with SIGPIPE pretty reliably. I've fixed the > issue locally by setting SO_NOSIGPIPE on the socket in > __open_cached_connection() in lib/libc/net/nscachedcli.c and I've been > running with the fix for few months now. Any chance of getting a backtrace from an unpatched nscd? Ideally with the change described here: http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dbin/136073#reply1 To test, stop nscd, then run it from the command line like so: $ su - # cd /tmp # ulimit -c 0 # /usr/sbin/nscd -nst (do something in another terminal that causes it to crash) # echo backtrace | gdb -batch -x /dev/stdin /usr/sbin/nscd nscd.core and send me the output from both nscd and gdb once it crashes.=20=20 DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 17:56:44 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D62CA106566C for ; Tue, 4 Oct 2011 17:56:44 +0000 (UTC) (envelope-from sean@coreitpro.com) Received: from masakari.coreitpro.com (masakari.coreitpro.com [38.98.245.188]) by mx1.freebsd.org (Postfix) with ESMTP id 848158FC14 for ; Tue, 4 Oct 2011 17:56:44 +0000 (UTC) Received: from Uller.local (c-76-124-116-189.hsd1.pa.comcast.net [76.124.116.189]) (authenticated bits=0) by masakari.coreitpro.com (8.14.4/8.14.4) with ESMTP id p94HugiP070939 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 4 Oct 2011 13:56:42 -0400 (EDT) (envelope-from sean@coreitpro.com) Message-ID: <4E8B48D5.3080307@coreitpro.com> Date: Tue, 04 Oct 2011 13:56:37 -0400 From: "Sean M. Collins" Organization: Core IT Pro User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <86sjn84wco.fsf@ds4.des.no> <4E8B2AA6.90204@coreitpro.com> In-Reply-To: X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: realbushman@gmail.com Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 17:56:44 -0000 Oops - I latched on to the wrong parts of the manpage when I was reading and sent my first message. Thanks for pointing this out. However: How useful is the caching of users and groups? I still believe that for caching DNS, BIND or another DNS server running locally or on the same LAN is the common practice. -- Sean M. Collins From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 18:02:41 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3F80106566B for ; Tue, 4 Oct 2011 18:02:41 +0000 (UTC) (envelope-from realbushman@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7A1D28FC08 for ; Tue, 4 Oct 2011 18:02:41 +0000 (UTC) Received: by qadz30 with SMTP id z30so774855qad.13 for ; Tue, 04 Oct 2011 11:02:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=FIJCK3lEGdmbfRtgNXXz3v5sAlkjVS3ylGdX4C4cJr8=; b=sDaiNRDOkXc9HtLJ1qthVIMTeH8099wJShhwUEfzKZuReN+8HNXuPUNSswi2dIQS5w pDIXR8RLtDLhTOTCBPPH+lbDuLBpzY+QtGG1wm7qZ6GrxstsIVnYKLwAWsi2kvjYH2EG aqa69V7zCubg4PPrAEExm/oS4eGC5TA0UTi4M= MIME-Version: 1.0 Received: by 10.229.242.84 with SMTP id lh20mr1232468qcb.211.1317751360431; Tue, 04 Oct 2011 11:02:40 -0700 (PDT) Sender: realbushman@gmail.com Received: by 10.229.8.66 with HTTP; Tue, 4 Oct 2011 11:02:40 -0700 (PDT) In-Reply-To: <4E8B48D5.3080307@coreitpro.com> References: <86sjn84wco.fsf@ds4.des.no> <4E8B2AA6.90204@coreitpro.com> <4E8B48D5.3080307@coreitpro.com> Date: Tue, 4 Oct 2011 20:02:40 +0200 X-Google-Sender-Auth: 3D4FA1vSz-g4PDKJLYPIRX4PUuA Message-ID: From: Michael Bushkov To: "Sean M. Collins" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 18:02:41 -0000 Users/groups caching can be pretty much useful when you have large LDAP or NIS setup. -- Michael On Tue, Oct 4, 2011 at 7:56 PM, Sean M. Collins wrote: > Oops - I latched on to the wrong parts of the manpage when I was reading > and sent my first message. Thanks for pointing this out. > > However: > > How useful is the caching of users and groups? =A0I still believe that fo= r > caching DNS, BIND or another DNS server running locally or on the same > LAN is the common practice. > > -- > Sean M. Collins > From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 18:14:55 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62500106566C for ; Tue, 4 Oct 2011 18:14:55 +0000 (UTC) (envelope-from sean@coreitpro.com) Received: from masakari.coreitpro.com (masakari.coreitpro.com [38.98.245.188]) by mx1.freebsd.org (Postfix) with ESMTP id 285568FC0A for ; Tue, 4 Oct 2011 18:14:54 +0000 (UTC) Received: from Uller.local (c-76-124-116-189.hsd1.pa.comcast.net [76.124.116.189]) (authenticated bits=0) by masakari.coreitpro.com (8.14.4/8.14.4) with ESMTP id p94IEqf8071684 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 4 Oct 2011 14:14:53 -0400 (EDT) (envelope-from sean@coreitpro.com) Message-ID: <4E8B4D17.4010903@coreitpro.com> Date: Tue, 04 Oct 2011 14:14:47 -0400 From: "Sean M. Collins" Organization: Core IT Pro User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <86sjn84wco.fsf@ds4.des.no> <4E8B2AA6.90204@coreitpro.com> <4E8B48D5.3080307@coreitpro.com> In-Reply-To: X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Michael Bushkov Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 18:14:55 -0000 On 10/4/11 2:02 PM, Michael Bushkov wrote: > Users/groups caching can be pretty much useful when you have large > LDAP or NIS setup. Agreed, and forgive me for hammering you (I freely admit I don't have any significant contributions to FreeBSD) but it would be far more useful if it was cached machine-wide instead of per user, yes? I guess I'm just playing devil's advocate - none the less you have my respect. -- Sean M. Collins From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 18:48:52 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 53038106564A for ; Tue, 4 Oct 2011 18:48:52 +0000 (UTC) (envelope-from faust64@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id F2D4A8FC18 for ; Tue, 4 Oct 2011 18:48:51 +0000 (UTC) Received: by vws11 with SMTP id 11so945609vws.13 for ; Tue, 04 Oct 2011 11:48:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=mcPqaazE0xL7WHXUdmP7QqT8XDSVIYP8jGffoVhV4Po=; b=YSgAaOQ/wTbgAhXuTRNpFKsjjzkdWcn2xOneTiJGJQog5uZEuazhmLkokYhihP0d+X wsyBD5rEyvMIR3hgJlhmXyIjz9mTUl3rIJSm6yegtnapv0gqDjNqoiVMnM22PBy3fZtX Qk6mWarEMOds3Bs9E2TYIwuXZTkmi6W8INTGI= Received: by 10.52.24.109 with SMTP id t13mr1507630vdf.187.1317752645227; Tue, 04 Oct 2011 11:24:05 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.193.138 with HTTP; Tue, 4 Oct 2011 11:23:25 -0700 (PDT) In-Reply-To: <86sjn84wco.fsf@ds4.des.no> References: <86sjn84wco.fsf@ds4.des.no> From: =?ISO-8859-1?Q?Samuel_Mart=EDn_Moro?= Date: Tue, 4 Oct 2011 20:23:25 +0200 Message-ID: To: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 18:48:52 -0000 Using it since a few, almost without any problem. Just one thing: while connected to some external intranet, nscd still try t= o contact my LDAP. So I made a few tests using sssd instead but I'm quite disapointed: even being in my office, I often get the 'authenticated using cached credentials' message (after waiting for 10 seconds, of course... definitely shoud tune timeouts). Why should one choose to use sssd over nscd? Is someone actually using it? Regards. --=20 Samuel Mart=EDn Moro {EPITECH.} SMILE - Open Source Solutions From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 18:51:26 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B0DE106564A for ; Tue, 4 Oct 2011 18:51:26 +0000 (UTC) (envelope-from realbushman@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 1BA478FC12 for ; Tue, 4 Oct 2011 18:51:25 +0000 (UTC) Received: by qadz30 with SMTP id z30so833743qad.13 for ; Tue, 04 Oct 2011 11:51:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=fuLZBNObkJqNvZ3g6xf1G0MsnGDBMEf43MTA+rt1y+U=; b=lNeiEICBX+9M+Qp7//PgDnKlIL5sD8ChvnHDm2ORiurgC9gjPZrtjDLIMtV1vreyZJ V5g45NZC2wAq0an+cxb9nl6LsTo+x7+t2vagRE9plvkmXYcVnO+8kiucid0QIuubPPNv Xi0WBGUGv6kcqoQOBoRjbv1ZYjcqSo2V1Zi3M= MIME-Version: 1.0 Received: by 10.229.64.204 with SMTP id f12mr1257162qci.249.1317754285329; Tue, 04 Oct 2011 11:51:25 -0700 (PDT) Sender: realbushman@gmail.com Received: by 10.229.8.66 with HTTP; Tue, 4 Oct 2011 11:51:25 -0700 (PDT) In-Reply-To: <4E8B4D17.4010903@coreitpro.com> References: <86sjn84wco.fsf@ds4.des.no> <4E8B2AA6.90204@coreitpro.com> <4E8B48D5.3080307@coreitpro.com> <4E8B4D17.4010903@coreitpro.com> Date: Tue, 4 Oct 2011 20:51:25 +0200 X-Google-Sender-Auth: 8sPqYqmIYe4nQwLV-dVbOLcvlk8 Message-ID: From: Michael Bushkov To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 18:51:26 -0000 On Tue, Oct 4, 2011 at 8:14 PM, Sean M. Collins wrote: > On 10/4/11 2:02 PM, Michael Bushkov wrote: >> Users/groups caching can be pretty much useful when you have large >> LDAP or NIS setup. > > Agreed, and forgive me for hammering you (I freely admit I don't have > any significant contributions to FreeBSD) but it would be far more > useful if it was cached machine-wide instead of per user, yes? > No problem ) Technically, yes, but machine-wide cache leads to a security issue - malicious user can poison the global cache with arbitrary data. Cheers, Michael > I guess I'm just playing devil's advocate - none the less you have my > respect. > > -- > Sean M. Collins > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 18:56:39 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96A7F1065675 for ; Tue, 4 Oct 2011 18:56:39 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 57E758FC1B for ; Tue, 4 Oct 2011 18:56:39 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 712C31FFC36; Tue, 4 Oct 2011 18:56:38 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 4304B845E2; Tue, 4 Oct 2011 20:56:38 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Samuel =?utf-8?Q?Mart=C3=ADn?= Moro References: <86sjn84wco.fsf@ds4.des.no> Date: Tue, 04 Oct 2011 20:56:38 +0200 In-Reply-To: ("Samuel =?utf-8?Q?Mart=C3=ADn?= Moro"'s message of "Tue, 4 Oct 2011 20:23:25 +0200") Message-ID: <86botw4mc9.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 18:56:39 -0000 Samuel Mart=C3=ADn Moro writes: > Using it since a few, almost without any problem. Just one thing: > while connected to some external intranet, nscd still try to contact > my LDAP. Well, by default, nscd caches hits for an hour and misses for a minute. One could imagine an option to have nscd return the cached entry if it can't contact the server, even if it has technically expired, but the problem is that nscd doesn't know what the backend is and can't reliably tell whether the lookup failed because the server is unreachable or just because the entry does not exist; and even if it could, it would still have to query the backend every time, so you might still get a longish timeout for every lookup, depending on the type of backend and the reason it failed. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 20:02:20 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91275106564A for ; Tue, 4 Oct 2011 20:02:20 +0000 (UTC) (envelope-from uffe@uffe.org) Received: from mail.starion.dk (mx0.starion.dk [93.162.70.34]) by mx1.freebsd.org (Postfix) with SMTP id C5D408FC0A for ; Tue, 4 Oct 2011 20:02:19 +0000 (UTC) Received: (qmail 33004 invoked by uid 1004); 4 Oct 2011 20:02:28 -0000 Message-ID: <4E8B661A.7080307@uffe.org> Date: Tue, 04 Oct 2011 22:01:57 +0200 From: Uffe Jakobsen X-Mozilla-Draft-Info: internal/draft; vcard=0; receipt=0; DSN=0; uuencode=0 User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110922 Lightning/1.0b2 Mnenhy/0.8.3 Thunderbird/3.1.15 MIME-Version: 1.0 To: Julian Elischer , hackers@freebsd.org References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> <4E8B3C17.7000902@freebsd.org> In-Reply-To: <4E8B3C17.7000902@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 20:02:20 -0000 On 2011-10-04 19:02, Julian Elischer wrote: > On 10/4/11 9:51 AM, Dag-Erling Smørgrav wrote: >> Trond Endrestøl writes: >>> It's in daily use at Gjøvik Technical College (Fagskolen i Gjøvik), >>> here in Norway. Both the mail and web servers authenticates our users >>> by LDAP, and nscd certainly speeds up the lookups. >> OK. No trouble with clients dying of SIGPIPE? I could never reproduce >> the bug, but both users who reported problems used ldap, and I don't >> have an LDAP server to test against, so I thought it might be specific >> to LDAP. >> > I had never heard of it until now but it looks as though I could have used > it several times in the past. > > We should have people announce new features just like new committers. > > "Hi, my name is nscd, I cache data that is accessed through the nsswitch > system.... etc." > FYI: If you've ever used a Solaris box then you've used without knowing it. Solaris has used nscd by default the last 16 years - since Solaris 2.5 And yes it serverely speeds up nameservice lookups on large installations wheter it be plain files, NIS, NIS+, LDAP ot the like... /Uffe From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 4 23:17:26 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DF7391065672 for ; Tue, 4 Oct 2011 23:17:26 +0000 (UTC) (envelope-from brodbd@uw.edu) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 798408FC17 for ; Tue, 4 Oct 2011 23:17:26 +0000 (UTC) Received: by eyg7 with SMTP id 7so1431242eyg.13 for ; Tue, 04 Oct 2011 16:17:25 -0700 (PDT) MIME-Version: 1.0 Received: by 10.213.28.78 with SMTP id l14mr1680971ebc.19.1317768353571; Tue, 04 Oct 2011 15:45:53 -0700 (PDT) Received: by 10.213.10.4 with HTTP; Tue, 4 Oct 2011 15:45:53 -0700 (PDT) In-Reply-To: <20111004160043.GA16034@lpthe.jussieu.fr> References: <20111004160043.GA16034@lpthe.jussieu.fr> Date: Tue, 4 Oct 2011 15:45:53 -0700 Message-ID: From: David Brodbeck To: hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 23:17:27 -0000 On Tue, Oct 4, 2011 at 9:00 AM, Michel Talon wrote: > Des wrote: > > Does anyone actually use nscd? > > I am using it since a lot of time. I have not experienced annoying bugs > in all that time. The last time i have been hit is when installing some > new softs which require adding some user and some group with pw. Of > course this doesn't work well with caching these data, and i had > completely forgotten i was using a cache. This is very perplexing. > It can also be problematic on Samba domain controllers, when adding new machines to the domain -- Samba creates an account for the new machine, then gets confused when a subsequent lookup of that account fails. -- David Brodbeck System Administrator, Linguistics University of Washington From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 03:01:13 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 499391065677 for ; Wed, 5 Oct 2011 03:01:13 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id C64B98FC08 for ; Wed, 5 Oct 2011 03:01:05 +0000 (UTC) Received: from ur.gsoft.com.au (Ur.gsoft.com.au [203.31.81.44]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id p952OvPe095803 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 5 Oct 2011 12:55:03 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=us-ascii From: "Daniel O'Connor" In-Reply-To: <20111004160043.GA16034@lpthe.jussieu.fr> Date: Wed, 5 Oct 2011 12:54:57 +1030 Content-Transfer-Encoding: quoted-printable Message-Id: <051853CE-03EC-4EEC-A5AC-C380131B28E4@gsoft.com.au> References: <20111004160043.GA16034@lpthe.jussieu.fr> To: Michel Talon X-Mailer: Apple Mail (2.1244.3) X-Spam-Score: -4.391 () ALL_TRUSTED,BAYES_00,RP_MATCHES_RCVD X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 03:01:13 -0000 On 05/10/2011, at 2:30, Michel Talon wrote: > Des wrote: >> Does anyone actually use nscd? >=20 > I am using it since a lot of time. I have not experienced annoying = bugs > in all that time. The last time i have been hit is when installing = some > new softs which require adding some user and some group with pw. Of > course this doesn't work well with caching these data, and i had > completely forgotten i was using a cache. This is very perplexing. >=20 In my experience ncsd seems to cache negative hits forever, regardless = of the setting for negative-time-to-live. If I am installing ports which create a new user or group I have to = restart nscd. I also find if openldap dies (not infrequent) I have to = restart nscd after restarting openldap.. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 03:02:42 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AAE54106566C for ; Wed, 5 Oct 2011 03:02:42 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 3228C8FC18 for ; Wed, 5 Oct 2011 03:02:41 +0000 (UTC) Received: from ur.gsoft.com.au (Ur.gsoft.com.au [203.31.81.44]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id p952QJDi095884 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 5 Oct 2011 12:56:25 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=iso-8859-1 From: "Daniel O'Connor" In-Reply-To: <86sjn84wco.fsf@ds4.des.no> Date: Wed, 5 Oct 2011 12:56:19 +1030 Content-Transfer-Encoding: quoted-printable Message-Id: <53BBCF50-2ACB-431B-9EED-0533A3F1BE78@gsoft.com.au> References: <86sjn84wco.fsf@ds4.des.no> To: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1244.3) X-Spam-Score: -4.391 () ALL_TRUSTED,BAYES_00,RP_MATCHES_RCVD X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 03:02:42 -0000 On 05/10/2011, at 1:50, Dag-Erling Sm=F8rgrav wrote: > I ask because when I cleaned up a slew of aliasing bugs a couple of > years ago, I believe I may have introduced a bug; I got exactly two > complaints, and neither of the complainants could be bothered to try = the > workaround I suggested and report back. >=20 > Although the code quality is atrocious, nscd is actually a pretty good > idea. I suspect the reason why nobody uses it is that it's off by > default and people simply don't know about it. Besides nuking it, = which > would be a shame, we have a range of options, from "just fixing the = bug > so those who want to use it can" in one end to "finding someone = willing > to clean it up and maintain it and enable it by default" in the other. >=20 > (no, I'm not volunteering to maintain it) I'd be interested in testing your workaround(s) :) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 05:35:45 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E37B1106566C for ; Wed, 5 Oct 2011 05:35:45 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id AB8C38FC08 for ; Wed, 5 Oct 2011 05:35:45 +0000 (UTC) Received: from lstewart.caia.swin.edu.au (lstewart.caia.swin.edu.au [136.186.229.95]) by lauren.room52.net (Postfix) with ESMTPSA id 3EEC97E84F; Wed, 5 Oct 2011 16:35:44 +1100 (EST) Message-ID: <4E8BECAF.8030204@freebsd.org> Date: Wed, 05 Oct 2011 16:35:43 +1100 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0) Gecko/20110929 Thunderbird/7.0 MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <86sjn84wco.fsf@ds4.des.no> In-Reply-To: <86sjn84wco.fsf@ds4.des.no> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on lauren.room52.net Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 05:35:46 -0000 On 10/05/11 02:20, Dag-Erling Smørgrav wrote: > Does anyone actually use nscd? Yes, particularly for caching LDAP data. > I ask because when I cleaned up a slew of aliasing bugs a couple of > years ago, I believe I may have introduced a bug; I got exactly two > complaints, and neither of the complainants could be bothered to try the > workaround I suggested and report back. I haven't seen these bugs. There is a different bug we hit fairly regularly related to -ve caching. If the machine running nscd loses connectivity with the DNS server for a while and does a DNS lookup during that time, nscd will cache the -ve reply indefinitely for all users, which breaks all sorts of crap. Have to forcibly run "nscd -I all" to fix. I will find and fix this bug one day if noone beats me to it... > Although the code quality is atrocious, nscd is actually a pretty good > idea. I suspect the reason why nobody uses it is that it's off by > default and people simply don't know about it. Besides nuking it, which > would be a shame, we have a range of options, from "just fixing the bug > so those who want to use it can" in one end to "finding someone willing > to clean it up and maintain it and enable it by default" in the other. I'd like to see it stay in base. Moving it (slowly) towards a point where we can turn it on by default would be cool. Cheers, Lawrence From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 08:39:55 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FCB7106566B for ; Wed, 5 Oct 2011 08:39:55 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id C4A478FC12 for ; Wed, 5 Oct 2011 08:39:54 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id C61A91FFC35; Wed, 5 Oct 2011 08:39:53 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id AA632845C9; Wed, 5 Oct 2011 10:39:53 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Lawrence Stewart References: <86sjn84wco.fsf@ds4.des.no> <4E8BECAF.8030204@freebsd.org> Date: Wed, 05 Oct 2011 10:39:53 +0200 In-Reply-To: <4E8BECAF.8030204@freebsd.org> (Lawrence Stewart's message of "Wed, 05 Oct 2011 16:35:43 +1100") Message-ID: <86ipo3hlwm.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 08:39:55 -0000 Lawrence Stewart writes: > If the machine running nscd loses connectivity with the DNS server for > a while and does a DNS lookup during that time, nscd will cache the > -ve reply indefinitely for all users, which breaks all sorts of > crap. Have to forcibly run "nscd -I all" to fix. I will find and fix > this bug one day if noone beats me to it... Definitely a bug, nscd is only supposed to cache negative responses for 60 seconds. I hope you find the time to track it down :) Is it 100% reproducable? How long does the DNS server have to be unreachable before it happens? > I'd like to see it stay in base. Moving it (slowly) towards a point > where we can turn it on by default would be cool. Agreed, in principle. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 08:43:28 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10579106564A for ; Wed, 5 Oct 2011 08:43:28 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id C71E18FC17 for ; Wed, 5 Oct 2011 08:43:27 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id D99CF1FFC37; Wed, 5 Oct 2011 08:43:26 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id C4328845C9; Wed, 5 Oct 2011 10:43:26 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: "Daniel O'Connor" References: <86sjn84wco.fsf@ds4.des.no> <53BBCF50-2ACB-431B-9EED-0533A3F1BE78@gsoft.com.au> Date: Wed, 05 Oct 2011 10:43:26 +0200 In-Reply-To: <53BBCF50-2ACB-431B-9EED-0533A3F1BE78@gsoft.com.au> (Daniel O'Connor's message of "Wed, 5 Oct 2011 12:56:19 +1030") Message-ID: <86ehyrhlqp.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 08:43:28 -0000 "Daniel O'Connor" writes: > I'd be interested in testing your workaround(s) :) It wasn't a workaround, actually, just a one-line change that enables additional logging (when running with from the console -nst) which might help me figure out why it crashes. See my reply to Artem Belevich earlier in this thread. While we're at it, I'd be very grateful if someone could email me a quick and dirty guide to setting up an LDAP server for testing. I have too much on my plate right now to start reading documentation... DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 08:46:12 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C99B6106564A for ; Wed, 5 Oct 2011 08:46:12 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 4FB868FC23 for ; Wed, 5 Oct 2011 08:46:11 +0000 (UTC) Received: from ur.dons.net.au (ppp118-210-240-39.lns20.adl6.internode.on.net [118.210.240.39]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id p958isW9028861 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 5 Oct 2011 19:15:00 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=windows-1252 From: "Daniel O'Connor" In-Reply-To: <86ehyrhlqp.fsf@ds4.des.no> Date: Wed, 5 Oct 2011 19:14:54 +1030 Content-Transfer-Encoding: quoted-printable Message-Id: <21CBFE57-CA9B-4706-A2BA-CD21A2F0E468@gsoft.com.au> References: <86sjn84wco.fsf@ds4.des.no> <53BBCF50-2ACB-431B-9EED-0533A3F1BE78@gsoft.com.au> <86ehyrhlqp.fsf@ds4.des.no> To: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1244.3) X-Spam-Score: 2.162 (**) BAYES_00,KHOP_DYNAMIC,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 08:46:12 -0000 On 05/10/2011, at 19:13, Dag-Erling Sm=F8rgrav wrote: > "Daniel O'Connor" writes: >> I'd be interested in testing your workaround(s) :) >=20 > It wasn't a workaround, actually, just a one-line change that enables > additional logging (when running with from the console -nst) which = might > help me figure out why it crashes. See my reply to Artem Belevich > earlier in this thread. OK I'll dig it up.. > While we're at it, I'd be very grateful if someone could email me a > quick and dirty guide to setting up an LDAP server for testing. I = have > too much on my plate right now to start reading documentation=85 Hmm, most of the heavy lifting for me was done by the net/smbldap-tools = port but it was still fiddly and it was a while ago. Hopefully an LDAP guru can offer more :( -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 09:37:00 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A7E6106564A for ; Wed, 5 Oct 2011 09:37:00 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3fd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id C6D6A8FC12 for ; Wed, 5 Oct 2011 09:36:59 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.187.76.163]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id p959ZddH029908 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 5 Oct 2011 10:35:40 +0100 (BST) (envelope-from m.seaman@infracaninophile.co.uk) X-DKIM: Sendmail DKIM Filter v2.8.3 smtp.infracaninophile.co.uk p959ZddH029908 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infracaninophile.co.uk; s=201001-infracaninophile; t=1317807340; bh=K5x3gdYNmw+iRnsPH7uSkVKq2Ei+LFF283BBW5bo1tw=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Cc:Content-Type:Date:From:In-Reply-To: Message-ID:Mime-Version:References:To; z=Message-ID:=20<4E8C24E3.3010407@infracaninophile.co.uk>|Date:=20W ed,=2005=20Oct=202011=2010:35:31=20+0100|From:=20Matthew=20Seaman= 20|User-Agent:=20Mozilla/5.0=20(M acintosh=3B=20Intel=20Mac=20OS=20X=2010.6=3B=20rv:7.0.1)=20Gecko/2 0110929=20Thunderbird/7.0.1|MIME-Version:=201.0|To:=20=3D?UTF-8?B? RGFnLUVybGluZyBTbcO4cmdyYXY=3D?=3D=20|CC:=20Daniel=20O 'Connor=20,=20hackers@freebsd.org|Subject:= 20Re:=20Does=20anyone=20use=20nscd?|References:=20<86sjn84wco.fsf@ ds4.des.no>=20<53BBCF50-2ACB-431B-9EED-0533A3F1BE78@gsoft.com.au>= 20<86ehyrhlqp.fsf@ds4.des.no>|In-Reply-To:=20<86ehyrhlqp.fsf@ds4.d es.no>|X-Enigmail-Version:=201.3.2|OpenPGP:=20id=3D60AE908C|Conten t-Type:=20multipart/signed=3B=20micalg=3Dpgp-sha1=3B=0D=0A=20proto col=3D"application/pgp-signature"=3B=0D=0A=20boundary=3D"--------- ---enig73C7C15DBDBABA80C406AABD"; b=gQY4I4PzNnBlg+OmQH6cqeblABEyKc0G0PWZpAbmz6FZxpX24K3UL1xCGbwEmn2gm es3TJXXjaecRTyxNcUK2TsSkKwczOEoWcWTnwdAhBzNQhJ0IiOHZBTS+Yrn2cUlv+T DOG7wVCdoFcDbwAOf6oq7MDE/pPCvlf6i0klBtZ8= Message-ID: <4E8C24E3.3010407@infracaninophile.co.uk> Date: Wed, 05 Oct 2011 10:35:31 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <86sjn84wco.fsf@ds4.des.no> <53BBCF50-2ACB-431B-9EED-0533A3F1BE78@gsoft.com.au> <86ehyrhlqp.fsf@ds4.des.no> In-Reply-To: <86ehyrhlqp.fsf@ds4.des.no> X-Enigmail-Version: 1.3.2 OpenPGP: id=60AE908C Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig73C7C15DBDBABA80C406AABD" X-Virus-Scanned: clamav-milter 0.97.2 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-0.1 required=5.0 tests=BAYES_20,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,SPF_FAIL autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 09:37:00 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig73C7C15DBDBABA80C406AABD Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 05/10/2011 09:43, Dag-Erling Sm=C3=B8rgrav wrote: > While we're at it, I'd be very grateful if someone could email me a > quick and dirty guide to setting up an LDAP server for testing. I have= > too much on my plate right now to start reading documentation... The Quick Start guide on the OpenLDAP site is pretty good: http://www.openldap.org/doc/admin24/quickstart.html although steps 1 -- 8 just boil down to 'install from ports' on FreeBSD. Notes: 1) Don't enable SASL -- it adds a lot of complexity but doesn't change anything fundamental in the way LDAP works for testing purposes. 2) The default schema include inetOrgPerson and Posix which is enough to deal with basic Unix users and groups. If you want to do anything more advanced (eg. sudo related or OpenSSH LPK patches) then you'll need to import some external schema. I recommend always copying the schema files into $PREFIX/etc/openldap/schema or else casually removing a port could prevent your slapd from restarting days or weeks later... 3) The structure of an LDAP tree is site-specific and can be quite different between different organizations, but in essence it consists of sorting and grouping various classes of objects into various subdirectories of your directory tree. For testing purposes, impose at least a minimal amount of structure. As the quick start guide suggests, use the dc=3Dexample,dc=3Dcom form based on your domain name to root your= LDAP tree. Within that, create some sub-directories 'ou=3DUsers', 'ou=3DGroups', 'ou=3DHosts' for storing objects of the appropriate types.= This should provide a reasonable parallel to what most people would use in production. 4) ACLs and permissions are pretty complex in LDAP. This is something where you are going to have to spend some quality time with the manuals I'm afraid. 5) phpldapadmin is a pretty good tool for populating a directory with test data. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matthew@infracaninophile.co.uk Kent, CT11 9PW --------------enig73C7C15DBDBABA80C406AABD Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6MJOoACgkQ8Mjk52CukIxQnQCcCo/cp0RKKZEMZkfAqDX3VMwK EjAAn2QUbCANK2rABVsDjoYWc4eEgpMr =1tOB -----END PGP SIGNATURE----- --------------enig73C7C15DBDBABA80C406AABD-- From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 09:48:24 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DA3A106564A for ; Wed, 5 Oct 2011 09:48:24 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 556DB8FC18 for ; Wed, 5 Oct 2011 09:48:23 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id p959kqsq045082; Wed, 5 Oct 2011 16:46:53 +0700 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <4E8C2787.9000009@rdtc.ru> Date: Wed, 05 Oct 2011 16:46:47 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: "Sean M. Collins" References: <86sjn84wco.fsf@ds4.des.no> <4E8B2AA6.90204@coreitpro.com> In-Reply-To: <4E8B2AA6.90204@coreitpro.com> Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Wed, 05 Oct 2011 11:03:42 +0000 Cc: freebsd-hackers@freebsd.org, des@des.no Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 09:48:24 -0000 04.10.2011 22:47, Sean M. Collins ÐÉÛÅÔ: > I've never heard of the utility until you mentioned it. > > I'd nuke it, since really there are more popular alternatives like Redis > and Memcached in the ports tree that most people will reach for first. > Please look at http://www.freebsd.org/cgi/query-pr.cgi?pr=75855 before making final decision. From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 15:12:11 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E404F106564A for ; Wed, 5 Oct 2011 15:12:11 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from agogare.doit.wisc.edu (agogare.doit.wisc.edu [144.92.197.211]) by mx1.freebsd.org (Postfix) with ESMTP id B899E8FC15 for ; Wed, 5 Oct 2011 15:12:11 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from avs-daemon.smtpauth2.wiscmail.wisc.edu by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0LSL00B00M8ATI00@smtpauth2.wiscmail.wisc.edu> for freebsd-hackers@freebsd.org; Wed, 05 Oct 2011 10:12:11 -0500 (CDT) Received: from comporellon.tachypleus.net ([unknown] [76.210.75.72]) by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0LSL008WSM85D700@smtpauth2.wiscmail.wisc.edu> for freebsd-hackers@freebsd.org; Wed, 05 Oct 2011 10:12:06 -0500 (CDT) Date: Wed, 05 Oct 2011 10:12:05 -0500 From: Nathan Whitehorn In-reply-to: <4E86DC86.3040204@FreeBSD.org> To: freebsd-hackers@freebsd.org Message-id: <4E8C73C5.3020809@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=76.210.75.72 X-Spam-PmxInfo: Server=avs-11, Version=5.6.1.2065439, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.10.5.145714, SenderIP=76.210.75.72 References: <4E86DC86.3040204@FreeBSD.org> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0) Gecko/20110928 Thunderbird/7.0 Subject: Re: how are callouts handled in cpu_idle() ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 15:12:12 -0000 On 10/01/11 04:25, Alexander Motin wrote: > Hi. > > Adrian Chadd wrote: >> What happens if this occurs: >> >> * cpu_idle() is entered >> * you enter critical_enter() and call cpu_idleclock() >> * an interrupt occurs, scheduling an immediate callout >> * .. but you've already set the clock register, so it won't be >> serviced until the wait returns. >> >> Perhaps interrupts have to be disabled before critical_enter() and >> cpu_idletick() to ensure an interrupt-driven callout doesn't get >> delayed? > Use of critical section in cpu_idle() from the beginning was based on > number of assumptions about filter interrupt handler's limitations. > These handlers are not guarantied to get updated system time/ticks and > they are discouraged to use callouts. If callout scheduled from > interrupt filter during system wake up, system has no ticks counter > updated yet and you may get callout scheduled into the past (it will be > run immediately) or at least much earlier (up to 250ms) then requested. > In your case callout indeed may get delayed (up to the same 250ms). All > that is not a problem for regular interrupt threaded interrupts -- > interrupt thread execution will be delayed until all stuff get updated. > Can you explain why the critical section is there in more detail? It seems like all of our problems arise because of it. -Nathan From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 15:50:32 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A12901065670 for ; Wed, 5 Oct 2011 15:50:32 +0000 (UTC) (envelope-from ken@mthelicon.com) Received: from hercules.mthelicon.com (hercules.mthelicon.com [66.90.118.40]) by mx1.freebsd.org (Postfix) with ESMTP id 733228FC20 for ; Wed, 5 Oct 2011 15:50:32 +0000 (UTC) Received: from feathers.peganest.com ([89.192.128.16]) (authenticated bits=0) by hercules.mthelicon.com (8.14.3/8.14.3) with ESMTP id p95FaJxh042553 for ; Wed, 5 Oct 2011 15:36:21 GMT (envelope-from ken@mthelicon.com) From: Pegasus Mc Cleaft Organization: Feathers To: freebsd-hackers@freebsd.org Date: Wed, 5 Oct 2011 16:31:32 +0100 User-Agent: KMail/1.13.7 (FreeBSD/10.0-CURRENT; KDE/4.6.5; amd64; ; ) References: <86sjn84wco.fsf@ds4.des.no> <4E8B2AA6.90204@coreitpro.com> <4E8C2787.9000009@rdtc.ru> In-Reply-To: <4E8C2787.9000009@rdtc.ru> MIME-Version: 1.0 Content-Type: Text/Plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Message-Id: <201110051631.32756.ken@mthelicon.com> X-Spam-Status: No, score=-1.1 required=15.0 tests=BAYES_00,RDNS_NONE autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on hercules.mthelicon.com Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 15:50:32 -0000 On Wednesday 05 October 2011 10:46:47 Eugene Grosbein wrote: > 04.10.2011 22:47, Sean M. Collins =D0=C9=DB=C5=D4: > > I've never heard of the utility until you mentioned it. > >=20 > > I'd nuke it, since really there are more popular alternatives like Redis > > and Memcached in the ports tree that most people will reach for first. >=20 I had not used it before I saw this thread.. But now I am 8-) I kind of= =20 like it, actually.. Especially for DNS caching..=20 From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 16:55:09 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF71D106566C for ; Wed, 5 Oct 2011 16:55:09 +0000 (UTC) (envelope-from cattaneo.riccardo@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5B8E38FC13 for ; Wed, 5 Oct 2011 16:55:09 +0000 (UTC) Received: by wyj26 with SMTP id 26so2591301wyj.13 for ; Wed, 05 Oct 2011 09:55:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:content-type:content-transfer-encoding:subject:date:message-id :to:mime-version:x-mailer; bh=/2mDLR9SqIxXj6HVI/+AKJbeh+I6hyPQnfI/PzrBbR4=; b=Zh2L7Sbax49h0EtW7dp2KuCDEBQmWnKOm28xhfktfJreQVrtUUKGB2SrpX7nLgvZew PsIjgEl9xdzh5AvELwcCNgRpufmPRoTugK+WgPn/A2GusuzvTaRnApfIIJY8MvsMaOvt mKDvP9iDRR3ePunfspmOT+K0OsqOoPxq/2EdA= Received: by 10.227.208.20 with SMTP id ga20mr366765wbb.10.1317833708226; Wed, 05 Oct 2011 09:55:08 -0700 (PDT) Received: from [192.168.0.9] (micro.elet.polimi.it. [131.175.127.118]) by mx.google.com with ESMTPS id i29sm3849611wbp.22.2011.10.05.09.54.59 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 05 Oct 2011 09:55:02 -0700 (PDT) From: Riccardo Cattaneo Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Wed, 5 Oct 2011 18:54:57 +0200 Message-Id: <127B21EE-2730-4463-8921-B79CCE2B5ECB@gmail.com> To: freebsd-hackers@freebsd.org Mime-Version: 1.0 (Apple Message framework v1244.3) X-Mailer: Apple Mail (2.1244.3) Subject: 4.4BSD timeslice management X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 16:55:09 -0000 Hello, I'm looking into the code of 4.4BSD scheduler (and related files) of = FreeBSD version 7.2-RELEASE on an amd64 machine in order to understand = how to manage time slices. I'm trying to dynamically change the length = of the time slice on a thread by thread basis to "give more priority" or = "less priority" to threads. In function sched_clock (sched_4bsd.c) I found a piece of code in which = thread's flags are added with TDF_NEEDRESCHED when the time slice = (quantum) expires but I failed to notice any point in the code that = actually preempt the thread. Adding a per-thread parameter (e.g., in struct thread) I could set = TDF_NEEDRESCHED when the dynamic time slice expires but how can I be = sure the thread gets preempted? Where can I set a per-thread parameter = (maybe after cpu_switch?!)? Is there a timer periodically calling a sort = of context switch function or the context switches happen only when = triggered by the kernel code? Thanks Riccardo Cattaneo= From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 16:59:01 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36F361065670; Wed, 5 Oct 2011 16:59:01 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id CA38A8FC12; Wed, 5 Oct 2011 16:59:00 +0000 (UTC) Received: by vws11 with SMTP id 11so2115563vws.13 for ; Wed, 05 Oct 2011 09:58:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=kE6uf3Z9jkJu8UGpwkkg6aoI0JWDpUELHU3NZMvAQu0=; b=BvsUVYGRNG1QmkfKpwTIhA1grPfQC3c0t926IB+EtDYJaqrvBPhtVnQJYQdqOn2J4G A57LobvVZnEwt1Fxi/O7CTyHPa9T8FW0lojtSLysqsR2FDP9Pk0TPCIDhkHdb9jF6ChR YZqqlehHF63eQI22tGRWFeFtQ4CcMvsnajOaM= MIME-Version: 1.0 Received: by 10.52.73.33 with SMTP id i1mr2499123vdv.261.1317833939731; Wed, 05 Oct 2011 09:58:59 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.52.161.138 with HTTP; Wed, 5 Oct 2011 09:58:59 -0700 (PDT) In-Reply-To: <4E8C73C5.3020809@freebsd.org> References: <4E86DC86.3040204@FreeBSD.org> <4E8C73C5.3020809@freebsd.org> Date: Thu, 6 Oct 2011 00:58:59 +0800 X-Google-Sender-Auth: JQdvTJCD9AFQWj-2whSsd16nS3E Message-ID: From: Adrian Chadd To: Nathan Whitehorn Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org Subject: Re: how are callouts handled in cpu_idle() ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 16:59:01 -0000 On 5 October 2011 23:12, Nathan Whitehorn wrote: > Can you explain why the critical section is there in more detail? It seems > like all of our problems arise because of it. It seems to make some of my MIPS cases more problematic. I don't know about the i386. DES reports that his weird/crashing fileserver issues are fixed by setting idletick=1. I'd love to hear from other users who have fixed their HEAD/9.0 issues by setting kernel.eventtimer.idletick=1. I have a feeling that some corner cases in interrupt handling and task scheduling are now (more) exposed due to the timer code in 9. I'm hoping to replicate some issues on my i386 eeepc (which uses ACPI for halting, rather than HLT) and see if that also contributes. Adrian From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 18:03:28 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8125B1065676 for ; Wed, 5 Oct 2011 18:03:28 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3F4FD8FC14 for ; Wed, 5 Oct 2011 18:03:28 +0000 (UTC) Received: by ywp17 with SMTP id 17so2316274ywp.13 for ; Wed, 05 Oct 2011 11:03:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=M9/L5PejkQpal+bqnEhQAb213Ijlr7E3ZTyV6lfNV7c=; b=cEAgKiMfrsjTwor8BzSG9RUydphLyQvLz9L+dsaD8IgQ3Z5Oy01vbNp/cVSzqMuOxM 0MuORVfpS2AVIBREPbM6E1v/KeqfQlXbjcb8844FNiXErfvzYYD6pAinVINRAs1qXRpK s+4woDPa8Iy1mo6oqZFoxYJwXg9Lh2VQV5ECY= MIME-Version: 1.0 Received: by 10.236.186.35 with SMTP id v23mr15532083yhm.80.1317837807265; Wed, 05 Oct 2011 11:03:27 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.236.103.33 with HTTP; Wed, 5 Oct 2011 11:03:27 -0700 (PDT) In-Reply-To: <86fwj84p8i.fsf@ds4.des.no> References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> <86fwj84p8i.fsf@ds4.des.no> Date: Wed, 5 Oct 2011 11:03:27 -0700 X-Google-Sender-Auth: c1GV_fPHohYcJtOdgT0dQX_cbgw Message-ID: From: Artem Belevich To: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org, =?ISO-8859-1?Q?Trond_Endrest=F8l?= Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 18:03:28 -0000 2011/10/4 Dag-Erling Sm=F8rgrav : > Any chance of getting a backtrace from an unpatched nscd? =A0Ideally with > the change described here: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dbin/136073#reply1 > > To test, stop nscd, then run it from the command line like so: > > $ su - > # cd /tmp > # ulimit -c 0 > # /usr/sbin/nscd -nst > (do something in another terminal that causes it to crash) > # echo backtrace | gdb -batch -x /dev/stdin /usr/sbin/nscd nscd.core > > and send me the output from both nscd and gdb once it crashes. In my case it's top that dies with SIGPIPE. nscd keeps running just fine. So, there's no backtrace from nscd. top receives SIGPIPE after it tries to write to the socket with nscd on the other end. Apparently nscd closes connection on its end. Running ktrace on top I see that before the write to nscd socket, there's a read that returned 0 bytes. Here's top's backtrace. Alas I don't have libc with debug symbols handy: Program received signal SIGPIPE, Broken pipe. 0x0000000800abe8cc in write () from /lib/libc.so.7 (gdb) where #0 0x0000000800abe8cc in write () from /lib/libc.so.7 #1 0x0000000800aa3f44 in ftell () from /lib/libc.so.7 #2 0x0000000800aa415f in ftell () from /lib/libc.so.7 #3 0x0000000800aa2031 in __h_errno () from /lib/libc.so.7 #4 0x0000000800a98311 in nsdispatch () from /lib/libc.so.7 #5 0x0000000800a84d95 in getpwent_r () from /lib/libc.so.7 #6 0x0000000800a84911 in acl_get_brand_np () from /lib/libc.so.7 #7 0x0000000000404f7b in machine_init (statics=3D0x7fffffffe770, do_unames=3D1 '\001') at /usr/srcdir/src.git/usr.bin/top/machine.c:258 #8 0x000000000040a9ab in main (argc=3D1, argv=3D0x7fffffffe8c8) at /usr/srcdir/src.git/usr.bin/top/../../contrib/top/top.c:464 --Artem From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 19:23:05 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 54F2C106566B for ; Wed, 5 Oct 2011 19:23:05 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 100628FC12 for ; Wed, 5 Oct 2011 19:23:04 +0000 (UTC) Received: by yxk36 with SMTP id 36so2410708yxk.13 for ; Wed, 05 Oct 2011 12:23:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=vJw6qvrwAafia23Qqrsax7oyLvUt0FSkSLnjnSvbEkc=; b=Bxy/AkM2JXO451eyP0V7G9SmZ6UvAlhV/daRVVBx6JtrLe+YDDsytC+o//PfutoFSu GeRmxwfvEiZxCinZgKjVb1jihL5fm6qg2eMRu23umgmavLKxLVPGRwcMcGUIGNAcyNqX k96TjmZtzBG4MmLIfhKG3QF05wo9EjyFYyGHg= MIME-Version: 1.0 Received: by 10.236.186.35 with SMTP id v23mr15946219yhm.80.1317842584176; Wed, 05 Oct 2011 12:23:04 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.236.103.33 with HTTP; Wed, 5 Oct 2011 12:23:04 -0700 (PDT) In-Reply-To: References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> <86fwj84p8i.fsf@ds4.des.no> Date: Wed, 5 Oct 2011 12:23:04 -0700 X-Google-Sender-Auth: NeoRm2aSfIy5p1ECPsxBfnafAlI Message-ID: From: Artem Belevich To: Michael Bushkov Content-Type: text/plain; charset=ISO-8859-1 Cc: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= , hackers@freebsd.org, =?ISO-8859-1?Q?Trond_Endrest=F8l?= Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 19:23:05 -0000 2011/10/5 Michael Bushkov : > There are probably 2 things here: > 1. There's some error in nsswitch<->nscd communication protocol that > causes nsswitch to write into the closed socket. This is not trivial > to investigate and will require analyzing nscd and client process logs > side by side (and possibly adding some more logging). > 2. Consequences of the aforementioned problem can probably be > corrected by using _setsockopt(..., SO_NOSIGPIPE) in > __open_cached_connection() in nscachedcli.c > (http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/net/nscachedcli.c?rev=1.3). > > I have no access to FreeBSD desktop at the moment - Artem, it would be > cool if you can try the second solution. That's exactly what I did and it fixed the problem on the client side. I've posted the patch setting SO_NOSIGPIPE earlier in this thread: http://lists.freebsd.org/pipermail/freebsd-hackers/2011-October/036539.html --Artem From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 19:27:05 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D4244106566B for ; Wed, 5 Oct 2011 19:27:05 +0000 (UTC) (envelope-from realbushman@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 88DE38FC08 for ; Wed, 5 Oct 2011 19:27:05 +0000 (UTC) Received: by qadz30 with SMTP id z30so2017816qad.13 for ; Wed, 05 Oct 2011 12:27:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=z8rYiqdjq8nIX27KbjS1m1p4obLCUJZzVEIoEHqCLkU=; b=d6BvY2WG4iBvP9KPBPCH31DUP86493SPZgCNvht2o88yQKRwI0oTqqW9OpHY4EPW32 Lx1xDOJEn0LbV9Vh8XF0xpNQ0SeYm0SwwhvZqskIy/W90t7uZUGAMR3TyJAWd9usjYp1 7Y9pVmvmZAS8Ec/CKBT6MzOyOBHSViuUdFvAU= MIME-Version: 1.0 Received: by 10.229.242.84 with SMTP id lh20mr2256062qcb.211.1317841340955; Wed, 05 Oct 2011 12:02:20 -0700 (PDT) Sender: realbushman@gmail.com Received: by 10.229.8.66 with HTTP; Wed, 5 Oct 2011 12:02:17 -0700 (PDT) In-Reply-To: References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> <86fwj84p8i.fsf@ds4.des.no> Date: Wed, 5 Oct 2011 21:02:17 +0200 X-Google-Sender-Auth: Pa8XF0LjD6tY2Ni-nN-3o1uFRIA Message-ID: From: Michael Bushkov To: Artem Belevich Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= , hackers@freebsd.org, =?ISO-8859-1?Q?Trond_Endrest=F8l?= Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 19:27:06 -0000 There are probably 2 things here: 1. There's some error in nsswitch<->nscd communication protocol that causes nsswitch to write into the closed socket. This is not trivial to investigate and will require analyzing nscd and client process logs side by side (and possibly adding some more logging). 2. Consequences of the aforementioned problem can probably be corrected by using _setsockopt(..., SO_NOSIGPIPE) in __open_cached_connection() in nscachedcli.c (http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/net/nscachedcli.c?rev= =3D1.3). I have no access to FreeBSD desktop at the moment - Artem, it would be cool if you can try the second solution. Cheers, Michael 2011/10/5 Artem Belevich : > 2011/10/4 Dag-Erling Sm=F8rgrav : >> Any chance of getting a backtrace from an unpatched nscd? =A0Ideally wit= h >> the change described here: >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dbin/136073#reply1 >> >> To test, stop nscd, then run it from the command line like so: >> >> $ su - >> # cd /tmp >> # ulimit -c 0 >> # /usr/sbin/nscd -nst >> (do something in another terminal that causes it to crash) >> # echo backtrace | gdb -batch -x /dev/stdin /usr/sbin/nscd nscd.core >> >> and send me the output from both nscd and gdb once it crashes. > > In my case it's top that dies with SIGPIPE. nscd keeps running just > fine. So, there's no backtrace from nscd. > > top receives SIGPIPE after it tries to write to the socket with nscd > on the other end. Apparently nscd closes connection on its end. > Running ktrace on top I see that before the write to nscd socket, > there's a read that returned 0 bytes. > > Here's top's backtrace. Alas I don't have libc with debug symbols handy: > > Program received signal SIGPIPE, Broken pipe. > 0x0000000800abe8cc in write () from /lib/libc.so.7 > (gdb) where > #0 =A00x0000000800abe8cc in write () from /lib/libc.so.7 > #1 =A00x0000000800aa3f44 in ftell () from /lib/libc.so.7 > #2 =A00x0000000800aa415f in ftell () from /lib/libc.so.7 > #3 =A00x0000000800aa2031 in __h_errno () from /lib/libc.so.7 > #4 =A00x0000000800a98311 in nsdispatch () from /lib/libc.so.7 > #5 =A00x0000000800a84d95 in getpwent_r () from /lib/libc.so.7 > #6 =A00x0000000800a84911 in acl_get_brand_np () from /lib/libc.so.7 > #7 =A00x0000000000404f7b in machine_init (statics=3D0x7fffffffe770, > do_unames=3D1 '\001') at /usr/srcdir/src.git/usr.bin/top/machine.c:258 > #8 =A00x000000000040a9ab in main (argc=3D1, argv=3D0x7fffffffe8c8) at > /usr/srcdir/src.git/usr.bin/top/../../contrib/top/top.c:464 > > --Artem > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > > > From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 21:41:05 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B64A4106564A; Wed, 5 Oct 2011 21:41:05 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 17BE18FC0C; Wed, 5 Oct 2011 21:41:05 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 255DC1FFC35; Wed, 5 Oct 2011 21:41:04 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 1A3DA845D5; Wed, 5 Oct 2011 23:41:04 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Michael Bushkov References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> <86fwj84p8i.fsf@ds4.des.no> Date: Wed, 05 Oct 2011 23:41:04 +0200 In-Reply-To: (Michael Bushkov's message of "Wed, 5 Oct 2011 21:02:17 +0200") Message-ID: <86wrcjf767.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Artem Belevich , Trond, hackers@freebsd.org, =?utf-8?Q?Endrest=C3=B8l?= Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 21:41:05 -0000 Michael Bushkov writes: > 2. Consequences of the aforementioned problem can probably be > corrected by using _setsockopt(..., SO_NOSIGPIPE) in > __open_cached_connection() in nscachedcli.c That sounds like a workaround rather than a fix... DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 5 22:54:01 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D551106564A; Wed, 5 Oct 2011 22:54:01 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4436B8FC21; Wed, 5 Oct 2011 22:54:00 +0000 (UTC) Received: by gyf2 with SMTP id 2so2619792gyf.13 for ; Wed, 05 Oct 2011 15:54:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=+HtX66Ohi0hIFSl2juMBVwQUbqGYiXeNGBEq6zCxTRc=; b=nNY6XCDY6n3JYympO4OrNOGO1ucHrUL/caG3RUo/sh5oQMiU2UPtQExsVf98ed/U4z LHUM4fFuAlmL8REluaGcly4/pHR0Ie+liVLPr0uSl4ZPWTy0kjAdkO0ljiMDSuUFjtST 5PG6OnIL/NxsKQdj30SFLJ6q4X64c4NU9SjzM= MIME-Version: 1.0 Received: by 10.236.123.43 with SMTP id u31mr18948yhh.97.1317855240520; Wed, 05 Oct 2011 15:54:00 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.236.103.33 with HTTP; Wed, 5 Oct 2011 15:54:00 -0700 (PDT) In-Reply-To: <86wrcjf767.fsf@ds4.des.no> References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> <86fwj84p8i.fsf@ds4.des.no> <86wrcjf767.fsf@ds4.des.no> Date: Wed, 5 Oct 2011 15:54:00 -0700 X-Google-Sender-Auth: sYDSoIZMseBBXLUzIfNhNROKxY4 Message-ID: From: Artem Belevich To: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Michael Bushkov , hackers@freebsd.org, =?ISO-8859-1?Q?Trond_Endrest=F8l?= Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2011 22:54:01 -0000 2011/10/5 Dag-Erling Sm=F8rgrav : > Michael Bushkov writes: >> 2. Consequences of the aforementioned problem can probably be >> corrected by using _setsockopt(..., SO_NOSIGPIPE) in >> __open_cached_connection() in nscachedcli.c > > That sounds like a workaround rather than a fix... Not necessarily. Using SO_NOSIGPIPE is a valid option when someone wants to see read/write on a closed socket fail and return -1 with errno=3DEPIPE. Quick grep in libc shows that resolver code in lib/libc/resolv/res_send.c also sets SO_NOSIGPIPE for exactly that reason. --Artem From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 01:02:24 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B6B7106567C for ; Thu, 6 Oct 2011 01:02:24 +0000 (UTC) (envelope-from trond@fagskolen.gjovik.no) Received: from smtp.fagskolen.gjovik.no (smtp.fagskolen.gjovik.no [128.39.174.11]) by mx1.freebsd.org (Postfix) with ESMTP id 618558FC12 for ; Thu, 6 Oct 2011 01:02:22 +0000 (UTC) Received: from mail.fig.ol.no (localhost [127.0.0.1]) by mail.fig.ol.no (8.14.4/8.14.4) with ESMTP id p94IrRld025521 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 4 Oct 2011 20:53:28 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) Received: from localhost (trond@localhost) by mail.fig.ol.no (8.14.4/8.14.4/Submit) with ESMTP id p94IrReu025518; Tue, 4 Oct 2011 20:53:27 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) X-Authentication-Warning: mail.fig.ol.no: trond owned process doing -bs Date: Tue, 4 Oct 2011 20:53:27 +0200 (CEST) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= Sender: Trond.Endrestol@fagskolen.gjovik.no To: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= In-Reply-To: <86obxw4s4w.fsf@ds4.des.no> Message-ID: References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) Organization: =?ISO-8859-1?Q?Fagskolen_i_Gj=F8vik?= OpenPGP: url=http://fig.ol.no/~trond/trond.key MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="2055831798-642281578-1317754407=:18373" X-Spam-Status: No, score=-2.8 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on mail.fig.ol.no Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 01:02:24 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --2055831798-642281578-1317754407=:18373 Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT On Tue, 4 Oct 2011 18:51+0200, Dag-Erling Smørgrav wrote: > Trond Endrestøl writes: > > It's in daily use at Gjøvik Technical College (Fagskolen i Gjøvik), > > here in Norway. Both the mail and web servers authenticates our users > > by LDAP, and nscd certainly speeds up the lookups. > > OK. No trouble with clients dying of SIGPIPE? I could never reproduce > the bug, but both users who reported problems used ldap, and I don't > have an LDAP server to test against, so I thought it might be specific > to LDAP. Not in my (somewhat limited) experience. Trond. -- ---------------------------------------------------------------------- Trond Endrestøl | Trond.Endrestol@fagskolen.gjovik.no ACM, NAS, NUUG, SAGE, USENIX | FreeBSD 8.2-STABLE & Alpine 2.00 --2055831798-642281578-1317754407=:18373-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 01:05:34 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42CDD106566C for ; Thu, 6 Oct 2011 01:05:34 +0000 (UTC) (envelope-from trond@fagskolen.gjovik.no) Received: from smtp.fagskolen.gjovik.no (smtp.fagskolen.gjovik.no [128.39.174.11]) by mx1.freebsd.org (Postfix) with ESMTP id 52E5E8FC1B for ; Thu, 6 Oct 2011 01:05:32 +0000 (UTC) Received: from mail.fig.ol.no (localhost [127.0.0.1]) by mail.fig.ol.no (8.14.4/8.14.4) with ESMTP id p957cqKF020417 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 5 Oct 2011 09:38:52 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) Received: from localhost (trond@localhost) by mail.fig.ol.no (8.14.4/8.14.4/Submit) with ESMTP id p957cosk020414; Wed, 5 Oct 2011 09:38:52 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) X-Authentication-Warning: mail.fig.ol.no: trond owned process doing -bs Date: Wed, 5 Oct 2011 09:38:50 +0200 (CEST) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= Sender: Trond.Endrestol@fagskolen.gjovik.no To: "Daniel O'Connor" In-Reply-To: <051853CE-03EC-4EEC-A5AC-C380131B28E4@gsoft.com.au> Message-ID: References: <20111004160043.GA16034@lpthe.jussieu.fr> <051853CE-03EC-4EEC-A5AC-C380131B28E4@gsoft.com.au> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) Organization: =?ISO-8859-1?Q?Fagskolen_i_Gj=F8vik?= OpenPGP: url=http://fig.ol.no/~trond/trond.key MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="2055831798-1450607061-1317800332=:18373" X-Spam-Status: No, score=-1.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_40 autolearn=ham version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on mail.fig.ol.no Cc: hackers@freebsd.org, Michel Talon Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 01:05:34 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --2055831798-1450607061-1317800332=:18373 Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT On Wed, 5 Oct 2011 12:54+1030, Daniel O'Connor wrote: > On 05/10/2011, at 2:30, Michel Talon wrote: > > > Des wrote: > >> Does anyone actually use nscd? > > > > I am using it since a lot of time. I have not experienced annoying bugs > > in all that time. The last time i have been hit is when installing some > > new softs which require adding some user and some group with pw. Of > > course this doesn't work well with caching these data, and i had > > completely forgotten i was using a cache. This is very perplexing. > > In my experience ncsd seems to cache negative hits forever, > regardless of the setting for negative-time-to-live. I'm glad to see I'm not the only one who has noticed this odd behaviour of nscd. Shame on me for not speaking up sooner, but I feared I might be proved wrong (again), and yes, that's a lame excuse. :-/ > If I am installing ports which create a new user or group I have to > restart nscd. I also find if openldap dies (not infrequent) I have > to restart nscd after restarting openldap.. After bulk loading ~250 students into our LDAP (Novell eDirectory) each fall, and deleting the graduated students, I restart nscd on our servers just to make sure the caches doesn't contain any negative results. Maybe I should set up a cron job to restart nscd once a day until the source code is cleaned up. Trond. -- ---------------------------------------------------------------------- Trond Endrestøl | Trond.Endrestol@fagskolen.gjovik.no ACM, NAS, NUUG, SAGE, USENIX | FreeBSD 8.2-STABLE & Alpine 2.00 --2055831798-1450607061-1317800332=:18373-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 01:05:35 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E80D106564A for ; Thu, 6 Oct 2011 01:05:35 +0000 (UTC) (envelope-from trond@fagskolen.gjovik.no) Received: from smtp.fagskolen.gjovik.no (smtp.fagskolen.gjovik.no [128.39.174.11]) by mx1.freebsd.org (Postfix) with ESMTP id D53E48FC08 for ; Thu, 6 Oct 2011 01:05:34 +0000 (UTC) Received: from mail.fig.ol.no (localhost [127.0.0.1]) by mail.fig.ol.no (8.14.4/8.14.4) with ESMTP id p94GJcvs023171 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 4 Oct 2011 18:19:38 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) Received: from localhost (trond@localhost) by mail.fig.ol.no (8.14.4/8.14.4/Submit) with ESMTP id p94GJcxE023168; Tue, 4 Oct 2011 18:19:38 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) X-Authentication-Warning: mail.fig.ol.no: trond owned process doing -bs Date: Tue, 4 Oct 2011 18:19:38 +0200 (CEST) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= Sender: Trond.Endrestol@fagskolen.gjovik.no To: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= In-Reply-To: <86sjn84wco.fsf@ds4.des.no> Message-ID: References: <86sjn84wco.fsf@ds4.des.no> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) Organization: =?ISO-8859-1?Q?Fagskolen_i_Gj=F8vik?= OpenPGP: url=http://fig.ol.no/~trond/trond.key MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="2055831798-1033927557-1317745178=:18373" X-Spam-Status: No, score=-2.8 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on mail.fig.ol.no Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 01:05:35 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --2055831798-1033927557-1317745178=:18373 Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT On Tue, 4 Oct 2011 17:20+0200, Dag-Erling Smørgrav wrote: > Does anyone actually use nscd? It's in daily use at Gjøvik Technical College (Fagskolen i Gjøvik), here in Norway. Both the mail and web servers authenticates our users by LDAP, and nscd certainly speeds up the lookups. Trond. -- ---------------------------------------------------------------------- Trond Endrestøl | Trond.Endrestol@fagskolen.gjovik.no ACM, NAS, NUUG, SAGE, USENIX | FreeBSD 8.2-STABLE & Alpine 2.00 --2055831798-1033927557-1317745178=:18373-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 06:00:19 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B44C106566B for ; Thu, 6 Oct 2011 06:00:19 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email2.allantgroup.com (email2.emsphone.com [199.67.51.116]) by mx1.freebsd.org (Postfix) with ESMTP id 360A98FC13 for ; Thu, 6 Oct 2011 06:00:05 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by email2.allantgroup.com (8.14.4/8.14.4) with ESMTP id p965iBYf019079 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 6 Oct 2011 00:44:11 -0500 (CDT) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.5/8.14.5) with ESMTP id p965iBS3026934 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 6 Oct 2011 00:44:11 -0500 (CDT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.5/8.14.5/Submit) id p965iAAg026933; Thu, 6 Oct 2011 00:44:10 -0500 (CDT) (envelope-from dan) Date: Thu, 6 Oct 2011 00:44:10 -0500 From: Dan Nelson To: Trond Endrestol Message-ID: <20111006054409.GS9801@dan.emsphone.com> References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 8.2-STABLE User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.97.2 at email2.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (email2.allantgroup.com [199.67.51.78]); Thu, 06 Oct 2011 00:44:11 -0500 (CDT) X-Scanned-By: MIMEDefang 2.68 on 199.67.51.78 Cc: Dag-Erling Smorgrav , hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 06:00:19 -0000 In the last episode (Oct 04), Trond Endrestol said: > On Tue, 4 Oct 2011 18:51+0200, Dag-Erling Smorgrav wrote: > > Trond Endrestol writes: > > > It's in daily use at Gjovik Technical College (Fagskolen i Gjovik), > > > here in Norway. Both the mail and web servers authenticates our users > > > by LDAP, and nscd certainly speeds up the lookups. > > > > OK. No trouble with clients dying of SIGPIPE? I could never reproduce > > the bug, but both users who reported problems used ldap, and I don't > > have an LDAP server to test against, so I thought it might be specific > > to LDAP. > > Not in my (somewhat limited) experience. On a tangent, I also heavily recommend using the nss-pam-ldapd port instead of nss_ldap. It includes a daemon called nslcd which is the only process that links to the ldap libary. The nss module is a tiny plug that talks to nslcd using a simple protocol. It really reduces the socket count to your ldap server, and removes the potential namespace problems caused by dlopening libldap.so in every process. -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 10:31:06 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 74F5C106566C for ; Thu, 6 Oct 2011 10:31:06 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id F1F9F8FC0A for ; Thu, 6 Oct 2011 10:31:05 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1RBlEG-0000WD-Qs for freebsd-hackers@freebsd.org; Thu, 06 Oct 2011 12:31:04 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 06 Oct 2011 12:31:04 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 06 Oct 2011 12:31:04 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Thu, 06 Oct 2011 12:30:53 +0200 Lines: 52 Message-ID: References: <20111004160043.GA16034@lpthe.jussieu.fr> <051853CE-03EC-4EEC-A5AC-C380131B28E4@gsoft.com.au> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigBA91D3F6C65E1DD9045A1A7A" X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111004 Thunderbird/7.0.1 In-Reply-To: X-Enigmail-Version: 1.1.2 Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 10:31:06 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigBA91D3F6C65E1DD9045A1A7A Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 05/10/2011 09:38, Trond Endrest=C3=B8l wrote: > On Wed, 5 Oct 2011 12:54+1030, Daniel O'Connor wrote: >=20 >> On 05/10/2011, at 2:30, Michel Talon wrote: >> >>> Des wrote: >>>> Does anyone actually use nscd? >>> >>> I am using it since a lot of time. I have not experienced annoying bu= gs >>> in all that time. The last time i have been hit is when installing so= me >>> new softs which require adding some user and some group with pw. Of >>> course this doesn't work well with caching these data, and i had >>> completely forgotten i was using a cache. This is very perplexing. >> >> In my experience ncsd seems to cache negative hits forever,=20 >> regardless of the setting for negative-time-to-live. >=20 > I'm glad to see I'm not the only one who has noticed this odd=20 > behaviour of nscd. Shame on me for not speaking up sooner, but I=20 > feared I might be proved wrong (again), and yes, that's a lame excuse. = > :-/ +1. It's very annoying when installing ports which add users - the port adds it then in some future code checks it and it fails. I've noticed it with at least CUPS. --------------enigBA91D3F6C65E1DD9045A1A7A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6Ng10ACgkQldnAQVacBchoOACbBnyx6jw2hoJBkzVa9qU4c0KQ WA0AnR4YDKKB7RJo5WGNq6rKIFgf5VD9 =XewQ -----END PGP SIGNATURE----- --------------enigBA91D3F6C65E1DD9045A1A7A-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 13:11:39 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0585B1065673 for ; Thu, 6 Oct 2011 13:11:39 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 26F898FC0A for ; Thu, 6 Oct 2011 13:11:36 +0000 (UTC) Received: from lstewart1.loshell.room52.net (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id EF2537E8C8; Fri, 7 Oct 2011 00:11:32 +1100 (EST) Message-ID: <4E8DA904.5070907@freebsd.org> Date: Fri, 07 Oct 2011 00:11:32 +1100 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0.2) Gecko/20110914 Thunderbird/6.0.2 MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <86sjn84wco.fsf@ds4.des.no> <53BBCF50-2ACB-431B-9EED-0533A3F1BE78@gsoft.com.au> <86ehyrhlqp.fsf@ds4.des.no> In-Reply-To: <86ehyrhlqp.fsf@ds4.des.no> Content-Type: multipart/mixed; boundary="------------050705030408060200020104" X-Spam-Status: No, score=0.4 required=5.0 tests=FILL_THIS_FORM_FRAUD_PHISH, T_FILL_THIS_FORM_SHORT,UNPARSEABLE_RELAY autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on lauren.room52.net Cc: hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 13:11:39 -0000 This is a multi-part message in MIME format. --------------050705030408060200020104 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 10/05/11 19:43, Dag-Erling Smørgrav wrote: [snip] > > While we're at it, I'd be very grateful if someone could email me a > quick and dirty guide to setting up an LDAP server for testing. I have > too much on my plate right now to start reading documentation... A bit dated, but this build guide is reasonably complete and should pretty much all still be relevant. It has some bits you won't need if you're just setting something up for quick and dirty testing - I think they'll be obvious as you're working through it. Cheers, Lawrence --------------050705030408060200020104 Content-Type: text/plain; name="ldap_server_buildguide_20081212.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="ldap_server_buildguide_20081212.txt" Ci0gQmFzZWQgbW9zdGx5IG9uIGluZm9ybWF0aW9uIGZyb20gdGhlc2UgcGxhY2VzIChhbW9u ZyBtYW55IG90aGVycyEpOgotIGh0dHA6Ly93d3cuZnJlZWJzZC5vcmcvZG9jL2VuL2FydGlj bGVzL2xkYXAtYXV0aC9pbmRleC5odG1sCi0gaHR0cDovL3d3dy5vcGVubGRhcC5vcmcvZG9j L2FkbWluMjQvCgoKIyMgVVNFRlVMIFJBTkRPTSBUSURCSVRTICMjCgotIFRoZXNlIHBlYXJs cyBvZiB3aXNkb20gYXJlIHVzZWZ1bCBkdXJpbmcgc2V0IHVwIGFuZCBkZWJ1Z2dpbmcKCgot IFVzZSB0aGUgInNsYXBjYXQiIHRvb2wgb24gdGhlIHNlcnZlciBjb21tYW5kIGxpbmUgdG8g c2hvdyB5b3UgdGhlIGNvbnRlbnRzCi0gb2YgdGhlIGxkYXAgZGF0YWJhc2UgaW4gbGRpZiBm b3JtYXQuIFRoaXMgaXMgdXNlZnVsIGZvciByb2xsaW5nIGJhY2t1cHMuCi0gSGF2ZSBhIGNy b24gam9iIGNhbGwgc2xhcGNhdCBldmVyeSAzMCBtaW5zIGFuZCB3cml0ZSB0aGUgb3V0cHV0 IG92ZXIgdGhlIHRvcAotIG9mIGEgZmlsZSB3aGljaCBpcyBpbmNyZW1lbnRhbGx5IGJhY2tl ZCB1cAoKLSBFeGFtcGxlIG9mIHVwZGF0aW5nIHRoZSAidGVzdCIgdXNlcidzIHBhc3N3b3Jk IGFzIHRoZSBEQiBhZG1pbgpsZGFwcGFzc3dkIC14IC1TIC1IIGxkYXA6Ly8xMjcuMC4wLjEg LUQgJ2NuPXJvb3QsZGM9ZXhhbXBsZSxkYz1jb20nIC1XICd1aWQ9dGVzdCxvdT1wZW9wbGUs ZGM9ZXhhbXBsZSxkYz1jb20nCgoKLSBMREFQIGRhdGEgaXMgb2Z0ZW4gc3RvcmVkIGJhc2U2 NCBlbmNvZGVkIChlLmcuIHVzZXJQYXNzd29yZCBmaWVsZCkuIFVzZSB0aGlzIHRvIGRlY29k ZToKZWNobyAiZTBOU1dWQlVmU1F4SkV0NVQzRnhZbWRsSkRRZWMzVlRMblpVWTIxclR6UkdX RU5CVlZCVE1qRT0iIHwgcGVybCAtTU1JTUU6OkJhc2U2NCAtbmUgJ3ByaW50IGRlY29kZV9i YXNlNjQoJF8pIC4gIlxuIjsnCgotVG8gbWFudWFsbHkgdXBkYXRlIGFuIGV4aXN0aW5nIGVu dHJ5J3MgYXR0cmlidXRlCmxkYXBtb2RpZnkgLXggLUggbGRhcDovLzEyNy4wLjAuMSAtRCAn Y249cm9vdCxkYz1leGFtcGxlLGRjPWNvbScgLVcgLWYgdGVzdHVzZXIubGRpZgotIFdpdGgg dGVzdHVzZXIubGRpZiBjb250ZW50czoKZG46IHVpZD10ZXN0LG91PXBlb3BsZSxkYz1leGFt cGxlLGRjPWNvbQpjaGFuZ2V0eXBlOiBtb2RpZnkKcmVwbGFjZTogdXNlclBhc3N3b3JkCnVz ZXJQYXNzd29yZDoge2NyeXB0fSQxJHBVWHlzVFVaJDk3cjI3TDZGRTIxTkR0WmROVVlSYzEK CgotVG8gZGVsZXRlIGFuIGV4aXN0aW5nIGVudHJ5CmxkYXBtb2RpZnkgLXggLUggbGRhcDov LzEyNy4wLjAuMSAtRCAnY249cm9vdCxkYz1leGFtcGxlLGRjPWNvbScgLVcgLWYgdGVzdHVz ZXIubGRpZgotIFdpdGggdGVzdHVzZXIubGRpZiBjb250ZW50czoKZG46IHVpZD10ZXN0LG91 PXBlb3BsZSxkYz1leGFtcGxlLGRjPWNvbQpjaGFuZ2V0eXBlOiBkZWxldGUKCi0gRW5hYmxl IHRoZSBtb25pdG9yIGJhY2tlbmQgaW4gc2xhcGQuY29uZiBhbmQgcnVuIHRoZSBmb2xsb3dp bmcgcXVlcnkgdG8gZ2V0IG9wZXJhdGlvbmFsIGluZm8gYWJvdXQgc2xhcGQKbGRhcHNlYXJj aCAteCAtSCBsZGFwOi8vMTI3LjAuMC4xIC1iICdjbj1Nb25pdG9yJyArCgotIENoYW5nZSB0 aGUgImxvZ2xldmVsIiBkaXJlY3RpdmUgaW4gc2xhcGQuY29uZiB0byAiY29ubnMgZmlsdGVy IHN0YXRzIiB0byBnZXQgdXNlZnVsIGRlYnVnZ2luZyBpbmZvIGluIC92YXIvbG9nL2RlYnVn LmxvZwoKLSBTZXR0aW5nICJkZWJ1ZyAxIiBpbiBwYW1fbGRhcC5jb25mIGFuZC9vciBuc3Nf bGRhcC5jb25mIHdpbGwgZ2l2ZSB5b3Ugc29tZSB1c2VmdWwgZGVidWdnaW5nIGluZm8gb24g dGhlIGNsaWVudCBzaWRlIGluIC92YXIvbG9nL2RlYnVnLmxvZwoKLSAKIyMgRU5EIFVTRUZV TCBSQU5ET00gVElEQklUUyAjIwoKCgoKCgoKCiMjIFNFUlZFUiBDT05GSUcgIyMKCi0gQWRk IHRoZSBmb2xsb3dpbmcgbGluZXMgdG8gL2V0Yy9tYWtlLmNvbmYgdG8gbWFrZSB0aGUgRnJl ZUJTRCBwb3J0cyBzeXN0ZW0KLSB1c2UgTERBUCAyLjQueCBmb3IgcG9ydHMgdGhhdCByZXF1 aXJlIExEQVAgKGRlZmF1bHQgdXNlcyAyLjMueCkKCiMgVXNlIE9wZW5MREFQIDIuNC54CldB TlRfT1BFTkxEQVBfVkVSPTI0CgoKCi0gSW5zdGFsbCBvcGVubGRhcCBzZXJ2ZXIKY2QgL3Vz ci9wb3J0cy9uZXQvb3BlbmxkYXAyNC1zZXJ2ZXIKCm1ha2UgaW5zdGFsbCBjbGVhbgoKLSBT ZWxlY3Qgb3B0aW9uczogU0FTTCwgUEFTU1dELCBQRVJMLCBPREJDLCBUQ1BfV1JBUFBFUlMs IEJEQiwgU0VRTU9ELCBTWU5DUFJPViwgRFlOQU1JQ19CQUNLRU5EUwoKCi0gQ3JlYXRlIGEg Y2VydGlmaWNhdGUva2V5IGZvciBlbmNyeXB0ZWQgY29tbXMgd2l0aCB0aGUgTERBUCBzZXJ2 ZXIKCm1rZGlyIC91c3IvbG9jYWwvY2VydHMKCm9wZW5zc2wgcmVxIC1uZXcgLXg1MDkgLWRh eXMgMTgyNSAtbm9kZXMgLW91dCAvdXNyL2xvY2FsL2NlcnRzL2xkYXBzZXJ2ZXIucGVtIC1r ZXlvdXQgL3Vzci9sb2NhbC9jZXJ0cy9sZGFwc2VydmVyLnBlbQoKLSBUaGUgZGlhbG9nIHdp dGggb3BlbnNzbCBzaG91bGQgZ28gc29tZXRoaW5nIGxpa2UgdGhpczoKCkNvdW50cnkgTmFt ZSAoMiBsZXR0ZXIgY29kZSkgW0FVXTpBVQpTdGF0ZSBvciBQcm92aW5jZSBOYW1lIChmdWxs IG5hbWUpIFtTb21lLVN0YXRlXTpWaWN0b3JpYQpMb2NhbGl0eSBOYW1lIChlZywgY2l0eSkg W106TWVsYm91cm5lCk9yZ2FuaXphdGlvbiBOYW1lIChlZywgY29tcGFueSkgW0ludGVybmV0 IFdpZGdpdHMgUHR5IEx0ZF06IE9yZwpPcmdhbml6YXRpb25hbCBVbml0IE5hbWUgKGVnLCBz ZWN0aW9uKSBbXTogT3JnIFVuaXQKQ29tbW9uIE5hbWUgKGVnLCBZT1VSIG5hbWUpIFtdOmxk YXBzZXJ2ZXIuYmxhaC5jb20uYXUKRW1haWwgQWRkcmVzcyBbXTp5b3VydXNlckBibGFoLmNv bS5hdQoKUGxlYXNlIGVudGVyIHRoZSBmb2xsb3dpbmcgJ2V4dHJhJyBhdHRyaWJ1dGVzCnRv IGJlIHNlbnQgd2l0aCB5b3VyIGNlcnRpZmljYXRlIHJlcXVlc3QKQSBjaGFsbGVuZ2UgcGFz c3dvcmQgW106CkFuIG9wdGlvbmFsIGNvbXBhbnkgbmFtZSBbXToKCgojLSBDaGVjayB0aGF0 IHRoZSBiYXNpYyBkZXRhaWxzIGFyZSBhbGwgZ29vZApvcGVuc3NsIHg1MDkgLXN1YmplY3Qg LWRhdGVzIC1maW5nZXJwcmludCAtbm9vdXQgLWluIC91c3IvbG9jYWwvY2VydHMvbGRhcHNl cnZlci5wZW0KCiMtIFNldCBwZXJtcyBvbiBzZW5zaXRpdmUgZmlsZXMKI2NyZWF0ZSAnY2Vy dHMnIGdyb3VwIGFuZCBhZGQgbGRhcCBhbmQgd3d3IHVzZXJzOgoKcHcgZ3JvdXBhZGQgY2Vy dHMKcHcgZ3JvdXBtb2QgY2VydHMgLU0gd3d3IGxkYXAKCmNobW9kIDA0NDAgL3Vzci9sb2Nh bC9jZXJ0cy9sZGFwc2VydmVyLnBlbQpjaGdycCBjZXJ0cyAvdXNyL2xvY2FsL2NlcnRzL2xk YXBzZXJ2ZXIucGVtCgojLSBDb25maWd1cmUgc2xhcGQsIHRoZSBsZGFwIHNlcnZlcgoKbWtk aXIgL3Zhci9kYi9zbGFwZApjaG93biBsZGFwOmxkYXAgL3Zhci9kYi9zbGFwZAoKY3AgL3Vz ci9sb2NhbC9ldGMvb3BlbmxkYXAvREJfQ09ORklHLmV4YW1wbGUgL3Zhci9kYi9zbGFwZC9E Ql9DT05GSUcKCmNkIC91c3IvbG9jYWwvZXRjL29wZW5sZGFwCgoKCmVkaXQgc2xhcGQuY29u ZgoKLSBNYWtlIHRoZSBmaWxlIGxvb2sgbGlrZSB0aGlzCgojIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCmluY2x1ZGUgICAgICAgICAv dXNyL2xvY2FsL2V0Yy9vcGVubGRhcC9zY2hlbWEvY29yZS5zY2hlbWEKaW5jbHVkZSAgICAg ICAgIC91c3IvbG9jYWwvZXRjL29wZW5sZGFwL3NjaGVtYS9jb3NpbmUuc2NoZW1hCmluY2x1 ZGUgICAgICAgICAvdXNyL2xvY2FsL2V0Yy9vcGVubGRhcC9zY2hlbWEvaW5ldG9yZ3BlcnNv bi5zY2hlbWEKaW5jbHVkZSAgICAgICAgIC91c3IvbG9jYWwvZXRjL29wZW5sZGFwL3NjaGVt YS9uaXMuc2NoZW1hICAgICAgICAgIAppbmNsdWRlICAgICAgICAgL3Vzci9sb2NhbC9zaGFy ZS9leGFtcGxlcy9zYW1iYS9MREFQL3NhbWJhLnNjaGVtYSAgCgpsb2dsZXZlbCAgICAgICAg Y29ubnMgZmlsdGVyIHN0YXRzCiNsb2dsZXZlbCAgICAgICBub25lCnBpZGZpbGUgICAgICAg ICAvdmFyL3J1bi9vcGVubGRhcC9zbGFwZC5waWQKYXJnc2ZpbGUgICAgICAgIC92YXIvcnVu L29wZW5sZGFwL3NsYXBkLmFyZ3MKbW9kdWxlcGF0aCAgICAgIC91c3IvbG9jYWwvbGliZXhl Yy9vcGVubGRhcAptb2R1bGVsb2FkICAgICAgYmFja19iZGIKZGVmYXVsdHNlYXJjaGJhc2Ug ImRjPWJsYWgsZGM9Y29tLGRjPWF1IgpwYXNzd29yZC1oYXNoICAge0NSWVBUfQpwYXNzd29y ZC1jcnlwdC1zYWx0LWZvcm1hdCAiJDEkJS44cyIKc2VjdXJpdHkgc3NmPTEyOApUTFNDZXJ0 aWZpY2F0ZUZpbGUgL3Vzci9sb2NhbC9jZXJ0cy9sZGFwc2VydmVyLnBlbQpUTFNDZXJ0aWZp Y2F0ZUtleUZpbGUgL3Vzci9sb2NhbC9jZXJ0cy9sZGFwc2VydmVyLnBlbQpUTFNDQUNlcnRp ZmljYXRlRmlsZSAvdXNyL2xvY2FsL2NlcnRzL2xkYXBzZXJ2ZXIucGVtCgpkYXRhYmFzZSAg ICAgICAgYmRiCmRpcmVjdG9yeSAgICAgICAvdmFyL2RiL3NsYXBkCnN1ZmZpeCAgICAgICAg ICAiZGM9YmxhaCxkYz1jb20sZGM9YXUiCnJvb3RkbiAgICAgICAgICAiY249cm9vdCxkYz1i bGFoLGRjPWNvbSxkYz1hdSIKcm9vdHB3ICAgICAgICAgIDxwYXNzd29yZCBmcm9tIG5leHQg c3RlcD4KY2hlY2twb2ludCAgICAgIDAgNQppbmRleCAgICAgICAgICAgb2JqZWN0Q2xhc3Mg ICAgIGVxCmluZGV4ICAgICAgICAgICB1aWQgICAgICAgICAgICAgZXEKaW5kZXggICAgICAg ICAgIHVpZE51bWJlciAgICAgICBlcQppbmRleCAgICAgICAgICAgZ2lkTnVtYmVyICAgICAg IGVxCgphY2Nlc3MgdG8gZG4uYmFzZT0ic2FtYmFEb21haW5OYW1lPUxEQVBTRVJWRVIsb3U9 c2FtYmEsZGM9YmxhaCxkYz1jb20sZGM9YXUiCiBieSBkbi5iYXNlPSJ1aWQ9c2FtYmEsb3U9 YWRtaW5zLGRjPWJsYWgsZGM9Y29tLGRjPWF1IiB3cml0ZQoKYWNjZXNzIHRvIGF0dHJzPXVz ZXJQYXNzd29yZCxzYW1iYU5UUGFzc3dvcmQsc2FtYmFMTVBhc3N3b3JkLHNhbWJhUGFzc3dv cmRIaXN0b3J5LHNhbWJhUHdkTGFzdFNldCxzYW1iYUFjY3RGbGFncwogYnkgc2VsZiA9eHcK IGJ5IGRuLmJhc2U9InVpZD1zYW1iYSxvdT1hZG1pbnMsZGM9YmxhaCxkYz1jb20sZGM9YXUi IHdyaXRlCiBieSAqIGF1dGgKCmFjY2VzcyB0byAqCiBieSBzZWxmIHdyaXRlCiBieSAqIHJl YWQKCgojIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjCgojLSBDcmVhdGUgYSBzbGFwZCByb290IHBhc3N3b3JkLCByZXBsYWNpbmcgInRl c3QiIHdpdGggcmVhbCBwYXNzd29yZApzbGFwcGFzc3dkIC1zICJ0ZXN0IgoKLSBUaGlzIHdp bGwgc3BpdCBhIHN0cmluZyBvdXQgdG8gdGhlIGNvbnNvbGUgdGhhdCBsb29rcyBsaWtlICJ7 U1NIQX1MNGtxM0MxVFdDcFZuV0txOG9lSHRaVjg4M2xqaVZpTSIKLSBDb3B5IHRoZSBlbnRp cmUgc3RyaW5nIGFuZCBwYXN0ZSBpdCBpbnRvIHRoZSBzbGFwZCBjb25maWcgZmlsZSBhdCB0 aGUgInJvb3RwdyIgbGluZQoKCgoKIyAtIGVkaXQgL3Vzci9sb2NhbC9ldGMvb3BlbmxkYXAv bGRhcC5jb25mIC0gYWRkIGZvbG93aW5nIGxpbmU6CgplY2hvICJUTFNfUkVRQ0VSVCBhbGxv dyIgPj4gL3Vzci9sb2NhbC9ldGMvb3BlbmxkYXAvbGRhcC5jb25mCgojLSBNYWtlIGxkYXAg c2VydmVyIHN0YXJ0IG9uIGJvb3QKI2VkaXQgL2V0Yy9yYy5jb25mCiMtIEFkZCB0aGUgZm9s bG93aW5nIGxpbmUKCmVjaG8gJ3NsYXBkX2VuYWJsZT1cIllFU1wiJyA+PiAvZXRjL3JjLmNv bmYKCi0gU3RhcnQgdGhlIHNlcnZlciBub3cKL3Vzci9sb2NhbC9ldGMvcmMuZC9zbGFwZCBz dGFydAoKCgoKIyMgQkVHSU4gT1BUSU9OQUwgU0VSVkVSIENPTkZJRyAjIwoKLSBJbnN0YWxs IFBIUCBMREFQIGFkbWluIChwaHBsZGFwYWRtaW4tMS4xLjAuNV8yLDEpCgpjZCAvdXNyL3Bv cnRzL3d3dy9hcGFjaGUyMgptYWtlIGluc3RhbGwgY2xlYW4KCi0gc2VsZWN0IGRlZmF1bHQg b3B0aW9ucwoKY2QgL3Vzci9wb3J0cy9uZXQvcGhwbGRhcGFkbWluCm1ha2UgaW5zdGFsbCBj bGVhbgoKLSBzZWxlY3Qgb3B0aW9uczogQ0xJLCBDR0ksIEFQQUNIRSwgU1VIT1NJTiwgTVVM VElCWVRFLCBJUFY2LCBGQVNUQ0dJLCBQQVRISU5GTyBmb3IgcGhwNS01LjIuNgoKLSBNYWtl IGFwYWNoZSBoYW5kbGUgLnBocCBmaWxlcyB2aWEgdGhlIHBocCBpbnRlcnByZXRlcgoKZWRp dCAvdXNyL2xvY2FsL2V0Yy9hcGFjaGUyMi9odHRwZC5jb25mCgotIEFkZCB0aGUgZm9sbG93 aW5nIGxpbmVzIHVuZGVyIHRoZSBleGlzdGluZyBsaW5lICJBZGRUeXBlIGFwcGxpY2F0aW9u L3gtZ3ppcCAuZ3ogLnRneiIKCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMKQWRkVHlwZSBhcHBsaWNhdGlvbi94LWh0dHBkLXBocCAu cGhwCkFkZFR5cGUgYXBwbGljYXRpb24veC1odHRwZC1waHAtc291cmNlIC5waHBzCiMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKCi0g VW5jb21tZW50IHRoZSBmb2xsb3dpbmcgbGluZQoKI0luY2x1ZGUgZXRjL2FwYWNoZTIyL2V4 dHJhL2h0dHBkLXNzbC5jb25mCgoKCgotIENvbmZpZ3VyZSBTU0wvVExTCgplZGl0IC91c3Iv bG9jYWwvZXRjL2FwYWNoZTIyL2V4dHJhL2h0dHBkLXNzbC5jb25mCgotIENvbW1lbnQgb3V0 IHRoZSBmb2xsb3dpbmcgbGluZXMKI1NlcnZlck5hbWUgd3d3LmV4YW1wbGUuY29tOjQ0Mwoj U2VydmVyQWRtaW4geW91QGV4YW1wbGUuY29tCgotIENoYW5nZSB0aGUgZm9sbG93aW5nIGxp bmVzIGFwcHJvcHJpYXRlbHkKClNTTENlcnRpZmljYXRlRmlsZSAiL3Vzci9sb2NhbC9jZXJ0 cy9sZGFwc2VydmVyLnBlbSIKU1NMQ2VydGlmaWNhdGVLZXlGaWxlICIvdXNyL2xvY2FsL2Nl cnRzL2xkYXBzZXJ2ZXIucGVtIgoKCgoKCi0gTWFrZSBhcGFjaGUgc2VydmUgcGhwbGRhcGFk bWluIHZpYSBIVFRQUwoKZWRpdCAvdXNyL2xvY2FsL2V0Yy9hcGFjaGUyMi9JbmNsdWRlcy9w aHBsZGFwYWRtaW4uY29uZgoKLSBBZGQgdGhlIGZvbGxvd2luZyBsaW5lczoKCiMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKUmV3cml0 ZUVuZ2luZSBvbgpSZXdyaXRlQ29uZCAgICV7U0VSVkVSX1BPUlR9ICAhNDQzJApSZXdyaXRl UnVsZSBeLz8ocGhwbGRhcGFkbWluLy4qKSBodHRwczovLyV7U0VSVkVSX05BTUV9LyQxIFtS PTMwMSxMXQoKQWxpYXMgL3BocGxkYXBhZG1pbiAvdXNyL2xvY2FsL3d3dy9waHBsZGFwYWRt aW4vaHRkb2NzCgo8RGlyZWN0b3J5IC91c3IvbG9jYWwvd3d3L3BocGxkYXBhZG1pbj4KICAg IE9yZGVyIGFsbG93LGRlbnkKICAgIEFsbG93IGZyb20gYWxsCiAgICBEaXJlY3RvcnlJbmRl eCBpbmRleC5waHAKPC9EaXJlY3Rvcnk+CiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKCgotIENvbmZpZ3VyZSBwaHBsZGFwYWRtaW4K CmVkaXQgL3Vzci9sb2NhbC93d3cvcGhwbGRhcGFkbWluL2NvbmZpZy9jb25maWcucGhwCgot IE1ha2UgdGhlIGZpbGUgbG9vayBsaWtlIHRoaXM6CgojIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCjw/cGhwCiRpPTA7CiRsZGFwc2Vy dmVycyA9IG5ldyBMREFQU2VydmVyczsKJGxkYXBzZXJ2ZXJzLT5TZXRWYWx1ZSgkaSwnc2Vy dmVyJywnbmFtZScsJ0xEQVAgU2VydmVyJyk7CiRsZGFwc2VydmVycy0+U2V0VmFsdWUoJGks J3NlcnZlcicsJ2hvc3QnLCcxMjcuMC4wLjEnKTsKJGxkYXBzZXJ2ZXJzLT5TZXRWYWx1ZSgk aSwnc2VydmVyJywnYXV0aF90eXBlJywnc2Vzc2lvbicpOwokbGRhcHNlcnZlcnMtPlNldFZh bHVlKCRpLCdzZXJ2ZXInLCd0bHMnLHRydWUpOwokbGRhcHNlcnZlcnMtPlNldFZhbHVlKCRp LCdzZXJ2ZXInLCdzYXNsX2F1dGgnLGZhbHNlKTsKCiRmcmllbmRseV9hdHRycyA9IGFycmF5 KCk7CiRmcmllbmRseV9hdHRyc1snZmFjc2ltaWxlVGVsZXBob25lTnVtYmVyJ10gPSAnRmF4 JzsKJGZyaWVuZGx5X2F0dHJzWyd0ZWxlcGhvbmVOdW1iZXInXSAgICAgICAgICA9ICdQaG9u ZSc7CiRmcmllbmRseV9hdHRyc1sndWlkJ10gICAgICAgICAgICAgICAgICAgICAgPSAnVXNl ciBOYW1lJzsKCiRxPTA7CiRxdWVyaWVzID0gYXJyYXkoKTsKJHF1ZXJpZXNbJHFdWyduYW1l J10gPSAnVXNlciBMaXN0JzsKJHF1ZXJpZXNbJHFdWydiYXNlJ10gPSAnZGM9YmxhaCxkYz1j b20sZGM9YXUnOwokcXVlcmllc1skcV1bJ3Njb3BlJ10gPSAnc3ViJzsKJHF1ZXJpZXNbJHFd WydmaWx0ZXInXSA9ICcoJihvYmplY3RDbGFzcz1wb3NpeEFjY291bnQpKHVpZD0qKSknOwok cXVlcmllc1skcV1bJ2F0dHJpYnV0ZXMnXSA9ICdjbiwgdWlkLCBob21lRGlyZWN0b3J5LCB0 ZWxlcGhvbmVudW1iZXIsIGpwZWdwaG90byc7Cj8+CiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKCgotIE1ha2UgYXBhY2hlIHN0YXJ0 IG9uIGJvb3QKCmVkaXQgL2V0Yy9yYy5jb25mCgotIEFkZCB0aGUgZm9sbG93aW5nIGxpbmUK CmFwYWNoZTIyX2VuYWJsZT0iWUVTIgoKCi0gU3RhcnQgYXBhY2hlCgphcGFjaGVjdGwgc3Rh cnQKCgotIFlvdSBzaG91bGQgbm93IGJlIGFibGUgdG8gYWNjZXNzIGh0dHA6Ly88aG9zdG5h bWU+L3BocGxkYXBhZG1pbiBhbmQgbG9naW4gd2l0aCB5b3VyCi0gTERBUCBzZXJ2ZXIncyBy b290ZG4gY3JlZGVudGlhbHMKLSBlLmcuIHVzZXJuYW1lICJjbj1yb290LGRjPWJsYWgsZGM9 Y29tLGRjPWF1IiBhbmQgcGFzc3dvcmQgaXMgdGhlIGNsZWFyIHRleHQgdmVyc2lvbgotIHlv dSB0eXBlZCBpbnRvIHRoZSBzbGFwcGFzc3dkIGNvbW1hbmQgcHJldmlvdXNseSB0byBnZW5l cmF0ZSB0aGUgcm9vdHB3IG9wdGlvbgoKLSAiSW1wb3J0IiBhY3Rpb24gaW4gcGhwbGRhcGFk bWluIGNhbiBiZSB1c2VkIHRvIHJ1biBhbiBsZGlmIGZpbGUgYWdhaW5zdCB0aGUgc2VydmVy CgoKIyMgRU5EIE9QVElPTkFMIFNFUlZFUiBDT05GSUcgIyMKCgoKCgoKLSBDcmVhdGUgYW4g bGRpZiBmaWxlIHRvIHNlZWQgdGhlIExEQVAgZGIgd2l0aCBvdXIgcmVxdWlyZWQgc3RydWN0 dXJlCi0gV2UnbGwgdXNlICJwZW9wbGUiIGFzIHRoZSBPVSBmb3IgdXNlciBhY2NvdW50cyBh bmQgImdyb3VwcyIgYXMgdGhlIE9VIGZvciBncm91cHMKCmVkaXQgL3Vzci9sb2NhbC9ldGMv b3BlbmxkYXAvaW5pdGxkYXAubGRpZgoKLSBzZWVkIGxkaWYgZmlsZSBjb250ZW50czoKCiMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMK I29yZwpkbjogZGM9YmxhaCxkYz1jb20sZGM9YXUKb2JqZWN0Q2xhc3M6IGRjT2JqZWN0Cm9i amVjdENsYXNzOiBvcmdhbml6YXRpb24KZGM6IG9yZwpvOiBPUkcgTkFNRQpkZXNjcmlwdGlv bjogQmxhaAoKIyBDcmVhdGUgYW4gT1UgZm9yIHBlb3BsZQpkbjogb3U9cGVvcGxlLGRjPWJs YWgsZGM9Y29tLGRjPWF1Cm91OiBwZW9wbGUKb2JqZWN0Q2xhc3M6IHRvcApvYmplY3RDbGFz czogb3JnYW5pemF0aW9uYWxVbml0CgojIENyZWF0ZSBhbiBPVSBmb3IgZ3JvdXBzCmRuOiBv dT1ncm91cHMsZGM9YmxhaCxkYz1jb20sZGM9YXUKb2JqZWN0Q2xhc3M6IHRvcApvYmplY3RD bGFzczogb3JnYW5pemF0aW9uYWxVbml0Cm91OiBncm91cHMKIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIwoKCi0gQXBwbHkgdGhlIGNv bnRlbnRzIG9mIHRoZSBsZGlmIGZpbGUgYWdhaW5zdCB0aGUgREIgKG9yIHVzZSB3ZWIgaW50 ZXJmYWNlKQoKbGRhcGFkZCAtYyAteCAtZiAvdXNyL2xvY2FsL2V0Yy9vcGVubGRhcC9pbml0 bGRhcC5sZGlmIC1IIGxkYXA6Ly8xMjcuMC4wLjEgLUQgJ2NuPXJvb3QsZGM9YmxhaCxkYz1j b20sZGM9YXUnIC1XIC1aWgoKCiMjIEVORCBTRVJWRVIgQ09ORklHICMjCgoKCgoKCgoKCgoK CgoKCgojIyBDTElFTlQgQ09ORklHICMjCgotIEFkZCB0aGUgZm9sbG93aW5nIGxpbmVzIHRv IC9ldGMvbWFrZS5jb25mIHRvIG1ha2UgdGhlIEZyZWVCU0QgcG9ydHMgc3lzdGVtCi0gdXNl IExEQVAgMi40LnggZm9yIHBvcnRzIHRoYXQgcmVxdWlyZSBMREFQIChkZWZhdWx0IHVzZXMg Mi4zLngpCgojIFVzZSBPcGVuTERBUCAyLjQueApXQU5UX09QRU5MREFQX1ZFUj0yNAoKCi0g SW5zdGFsbCByZXF1aXJlZCBwb3J0cwoKClBhc3RlIHRoZSBmb2xsb3dpbmcgcGF0Y2ggaW50 byB0aGUgZmlsZSBzaG93biBiZWxvdzoKL3Vzci9wb3J0cy9zZWN1cml0eS9wYW1fbGRhcC9m aWxlcy9wYXRjaC1ncm91cGRuZml4CiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKCi0tLSBwYW1fbGRhcC5jLm9yaWcg ICAgIDIwMDgtMDYtMjcgMTc6NTE6MzcuMDAwMDAwMDAwICsxMDAwCisrKyBwYW1fbGRhcC5j ICAyMDA4LTA2LTI3IDE3OjQ1OjExLjAwMDAwMDAwMCArMTAwMApAQCAtNDAyMCw3ICs0MDIw LDcgQEAKICAgICB7CiAgICAgICByYyA9IGxkYXBfY29tcGFyZV9zIChzZXNzaW9uLT5sZCwK ICAgICAgICAgICAgICAgICAgICAgICAgICAgc2Vzc2lvbi0+Y29uZi0+Z3JvdXBkbiwKLSAg ICAgICAgICAgICAgICAgICAgICAgICAgc2Vzc2lvbi0+Y29uZi0+Z3JvdXBhdHRyLCBzZXNz aW9uLT5pbmZvLT51c2VyZG4pOworICAgICAgICAgICAgICAgICAgICAgICAgICBzZXNzaW9u LT5jb25mLT5ncm91cGF0dHIsIHNlc3Npb24tPmluZm8tPnVzZXJuYW1lKTsKICAgICAgIGlm IChyYyAhPSBMREFQX0NPTVBBUkVfVFJVRSkKICAgICAgICB7CiAgICAgICAgICBzbnByaW50 ZiAoYnVmLCBzaXplb2YgYnVmLCAiWW91IG11c3QgYmUgYSAlcyBvZiAlcyB0byBsb2dpbi4i LAoKIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIwoKY2QgL3Vzci9wb3J0cy9zZWN1cml0eS9wYW1fbGRhcAptYWtlIGlu c3RhbGwgY2xlYW4KCmNkIC91c3IvcG9ydHMvbmV0L25zc19sZGFwCm1ha2UgaW5zdGFsbCBj bGVhbgoKY2QgL3Vzci9wb3J0cy9zZWN1cml0eS9wYW1fbWtob21lZGlyCm1ha2UgV0lUSF9Q Uk9GSUxFPXllcyBpbnN0YWxsIGNsZWFuCgoKLSBDb25maWd1cmUgcGFtIHRvIGF0dGVtcHQg bGRhcCBhdXRoIGJlZm9yZSBmYWxsaW5nIGJhY2sgb24gbG9jYWwgZmlsZSBhdXRoCgplZGl0 IC9ldGMvcGFtLmQvc3NoZAoKLSBBZGQgdGhlIGZvbGxvd2luZyBsaW5lIGJlZm9yZSAiYXV0 aCByZXF1aXJlZCBwYW1fdW5peC5zbyBub193YXJuIHRyeV9maXJzdF9wYXNzIgoKYXV0aCAg ICAgICAgICAgIHN1ZmZpY2llbnQgICAgICAvdXNyL2xvY2FsL2xpYi9wYW1fbGRhcC5zbyAg ICAgIG5vX3dhcm4gY29uZmlnPS91c3IvbG9jYWwvZXRjL3BhbV9sZGFwLmNvbmYgdHJ5X2Zp cnN0X3Bhc3MKCgotIEFkZCB0aGUgZm9sbG93aW5nIGxpbmUgYmVmb3JlICJhY2NvdW50IHJl cXVpcmVkIHBhbV91bml4LnNvIgphY2NvdW50ICAgICAgICAgc3VmZmljaWVudCAgICAgICAg L3Vzci9sb2NhbC9saWIvcGFtX2xkYXAuc28gICAgICBub193YXJuIGlnbm9yZV9hdXRoaW5m b191bmF2YWlsIGlnbm9yZV91bmtub3duX3VzZXIKCgotIEFkZCB0aGUgZm9sbG93aW5nIGxp bmUgYmVmb3JlICJzZXNzaW9uIHJlcXVpcmVkIHBhbV9wZXJtaXQuc28iCgpzZXNzaW9uICAg ICAgICAgcmVxdWlyZWQgICAgICAgIC91c3IvbG9jYWwvbGliL3BhbV9ta2hvbWVkaXIuc28K CgoKLSBNYWtlIHN1cmUgL2hvbWUgZXhpc3RzIGVpdGhlciBhcyBhIHN5bWxpbmsgdG8gYSBk aXJlY3Rvcnkgb3IgYW4gYWN0dWFsIGRpcmVjdG9yeQoKZWRpdCAvdXNyL2xvY2FsL2V0Yy9w YW1fbGRhcC5jb25mCgotIE1ha2UgdGhlIGZpbGUgY29udGVudHMgdGhpczoKCiMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKdXJpIGxk YXA6Ly9sZGFwc2VydmVyLmJsYWguY29tLmF1CmJhc2UgZGM9YmxhaCxkYz1jb20sZGM9YXUK c3NsIHN0YXJ0X3Rscwp0bHNfY2hlY2twZWVyIG5vCiNkZWJ1ZyAxCiMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKCgotIENvbmZpZ3Vy ZSBuc3MgdG8gdXNlIGxkYXAgdG8gcmVzb2x2ZSB1aWQvZ2lkIGluZm8KCmVkaXQgL2V0Yy9u c3N3aXRjaC5jb25mCgotIENoYW5nZSB0aGUgZm9sbG93aW5nIGxpbmVzIGFjY29yZGluZ2x5 Cgpncm91cDogZmlsZXMgbGRhcApwYXNzd2Q6IGZpbGVzIGxkYXAKCi0gbnNzX2xkYXAgaXMg d3JpdHRlbiBieSB0aGUgYXV0aG9ycyBvZiBwYW1fbGRhcCBhbmQgdGhleSBjYW4gc2hhcmUg Y29uZmlnIGZpbGVzCmxuIC1zIC91c3IvbG9jYWwvZXRjL3BhbV9sZGFwLmNvbmYgL3Vzci9s b2NhbC9ldGMvbnNzX2xkYXAuY29uZgoKLSBJZiBuc3MgaXMgd29ya2luZywgInRvdWNoIHRl c3QuZmlsZSA7IGNob3duIGxkYXB1aWQ6bGRhcGdpZCB0ZXN0LmZpbGUiIHdoZXJlICJsZGFw dWlkIiBpcyBhIHVpZCBvZiBhIHVzZXIgaW4gbGRhcCB0aGF0IGlzbid0Ci0gaW4gdGhlIGxv Y2FsIHN5c3RlbSdzIC9ldGMvcGFzc3dkIGZpbGUgYW5kICJsZGFwZ2lkIiBpcyBhIGdpZCBv ZiBhIGdyb3VwIGluIGxkYXAgdGhhdCBpc24ndAotIGluIHRoZSBsb2NhbCBzeXN0ZW0ncyAv ZXRjL2dyb3VwIGZpbGUKLSBUaGVuIHJ1biAibHMgLWwgdGVzdC5maWxlIiBhbmQgeW91IHNo b3VsZCBzZWUgdGhlIHVzZXIgbmFtZSBhbmQgZ3JvdXAgbmFtZSBkZXRhaWxzIGZyb20gbGRh cCBuZXh0IHRvIHRoZSBmaWxlCgoKCgojIyMgTG9jYWwgbG9naW5zClRoZSBvcmlnaW5hbCBj b25maWcgb25seSBzZXRzIHVwIGxkYXAgYXV0aCBmb3IgU1NIIGxvZ2lucwpBZGQgc3VwcG9y dCBmb3IgbG9jYWwgTERBUCBsb2dpbnMgaWYgcmVxdWlyZWQ6CmVkaXQgL2V0Yy9wYW0uZC9s b2dpbiB3aXRoIHRoZSBzYW1lIG9wdGlvbnMgYXMgeW91IGRpZCBmb3IgL2V0Yy9wYW0uZC9z c2hkIHRvIGFsbG93IGxvY2FsIGxkYXAgbG9naW5zCgoKCgojIyMgQWRkaW5nIG5zc3dpdGNo IGNhY2hpbmcgd2l0aCBuc2NkCldpdGhvdXQgY2FjaGluZywgYW4gb2ZmbGluZSBMREFQIHNl cnZlciByZW5kZXJzIHRoZSB3b3Jrc3RhdGlvbiBhbG1vc3QgdXNlbGVzcywgYXMgZWFjaCB0 aW1lIGEgdXNlcm5hbWUgaXMgcmVxdWlyZWQsIGVnICJscyAtYWwiIHRoZSBzeXN0ZW0gd2ls bCB0cnkgYW5kIGNvbm5lY3QgbyB0aGUgTERBUCBzZXJ2ZXIgdG8gcmVzb2x2ZSB0aGUgbmFt ZS4KClRoZSBzb2x1dGlvbiBpcyB0aGUgTmFtZSBTZXJ2aWNlIENhY2hpbmcgRGFlbW9uLiBU aGlzIGJhc2ljIGNvbmZpZ3VyYXRpb24gd2lsbCBtZWFuIHRoYXQgb25jZSBsb2dnZWQgaW50 byBhIG1hY2hpbmUgKGF1dGggaXMgZG9uZSkgdWlkL3VzZXJuYW1lIHBhaXJzIHdpbGwgYmUg Y2FjaGVkIGxvY2FsbHkuIAoKCmNvbmZpcm0gL2V0Yy9uc2NkLmNvbmYgY29udGFpbnMgdGhl IGZvbGxvd2luZwoKXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl4KZW5h YmxlLWNhY2hlIHBhc3N3ZCB5ZXMKZW5hYmxlLWNhY2hlIGdyb3VwIHllcwplbmFibGUtY2Fj aGUgaG9zdHMgeWVzCmVuYWJsZS1jYWNoZSBzZXJ2aWNlcyB5ZXMKZW5hYmxlLWNhY2hlIHBy b3RvY29scyB5ZXMKZW5hYmxlLWNhY2hlIHJwYyB5ZXMKZW5hYmxlLWNhY2hlIG5ldHdvcmtz IHllcwoKXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl4KVGhpcyB3aWxs IGNvbmZpZ3VyZSBuc2NkIHRvIGNhY2hlIGV2ZXJ5dGhpbmcgaXQgY2FuLgpCeSBkZWZhdWx0 LCBjYWNoZXMgYXJlIGtlcHQgZm9yIDEgaG91ci4KVGhpcyBjYW4gYmUgY2hhbmdlZCBieSBh ZGRpbmcgYW4gZW50cnkgYXMgYmVsb3cuCkEgbGluZSBpcyBuZWVkZWQgZm9yIGVhY2ggY2hh Y2hlIHR5cGUgeW91IHdvdWxkIGxpa2UgdG8gYWx0ZXIuCl5eXl5eXl5eXl5eXl5eXl5eXl5e Xl5eXl5eXl5eXl5eXl5eXl5eCnBvc2l0aXZlLXRpbWUtdG8tbGl2ZSBwYXNzd2QgNzIwMAoK Xl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl4KCgplZGl0IC9ldGMvcmMu Y29uZgphZGQ6CgpeXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXgpuc2Nk X2VuYWJsZT0iWUVTIgoKCl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5e CmVkaXQgL2V0Yy9uc3Nzd2l0Y2guY29uZgphZGQgJ2NhY2hlJyBhcyB0aGUgZmlyc3Qgc291 cmNlIGZvciBncm91cCwgaG9zdHMsIHBhc3N3ZCwgc2VydmljZXMsIHByb3RvY2FvbHMsIHJw YyBhbmQgbmV0d29ya3MuCgpUaGlzIHdpbGwgZW5hYmxlIGNhY2hpbmcgb2YgdXNlcnMgYW5k IHBhc3N3b3JkcyBmcm9tIGxkYXAgYXMgd2VsbCBhcyBETlMgaG9zdCBsb29rdXBzCgoKCgoK IyMjIExEQVAgUGFzc3dvcmQgY2hhbmdlcyB3aXRoICdwYXNzd2QnCmVkaXQgL3Vzci9sb2Nh bC9ldGMvcGFtX2xkYXAuY29uZgoKXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5e Xl5eXl4KYWRkICJwYW1fcGFzc3dkIGV4b3AiCgpeXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5e Xl5eXl5eXl5eXl5eXgpub3RlOiBXaXRoIHRoZSAnZXhvcCcgb3B0aW9uLCB0aGUgcGFzc3dv cmQgaXMgc2VudCBpbiBjbGVhciB0ZXh0IChCVVQgaXQgaXMgb3ZlciBUTFMgc28gd2lsbCBz dGlsbCBiZSBzZWN1cmUpIGFuZCB1cG9uIHJlY2VpdmluZyB0aGUgcGFzc3dvcmQsIHRoZSBz ZXJ2ZXIgZW5jcnlwdHMgaXQgaW50byB0aGUgcHJvcGVyIGZvcm1hdC4gJ2NyeXB0JyBhbHNv IHdvcmtzLCBhbmQgc2VuZHMgYW4gZW5jcnlwdGVkIHBhc3N3b3JkIHRvIGxkYXAsIGJ1dCBp dCBpcyBpZGVhbCB0byBrZWVwIHBhc3N3b3JkcyBpbiBleGFjdGx5IHRoZSBzYW1lIGZvcm1h dCBpbiBsZGFwIGFzIGlzIGtlcHQgaW4gYSBsb2NhdCBtYXN0ZXIucGFzc3dkIGZpbGUuCgpU aGUgc3lzdGVtIHBhc3N3b3JkIGNvbW1hbmQgaXMgdmVyeSBvbGQgYW5kIGlzbid0IGF3YXJl IHRoYXQgcGFzc3dvcmRzIGNhbiBiZSBjaGFuZ2VkIGZvciBMREFQIHVzZXJzLgpXZSB0aGVy ZWZvcmUgaGF2ZSB0byBhZGQgYSBoYWNrIHRvIGJ5cGFzcyB0aGlzIGxpbWl0YXRpb24uIChU aGVyZSBpcyBubyBmZWFzYWJsZSBhbHRlcm5hdGUgc29sdXRpb24gYXQgdGhpcyBzdGFnZSku CgplZGl0IC91c3Ivc3JjL3Vzci5iaW4vcGFzc3dkL3Bhc3N3ZC5jClJlcGxhY2U6CiAgICBl cnJ4KDEsICJTb3JyeSwgYHBhc3N3ZCcgY2FuIG9ubHkgY2hhbmdlIHBhc3N3b3JkcyBmb3Ig bG9jYWwgb3IgTklTIHVzZXJzLiIpOwpXaXRoOgogICAgICAgICAgICAgICAgZnByaW50Zihz dGRlcnIsICJOb3cgeW91IGNhbiBjaGFuZ2UgTERBUCBwYXNzd29yZCB2aWEgUEFNXG4iKTsK Cm1ha2UgaW5zdGFsbCBpbiB0aGF0IGRpcmVjdG9yeS4KCgplZGl0IC9ldGMvcGFtLmQvcGFz c3dkCmFkZCB0aGUgZm9sbG93aW5nIGxpbmUgYmVmb3JlIHRoZSBwYW1fdW5peC5zbyBlbnRy eQoKXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl4KcGFzc3dvcmQgICAg ICAgIHN1ZmZpY2llbnQgICAgICAvdXNyL2xvY2FsL2xpYi9wYW1fbGRhcC5zbyAgICAgIG5v X3dhcm4gdHJ5X2ZpcnN0X3Bhc3MgY29uZmlnPS91c3IvbG9jYWwvZXRjL3BhbV9sZGFwLmNv bmYKCl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl4KCgoKCiMjIyBSZXN0 cmljdGluZyBMb2dpbnMgdG8gYSBwYXJ0aWN1bGFyIGdyb3VwCgpodHRwOi8vbGlzdHMuZnJl ZWJzZC5vcmcvcGlwZXJtYWlsL2ZyZWVic2QtcXVlc3Rpb25zLzIwMDUtT2N0b2Jlci8xMDA4 MTYuaHRtbAoKZWRpdCAvdXNyL2xvY2FsL2V0Yy9wYW1fbGRhcC5jb25mCl5eXl5eXl5eXl5e Xl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl4KcGFtX2dyb3VwZG4gY249bG9naW5fZ3JvdXAs b3U9Z3JvdXBzLGRjPWJsYWgsZGM9Y29tLGRjPWF1CnBhbV9tZW1iZXJfYXR0cmlidXRlIG1l bWJlclVpZAoKXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXgpXQVJOSU5H OiBGb3IgdGhpcyB0byB3b3JrIHRoZSBGVUxMIEROIG9mIHRoZSB1c2VyIG11c3QgYmUgbGlz dGVkIGluIHRoZSBtZW1iZXJVaWQgYXR0cmlidXRlIG9mIHRoZSBwb3NpeCBncm91cCBzcGVj aWZpZWQgaW4gcGFtX2dyb3VwZG4gY29uZmlnLgoKU3VnZ2VzdGlvbiwgdXNlIGEgc2VwYXJh dGUgT1UgZm9yIGhvc3RzLCB3aGljaCBjb250YWluIGdyb3VwcyB3aXRoIG1hY2hpbmUgbmFt ZXMgLT4gZnVsbCBkbiBvZiBhbGxvd2VkIHVzZXJzIG9uIHRob3NlIG1hY2hpbmVzIHNob3Vs ZCBiZSBzdG9yZWQgaW4gbWVtYmVyVWlkCk9SCmFkZCBhIHNlcGFyYXRlIGF0dHJpYnV0ZSAn bG9naW5ETicgdG8gcmVsYXZhbnQgZ3JvdXBzIHRoYXQgd2lsbCBiZSB1c2VkIGZvciBsb2dp biByZXN0cmljdGlvbnMuIGNoYW5nZSBwYW1fbWVtYmVyX2F0cnJpYnV0ZSBmcm9tIG1lbWJl clVpZCB0byBsb2dpbkROCgojIyMgUHJldmVudGluZyBsb25nIGRlbGF5cyBkdXJpbmcgQXV0 aCB3aGVuIExEQVAgc2VydmVyIGlzbid0IGF2YWlsYWJsZQoKQnkgZGVmYXVsdCwgd2hlbiB0 aGUgTERBUCBzZXJ2ZXIgaXNuJ3QgYXZhaWxhYmxlLCB0aGUgY2xpZW50IHdpbGwgbm90IGdp dmUgdXAgdHJ5aW5nIHRvIGNvbm5lY3QgdG8gdGhlIExEQVAgc2VydmVyLgpDaGFuZ2luZyB0 aGUgYmluZCBwb2xpY3kgdG8gJ3NvZnQnIHdpbGwgY2F1c2UgdGhlIGNsaWVudCB0byBnaXZl IHVwIGltbWVkaWF0ZWx5IGlmIHRoZSBzZXJ2ZXIgaXMgbm90IGF2YWlsYWJsZS4KCiMgUmVj b25uZWN0IHBvbGljeToKIyAgaGFyZF9vcGVuOiByZWNvbm5lY3QgdG8gRFNBIHdpdGggZXhw b25lbnRpYWwgYmFja29mZiBpZgojICAgICAgICAgICAgIG9wZW5pbmcgY29ubmVjdGlvbiBm YWlsZWQKIyAgaGFyZF9pbml0OiByZWNvbm5lY3QgdG8gRFNBIHdpdGggZXhwb25lbnRpYWwg YmFja29mZiBpZgojICAgICAgICAgICAgIGluaXRpYWxpemluZyBjb25uZWN0aW9uIGZhaWxl ZAojICBoYXJkOiAgICAgIGFsaWFzIGZvciBoYXJkX29wZW4KIyAgc29mdDogICAgICByZXR1 cm4gaW1tZWRpYXRlbHkgb24gc2VydmVyIGZhaWx1cmUKCmVkaXQgL3Vzci9sb2NhbC9ldGMv cGFtX2xkYXAuY29uZgoKCl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eCmJp bmRfcG9saWN5IHNvZnQgCgpeXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXgoK IyMjIEFsbG93aW5nIG1hY2hpbmUgbG9naW5zIHRvIGJlIHJlc3RyaWN0ZWQgYnkgZ3JvdXBz CgpQYXN0ZSB0aGUgZm9sbG93aW5nIHBhdGNoIGludG8gdGhlIGZpbGUgc2hvd24gYmVsb3c6 Ci91c3IvcG9ydHMvc2VjdXJpdHkvcGFtX2xkYXAvZmlsZXMvcGF0Y2gtZ3JvdXBkbmZpeAoK LS0tIHBhbV9sZGFwLmMub3JpZyAgICAgMjAwOC0wNi0yNyAxNzo1MTozNy4wMDAwMDAwMDAg KzEwMDAKKysrIHBhbV9sZGFwLmMgIDIwMDgtMDYtMjcgMTc6NDU6MTEuMDAwMDAwMDAwICsx MDAwCkBAIC00MDIwLDcgKzQwMjAsNyBAQAogICAgIHsKICAgICAgIHJjID0gbGRhcF9jb21w YXJlX3MgKHNlc3Npb24tPmxkLAogICAgICAgICAgICAgICAgICAgICAgICAgICBzZXNzaW9u LT5jb25mLT5ncm91cGRuLAotICAgICAgICAgICAgICAgICAgICAgICAgICBzZXNzaW9uLT5j b25mLT5ncm91cGF0dHIsIHNlc3Npb24tPmluZm8tPnVzZXJkbik7CisgICAgICAgICAgICAg ICAgICAgICAgICAgIHNlc3Npb24tPmNvbmYtPmdyb3VwYXR0ciwgc2Vzc2lvbi0+aW5mby0+ dXNlcm5hbWUpOwogICAgICAgaWYgKHJjICE9IExEQVBfQ09NUEFSRV9UUlVFKQogICAgICAg IHsKICAgICAgICAgIHNucHJpbnRmIChidWYsIHNpemVvZiBidWYsICJZb3UgbXVzdCBiZSBh ICVzIG9mICVzIHRvIGxvZ2luLiIsCmNkIC91c3IvcG9ydHMvCm1ha2UgY2xlYW4KCgojIyBF TkQgQ0xJRU5UIENPTkZJRyAjIwoKCgojIyBCRUdJTiBTQU1CQSBTRVJWRVIgQ09ORklHICMj CgogICBwYXNzZGIgYmFja2VuZCA9IGxkYXBzYW06bGRhcDovL2xkYXBzZXJ2ZXIuYmxhaC5j b20uYXUKICAgbGRhcCBhZG1pbiBkbiA9IHVpZD1zYW1iYSxvdT1hZG1pbnMsZGM9YmxhaCxk Yz1jb20sZGM9YXUKICAgbGRhcCBzc2wgPSBubwogICBsZGFwIGRlbGV0ZSBkbiA9IG5vCiAg IGxkYXAgdXNlciBzdWZmaXggPSBvdT1wZW9wbGUKICAgbGRhcCBncm91cCBzdWZmaXggPSBv dT1ncm91cHMKICAgbGRhcCBtYWNoaW5lIHN1ZmZpeCA9IG91PWNvbXB1dGVycwogICBsZGFw IHN1ZmZpeCA9IGRjPWJsYWgsZGM9Y29tLGRjPWF1CgojIyBFTkQgU0FNQkEgU0VSVkVSIENP TkZJRyAjIwo= --------------050705030408060200020104-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 15:12:47 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 37DA8106566B for ; Thu, 6 Oct 2011 15:12:47 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 998CB8FC17 for ; Thu, 6 Oct 2011 15:12:46 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.4) with ESMTP id p96Efk3u015561; Thu, 6 Oct 2011 16:41:46 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.4/Submit) with ESMTP id p96Efjs8015558; Thu, 6 Oct 2011 16:41:45 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Thu, 6 Oct 2011 16:41:45 +0200 (CEST) From: Wojciech Puchar To: hackers@freebsd.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Thu, 06 Oct 2011 16:41:46 +0200 (CEST) Cc: Grzegorz Kulewski Subject: mmap performance and memory use X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 15:12:47 -0000 i have few questions. 1) suppose i map 1TB of address space as anonymous and touch just one page. how much memory is used to manage this? 2) suppose we have 1TB file on disk without holes and 100000 processes mmaps this file to it's address space. are just pages shared or can pagetables be shared too? how much memory is used to manage such situation? yes this is a real question - assume most of these processes are mostly sleeping but every now and then do something and work of some set of pages from this file and there is enough memory in computer to keep this working set, but only if managing it by OS will not overuse memory. From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 16:29:46 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 74710106566C for ; Thu, 6 Oct 2011 16:29:46 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id E70398FC12 for ; Thu, 6 Oct 2011 16:29:45 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p96G206A024008 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 6 Oct 2011 19:02:00 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id p96G1x0p098608; Thu, 6 Oct 2011 19:01:59 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id p96G1xel098607; Thu, 6 Oct 2011 19:01:59 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 6 Oct 2011 19:01:59 +0300 From: Kostik Belousov To: Wojciech Puchar Message-ID: <20111006160159.GQ1511@deviant.kiev.zoral.com.ua> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="1jIccons5TjyLah6" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: hackers@freebsd.org, Grzegorz Kulewski Subject: Re: mmap performance and memory use X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 16:29:46 -0000 --1jIccons5TjyLah6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 06, 2011 at 04:41:45PM +0200, Wojciech Puchar wrote: > i have few questions. >=20 > 1) suppose i map 1TB of address space as anonymous and touch just one=20 > page. how much memory is used to manage this? I am not sure how deep the enumeration you want to know, but the first approximation will be: one struct vm_map_entry one struct vm_object one pv_entry Page table structures need four pages for directories and page table proper. >=20 > 2) suppose we have 1TB file on disk without holes and 100000 processes=20 > mmaps this file to it's address space. are just pages shared or can=20 > pagetables be shared too? how much memory is used to manage such=20 > situation? Only pages are shared. Pagetables are not. For one thing, this indeed causes more memory use for the OS. This is somewhat mitigated by automatic use of superpages. Superpage promotion still keeps the 4KB page table around, so most savings from the superpages are due to more efficient use of TLB. On the other hand, having non-shared page tables allows for much more accurate tracking of the accesses and writes, which can result in better pageout performance. For the situation 1TB/100000 processes, you will probably need to tune the amount of pv entries, see sysctl vm.pmap.pv*. >=20 > yes this is a real question - assume most of these processes are mostly= =20 > sleeping but every now and then do something and work of some set of=20 > pages from this file and there is enough memory in computer to keep this= =20 > working set, but only if managing it by OS will not overuse memory. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" --1jIccons5TjyLah6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk6N0PcACgkQC3+MBN1Mb4im0wCfbSpLN473jBiTzy/mOQflRhvO i3cAoNVkxpInQLGVouyO7LjIC1i7KAYG =qYXK -----END PGP SIGNATURE----- --1jIccons5TjyLah6-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 16:51:15 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25B9E106564A for ; Thu, 6 Oct 2011 16:51:15 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id B894B8FC16 for ; Thu, 6 Oct 2011 16:51:14 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:b9fd:2f11:cd06:1a6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id D48664AC2D for ; Thu, 6 Oct 2011 20:51:12 +0400 (MSD) Date: Thu, 6 Oct 2011 20:51:03 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <1034127827.20111006205103@serebryakov.spb.ru> To: hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: Subject: how to debug RB_TREE for memory corruption? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 16:51:15 -0000 Hello, Hackers. I'm writing some code, which uses RB_TREE from . At some momoent, it crashes within REMOVE method with "elm" 0xa5a5a5a5 (I have malloc() debug options turned on). So, it seems, that free()ed element presents somewhere in the tree, am I right? Ok, I add printing of whole tree BEFORE removal call with simple recursive function. It doesn't crash and doesn't print any invalid pointers! How could it happen!? Tree is perfectly valid at line BEFORE RB_DELETE() call and crashes with bad pointer in this method! I could (theoretically!) belive, that my code forget to delete node from tree in some situations. But in such case tree printing function will crash (or print "0xa5a5a5a5" pointer) before RB_DELETE crash! Any hints how to debug such strange situation? --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 16:58:55 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66E191065670; Thu, 6 Oct 2011 16:58:55 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7BE808FC14; Thu, 6 Oct 2011 16:58:54 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA06513; Thu, 06 Oct 2011 19:58:52 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E8DDE4B.4040608@FreeBSD.org> Date: Thu, 06 Oct 2011 19:58:51 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111003 Thunderbird/7.0.1 MIME-Version: 1.0 To: lev@FreeBSD.org References: <1034127827.20111006205103@serebryakov.spb.ru> In-Reply-To: <1034127827.20111006205103@serebryakov.spb.ru> X-Enigmail-Version: undefined Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: hackers@FreeBSD.org Subject: Re: how to debug RB_TREE for memory corruption? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 16:58:55 -0000 on 06/10/2011 19:51 Lev Serebryakov said the following: > Hello, Hackers. > > I'm writing some code, which uses RB_TREE from . At some > momoent, it crashes within REMOVE method with "elm" 0xa5a5a5a5 (I have > malloc() debug options turned on). > So, it seems, that free()ed element presents somewhere in the tree, > am I right? > Ok, I add printing of whole tree BEFORE removal call with simple > recursive function. It doesn't crash and doesn't print any invalid pointers! > > How could it happen!? Tree is perfectly valid at line BEFORE > RB_DELETE() call and crashes with bad pointer in this method! > > I could (theoretically!) belive, that my code forget to delete node > from tree in some situations. But in such case tree printing function > will crash (or print "0xa5a5a5a5" pointer) before RB_DELETE crash! > > Any hints how to debug such strange situation? A trivial check first - does the element on which you call RB_DELETE actually belong to the tree in question? -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 17:25:40 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 818701065672; Thu, 6 Oct 2011 17:25:40 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 479458FC15; Thu, 6 Oct 2011 17:25:40 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:b9fd:2f11:cd06:1a6]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 788EF4AC1C; Thu, 6 Oct 2011 21:25:39 +0400 (MSD) Date: Thu, 6 Oct 2011 21:25:29 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <155347808.20111006212529@serebryakov.spb.ru> To: Andriy Gapon In-Reply-To: <4E8DDE4B.4040608@FreeBSD.org> References: <1034127827.20111006205103@serebryakov.spb.ru> <4E8DDE4B.4040608@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: hackers@FreeBSD.org Subject: Re: how to debug RB_TREE for memory corruption? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 17:25:40 -0000 Hello, Andriy. You wrote 6 =EE=EA=F2=FF=E1=F0=FF 2011 =E3., 20:58:51: >> Any hints how to debug such strange situation? > A trivial check first - does the element on which you call RB_DELETE actu= ally > belong to the tree in question? Yep, I've checked it after sending message. Doesn't belong. Logic bug earlier... Sorry for noise on list :) --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 6 17:36:25 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28885106566C for ; Thu, 6 Oct 2011 17:36:25 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id DE3668FC16 for ; Thu, 6 Oct 2011 17:36:24 +0000 (UTC) Received: by vws11 with SMTP id 11so3472186vws.13 for ; Thu, 06 Oct 2011 10:36:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=x/3Mkgx6+RV5OqEHsp0y7s5yoKbIOiGOIGOFXtBbkrM=; b=C2wNnE7S0yjlekt6PHbnUHjZddQNXeTmU28O9tvnDuTnWKnrfR2gywN4yG1KkvXiTK bkB5pApa/aHVBgrJmj43jbzoQquWW+ai2bZmZpVv7aCEYv/55um2IXrlKNG7F50gjuFI ouU+6uBifT3+PsoFoCNTVcyyUABmaakBv+2LA= MIME-Version: 1.0 Received: by 10.52.67.52 with SMTP id k20mr876732vdt.383.1317920699555; Thu, 06 Oct 2011 10:04:59 -0700 (PDT) Received: by 10.52.29.228 with HTTP; Thu, 6 Oct 2011 10:04:59 -0700 (PDT) In-Reply-To: <1034127827.20111006205103@serebryakov.spb.ru> References: <1034127827.20111006205103@serebryakov.spb.ru> Date: Thu, 6 Oct 2011 19:04:59 +0200 Message-ID: From: Davide Italiano To: lev@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: hackers@freebsd.org Subject: Re: how to debug RB_TREE for memory corruption? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2011 17:36:25 -0000 Was the node you're removing actually part of the tree? I had a similar issue some time ago because I've tried insert two nodes w/ the same key and then remove then. In practice, the second INSERT operation failed (due to the definition of key in a BST), and so I was trying to remove a node that wasn't actually inserted. Can you provide a snippet of code o some minimal testcase in order to reproduce the error? Regards Davide From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 00:46:52 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9FA2B106566B for ; Fri, 7 Oct 2011 00:46:52 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 1313A8FC13 for ; Fri, 7 Oct 2011 00:46:51 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.4) with ESMTP id p970kpfH027259; Fri, 7 Oct 2011 02:46:51 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.4/Submit) with ESMTP id p970kp5I027255; Fri, 7 Oct 2011 02:46:51 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Fri, 7 Oct 2011 02:46:51 +0200 (CEST) From: Wojciech Puchar To: Kostik Belousov In-Reply-To: <20111006160159.GQ1511@deviant.kiev.zoral.com.ua> Message-ID: References: <20111006160159.GQ1511@deviant.kiev.zoral.com.ua> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Fri, 07 Oct 2011 02:46:51 +0200 (CEST) Cc: hackers@freebsd.org, Grzegorz Kulewski Subject: Re: mmap performance and memory use X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 00:46:52 -0000 >> page. how much memory is used to manage this? > I am not sure how deep the enumeration you want to know, but the first > approximation will be: > one struct vm_map_entry > one struct vm_object > one pv_entry actually i don't need precise answer but algorithms. > > Page table structures need four pages for directories and page table proper. >> >> 2) suppose we have 1TB file on disk without holes and 100000 processes >> mmaps this file to it's address space. are just pages shared or can >> pagetables be shared too? how much memory is used to manage such >> situation? > Only pages are shared. Pagetables are not. this is what i really asked, thank you for an answer. My example was rather extreme but datasets of tens of gigabytes would be used. > superpages are due to more efficient use of TLB. actually this is not really working at least a while ago (but already in FreeBSD 8) i tested it. Even with 1GB squid process without any swapping it wasn't often allocating them. Even with working case it probably will not help much here unless completely all data is in RAM, and following explains why > accurate tracking of the accesses and writes, which can result in better > pageout performance. > > For the situation 1TB/100000 processes, you will probably need to tune > the amount of pv entries, see sysctl vm.pmap.pv*. so there is a workaround but causing lots of soft page faults as there would be no more than few hundreds or so instructions between touching different pages. What i want to do is database library (but no SQL!). It will be something alike (but definitely not the same and NOT compatible) CA-Clipper/Harbour or harbour but with higher performance and to use it including heavy cases. With this system one user is one process, one thread. if used as WWW/something alike it will be this+some other thing doing WWW interface but still one logged user=exactly one process As properly planned database tables should not be huge i assume most of them (possibly excluded parts that are mostly not used) will be kept in memory by VM subsystem. So hard faults and disk I/O will not be a deciding factor. To avoid system calls i just want to mmap tables and indexes. All semaphores can be done from userspace too, and i already know how to avoid lock contention well. Using indexes means doing lots of memory reads from different pages, but for every process it will be usually not all pages touched but small subset. So it MAY work well this way, or may end with 95% system CPU time mostly doing soft faults. But future question - is something for that case planned in FreeBSD? I think i am not the only one about that, not all people on earth use computers for few processes or personal usage and there are IMHO many cases when programs need to share huge dataset using mmap, while doing heavy timesharing. I understand that mmap works that way because it may be mapped in different places and even with parts of single file in different places as this is what mmap allows. But is it possible to make different mmap in kernel like that mmap_fullfile(fd,maxsize) which (taking amd64 case) will map file at 2MB boundary if maxsize<=2MB, 1GB boundary if maxsize<=1GB, 512GB boundary otherwise, with subsequent multiple 512GB address blocks if needed, and sharing everything? it is completely no problem that things like madvise from one process will clean madvise setting from other process, or other problems - as only one type of programs that are aware of this would use it. this way there will be practicaly no pagetable mapping overhead and actually simpler/faster OS duties. I don't really know how exactly VM subsystem works under FreeBSD but if it is not hard i may do this with some help from you. And no - i don't want to use any popular database systems for good reasons. From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 07:13:19 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6BF28106566B; Fri, 7 Oct 2011 07:13:19 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id 4AD578FC18; Fri, 7 Oct 2011 07:13:19 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id p977DIrm024799 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 7 Oct 2011 00:13:18 -0700 (PDT) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id p977DI0A024798; Fri, 7 Oct 2011 00:13:18 -0700 (PDT) Received: from fbsd81 ([192.168.200.81]) by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA00289; Fri, 7 Oct 11 00:06:40 PDT Date: Fri, 07 Oct 2011 07:05:48 -0700 From: perryh@pluto.rain.com To: ivoras@freebsd.org Message-Id: <4e8f073c.3g2aD/Zz9KdsWOKN%perryh@pluto.rain.com> References: <20111004160043.GA16034@lpthe.jussieu.fr> <051853CE-03EC-4EEC-A5AC-C380131B28E4@gsoft.com.au> In-Reply-To: User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 07:13:19 -0000 Ivan Voras wrote: > On 05/10/2011 09:38, Trond Endrest??l wrote: > > On Wed, 5 Oct 2011 12:54+1030, Daniel O'Connor wrote: > >> In my experience ncsd seems to cache negative hits forever, > >> regardless of the setting for negative-time-to-live. > > > > I'm glad to see I'm not the only one who has noticed this odd > > behaviour of nscd. Shame on me for not speaking up sooner, but > > I feared I might be proved wrong (again), and yes, that's a > > lame excuse. :-/ > > +1. > > It's very annoying when installing ports which add users - the > port adds it then in some future code checks it and it fails. > I've noticed it with at least CUPS. Sounds as if there ought to be a unified mechanism for ports to use when adding users, so that necessary notifications -- e.g. restarting nscd if it is running -- can be done in a standardized way and any necessary customizations can be done in a single place. From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 10:19:15 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E29A106564A for ; Fri, 7 Oct 2011 10:19:15 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 076238FC19 for ; Fri, 7 Oct 2011 10:19:14 +0000 (UTC) Received: by vcbf13 with SMTP id f13so4193144vcb.13 for ; Fri, 07 Oct 2011 03:19:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=3USHGkLKggriDpFMuOPrKMktZ0YnEUW+KiFOf/40AsM=; b=YYIyxr7qAyiU0/ntkIpskrVamzsI4moBWhaW1ynXLy2a4Yg1iCIV2UkvsUQ50bofVq eHd1Nw+sdRV2x+9cQpw2l/rS2zhxpWO6yrgiw8RaJX2YaoDp3zEDVMX6pD356CWU659R XyB0GZYQc260vmXfmLXs3ynbvpi3VkfYPuDIc= MIME-Version: 1.0 Received: by 10.52.65.205 with SMTP id z13mr1733897vds.87.1317981104837; Fri, 07 Oct 2011 02:51:44 -0700 (PDT) Received: by 10.52.111.201 with HTTP; Fri, 7 Oct 2011 02:51:44 -0700 (PDT) In-Reply-To: <4e8f073c.3g2aD/Zz9KdsWOKN%perryh@pluto.rain.com> References: <20111004160043.GA16034@lpthe.jussieu.fr> <051853CE-03EC-4EEC-A5AC-C380131B28E4@gsoft.com.au> <4e8f073c.3g2aD/Zz9KdsWOKN%perryh@pluto.rain.com> Date: Fri, 7 Oct 2011 10:51:44 +0100 Message-ID: From: Tom Evans To: perryh@pluto.rain.com Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 10:19:15 -0000 On Fri, Oct 7, 2011 at 3:05 PM, wrote: > Ivan Voras wrote: >> On 05/10/2011 09:38, Trond Endrest??l wrote: >> > On Wed, 5 Oct 2011 12:54+1030, Daniel O'Connor wrote: >> >> In my experience ncsd seems to cache negative hits forever, >> >> regardless of the setting for negative-time-to-live. >> > >> > I'm glad to see I'm not the only one who has noticed this odd >> > behaviour of nscd. Shame on me for not speaking up sooner, but >> > I feared I might be proved wrong (again), and yes, that's a >> > lame excuse. :-/ >> >> +1. >> >> It's very annoying when installing ports which add users - the >> port adds it then in some future code checks it and it fails. >> I've noticed it with at least CUPS. > > Sounds as if there ought to be a unified mechanism for ports > to use when adding users, so that necessary notifications -- > e.g. restarting nscd if it is running -- can be done in a > standardized way and any necessary customizations can be done > in a single place. Or nscd fixed to not permanently cache negative hits. Seems more correct. Cheers Tom From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 10:28:43 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0DF01065672 for ; Fri, 7 Oct 2011 10:28:43 +0000 (UTC) (envelope-from uqs@spoerlein.net) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 51B328FC1C for ; Fri, 7 Oct 2011 10:28:43 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by acme.spoerlein.net (8.14.4/8.14.4) with ESMTP id p97ASfOf076360 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 7 Oct 2011 12:28:42 +0200 (CEST) (envelope-from uqs@spoerlein.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=spoerlein.net; s=dkim200908; t=1317983322; bh=neWDSX9/GX792dCmW1cB+tEMHa0LYf2cU7SpgHXa5DU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Transfer-Encoding:In-Reply-To; b=BMPwrcuHMCOk+Ow2qvXdT8h77IvA0ZB+XSqSa/yqhLoiaXLwLALfB6PLs1tShf2CU +33xNUzPKL9J1H6nIjHckYRIbgIuBxMzKhOLaJDn3EBQ3ka3w2FGP/phbWj9W0YpIy e1yq2n/XL71HEHO+xRMUAz9Ony/z5Y17Ji36PxCU= Date: Fri, 7 Oct 2011 12:28:41 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Arnaud Lacombe Message-ID: <20111007102841.GG26743@acme.spoerlein.net> Mail-Followup-To: Arnaud Lacombe , hackers@freebsd.org References: <4E712D11.7040202@FreeBSD.org> <4E75B67E.1000802@FreeBSD.org> <20110922190535.GR26743@acme.spoerlein.net> <20110922212602.GS26743@acme.spoerlein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: hackers@freebsd.org Subject: Re: Fwd: my git development snapshot(s) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 10:28:43 -0000 On Fri, 2011-09-30 at 15:41:41 -0400, Arnaud Lacombe wrote: > Hi, > > On Mon, Sep 26, 2011 at 2:23 PM, Arnaud Lacombe wrote: > > Hi, > > > > On Thu, Sep 22, 2011 at 5:26 PM, Ulrich Spörlein wrote: > >> On Thu, 2011-09-22 at 15:52:43 -0400, Arnaud Lacombe wrote: > >>> Hi, > >>> > >>> On Thu, Sep 22, 2011 at 3:05 PM, Ulrich Spörlein wrote: > >>> > On Sun, 2011-09-18 at 12:14:38 +0300, Andriy Gapon wrote: > >>> >> > >>> >> Just decided to follow the global trends and trying to throw all of my > >>> >> local/private changes at you in hope that the "crowd-sourcing magic" might > >>> >> somehow happen :-)  This seems definitely easier than carefully producing the > >>> >> patch files and keeping them up-to-date. > >>> >> > >>> >> So, my newly cloned gitorious repository: > >>> >> https://gitorious.org/~avg/freebsd/avgbsd > >>> >> > >>> >> And the first branch of interest: > >>> >> https://gitorious.org/~avg/freebsd/avgbsd/commits/devel-20110915 > >>> > > >>> > I'll throw mine in as well: > >>> > > >>> > https://github.com/uqs/freebsd-head/branches > >>> > > >>> is that the same as tree as Fabien's, or a new snapshot ? > >>> > >>> If not, couldn't we agree to have all the same tree in order to ease > >>> code sharing between all of them ? > >>> > >>> I see there is already a https://github.com/freebsd/freebsd tree, > >>> which seem different than Fabien's tree on github. It really look like > >>> The FreeBSD Project (https://github.com/freebsd/) is not able to > >>> provide consistency. > >> > >> The freebsd-head tree is the same that everybody in the world can get by > >> simply running git svn clone against the FreeBSD subversion server > >> (you'd need a lot of patience, though). > >> > > FWIW, how comes that there is not yet any `stable/9' branch on the github tree ? > > > Ulrich, ping ? Oops, sorry for the delay! Fixed now, thanks for bringing it to my attention. I missed the push --all flag. :/ Uli From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 10:30:22 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 051911065677; Fri, 7 Oct 2011 10:30:22 +0000 (UTC) (envelope-from uqs@spoerlein.net) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 89F818FC17; Fri, 7 Oct 2011 10:30:21 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by acme.spoerlein.net (8.14.4/8.14.4) with ESMTP id p97AUKEK076421 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 7 Oct 2011 12:30:20 +0200 (CEST) (envelope-from uqs@spoerlein.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=spoerlein.net; s=dkim200908; t=1317983420; bh=08A94nYvAi4kW77tJ2OrQbIAG2R+HLlB+5d/Z1w6blE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Transfer-Encoding:In-Reply-To; b=t8775tE6yprTfNCNdxUmXzxvhcR4Eph1JmIn4OTo6ySorYaPuKF6l7nBBxV22Do8w /TJ7/tAOzJuq5WVW1dx67in/hlXCpqUWrOCRJFDoUmiMFbJdU3CzUcsJrplf/Gl1FY HJdY01aPSHO8HyEKgqdbcCYN9eNol8urb/dqOiy0= Date: Fri, 7 Oct 2011 12:30:20 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Arnaud Lacombe Message-ID: <20111007103020.GH26743@acme.spoerlein.net> Mail-Followup-To: Arnaud Lacombe , Andriy Gapon , hackers@freebsd.org References: <4E712D11.7040202@FreeBSD.org> <4E75B67E.1000802@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: hackers@freebsd.org, Andriy Gapon Subject: Re: my git development snapshot(s) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 10:30:22 -0000 On Wed, 2011-09-28 at 00:29:46 -0400, Arnaud Lacombe wrote: > Hi, > > On Sun, Sep 18, 2011 at 5:14 AM, Andriy Gapon wrote: > > > > Just decided to follow the global trends and trying to throw all of my > > local/private changes at you in hope that the "crowd-sourcing magic" might > > somehow happen :-)  This seems definitely easier than carefully producing the > > patch files and keeping them up-to-date. > > > > So, my newly cloned gitorious repository: > > https://gitorious.org/~avg/freebsd/avgbsd > > > and mine: > > https://github.com/lacombar/freebsd/branches > > not much stuff for now, but should fill up in the next few days. This > a fork from the freebsd/freebsd.git tree on github, with only stable > branches in it (ie. none of the >2k branches of the original clone), > plus my stuff. I think deleting the cvs2svn/vendor branch refs on github.com is the way to go. They'll still be converted and available from git.freebsd.your.org, just not pushed to github. This should unclutter the UI a bit (I think keeping user/ and projects/ is nice, as people can comment on commits that way). Cheers, Uli From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 10:30:34 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB922106570F for ; Fri, 7 Oct 2011 10:30:34 +0000 (UTC) (envelope-from realbushman@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 853D08FC0A for ; Fri, 7 Oct 2011 10:30:34 +0000 (UTC) Received: by qyk4 with SMTP id 4so3829122qyk.13 for ; Fri, 07 Oct 2011 03:30:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=BA/KoO3Yd6t2uveShNdQY2o/MRwFCh6hMFXTvDQ276A=; b=tNO0oR02yBZ/dr/lgPSTQpgQQUXedlYs9VLeRmWzZNxWPdp+fPY6Si4g3WW95Fjg5g Kmjvp1VDYKS3yWWGryn83KkrJ4VPeKJZpXOfb/bIiODys8a0pictttOsnTnmHbMzJDRz Hpgo6DwtJAegrhxJ42K4OKA+DdhA+r3uisjDk= MIME-Version: 1.0 Received: by 10.229.242.84 with SMTP id lh20mr1407438qcb.211.1317983433741; Fri, 07 Oct 2011 03:30:33 -0700 (PDT) Sender: realbushman@gmail.com Received: by 10.229.8.66 with HTTP; Fri, 7 Oct 2011 03:30:33 -0700 (PDT) In-Reply-To: References: <20111004160043.GA16034@lpthe.jussieu.fr> <051853CE-03EC-4EEC-A5AC-C380131B28E4@gsoft.com.au> <4e8f073c.3g2aD/Zz9KdsWOKN%perryh@pluto.rain.com> Date: Fri, 7 Oct 2011 12:30:33 +0200 X-Google-Sender-Auth: VTg4tTKEWgyikDqLgZIOq4cXews Message-ID: From: Michael Bushkov To: Tom Evans Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, perryh@pluto.rain.com Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 10:30:35 -0000 While I agree that nscd negative caching bug should be fixed, it won't help with the problem that users encounter during ports installation. When, for example, user "x" is added during port install, the following steps are involved: 1. Script checks if "x" is present in the users list. Nscd is queried, it returns negative and caches negative answer. 2. Script adds user "x". 3. Script checks that "x" have indeed been added. Nscd is queried, cachned negative answer is returned. Script fails as a result. So unless negative caching time is less than the time between steps 1) and 3) the issues during ports installation will persist. I like perryh@ idea of fixing it within ports. If we introduce some standard way of adding users/groups then this standard routine can take care of nscd. I don't know how much work this will require though... Cheers, Michael On Fri, Oct 7, 2011 at 11:51 AM, Tom Evans wrote= : > On Fri, Oct 7, 2011 at 3:05 PM, =A0 wrote: >> Ivan Voras wrote: >>> On 05/10/2011 09:38, Trond Endrest??l wrote: >>> > On Wed, 5 Oct 2011 12:54+1030, Daniel O'Connor wrote: >>> >> In my experience ncsd seems to cache negative hits forever, >>> >> regardless of the setting for negative-time-to-live. >>> > >>> > I'm glad to see I'm not the only one who has noticed this odd >>> > behaviour of nscd. Shame on me for not speaking up sooner, but >>> > I feared I might be proved wrong (again), and yes, that's a >>> > lame excuse. :-/ >>> >>> +1. >>> >>> It's very annoying when installing ports which add users - the >>> port adds it then in some future code checks it and it fails. >>> I've noticed it with at least CUPS. >> >> Sounds as if there ought to be a unified mechanism for ports >> to use when adding users, so that necessary notifications -- >> e.g. restarting nscd if it is running -- can be done in a >> standardized way and any necessary customizations can be done >> in a single place. > > Or nscd fixed to not permanently cache negative hits. Seems more correct. > > Cheers > > Tom > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > > From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 13:22:01 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DEEF81065670 for ; Fri, 7 Oct 2011 13:22:01 +0000 (UTC) (envelope-from uqs@spoerlein.net) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 6F6058FC13 for ; Fri, 7 Oct 2011 13:22:01 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by acme.spoerlein.net (8.14.4/8.14.4) with ESMTP id p97DKI3a080708 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 7 Oct 2011 15:20:19 +0200 (CEST) (envelope-from uqs@spoerlein.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=spoerlein.net; s=dkim200908; t=1317993619; bh=t5BYcygxA/DHtthPOa80AJ2DneS9gVdhz7yHcLTZvsI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=XyvcyskJCyDyg8m3pWhoPs7DaapPj3bQj/1+NzIPYHjZzcJ5ZaceFGZuni1niSw41 t+KGXBpeglI/r0i/Po5OZJB++jB2n6mibTPAYgwB6tPrKa5balyjuy0ZnI9+hJkGvP jRzAQEoFGqRP0NKt1zFy2zNCGJiOy+5P/qF8s/hA= Date: Fri, 7 Oct 2011 15:20:18 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Dan Nelson Message-ID: <20111007132017.GI26743@acme.spoerlein.net> Mail-Followup-To: Dan Nelson , Trond Endrestol , Dag-Erling Smorgrav , hackers@freebsd.org References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> <20111006054409.GS9801@dan.emsphone.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111006054409.GS9801@dan.emsphone.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Dag-Erling Smorgrav , hackers@freebsd.org, Trond Endrestol Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 13:22:01 -0000 On Thu, 2011-10-06 at 00:44:10 -0500, Dan Nelson wrote: > In the last episode (Oct 04), Trond Endrestol said: > > On Tue, 4 Oct 2011 18:51+0200, Dag-Erling Smorgrav wrote: > > > Trond Endrestol writes: > > > > It's in daily use at Gjovik Technical College (Fagskolen i Gjovik), > > > > here in Norway. Both the mail and web servers authenticates our users > > > > by LDAP, and nscd certainly speeds up the lookups. > > > > > > OK. No trouble with clients dying of SIGPIPE? I could never reproduce > > > the bug, but both users who reported problems used ldap, and I don't > > > have an LDAP server to test against, so I thought it might be specific > > > to LDAP. > > > > Not in my (somewhat limited) experience. > > On a tangent, I also heavily recommend using the nss-pam-ldapd port instead > of nss_ldap. It includes a daemon called nslcd which is the only process > that links to the ldap libary. The nss module is a tiny plug that talks to > nslcd using a simple protocol. It really reduces the socket count to your > ldap server, and removes the potential namespace problems caused by > dlopening libldap.so in every process. Seconded, I had endless troubles with leaked domain sockets and connection problems with nss_ldap and have found that only nss-pam-ldapd + nslcd will work somewhat reliably. Except it still manages to return empty results to sendmail every once in a while (for local delivery). Uli From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 14:48:44 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7294106567C for ; Fri, 7 Oct 2011 14:48:44 +0000 (UTC) (envelope-from s@samu.pl) Received: from samu.pl (samu.pl [IPv6:2001:41d0:1:f0cf::1]) by mx1.freebsd.org (Postfix) with ESMTP id 89F3E8FC16 for ; Fri, 7 Oct 2011 14:48:44 +0000 (UTC) Received: by samu.pl (Postfix, from userid 1001) id B3D24CDBBD; Fri, 7 Oct 2011 16:48:43 +0200 (CEST) To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Fri, 07 Oct 2011 16:48:43 +0200 From: s Message-ID: <7a4b6f4b2366333f640027303870e0e6@samu.pl> X-Sender: s@samu.pl User-Agent: RoundCube Webmail/0.5.1 Subject: MAC Framework, retreiving information from sysctl X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 14:48:45 -0000 Hi, Once again I would like to ask some question about kernel module programming using the MAC framework - but this time it may not be strictly related to MAC. So, I have made a simple security module (which I will publish as soon as I'll finish it), and now I'd like to have some nice tool to modify it's "rules" from the userland. Exactly the same thing is done by ugidfw, to modify bsdextended rules. But, here the problem appears - I don't know how to perform that. I've got a simple, two-dimensional array which contains these "rules" (if I can call them that way), and I would like to modify that array from the userland. So, I guess the best way to do so would be sysctl(3). This is what I've created, after reading mac_bsdextended source: kernel module: static int sysctl_rule(SYSCTL_HANDLER_ARGS) { // ... some code to parse what has been sent ... return (0); } SYSCTL_NODE(_security_somemodule, OID_AUTO, rules, CTLFLAG_MPSAFE | CTLFLAG_RW, sysctl_rule, "Some description"); And now I should be able to send some data using sysctl(3). But I really don't know how - and here my question comes. How should I send this data? How should I parse SYSCTL_HANDLER_ARGS (from struct.h: #define SYSCTL_HANDLER_ARGS struct sysctl_oid *oidp, void *arg1, int arg2, struct sysctl_req *req */) to edit my array? I will be grateful for any help - this is the only thing that remained in this project... ;) -- Pozdrawiam, Jakub 'samu' SzafraÅ„ski From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 09:10:45 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B1631065690 for ; Fri, 7 Oct 2011 09:10:45 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 36A288FC08 for ; Fri, 7 Oct 2011 09:10:43 +0000 (UTC) Received: by iage36 with SMTP id e36so1928812iag.13 for ; Fri, 07 Oct 2011 02:10:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=QTPqb6RzKxeQLISwBOZ8jQKlwJrBNZefmQyw5ZZM1hA=; b=lay+2qI83JLmbHvuBlzq4C/PC7XCavDXucZh6w9Cg+uFhVq2HCwunu6jgRoJvPgnFQ 8pgzlsm5lCWV5oVaMaE8ODd2Dl2tD7wO2v+wcY4P2uuUnyrkr0UmrOR1iOAC8H5MsGWY iTth+8nQkPHtLnogWc7cbYgafp5uGSFV8VzHE= MIME-Version: 1.0 Received: by 10.231.69.80 with SMTP id y16mr2967859ibi.34.1317978643002; Fri, 07 Oct 2011 02:10:43 -0700 (PDT) Received: by 10.231.35.194 with HTTP; Fri, 7 Oct 2011 02:10:40 -0700 (PDT) Received: by 10.231.35.194 with HTTP; Fri, 7 Oct 2011 02:10:40 -0700 (PDT) In-Reply-To: <4e8f073c.3g2aD/Zz9KdsWOKN%perryh@pluto.rain.com> References: <20111004160043.GA16034@lpthe.jussieu.fr> <051853CE-03EC-4EEC-A5AC-C380131B28E4@gsoft.com.au> <4e8f073c.3g2aD/Zz9KdsWOKN%perryh@pluto.rain.com> Date: Fri, 7 Oct 2011 10:10:40 +0100 Message-ID: From: Chris Rees To: perryh@pluto.rain.com X-Mailman-Approved-At: Fri, 07 Oct 2011 15:46:27 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, ivoras@freebsd.org Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 09:10:45 -0000 On 7 Oct 2011 08:13, wrote: > > Ivan Voras wrote: > > On 05/10/2011 09:38, Trond Endrest??l wrote: > > > On Wed, 5 Oct 2011 12:54+1030, Daniel O'Connor wrote: > > >> In my experience ncsd seems to cache negative hits forever, > > >> regardless of the setting for negative-time-to-live. > > > > > > I'm glad to see I'm not the only one who has noticed this odd > > > behaviour of nscd. Shame on me for not speaking up sooner, but > > > I feared I might be proved wrong (again), and yes, that's a > > > lame excuse. :-/ > > > > +1. > > > > It's very annoying when installing ports which add users - the > > port adds it then in some future code checks it and it fails. > > I've noticed it with at least CUPS. > > Sounds as if there ought to be a unified mechanism for ports > to use when adding users, so that necessary notifications -- > e.g. restarting nscd if it is running -- can be done in a > standardized way and any necessary customizations can be done > in a single place. There is, and I've been trying (with other people) to make older ports respect this framework. Chris From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 17:23:55 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A78BE106564A for ; Fri, 7 Oct 2011 17:23:55 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id E2CBC8FC13 for ; Fri, 7 Oct 2011 17:23:54 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.4) with ESMTP id p97HNoJ1008788; Fri, 7 Oct 2011 19:23:50 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.4/Submit) with ESMTP id p97HNoTg008785; Fri, 7 Oct 2011 19:23:50 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Fri, 7 Oct 2011 19:23:50 +0200 (CEST) From: Wojciech Puchar To: alc@freebsd.org In-Reply-To: Message-ID: References: <20111006160159.GQ1511@deviant.kiev.zoral.com.ua> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="1626729238-606807853-1318008230=:8664" X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Fri, 07 Oct 2011 19:23:50 +0200 (CEST) Cc: Kostik Belousov , hackers@freebsd.org, Grzegorz Kulewski Subject: Re: mmap performance and memory use X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 17:23:55 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --1626729238-606807853-1318008230=:8664 Content-Type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT > > You are correct about the page table page.  However, a superpage mapping consumes a single PV entry, in place of 512 or 1024 PV > entries.  This winds up saving about three physical pages worth of memory for every superpage mapping. does it actually work? simple test before (only idle system with 2GB RAM and most free) vm.pmap.pde.promotions: 921 vm.pmap.pde.p_failures: 21398 vm.pmap.pde.mappings: 299 vm.pmap.pde.demotions: 596 vm.pmap.shpgperproc: 200 vm.pmap.pv_entry_max: 696561 vm.pmap.pg_ps_enabled: 1 vm.pmap.pat_works: 1 and with that program running (==sleeping) #include int a[1<<24]; main() { int b; for(b=0;b<(1<<24);b++) a[b]=b; sleep(1000); } vm.pmap.pdpe.demotions: 0 vm.pmap.pde.promotions: 952 vm.pmap.pde.p_failures: 21398 vm.pmap.pde.mappings: 299 vm.pmap.pde.demotions: 596 vm.pmap.shpgperproc: 200 vm.pmap.pv_entry_max: 696561 vm.pmap.pg_ps_enabled: 1 vm.pmap.pat_works: 1 seems like i don't understand what these sysctl things mean (i did sysctl -d) or it doesn't really work. with program allocating and using linear 64MB chunk it should be 31 or 32 more mappings in vm.pmap.pde.mappings there are zero difference. --1626729238-606807853-1318008230=:8664-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 17:39:14 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E1BA1106564A for ; Fri, 7 Oct 2011 17:39:12 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9E6168FC14 for ; Fri, 7 Oct 2011 17:39:12 +0000 (UTC) Received: by vws11 with SMTP id 11so4722191vws.13 for ; Fri, 07 Oct 2011 10:39:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=bTGYv7796STHE4ErRqgwoQr7DJFRZPigHCq/G4eNzKE=; b=GT62QPNMuFDqZKt8gY5lNk6x3OOB41NRm+hsxXu9VEnddnJw0z76Z/CYWaqmFND3nH RNaw0kDmcL7RsbHsIUMVSaInlhr/e3lzdbHFuS0jJlMGgAzPkhINYCD3yH/YfUD7HGWf wD2Emma2IQJl18NlGyEKTdvYml22d1BoKIQ+Y= MIME-Version: 1.0 Received: by 10.68.208.229 with SMTP id mh5mr15208962pbc.124.1318007611585; Fri, 07 Oct 2011 10:13:31 -0700 (PDT) Received: by 10.142.166.3 with HTTP; Fri, 7 Oct 2011 10:13:31 -0700 (PDT) In-Reply-To: <20111006160159.GQ1511@deviant.kiev.zoral.com.ua> References: <20111006160159.GQ1511@deviant.kiev.zoral.com.ua> Date: Fri, 7 Oct 2011 12:13:31 -0500 Message-ID: From: Alan Cox To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Wojciech Puchar , hackers@freebsd.org, Grzegorz Kulewski Subject: Re: mmap performance and memory use X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alc@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 17:39:14 -0000 On Thu, Oct 6, 2011 at 11:01 AM, Kostik Belousov wrote: > On Thu, Oct 06, 2011 at 04:41:45PM +0200, Wojciech Puchar wrote: > > i have few questions. > > > > 1) suppose i map 1TB of address space as anonymous and touch just one > > page. how much memory is used to manage this? > I am not sure how deep the enumeration you want to know, but the first > approximation will be: > one struct vm_map_entry > one struct vm_object > one pv_entry > > Page table structures need four pages for directories and page table > proper. > > > > 2) suppose we have 1TB file on disk without holes and 100000 processes > > mmaps this file to it's address space. are just pages shared or can > > pagetables be shared too? how much memory is used to manage such > > situation? > Only pages are shared. Pagetables are not. > > For one thing, this indeed causes more memory use for the OS. This is > somewhat mitigated by automatic use of superpages. Superpage promotion > still keeps the 4KB page table around, so most savings from the > superpages are due to more efficient use of TLB. > > You are correct about the page table page. However, a superpage mapping consumes a single PV entry, in place of 512 or 1024 PV entries. This winds up saving about three physical pages worth of memory for every superpage mapping. Alan From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 18:33:13 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D232106566B for ; Fri, 7 Oct 2011 18:33:13 +0000 (UTC) (envelope-from resitsahin@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4F0798FC0A for ; Fri, 7 Oct 2011 18:33:13 +0000 (UTC) Received: by ggeq3 with SMTP id q3so3797906gge.13 for ; Fri, 07 Oct 2011 11:33:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=msUkzW+KY55L7oElZibuC3AiLCiRFltd6Zb9qlsG5ik=; b=eE3vGdaqo6VnZJbxUH8hTGsD9IUKv4eOs71M/HrqpBX3EFMqUHfuSqvkjVih2fg1eh 0NWQyuT/U9Ye+sFl11rtWVPTJWUYI45hBK3SvcktZQ/LAY9FnOzeM2XoAfWvNJIdctd4 ETFHyFL5ZsE09uchqQEa8bbEiGx53pBDlQKEw= Received: by 10.223.92.152 with SMTP id r24mr12216907fam.19.1318010895593; Fri, 07 Oct 2011 11:08:15 -0700 (PDT) Received: from [192.168.1.101] ([78.174.250.20]) by mx.google.com with ESMTPS id f10sm13860350fac.14.2011.10.07.11.08.14 (version=SSLv3 cipher=OTHER); Fri, 07 Oct 2011 11:08:15 -0700 (PDT) Message-ID: <4E8F400D.80305@gmail.com> Date: Fri, 07 Oct 2011 21:08:13 +0300 From: =?UTF-8?B?TXVzdGFmYSBSZcWfaXQgxZ5haGlu?= User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110922 Thunderbird/3.1.15 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Using Valgrind on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 18:33:13 -0000 Hello All, I am trying to run Valgrind on FreeBSD. I am getting the error about ksem_open which i stated below. I have searched for a solution to be able to solve this problem and found the calgrind patch. I could not found the instructions to apply this patch. How can i use valgrind on FreeBSD with ksem_open enabled? Kind Regards Resit Sahin I use valgrind version: valgrind-3.6.0 FreeBSD Version :FreeBSD 8.1-RELEASE-p2 FreeBSD 8.1-RELEASE-p2 #6: amd64 The error i get : (The valgrind output) --84521-- WARNING: unhandled syscall: 404 ==84521== at 0x144E2BC: __sys_ksem_init (in /lib/libc.so.7) ==84521== by 0x14422FE: sem_init (in /lib/libc.so.7) ==84521== by 0x1639BBB: snf__sem_ring_open (in /usr/local/opt/snf/lib/libsnf.so.0.3) ==84521== by 0x163A1AC: snf__open_endpoint_ring (in /usr/local/opt/snf/lib/libsnf.so.0.3) ==84521== by 0x163A5A3: snf__board_open (in /usr/local/opt/snf/lib/libsnf.so.0.3) ==84521== by 0x1637A7A: snf_open (in /usr/local/opt/snf/lib/libsnf.so.0.3) ==84521== by 0xF04DBA: snf_activate (in /usr/local/opt/snf/lib/libpcap.so.1.1.1) ==84521== by 0xF05C26: pcap_activate (in /usr/local/opt/snf/lib/libpcap.so.1.1.1) ==84521== by 0xF0624A: pcap_open_live (in /usr/local/opt/snf/lib/libpcap.so.1.1.1) ==84521== by 0x452B4D: ??? (in /usr/sbin/tcpdump) ==84521== by 0x402E4D: ??? (in /usr/sbin/tcpdump) ==84521== by 0x41FFF: ??? --84521-- You may be able to write your own handler. --84521-- Read the file README_MISSING_SYSCALL_OR_IOCTL. --84521-- Nevertheless we consider this a bug. Please report --84521-- it at http://valgrind.org/support/bug_reports.html. From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 18:38:52 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A03531065674 for ; Fri, 7 Oct 2011 18:38:52 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5DB488FC16 for ; Fri, 7 Oct 2011 18:38:52 +0000 (UTC) Received: by qadz30 with SMTP id z30so3998886qad.13 for ; Fri, 07 Oct 2011 11:38:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=2qlUFq34NirPDoHuNeVNed+WdMVegfYOkFKWiL6dL70=; b=QNUtQoZpoo0MMRV52IkBTMcc65G/hlIeWsmSJr6nBBah8hYgs7dUAqrWbJnXINcTpS mqSFmLrSKF5gjsJhuRyyURUetYCro4NRvy+bit4I06RcsihMhawwOs57ODiAHxUwUOPV pHKDvTv4lvliE4g1yhRUtqAEie8i1tMNqvUUQ= MIME-Version: 1.0 Received: by 10.229.65.165 with SMTP id j37mr1799624qci.205.1318012731801; Fri, 07 Oct 2011 11:38:51 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.229.83.196 with HTTP; Fri, 7 Oct 2011 11:38:51 -0700 (PDT) In-Reply-To: <4E8F400D.80305@gmail.com> References: <4E8F400D.80305@gmail.com> Date: Fri, 7 Oct 2011 11:38:51 -0700 X-Google-Sender-Auth: _lq8x_ps_MQHogTho_mpd9jTXS8 Message-ID: From: mdf@FreeBSD.org To: =?ISO-8859-9?Q?Mustafa_Re=FEit_=DEahin?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Using Valgrind on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 18:38:52 -0000 2011/10/7 Mustafa Re=C5=9Fit =C5=9Eahin : > I am trying to run Valgrind on FreeBSD. I am getting the error about > ksem_open which i stated below. I have searched for =C2=A0a solution to b= e able > to solve this problem and found the calgrind patch. I could not found the > instructions to apply this patch. How can i use valgrind on FreeBSD with > ksem_open enabled? It looks like that syscall is only available when the P1003_1B_SEMAPHORES option is in your kernel config. You will need to edit a kernel configuration file and rebuild / reinstall the kernel. Thanks, matthew > I use valgrind version: =C2=A0valgrind-3.6.0 > FreeBSD Version :FreeBSD 8.1-RELEASE-p2 FreeBSD 8.1-RELEASE-p2 #6: =C2=A0= amd64 > > The error i get : (The valgrind output) > > > --84521-- WARNING: unhandled syscall: 404 > =3D=3D84521=3D=3D =C2=A0 =C2=A0at 0x144E2BC: __sys_ksem_init (in /lib/lib= c.so.7) > =3D=3D84521=3D=3D =C2=A0 =C2=A0by 0x14422FE: sem_init (in /lib/libc.so.7) > =3D=3D84521=3D=3D =C2=A0 =C2=A0by 0x1639BBB: snf__sem_ring_open (in > /usr/local/opt/snf/lib/libsnf.so.0.3) > =3D=3D84521=3D=3D =C2=A0 =C2=A0by 0x163A1AC: snf__open_endpoint_ring (in > /usr/local/opt/snf/lib/libsnf.so.0.3) > =3D=3D84521=3D=3D =C2=A0 =C2=A0by 0x163A5A3: snf__board_open (in > /usr/local/opt/snf/lib/libsnf.so.0.3) > =3D=3D84521=3D=3D =C2=A0 =C2=A0by 0x1637A7A: snf_open (in > /usr/local/opt/snf/lib/libsnf.so.0.3) > =3D=3D84521=3D=3D =C2=A0 =C2=A0by 0xF04DBA: snf_activate (in > /usr/local/opt/snf/lib/libpcap.so.1.1.1) > =3D=3D84521=3D=3D =C2=A0 =C2=A0by 0xF05C26: pcap_activate (in > /usr/local/opt/snf/lib/libpcap.so.1.1.1) > =3D=3D84521=3D=3D =C2=A0 =C2=A0by 0xF0624A: pcap_open_live (in > /usr/local/opt/snf/lib/libpcap.so.1.1.1) > =3D=3D84521=3D=3D =C2=A0 =C2=A0by 0x452B4D: ??? (in /usr/sbin/tcpdump) > =3D=3D84521=3D=3D =C2=A0 =C2=A0by 0x402E4D: ??? (in /usr/sbin/tcpdump) > =3D=3D84521=3D=3D =C2=A0 =C2=A0by 0x41FFF: ??? > --84521-- You may be able to write your own handler. > --84521-- Read the file README_MISSING_SYSCALL_OR_IOCTL. > --84521-- Nevertheless we consider this a bug. =C2=A0Please report > --84521-- it at http://valgrind.org/support/bug_reports.html. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 7 21:44:15 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 203EA106566C for ; Fri, 7 Oct 2011 21:44:15 +0000 (UTC) (envelope-from lgj@usenix.org) Received: from lonestar.usenix.org (lonestar.usenix.org [131.106.3.102]) by mx1.freebsd.org (Postfix) with ESMTP id E34F18FC08 for ; Fri, 7 Oct 2011 21:44:14 +0000 (UTC) Received: from negroni.usenix.org (negroni.usenix.org [131.106.3.145]) (authenticated bits=0) by lonestar.usenix.org (8.14.2/8.14.2) with ESMTP id p97Lhngj014469 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Fri, 7 Oct 2011 14:44:12 -0700 (PDT) From: Lionel Garth Jones Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Date: Fri, 7 Oct 2011 14:44:10 -0700 Message-Id: <5DCC8663-FB82-4D41-9244-1E9BBFED7BAD@usenix.org> To: freebsd-hackers@freebsd.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) X-DCC-Usenix-Metrics: lonestar; whitelist X-Spam-Status: No, score=0.7 required=6.0 tests=ALL_TRUSTED, FH_DATE_PAST_20XX autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on lonestar X-Mailman-Approved-At: Fri, 07 Oct 2011 22:26:34 +0000 Subject: USENIX WebApps '12 Call for Papers Now Available X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2011 21:44:15 -0000 On behalf of the Program Committee, I would like to invite you to submit your work to the 3rd USENIX Conference on Web Application Development (WebApps '12). Like the inaugural WebApps '10 and '11, WebApps '12 seeks to attract cutting-edge research that advances the state of the art, not only of novel Web applications but also of infrastructure,frameworks, tools, and = techniques that support the development, analysis/testing, operation, or deployment of those applications. Possible topics include but are not limited to:=20 * Storage for Web-scale applications=20 * Techniques for testing and debugging=20 * Novel strategies for fault tolerance or high availability in Web apps=20= * The Web as an emerging platform in new application areas=20 * Making Web apps social and integrating with social utilities, e.g., Facebook, Twitter, Google+ * HCI techniques related specifically to Web apps=20 * Measurement, modeling, workload generation, and other tools to aid experimental research on Web apps=20 * New and unusual app features or implementation techniques=20 * Media delivery applications and infrastructure=20 * Client-side libraries, frameworks, toolkits, plug-ins=20 * Server-side frameworks=20 * Languages and language engineering advances relevant to Web app development * Deployment substrates and technologies (cloud computing, = infrastructure as a service, testing as a service, etc.) WebApps '12 will consist of a single track of refereed papers and, new this year, a set of short paper demos. Papers with practical significance and/or working prototypes will be preferred over purely theoretical results. Short paper demo submissions must have a working prototype. Submitted papers must be no longer than 12 pages for regular papers and 6 pages for demo papers, including figures, tables, and references. For more details on the submission process, advice on how to prepare a competitive paper, and templates to use with LaTeX, Word, etc., authors should consult http://www.usenix.org/events/webapps12/cfp/requirements.html Paper submissions are due by January 23, 2012, 11:59 p.m. PST. WebApps '12 will take place June 13-14, 2012, in Boston, MA. It is a co-located event that occurs during USENIX Federated Conferences Week, which spans June 12=9615, 2012. I look forward to receiving your submissions. Sincerely, Michael Maximilien, IBM Research WebApps '12 Program Chair webapps12chair@usenix.org ------------------------------------------------------------------------ Call for Papers: 3rd USENIX Conference on Web Application Development (WebApps '12)=20 June 13-14, 2012, Boston, MA http://www.usenix.org/webapps12/cfpa=20 Paper submissions deadline: January 23, 2012, 11:59 p.m. PST ------------------------------------------------------------------------= From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 8 00:15:13 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3438D106564A for ; Sat, 8 Oct 2011 00:15:13 +0000 (UTC) (envelope-from redcrash@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id C62098FC14 for ; Sat, 8 Oct 2011 00:15:12 +0000 (UTC) Received: by wwe3 with SMTP id 3so6329739wwe.31 for ; Fri, 07 Oct 2011 17:15:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=+XlbPH9J2699B1q6xw/ryvaHmATpceJ2tHdG+WMlvM4=; b=uFMP9ttrfuLddA5n2xjV+GFqKEt5G1maxr06RbsRB12JQb2S/d5hzIydGv3Ld/oWUt S/4IBXMeQ2N6Hc9kbLF13nDOiw81u4IoRu1WXzaVqt5fSuROrWpujtfgA83LEfUcrJXS DG9GYvjQlt45P/qa+p5wBCdkVbM0MgJ0WdS40= MIME-Version: 1.0 Received: by 10.216.199.81 with SMTP id w59mr3128217wen.32.1318031077655; Fri, 07 Oct 2011 16:44:37 -0700 (PDT) Received: by 10.216.45.83 with HTTP; Fri, 7 Oct 2011 16:44:37 -0700 (PDT) Date: Sat, 8 Oct 2011 01:44:37 +0200 Message-ID: From: Harald Servat To: FreeBSD Hackers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Power usage on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Oct 2011 00:15:13 -0000 Dear hackers, does anyone know if I can measure power consumption of the processor via software by using FreeBSD on a Intel Core i5? Any pointer on this, or other architectures, will be welcome. Thank you very much in advance! -- Fry: You can see how I lived before I met you. Bender: You lived before you met me?! Fry: Yeah, lots of people did. Bender: Really?! From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 8 07:24:53 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 071AA106564A; Sat, 8 Oct 2011 07:24:53 +0000 (UTC) (envelope-from omerfsen@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8F9D68FC14; Sat, 8 Oct 2011 07:24:52 +0000 (UTC) Received: by qyk4 with SMTP id 4so4699839qyk.13 for ; Sat, 08 Oct 2011 00:24:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=1BGL3vhTV7OrO5mtiyvjHyF0VMDPBKaSN2RMbEK4JZA=; b=wHHMBZX0yZ/eU2AvgDIaotoqtFYN/mnqixMn8oY8QRyvwOvtXjooQuHwTRsBeWUR3l VS7Yll21gJzNadI029hwxcbf/WFHr2QuJ21RvaUjHYYj8Ixbk//cjnhj0OmawGmqrwML JIx+OB5DLTi1DS1qb5u17zu5pJqeqKJnDAGgU= MIME-Version: 1.0 Received: by 10.229.241.135 with SMTP id le7mr2149933qcb.149.1318057064315; Fri, 07 Oct 2011 23:57:44 -0700 (PDT) Received: by 10.229.51.130 with HTTP; Fri, 7 Oct 2011 23:57:44 -0700 (PDT) In-Reply-To: References: <4E8F400D.80305@gmail.com> Date: Sat, 8 Oct 2011 09:57:44 +0300 Message-ID: From: Omer Faruk SEN To: mdf@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, romain@freebsd.org Subject: Re: Using Valgrind on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Oct 2011 07:24:53 -0000 Hi, May it be related with the patches for Valgrind that http://people.freebsd.org/~romain/valgrind/ written? It may require to rebuild valgrind with that patch but I am not sure. Maybe Romain can answer that question. Romain I have tried to use your patches against latest valgrind port: # Date created: April 19 2004 # Whom: Simon Barner # # $FreeBSD: ports/devel/valgrind/Makefile,v 1.50 2011/04/27 09:51:08 stas Exp $ # PORTNAME=3D valgrind PORTVERSION=3D 3.6.1 PORTREVISION=3D 2 PORTEPOCH=3D 1 but I got cc -DHAVE_CONFIG_H -I. -I.. -I.. -I../include -I../VEX/pub -DVGA_amd64=3D1 -DVGO_freebsd=3D1 -DVGP_amd64_freebsd=3D1 -I../coregrind -DVG_LIBDIR=3D"\"/usr/local/lib/valgrind"\" -DVG_PLATFORM=3D"\"amd64-freebs= d\"" -m64 -fomit-frame-pointer -O2 -g -Wall -Wmissing-prototypes -Wshadow -Wpointer-arith -Wstrict-prototypes -Wmissing-declarations -Wno-format-zero-length -fno-strict-aliasing -Wno-long-long -O2 -pipe -funroll-loops -fno-strict-aliasing -Wno-pointer-sign -fno-stack-protector -MT libcoregrind_amd64_freebsd_a-syswrap-freebsd.o -MD -MP -MF .deps/libcoregrind_amd64_freebsd_a-syswrap-freebsd.Tpo -c -o libcoregrind_amd64_freebsd_a-syswrap-freebsd.o `test -f 'm_syswrap/syswrap-freebsd.c' || echo './'`m_syswrap/syswrap-freebsd.c m_syswrap/syswrap-freebsd.c:3989: error: '__NR_mq_open' undeclared here (no= t in a function) m_syswrap/syswrap-freebsd.c:3989: error: array index in initializer not of integer type m_syswrap/syswrap-freebsd.c:3989: error: (near initialization for 'vgModuleLocal_syscall_table') m_syswrap/syswrap-freebsd.c:3990: error: '__NR_mq_getsetattr' undeclared here (not in a function) m_syswrap/syswrap-freebsd.c:3990: error: array index in initializer not of integer type m_syswrap/syswrap-freebsd.c:3990: error: (near initialization for 'vgModuleLocal_syscall_table') m_syswrap/syswrap-freebsd.c:3991: error: '__NR_mq_timedreceive' undeclared here (not in a function) m_syswrap/syswrap-freebsd.c:3991: error: array index in initializer not of integer type m_syswrap/syswrap-freebsd.c:3991: error: (near initialization for 'vgModuleLocal_syscall_table') m_syswrap/syswrap-freebsd.c:3993: error: '__NR_mq_timedsend' undeclared her= e (not in a function) m_syswrap/syswrap-freebsd.c:3993: error: array index in initializer not of integer type m_syswrap/syswrap-freebsd.c:3993: error: (near initialization for 'vgModuleLocal_syscall_table') m_syswrap/syswrap-freebsd.c:3994: error: '__NR_mq_notify' undeclared here (not in a function) m_syswrap/syswrap-freebsd.c:3994: error: array index in initializer not of integer type m_syswrap/syswrap-freebsd.c:3994: error: (near initialization for 'vgModuleLocal_syscall_table') m_syswrap/syswrap-freebsd.c:3995: error: '__NR_mq_unlink' undeclared here (not in a function) m_syswrap/syswrap-freebsd.c:3995: error: array index in initializer not of integer type m_syswrap/syswrap-freebsd.c:3995: error: (near initialization for 'vgModuleLocal_syscall_table') gmake[3]: *** [libcoregrind_amd64_freebsd_a-syswrap-freebsd.o] Error 1 gmake[3]: Leaving directory `/usr/ports/devel/valgrind/work/valgrind-freebsd-3.6.1-2/coregrind' gmake[2]: *** [all] Error 2 gmake[2]: Leaving directory `/usr/ports/devel/valgrind/work/valgrind-freebsd-3.6.1-2/coregrind' gmake[1]: *** [all-recursive] Error 1 gmake[1]: Leaving directory `/usr/ports/devel/valgrind/work/valgrind-freebsd-3.6.1-2' gmake: *** [all] Error 2 *** Error code 1 Stop in /usr/ports/devel/valgrind. *** Error code 1 Stop in /usr/ports/devel/valgrind. 2011/10/7 > 2011/10/7 Mustafa Re=C5=9Fit =C5=9Eahin : > > I am trying to run Valgrind on FreeBSD. I am getting the error about > > ksem_open which i stated below. I have searched for a solution to be > able > > to solve this problem and found the calgrind patch. I could not found t= he > > instructions to apply this patch. How can i use valgrind on FreeBSD wit= h > > ksem_open enabled? > > It looks like that syscall is only available when the > P1003_1B_SEMAPHORES option is in your kernel config. You will need to > edit a kernel configuration file and rebuild / reinstall the kernel. > > Thanks, > matthew > > > > I use valgrind version: valgrind-3.6.0 > > FreeBSD Version :FreeBSD 8.1-RELEASE-p2 FreeBSD 8.1-RELEASE-p2 #6: amd= 64 > > > > The error i get : (The valgrind output) > > > > > > --84521-- WARNING: unhandled syscall: 404 > > =3D=3D84521=3D=3D at 0x144E2BC: __sys_ksem_init (in /lib/libc.so.7) > > =3D=3D84521=3D=3D by 0x14422FE: sem_init (in /lib/libc.so.7) > > =3D=3D84521=3D=3D by 0x1639BBB: snf__sem_ring_open (in > > /usr/local/opt/snf/lib/libsnf.so.0.3) > > =3D=3D84521=3D=3D by 0x163A1AC: snf__open_endpoint_ring (in > > /usr/local/opt/snf/lib/libsnf.so.0.3) > > =3D=3D84521=3D=3D by 0x163A5A3: snf__board_open (in > > /usr/local/opt/snf/lib/libsnf.so.0.3) > > =3D=3D84521=3D=3D by 0x1637A7A: snf_open (in > > /usr/local/opt/snf/lib/libsnf.so.0.3) > > =3D=3D84521=3D=3D by 0xF04DBA: snf_activate (in > > /usr/local/opt/snf/lib/libpcap.so.1.1.1) > > =3D=3D84521=3D=3D by 0xF05C26: pcap_activate (in > > /usr/local/opt/snf/lib/libpcap.so.1.1.1) > > =3D=3D84521=3D=3D by 0xF0624A: pcap_open_live (in > > /usr/local/opt/snf/lib/libpcap.so.1.1.1) > > =3D=3D84521=3D=3D by 0x452B4D: ??? (in /usr/sbin/tcpdump) > > =3D=3D84521=3D=3D by 0x402E4D: ??? (in /usr/sbin/tcpdump) > > =3D=3D84521=3D=3D by 0x41FFF: ??? > > --84521-- You may be able to write your own handler. > > --84521-- Read the file README_MISSING_SYSCALL_OR_IOCTL. > > --84521-- Nevertheless we consider this a bug. Please report > > --84521-- it at http://valgrind.org/support/bug_reports.html. > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 8 10:42:03 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB52C106564A; Sat, 8 Oct 2011 10:42:03 +0000 (UTC) (envelope-from romain@blogreen.org) Received: from marvin.blogreen.org (smortex-1-pt.tunnel.tserv11.ams1.ipv6.he.net [IPv6:2001:470:1f14:7c2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 6E3B78FC13; Sat, 8 Oct 2011 10:42:03 +0000 (UTC) Received: by marvin.blogreen.org (Postfix, from userid 1001) id 4D4582487E; Sat, 8 Oct 2011 12:42:02 +0200 (CEST) Date: Sat, 8 Oct 2011 12:42:02 +0200 From: Romain =?iso-8859-1?Q?Tarti=E8re?= To: Omer Faruk SEN Message-ID: <20111008104202.GA9473@FreeBSD.org> References: <4E8F400D.80305@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="PEIAKu/WMn1b1Hv9" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://romain.blogreen.org/pubkey.asc User-Agent: Mutt/1.5.21 (2010-09-15) Cc: mdf@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: Using Valgrind on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Oct 2011 10:42:03 -0000 --PEIAKu/WMn1b1Hv9 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi! On Sat, Oct 08, 2011 at 09:57:44AM +0300, Omer Faruk SEN wrote: > May it be related with the patches for Valgrind that > http://people.freebsd.org/~romain/valgrind/ written? It may require to > rebuild valgrind with that patch but I am not sure. Maybe Romain can answ= er > that question. Long story short: I wrote some code I wanted to test with Valgrind and got the message saying that some syscall wrappers where not available. Since the IPC stuff was quite trivial and my concerns where about another part of the code, I started hacking these system calls in order to avoid Valgrind quitting when they where called so that it could check the 'important' part of my code. Then, I wondered how hard it would have been to make a complete IPC syscalls support for FreeBSD in Valgrind, unfortunately I could not get much time to take on this and I decided to push my patches 'as it' at some location I could get them back is necessary. =2E.. and it looks like Google indexed it. Basically, the code is copy-pasta with minor tweaks of other parts of Valgrind. It is NOT production-ready. It does not build because I pushed it after trying to add support for all message queues and semaphores syscalls after testing my program with the 2-3 system calls that where missing for it to run. I took a few minutes to fix the build (at least on 8-STABLE amd64, new patches are here: http://people.freebsd.org/~romain/valgrind/2011-10-08/ ) but I have not even tried to test anything, so beware! To use these patches, create a 'files' directory in the Valgrind port and put them there, then install the port as usual. If you would like to work on expanding IPC support in Valgrind on FreeBSD, please write me a line. Maybe we can coordinate some work ;-) Thanks! Romain --=20 Romain Tarti=E8re http://people.FreeBSD.org/~romain/ pgp: 8234 9A78 E7C0 B807 0B59 80FF BA4D 1D95 5112 336F (ID: 0x5112336F) (plain text =3Dnon-HTML=3D PGP/GPG encrypted/signed e-mail much appreciated) --PEIAKu/WMn1b1Hv9 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQGcBAEBAgAGBQJOkCj4AAoJELpNHZVREjNvWxgL/0hq4rsNWLjFDM962EmyLaqy d5qXRBcgvm28Zb9i+gUC+evLwrltNDJ3r1lVxmm0NEGbG47FTRsL9wzluGC4Ix/Q MUUveWXe4MKc2I26YZKdKRcNzXMpaZztErrGaolu8lXOu7t3lWUg85Wz8Xi28tEz HQsFya3HV85OKjzAgwCTrEFHaMxEpVLkMRT9k/ihfKu7sjZyIj4ui975KvdKCrBl VVjHDE1b4FujKCtGGv+xQLDLDmQJWzf9bBUcdgijLcx6tjLQo9JIhRwn1OSBlMje nfWKKE/3bXOKYiaNQXbZNGmtYgxn6hfo4DyNexp1q3SZgvwIb3s+9hQwZEov1+21 cTIp/oJipoXP6Djm/m9o82rJi1YMC4HITQphyYfM/ipMq2kverxnC2ui2DmhT5ix N7U7AOuKTPSOFrP+In1eC8Cc0Ggoz3YB0gFaOh2qax45q3iY1AFzjDJMC+LtWBNN rZ31xjvds4HA+qvewod6d+CvzdUvwqw/7t6iRuA0RA== =M7rS -----END PGP SIGNATURE----- --PEIAKu/WMn1b1Hv9-- From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 8 12:30:36 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A29B106566B for ; Sat, 8 Oct 2011 12:30:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id A22D38FC08 for ; Sat, 8 Oct 2011 12:30:33 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p98CUTJA053680 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 8 Oct 2011 15:30:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id p98CUTXk010286; Sat, 8 Oct 2011 15:30:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id p98CUTfZ010285; Sat, 8 Oct 2011 15:30:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 8 Oct 2011 15:30:29 +0300 From: Kostik Belousov To: Harald Servat Message-ID: <20111008123029.GD1511@deviant.kiev.zoral.com.ua> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Anc4Ty0odT0wy+qc" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: FreeBSD Hackers Subject: Re: Power usage on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Oct 2011 12:30:36 -0000 --Anc4Ty0odT0wy+qc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Oct 08, 2011 at 01:44:37AM +0200, Harald Servat wrote: > Dear hackers, >=20 > does anyone know if I can measure power consumption of the processor via > software by using FreeBSD on a Intel Core i5? Any pointer on this, or oth= er > architectures, will be welcome. >=20 > Thank you very much in advance! Measuring the power consumption by CPU would probably require you to get some NDA documentation from Intel. On the Core-class CPUs there are definitely the facilities related to the Turbo Mode, that show inner working of the power management microcontroller. See the Linux IPS driver, which controls both CPU and Northbridge/GPU. I was unable to find a public documentation about it. Some information is present in the CPU/PCH datasheets. On the other hand, if you are only concerned about whole system power consumption, use a laptop or a managed UPS. --Anc4Ty0odT0wy+qc Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk6QQmQACgkQC3+MBN1Mb4gJPgCgiU20WJkBo9I8Vk34vk6ep7yH 1y8AoK1bQfzLtNUbb7FmhUQy1ZxaRUHM =Q1vN -----END PGP SIGNATURE----- --Anc4Ty0odT0wy+qc-- From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 8 12:31:10 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B2691065673; Sat, 8 Oct 2011 12:31:10 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) by mx1.freebsd.org (Postfix) with ESMTP id E04DA8FC08; Sat, 8 Oct 2011 12:31:09 +0000 (UTC) Received: from toad.stack.nl (toad.stack.nl [IPv6:2001:610:1108:5010::135]) by mx1.stack.nl (Postfix) with ESMTP id C37E31DD436; Sat, 8 Oct 2011 14:31:08 +0200 (CEST) Received: by toad.stack.nl (Postfix, from userid 1677) id BBF0F3F46B; Sat, 8 Oct 2011 14:31:08 +0200 (CEST) Date: Sat, 8 Oct 2011 14:31:08 +0200 From: Jilles Tjoelker To: Artem Belevich Message-ID: <20111008123108.GA44476@stack.nl> References: <86sjn84wco.fsf@ds4.des.no> <86obxw4s4w.fsf@ds4.des.no> <86fwj84p8i.fsf@ds4.des.no> <86wrcjf767.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Michael Bushkov , Dag-Erling Sm?rgrav , hackers@freebsd.org, Trond Endrest?l Subject: Re: Does anyone use nscd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Oct 2011 12:31:10 -0000 On Wed, Oct 05, 2011 at 03:54:00PM -0700, Artem Belevich wrote: > 2011/10/5 Dag-Erling Sm?rgrav : > > Michael Bushkov writes: > >> 2. Consequences of the aforementioned problem can probably be > >> corrected by using _setsockopt(..., SO_NOSIGPIPE) in > >> __open_cached_connection() in nscachedcli.c > > That sounds like a workaround rather than a fix... > Not necessarily. Using SO_NOSIGPIPE is a valid option when someone > wants to see read/write on a closed socket fail and return -1 with > errno=EPIPE. > Quick grep in libc shows that resolver code in > lib/libc/resolv/res_send.c also sets SO_NOSIGPIPE for exactly that > reason. Disabling SIGPIPE is good anyway because a crashing/dying nscd should not cause applications to terminate. However, if EPIPE/SIGPIPE happens in normal operation, that is still a bug that should be fixed. By the way, SO_NOSIGPIPE is not in POSIX.1-2008 while the MSG_NOSIGNAL flag to send() is. It may be better to replace the write() call with send() with the MSG_NOSIGNAL flag and drop the setsockopt(). -- Jilles Tjoelker