From owner-freebsd-arch@FreeBSD.ORG Sun Mar 15 10:08:46 2009 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68185106564A for ; Sun, 15 Mar 2009 10:08:46 +0000 (UTC) (envelope-from robert@fledge.watson.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 18C538FC15 for ; Sun, 15 Mar 2009 10:08:46 +0000 (UTC) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id C5D3446B2D for ; Sun, 15 Mar 2009 06:08:45 -0400 (EDT) X-Return-Path: X-Received: from cyrus.watson.org ([unix socket]) by cyrus.watson.org (Cyrus v2.3.13) with LMTPA; Sun, 15 Mar 2009 06:04:58 -0400 X-Sieve: CMU Sieve 2.3 X-Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by cyrus.watson.org (Postfix) with ESMTP id 306D146B0D for ; Sun, 15 Mar 2009 06:04:58 -0400 (EDT) X-Received: from hub.freebsd.org (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 2AEA8161684; Sun, 15 Mar 2009 10:04:40 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) X-Received: from hub.freebsd.org (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 0C9FC10656C8; Sun, 15 Mar 2009 10:04:38 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) X-Delivered-To: current@FreeBSD.org X-Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 38FE3106564A; Sun, 15 Mar 2009 10:04:30 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) X-Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 1421D8FC12; Sun, 15 Mar 2009 10:04:30 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) X-Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 98BF546B0D; Sun, 15 Mar 2009 06:04:29 -0400 (EDT) Date: Sun, 15 Mar 2009 10:04:29 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: current@FreeBSD.org, net@FreeBSD.org In-Reply-To: Message-ID: References: <20080526110543.J26343@fledge.watson.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Sender: owner-freebsd-current@freebsd.org Errors-To: owner-freebsd-current@freebsd.org ReSent-Date: Sun, 15 Mar 2009 10:08:43 +0000 (GMT) ReSent-From: robert ReSent-To: arch@FreeBSD.org ReSent-Subject: Re: HEADS UP: IFF_NEEDSGIANT consumers to be disabled, removed (was: Re: Wiki page for non-MPSAFE network stack de-orbit scheduling) ReSent-Message-ID: ReSent-User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) Cc: Subject: Re: HEADS UP: IFF_NEEDSGIANT consumers to be disabled, removed (was: Re: Wiki page for non-MPSAFE network stack de-orbit scheduling) X-BeenThere: freebsd-arch@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2009 10:08:46 -0000 On Mon, 16 Feb 2009, Robert Watson wrote: > 16 February 2009 HEADS UP to lists (this e-mail) > 01 March 2009 Disable build of all IFF_NEEDSGIANT drivers in 8.x Slightly delayed while USB NDIS was sorted out, I will be moving ahead with disabling the build of all IFF_NEEDSGIANT drivers in HEAD later today. > 01 April 2009 Remove all IFF_NEEDSGIANT drivers from 8.x And this date is probably going to remain about the same. Anyone who is interested is welcome to step in and update/fix drivers on the removal path at any time, and drivers can also be restored later easily if drivers are updated outside the tree as well. Thanks, Robert N M Watson Computer Laboratory University of Cambridge _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sun Mar 15 20:03:17 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9BFC9106564A for ; Sun, 15 Mar 2009 20:03:17 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from gigi.cs.uoguelph.ca (gigi.cs.uoguelph.ca [131.104.94.210]) by mx1.freebsd.org (Postfix) with ESMTP id 5CA1C8FC1F for ; Sun, 15 Mar 2009 20:03:17 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by gigi.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id n2FJjqo1031516 for ; Sun, 15 Mar 2009 15:45:54 -0400 Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n2FJp9j22990 for ; Sun, 15 Mar 2009 15:51:09 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Sun, 15 Mar 2009 15:51:09 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: freebsd-arch@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanned-By: MIMEDefang 2.63 on 131.104.94.210 Subject: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2009 20:03:19 -0000 As some of you might already know, I have been working on an implementation of NFSv4.0 for BSD for some time now. My original interest in NFSv4.0 came about because I felt that a component of it called delegations might permit significant performance improvements along the lines of what I had hoped to achieve with an experimental protocol I called Not Quite NFS long long ago. To experiment with the use of delegations, I needed an implementation and I got to work on that around Fall 2001/Winter 2002. (If I had had any idea how long it was going to take to get to a near production quality implementation, I would have never started, but I'm finally there.:-) There has been some indication of interest in incorporating this into FreeBSD-CURRENT/8 and I am posting here to start a discussion of this. I feel (and Robert Watson seems to agree) that replacing the current NFS implementation would be impractical for at least one release cycle. It does include NFSv2 and 3 support in it, but it would take some time to gain experience with the implementation and incorporate all the features in the current release into it. As such, it seems appropriate to have it live side-by-side with the current implementation. Here's a brief overview of the current code base: - approximately 75,000 lines of C in the following subdirs nfscl - generic client code that can be shared by the ports. (This includes doing the RPCs using code that avoids most of my infamous macros and handling functions for the NFSv4 state.) nfsclient - a copy of sys/nfsclient tweaked so that it calls functions in nfscl instead of doing the v2,3 RPCs. (As such, it retains the caching semantics, etc of the current client.) nfs - functions shared by the client and server nfsd - the server code nfscrypto - a couple of small functions for helping with RPCSEC_GSS (these aren't required for Doug Rabson's new RPC code) Current work: I am currently in the process of converting it to use Doug Rabson's new RPC layer and re-organizing things so that the two servers can co-reside in the kernel (the client already does co-reside). This work should be completed in a few weeks. Beyond that, the only NFSv4.0 feature I might be looking at that isn't yet implemented is referrals. Code stability: The server code ran here in two FreeBSD NFS servers for a couple of years without difficulties. The client has been tested fairly extensively for the Mac OS X port (25 iMacs in a lab here + about 350 downloads by others). At this point I suspect bugs will be mostly related to architectures other than i386 and, maybe, issues running on fast, heavily SMP machines. (The code is SMP safe, to the best of my knowledge.) What to call it? I currently call it newnfs with options NFSCL and NFSD, but I could care less what it is called. Calling it nfsv4 seemed inaccurate, since it does have nfsv2 and v3 support in it. So, what do folks think? rick ps: I haven't even looked at NFSv4.1 implementation yet. From owner-freebsd-arch@FreeBSD.ORG Sun Mar 15 20:43:48 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66E061065670 for ; Sun, 15 Mar 2009 20:43:48 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id 319FE8FC31 for ; Sun, 15 Mar 2009 20:43:48 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: (from root@localhost) by kientzle.com (8.14.3/8.14.3) id n2FKMI4w085069; Sun, 15 Mar 2009 13:22:18 -0700 (PDT) (envelope-from kientzle@freebsd.org) Received: from [10.123.2.23] (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id d48wh833fbi3jrysapgbnhbmz6; Sun, 15 Mar 2009 13:22:17 -0700 (PDT) (envelope-from kientzle@freebsd.org) Message-ID: <49BD6378.9030501@freebsd.org> Date: Sun, 15 Mar 2009 13:22:16 -0700 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060422 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Rick Macklem References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2009 20:43:48 -0000 Ah, yes. I remember reading the NQNFS paper some years ago. Nice work! Q: Does this support NFSv4 ACLs? As I recall, trasz@ has NFSv4 patches in P4 that have yet to be merged; I've been waiting on that to work on NFSv4 support for tar/cpio. Q: How does this relate to the new NFS lockd recently committed? (by Doug Rabson? I can't remember now.) As for schedule, that depends. There was talk about branching 8.0 in April. If that's still the plan, then it might be best to just wait. But the website still announces August. If that's the plan, I'd be happy to push this in now with a build option to select which implementation to use. Have that default to the new implementation now in -CURRENT, but leave the option to ship 8.0 with the old implementation as the default. That would give a fair bit of production exposure during the 8.x cycle (some people will enable the new code, especially by 8.1 when the new code should have stabilized some). Then switch the default to the new code for 9.0. As for naming, I see no problems calling it (lowercase) nfs4 even if it does support (uppercase) NFSv2 and NFSv3. Tim Rick Macklem wrote: > As some of you might already know, I have been working on an > implementation of NFSv4.0 for BSD for some time now. My original > interest in NFSv4.0 came about because I felt that a component > of it called delegations might permit significant performance > improvements along the lines of what I had hoped to achieve with > an experimental protocol I called Not Quite NFS long long ago. > > To experiment with the use of delegations, I needed an implementation > and I got to work on that around Fall 2001/Winter 2002. (If I had > had any idea how long it was going to take to get to a near production > quality implementation, I would have never started, but I'm finally > there.:-) > > There has been some indication of interest in incorporating this > into FreeBSD-CURRENT/8 and I am posting here to start a discussion > of this. > > I feel (and Robert Watson seems to agree) that replacing the current > NFS implementation would be impractical for at least one release > cycle. It does include NFSv2 and 3 support in it, but it would take > some time to gain experience with the implementation and incorporate > all the features in the current release into it. As such, it seems > appropriate to have it live side-by-side with the current implementation. > > Here's a brief overview of the current code base: > - approximately 75,000 lines of C in the following subdirs > nfscl - generic client code that can be shared by the ports. > (This includes doing the RPCs using code that avoids most of my > infamous macros and handling functions for the NFSv4 state.) > nfsclient - a copy of sys/nfsclient tweaked so that it calls functions > in nfscl instead of doing the v2,3 RPCs. (As such, it retains the > caching semantics, etc of the current client.) > nfs - functions shared by the client and server > nfsd - the server code > nfscrypto - a couple of small functions for helping with RPCSEC_GSS > (these aren't required for Doug Rabson's new RPC code) > > Current work: > I am currently in the process of converting it to use Doug Rabson's new > RPC layer and re-organizing things so that the two servers can co-reside > in the kernel (the client already does co-reside). This work should be > completed in a few weeks. Beyond that, the only NFSv4.0 feature I might > be looking at that isn't yet implemented is referrals. > > Code stability: > The server code ran here in two FreeBSD NFS servers for a couple of years > without difficulties. The client has been tested fairly extensively for > the Mac OS X port (25 iMacs in a lab here + about 350 downloads by > others). At this point I suspect bugs will be mostly related to > architectures other than i386 and, maybe, issues running on fast, > heavily SMP machines. (The code is SMP safe, to the best of my > knowledge.) > > What to call it? > I currently call it newnfs with options NFSCL and NFSD, but I could > care less what it is called. Calling it nfsv4 seemed inaccurate, since > it does have nfsv2 and v3 support in it. > > So, what do folks think? rick > ps: I haven't even looked at NFSv4.1 implementation yet. > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > From owner-freebsd-arch@FreeBSD.ORG Sun Mar 15 20:56:24 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7EB47106564A for ; Sun, 15 Mar 2009 20:56:24 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from skerryvore.cs.uoguelph.ca (skerryvore.cs.uoguelph.ca [131.104.94.204]) by mx1.freebsd.org (Postfix) with ESMTP id 243468FC1B for ; Sun, 15 Mar 2009 20:56:23 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by skerryvore.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id n2FKuMuS004055; Sun, 15 Mar 2009 16:56:22 -0400 Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n2FL1eX02315; Sun, 15 Mar 2009 17:01:40 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Sun, 15 Mar 2009 17:01:40 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Tim Kientzle In-Reply-To: <49BD6378.9030501@freebsd.org> Message-ID: References: <49BD6378.9030501@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanned-By: MIMEDefang 2.63 on 131.104.94.204 Cc: freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2009 20:56:24 -0000 On Sun, 15 Mar 2009, Tim Kientzle wrote: > Q: Does this support NFSv4 ACLs? As I recall, > trasz@ has NFSv4 patches in P4 that have yet to > be merged; I've been waiting on that to work on > NFSv4 support for tar/cpio. > It works fine with it. Most of the work w.r.t. ACLs is in the local file systems on the server. (It just required a little bit of tweaking of my code, which was done a few months ago.) > Q: How does this relate to the new NFS lockd > recently committed? (by Doug Rabson? I can't > remember now.) > They will share the same RPC code, once I have completed the conversion. I basically just copied the client side lock code over into nfsclient, but have never tested it. As far as the server goes, I'd have to look. NFSv4 locking doesn't use lockd, but my server does grab byte range locks through the VFS and I suspect lockd sees those, just like any other process sees them. (One advantage of NFSv4 is integrated locking that seems to work well.) [good stuff w.r.t. integration snipped for brevity] rick From owner-freebsd-arch@FreeBSD.ORG Sun Mar 15 21:07:34 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 285DF106566B for ; Sun, 15 Mar 2009 21:07:34 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 183A98FC24 for ; Sun, 15 Mar 2009 21:07:33 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 0046F1A3C3B; Sun, 15 Mar 2009 13:52:29 -0700 (PDT) Date: Sun, 15 Mar 2009 13:52:29 -0700 From: Alfred Perlstein To: Rick Macklem Message-ID: <20090315205229.GV55200@elvis.mu.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2009 21:07:34 -0000 * Rick Macklem [090315 13:03] wrote: > As some of you might already know, I have been working on an > implementation of NFSv4.0 for BSD for some time now. My original > interest in NFSv4.0 came about because I felt that a component > of it called delegations might permit significant performance > improvements along the lines of what I had hoped to achieve with > an experimental protocol I called Not Quite NFS long long ago. ... > So, what do folks think? rick > ps: I haven't even looked at NFSv4.1 implementation yet. I think this is awesome! I think it wise to look at 4.1 and scoping that out before taking the time to integrate this to gain an understanding of: 1) what it would take to get to 4.1? 2) how we would interoperate with other machines until we get 4.1 (is everyone doing 4.0 or 4.1?). When will 4.1 become the defacto standard (is it already?)? 3) trade-off between taking the 4.0 code or how we can get 4.1 first before integration. This is important because the bug reports on 4.0 can swamp someone and prevent new feature development (4.1). You know more about it than most of us, so can you take some time to look at 4.1 and give some guestimates so we can gauge what to do next? Again, this is really cool and I apologize if 4.1 is something far out on the horizon, I just am approaching this from an integration perspective, not as an authority on NFSv4 standardization process. If the case is 4.1 is some number of years out, then I think my answer is "let's get this in ASAP!", otherwise we should think about it. thank you, -- - Alfred Perlstein From owner-freebsd-arch@FreeBSD.ORG Sun Mar 15 21:07:53 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0E791065676; Sun, 15 Mar 2009 21:07:53 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id ADBB28FC1A; Sun, 15 Mar 2009 21:07:53 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 9502F1A3C39; Sun, 15 Mar 2009 14:07:53 -0700 (PDT) Date: Sun, 15 Mar 2009 14:07:53 -0700 From: Alfred Perlstein To: Rick Macklem Message-ID: <20090315210753.GX55200@elvis.mu.org> References: <49BD6378.9030501@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: Tim Kientzle , freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2009 21:07:54 -0000 * Rick Macklem [090315 13:57] wrote: > > As far as the server goes, I'd have to look. NFSv4 locking > doesn't use lockd, but my server does grab byte range locks > through the VFS and I suspect lockd sees those, just like > any other process sees them. (One advantage of NFSv4 is > integrated locking that seems to work well.) I think the way that kernel server NFS lockd works is that it sits here: V TCPIP V lockd -> calls ASYNC locks down (expects callback) V VFS -> calls back to lockd when locks are acquired It's _not_ like this: V TCPIP V lockd -> calls SYNC locks down V VFS V lockd -> stuff so unless your server uses the async lock facility, you'll wind up with your server context blocked in VFS waiting for locks. (I think.) -Alfred From owner-freebsd-arch@FreeBSD.ORG Sun Mar 15 21:15:05 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D1445106564A; Sun, 15 Mar 2009 21:15:05 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from gigi.cs.uoguelph.ca (gigi.cs.uoguelph.ca [131.104.94.210]) by mx1.freebsd.org (Postfix) with ESMTP id 52F1D8FC0C; Sun, 15 Mar 2009 21:15:05 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by gigi.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id n2FLF4Vk029513; Sun, 15 Mar 2009 17:15:04 -0400 Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n2FLKKL04890; Sun, 15 Mar 2009 17:20:20 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Sun, 15 Mar 2009 17:20:20 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Alfred Perlstein In-Reply-To: <20090315205229.GV55200@elvis.mu.org> Message-ID: References: <20090315205229.GV55200@elvis.mu.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanned-By: MIMEDefang 2.63 on 131.104.94.210 Cc: freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2009 21:15:06 -0000 On Sun, 15 Mar 2009, Alfred Perlstein wrote: > > I think it wise to look at 4.1 and scoping that out before taking > the time to integrate this to gain an understanding of: NFSv4.1 is still way out there. It hasn't reached RFC stage yet and vendors are only testing bits and pieces of it. (The current draft of the "minor" revision is over 500 pages.) All the code vendors are currently shipping is running 4.0. > > 1) what it would take to get to 4.1? A lot. A required feature is something for handling RPC transport called sessions. One guy has been looking at doing sessions for FreeBSD (hopefully integrated with Doug Rabson's new RPC code), but I have no idea if he has made any progress. > 2) how we would interoperate with other machines until we > get 4.1 (is everyone doing 4.0 or 4.1?). When will 4.1 become > the defacto standard (is it already?)? Systems should still support 4.0 for a long time. I have no idea when 4.1 will become a defacto standard, but I'd guess years. (For 4.0, the RFC is dated 2003 and the Linux implementation of 4.0 is still listed as experimental, I believe. 4.0 shipped with Solaris10, which was the first production quality implementation, imho. Apple only shipped an incomplete, non-usable 4.0 with Leopard. I think that gives you some idea how long it has taken to get from RFC to shipping. 4.1 is almost an RFC, but not quite yet.) > 3) trade-off between taking the 4.0 code or how we can > get 4.1 first before integration. This is important because > the bug reports on 4.0 can swamp someone and prevent new feature > development (4.1). > > You know more about it than most of us, so can you take some time > to look at 4.1 and give some guestimates so we can gauge what to > do next? > I've tried reading the drafts and got swamped. Honestly, I think a 4.1 implementation would take man years of effort and is beyond what I am capable of. Once you have sessions (non trivial, from what I saw), pNFS is what everyone gets excited about, but it's a major project. To do a server requires back end server<->server protocols that aren't spelled out by the draft. A client side of pNFS needs to know how to handle the various layout schemes (there are 2 other drafts beyond the 500+ page document for 2 other schemes than the most basic whole file one). > Again, this is really cool and I apologize if 4.1 is something far > out on the horizon, I just am approaching this from an integration > perspective, not as an authority on NFSv4 standardization process. > > If the case is 4.1 is some number of years out, then I think my > answer is "let's get this in ASAP!", otherwise we should think > about it. > Yes, I think 4.1 is some years away. rick From owner-freebsd-arch@FreeBSD.ORG Mon Mar 16 00:37:50 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 900EB1065670 for ; Mon, 16 Mar 2009 00:37:50 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 7DA3D8FC1A for ; Mon, 16 Mar 2009 00:37:50 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 5745F1A3C39; Sun, 15 Mar 2009 17:37:50 -0700 (PDT) Date: Sun, 15 Mar 2009 17:37:50 -0700 From: Alfred Perlstein To: Rick Macklem Message-ID: <20090316003750.GY55200@elvis.mu.org> References: <20090315205229.GV55200@elvis.mu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2009 00:37:51 -0000 * Rick Macklem [090315 14:15] wrote: > > [[snipped but certainly appreciated]] > >If the case is 4.1 is some number of years out, then I think my > >answer is "let's get this in ASAP!", otherwise we should think > >about it. > > > Yes, I think 4.1 is some years away. That along with the fact that you stated, that everyone is shipping 4.0, means that it makes a lot of sense to proceed with your code. Thanks very much for the summary. -- - Alfred Perlstein From owner-freebsd-arch@FreeBSD.ORG Mon Mar 16 05:41:56 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D14F106566C for ; Mon, 16 Mar 2009 05:41:56 +0000 (UTC) (envelope-from jhein@timing.com) Received: from Daffy.timing.com (ns2.timing.com [206.168.13.218]) by mx1.freebsd.org (Postfix) with ESMTP id D71508FC0C for ; Mon, 16 Mar 2009 05:41:55 +0000 (UTC) (envelope-from jhein@timing.com) Received: from gromit.timing.com (gromit.timing.com [206.168.13.209]) by Daffy.timing.com (8.13.1/8.13.1) with ESMTP id n2G5fsnL075825 for ; Sun, 15 Mar 2009 23:41:55 -0600 (MDT) (envelope-from jhein@timing.com) Received: from gromit.timing.com (localhost [127.0.0.1]) by gromit.timing.com (8.14.3/8.14.3) with ESMTP id n2G5MUZG054713; Sun, 15 Mar 2009 23:22:30 -0600 (MDT) (envelope-from jhein@gromit.timing.com) Received: (from jhein@localhost) by gromit.timing.com (8.14.3/8.14.3/Submit) id n2G5MU73054710; Sun, 15 Mar 2009 23:22:30 -0600 (MDT) (envelope-from jhein) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18877.57878.136116.691250@gromit.timing.com> Date: Sun, 15 Mar 2009 23:22:30 -0600 From: John Hein To: "M. Warner Losh" In-Reply-To: <20090315.080814.669286040.imp@bsdimp.com> References: <20090313125115.GU31961@hoeg.nl> <20090313.090038.-890725739.imp@bsdimp.com> <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> X-Mailer: VM 7.19 under Emacs 22.3.1 X-Virus-Scanned: ClamAV version 0.91.2, clamav-milter version 0.91.2 on Daffy.timing.com X-Virus-Status: Clean Cc: ed@80386.nl, arch@freebsd.org Subject: Re: Final sanity pass: xdev X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2009 05:41:56 -0000 M. Warner Losh wrote at 08:08 +0900 on Mar 15, 2009: > In message: <18875.60334.947446.966085@gromit.timing.com> > John Hein writes: > : M. Warner Losh wrote at 09:00 -0600 on Mar 13, 2009: > : > In message: <20090313125115.GU31961@hoeg.nl> > : > Ed Schouten writes: > : > : Hi Warner, > : > : > : > : The last couple of days/weeks I've been working a lot with LLVM+Clang. > : > : I'm currently writing some BSD makefiles to see how hard it is to add > : > : Clang to our base system, as a proof of concept. > : > : > : > : A nice thing about Clang is that it seems to be very easy to compile in > : > : multiple backends into the same Clang binary. If we ever switch to Clang > : > : in the base system (who knows), it would be interesting to see whether > : > : it's useful to just compile in all backends by default. > : > : > : > : Anyway, I took a quick look at the xdev patch and it looks okay. :-) > : > > : > Woot! Please let me know if it causes problems. > : > : I meant to mention this earlier - OSREL (referenced in the _xi-links: > : target) is not defined. So ports looking for arm-freebsd8.0-cc (for > : instance) won't find it. > : > : > : An earlier patch of yours had: > : > : .if !defined(OSREL) > : OSREL!= uname -r | sed -e 's/[-(].*//' > : .endif > : > : But the latest (and the one committed) does not. > > You are right... Dang. I must have comitted the wrong thing... One more thing... Now that we've switched to bsd ar, installing gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built ports can't find arm-freebsd8.0-ar Adding usr.bin/ar to the cross tools does the job... Index: Makefile.inc1 =================================================================== RCS file: /base/FreeBSD-CVS/src/Makefile.inc1,v retrieving revision 1.620 diff -u -p -r1.620 Makefile.inc1 --- Makefile.inc1 13 Mar 2009 10:40:38 -0000 1.620 +++ Makefile.inc1 16 Mar 2009 05:11:12 -0000 @@ -1035,6 +1035,7 @@ cross-tools: .for _tool in \ gnu/usr.bin/binutils \ gnu/usr.bin/cc \ + usr.bin/ar \ usr.bin/sed \ usr.bin/xlint/lint1 usr.bin/xlint/lint2 usr.bin/xlint/xlint \ ${_btxld} \ @@ -1370,7 +1371,8 @@ _xb-build-tools: _xb-cross-tools: .for _tool in \ gnu/usr.bin/binutils \ - gnu/usr.bin/cc + gnu/usr.bin/cc \ + usr.bin/ar ${_+_}@${ECHODIR} "===> xdev ${_tool} (obj,depend,all)"; \ cd ${.CURDIR}/${_tool}; \ ${CDMAKE} DIRPRFX=${_tool}/ obj; \ @@ -1395,7 +1397,8 @@ _xi-cross-tools: @echo "_xi-cross-tools" .for _tool in \ gnu/usr.bin/binutils \ - gnu/usr.bin/cc + gnu/usr.bin/cc \ + usr.bin/ar ${_+_}@${ECHODIR} "===> xdev ${_tool} (install)"; \ cd ${.CURDIR}/${_tool}; \ ${CDMAKE} DIRPRFX=${_tool}/ install DESTDIR=${XDDESTDIR} From owner-freebsd-arch@FreeBSD.ORG Mon Mar 16 06:32:56 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15452106564A for ; Mon, 16 Mar 2009 06:32:56 +0000 (UTC) (envelope-from apache@ss37.shared.server-system.net) Received: from ss37.shared.server-system.net (ss37.shared.server-system.net [64.207.144.3]) by mx1.freebsd.org (Postfix) with ESMTP id EBB608FC20 for ; Mon, 16 Mar 2009 06:32:55 +0000 (UTC) (envelope-from apache@ss37.shared.server-system.net) Received: from ss37.shared.server-system.net (localhost.localdomain [127.0.0.1]) by ss37.shared.server-system.net (8.12.11.20060308/8.12.11) with ESMTP id n2G3OXZW008567 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sun, 15 Mar 2009 20:24:33 -0700 Received: (from apache@localhost) by ss37.shared.server-system.net (8.12.11.20060308/8.12.11/Submit) id n2G3OX9c008565; Sun, 15 Mar 2009 20:24:33 -0700 Date: Sun, 15 Mar 2009 20:24:33 -0700 Message-Id: <200903160324.n2G3OX9c008565@ss37.shared.server-system.net> X-MT-MESSAGEID: K2L2hvbWUvdmlydHVhbC9zaXRlNDEvZnN0L3Zhci93d3cvaHRtbC9zbGlkZWxsaG9tZXMuY29tL3dlYXRoZXIvaHczLnBocCwvL3dlYXRoZXIvaHczLnBocC8vaHczLnBocD9kYXlzb25seT0wKS5pbmNsdWRlKCRfR0VUJTViZmlsZSU1ZCkuKDAmZmlsZT1odHRwOi8vbWlkbmlnaHRjcjN3LmJ5LnJ1L2JhZC50eHQ/LDYwLjUwLjE0NC4xNDg= To: freebsd-arch@freebsd.org From: Visa CardŽ / MsnŽ INTERNATIONAL MEGA JACKPOT 2009 MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 8bit Subject: CONTACT CLAIMS ADMINISTRATOR X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Vicamsprdemet2009@hotmail.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2009 06:32:56 -0000 Visa CardŽ / MsnŽ INTERNATIONAL MEGA JACKPOT 2009 PROMOTION/CLAIMS DEPARTMENT 440 THE STRAND LONDON, WC2R 0QS ENGLAND, UNITED KINGDOM DIRECTOR:DR.GLENN EDWARD Attn:Winner Winning No:VCard/877/798/2009 Email Ref No:VCard/699/33/2009 Notification Date:26/01/2009 AMOUNT WON:500,000.00GBP (Five Hundred Thousand Great Britain Pounds). This email address has brought you an unexpected luck,Your e-mail address was selected and confirmed by our co-sponsor Msn International, through their latest internet software.You are therefore been approved by Visa Card Int./Msn UK the sum of 500,000.00 (Five Hundred Thousand Great Britain Pounds). =================================== CONTACT CLAIMS ADMINISTRATOR =================================== NAME: Mr.David Ronald Email: Vicamsprdemet2009@hotmail.com Phone # : +(44) 703 596 7375 Fax # :+(44) 454 464 9443 Visa CardŽ / MsnŽ Promotion Department Do email the above Claims Administrator, at once with all the claims requirements below.To avoid unnecessary delay.They are needed to proceed. Claims Requirements: 1. Full Name:_____________________ 2. Address:_____________________ 3. Nationality:___________Sex:________ 4. Age:________Date of Birth:___________ 5. Occupation:_________Martial Status_________ 6. Cell Phone:___________Fax:___________ 7. State of Origin:_________Country:_______ 8.Winning No:_______Email Ref No:______ PROCEEDURES / RIGHTS AND PRECAUTIONS Choose from payment options and Contact the Claims Administrator with all your claims requirements well filled @ Vicamsprdemet2009@hotmail.com (i).Bank Transfer (ii).Delivery of Prepaid Visa cardŽ Valued 500,000.00GBP by a registered Courier Company. Sincerely, Mrs.Dora Lazmon(Secretary) Visa CardŽ /MsnŽ Mega Jackpot Š2009 Microsoft Corporation.All rights reserved From owner-freebsd-arch@FreeBSD.ORG Mon Mar 16 11:06:52 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C08F106568E for ; Mon, 16 Mar 2009 11:06:52 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0F7118FC15 for ; Mon, 16 Mar 2009 11:06:52 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2GB6pck043187 for ; Mon, 16 Mar 2009 11:06:51 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2GB6ppi043183 for freebsd-arch@FreeBSD.org; Mon, 16 Mar 2009 11:06:51 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 16 Mar 2009 11:06:51 GMT Message-Id: <200903161106.n2GB6ppi043183@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2009 11:06:52 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 16 16:57:58 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F00D21065670; Mon, 16 Mar 2009 16:57:58 +0000 (UTC) (envelope-from lulf@FreeBSD.org) Received: from bene1.itea.ntnu.no (bene1.itea.ntnu.no [IPv6:2001:700:300:3::56]) by mx1.freebsd.org (Postfix) with ESMTP id D87308FC21; Mon, 16 Mar 2009 16:57:57 +0000 (UTC) (envelope-from lulf@FreeBSD.org) Received: from localhost (localhost [127.0.0.1]) by bene1.itea.ntnu.no (Postfix) with ESMTP id 20C1F16C73E; Mon, 16 Mar 2009 17:57:56 +0100 (CET) Received: from carrot (unknown [IPv6:2001:700:300:3::184]) by bene1.itea.ntnu.no (Postfix) with ESMTP id 6273D16C3A2; Mon, 16 Mar 2009 17:57:55 +0100 (CET) Date: Mon, 16 Mar 2009 16:58:00 +0100 From: Ulf Lilleengen To: freebsd-current@freebsd.org Message-ID: <20090316155800.GA2257@carrot> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KsGdsel6WgEHnImy" Content-Disposition: inline User-Agent: Mutt/1.5.19 (2009-01-05) X-Virus-Scanned: Debian amavisd-new at bene1.itea.ntnu.no Cc: freebsd-arch@freebsd.org, freebsd-geom@freebsd.org Subject: [HEADS UP] Merge of projects/gvinum to head X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2009 16:58:00 -0000 --KsGdsel6WgEHnImy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, This is a heads-up for a merge of gvinum project code into HEAD. This means that gvinum implementation will be changed some. The code is based on the work done by Lukas Ertl as well as the work I did before/during Google SoC 2007 and afterwards. It has been staying in p4 for some time, and then moved to the subversion project repository not long ago. The main reason for the delay of getting this into HEAD have been the lack of reviewers of the code, but after some discussion and help from testers, I've decided this is a good time to get it in (should perhaps have been merged a bit earlier) Testers have spotted several differences from the original gvinum, and I've tried to make it behave as the old implementation wherever that seemed the best way = to go. Luckily, the work has gotten a bit of attention lately, thanks to Rick = C. Petty for helping out with testing and bugfixing, as well as all others who have dared to run the new gvinum. So, what does this update offer? =46rom the user aspect: - Implementation of many of the missing commands from the old vinum: attach/detach, start (was partially implemented), stop (was partially implemented), concat, mirror, stripe, raid5 (shortcuts for creating volum= es with one plex of these organizations). - Support for fixing degraded plexes while mounted. - Support for growing for striped and raid5-plexes, meaning that one can extend the volumes for these plex types in addition to the concat type. Also works while the volume is mounted. - The parity check and rebuild no longer goes between userland/kernel, meaning that your gvinum command will not stay and wait forever for the rebuild to finish. You can instead watch the status with the list command. - Many problems with gvinum have been reported since 5.x, and some has been hard to fix due to the complicated architecture. Hopefully, it should be more stable and better handle edge cases that previously made gvinum crash. - Failed drives no longer disappears entirely, but now leave behind a dummy drive that makes sure the original state is not forgotten in case the system is rebooted between drive failures/swaps. =46rom the technical aspect: - Gvinum now uses one single workerthread instead of one thread for each volume and each plex. The reason for this is that the previous scheme was very complex, and was the cause of many of the bugs discovered in gvinum. Instead, gvinum now uses one worker thread with an event queue, quite similar to what used in gmirror. - The rebuild/grow/initialize/parity check routines no longer runs in separate threads, but are run as regular I/O requests with special flags. This made it easier to support on-mount growing and parity rebuild. Probably, there are many other issues that have been fixed, perhaps new issues introduced. This is why this is dropped in HEAD now, to give it more attention from the uses before the 8.0 branch. All in all, this will make gvinum an easier beast to handle for users. If you have any issues related = to this, send in problem reports or drop me an e-mail, and I'll take a look. For interested reviewers, the code resides in svn://svn.freebsd.org/base/projects/gvinum --=20 Ulf Lilleengen --KsGdsel6WgEHnImy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkm+dwAACgkQCILg8nMIdCV7TwCfYyxEvLTHgZqcscpXqPdldHIK 81EAnj4lc0hUj/O78iMd3gFEgs6rmRe+ =BAnc -----END PGP SIGNATURE----- --KsGdsel6WgEHnImy-- From owner-freebsd-arch@FreeBSD.ORG Mon Mar 16 16:59:57 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 698A8106566B; Mon, 16 Mar 2009 16:59:57 +0000 (UTC) (envelope-from lulf@FreeBSD.org) Received: from bene1.itea.ntnu.no (bene1.itea.ntnu.no [IPv6:2001:700:300:3::56]) by mx1.freebsd.org (Postfix) with ESMTP id 2140D8FC25; Mon, 16 Mar 2009 16:59:57 +0000 (UTC) (envelope-from lulf@FreeBSD.org) Received: from localhost (localhost [127.0.0.1]) by bene1.itea.ntnu.no (Postfix) with ESMTP id 4C06B16C73E; Mon, 16 Mar 2009 17:59:56 +0100 (CET) Received: from carrot (unknown [IPv6:2001:700:300:3::184]) by bene1.itea.ntnu.no (Postfix) with ESMTP id 0828E16C7B2; Mon, 16 Mar 2009 17:59:53 +0100 (CET) Date: Mon, 16 Mar 2009 16:59:57 +0100 From: Ulf Lilleengen To: freebsd-current@freebsd.org Message-ID: <20090316155957.GA2385@carrot> References: <20090316155800.GA2257@carrot> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090316155800.GA2257@carrot> User-Agent: Mutt/1.5.19 (2009-01-05) X-Virus-Scanned: Debian amavisd-new at bene1.itea.ntnu.no Cc: freebsd-geom@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [HEADS UP] Merge of projects/gvinum to head X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2009 16:59:58 -0000 On man, mar 16, 2009 at 04:58:00pm +0100, Ulf Lilleengen wrote: > Hello, > > This is a heads-up for a merge of gvinum project code into HEAD. This means > that gvinum implementation will be changed some. The code is based on the > work done by Lukas Ertl as well as the work I did before/during Google SoC > 2007 and afterwards. It has been staying in p4 for some time, and then moved > to the subversion project repository not long ago. The main reason for the > delay of getting this into HEAD have been the lack of reviewers of the code, > but after some discussion and help from testers, I've decided this is a good > time to get it in (should perhaps have been merged a bit earlier) Testers > have spotted several differences from the original gvinum, and I've tried to > make it behave as the old implementation wherever that seemed the best way to > go. Luckily, the work has gotten a bit of attention lately, thanks to Rick C. > Petty for helping out with testing and bugfixing, as well as all others who > have dared to run the new gvinum. So, what does this update offer? And I plan on importing it within 1-2 weeks :) -- Ulf Lilleengen From owner-freebsd-arch@FreeBSD.ORG Mon Mar 16 18:55:47 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75C45106564A for ; Mon, 16 Mar 2009 18:55:47 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 2696E8FC0C for ; Mon, 16 Mar 2009 18:55:47 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.2/8.14.1) with ESMTP id n2GIra3l041078; Mon, 16 Mar 2009 12:53:38 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Tue, 17 Mar 2009 03:54:05 +0900 (JST) Message-Id: <20090317.035405.1683322897.imp@bsdimp.com> To: jhein@timing.com From: "M. Warner Losh" In-Reply-To: <18877.57878.136116.691250@gromit.timing.com> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: ed@80386.nl, arch@freebsd.org Subject: Re: Final sanity pass: xdev X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2009 18:55:47 -0000 thanks! I'll check into this when I finish returning home... Warner In message: <18877.57878.136116.691250@gromit.timing.com> John Hein writes: : M. Warner Losh wrote at 08:08 +0900 on Mar 15, 2009: : > In message: <18875.60334.947446.966085@gromit.timing.com> : > John Hein writes: : > : M. Warner Losh wrote at 09:00 -0600 on Mar 13, 2009: : > : > In message: <20090313125115.GU31961@hoeg.nl> : > : > Ed Schouten writes: : > : > : Hi Warner, : > : > : : > : > : The last couple of days/weeks I've been working a lot with LLVM+Clang. : > : > : I'm currently writing some BSD makefiles to see how hard it is to add : > : > : Clang to our base system, as a proof of concept. : > : > : : > : > : A nice thing about Clang is that it seems to be very easy to compile in : > : > : multiple backends into the same Clang binary. If we ever switch to Clang : > : > : in the base system (who knows), it would be interesting to see whether : > : > : it's useful to just compile in all backends by default. : > : > : : > : > : Anyway, I took a quick look at the xdev patch and it looks okay. :-) : > : > : > : > Woot! Please let me know if it causes problems. : > : : > : I meant to mention this earlier - OSREL (referenced in the _xi-links: : > : target) is not defined. So ports looking for arm-freebsd8.0-cc (for : > : instance) won't find it. : > : : > : : > : An earlier patch of yours had: : > : : > : .if !defined(OSREL) : > : OSREL!= uname -r | sed -e 's/[-(].*//' : > : .endif : > : : > : But the latest (and the one committed) does not. : > : > You are right... Dang. I must have comitted the wrong thing... : : One more thing... Now that we've switched to bsd ar, installing : gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built : ports can't find arm-freebsd8.0-ar : : Adding usr.bin/ar to the cross tools does the job... : : Index: Makefile.inc1 : =================================================================== : RCS file: /base/FreeBSD-CVS/src/Makefile.inc1,v : retrieving revision 1.620 : diff -u -p -r1.620 Makefile.inc1 : --- Makefile.inc1 13 Mar 2009 10:40:38 -0000 1.620 : +++ Makefile.inc1 16 Mar 2009 05:11:12 -0000 : @@ -1035,6 +1035,7 @@ cross-tools: : .for _tool in \ : gnu/usr.bin/binutils \ : gnu/usr.bin/cc \ : + usr.bin/ar \ : usr.bin/sed \ : usr.bin/xlint/lint1 usr.bin/xlint/lint2 usr.bin/xlint/xlint \ : ${_btxld} \ : @@ -1370,7 +1371,8 @@ _xb-build-tools: : _xb-cross-tools: : .for _tool in \ : gnu/usr.bin/binutils \ : - gnu/usr.bin/cc : + gnu/usr.bin/cc \ : + usr.bin/ar : ${_+_}@${ECHODIR} "===> xdev ${_tool} (obj,depend,all)"; \ : cd ${.CURDIR}/${_tool}; \ : ${CDMAKE} DIRPRFX=${_tool}/ obj; \ : @@ -1395,7 +1397,8 @@ _xi-cross-tools: : @echo "_xi-cross-tools" : .for _tool in \ : gnu/usr.bin/binutils \ : - gnu/usr.bin/cc : + gnu/usr.bin/cc \ : + usr.bin/ar : ${_+_}@${ECHODIR} "===> xdev ${_tool} (install)"; \ : cd ${.CURDIR}/${_tool}; \ : ${CDMAKE} DIRPRFX=${_tool}/ install DESTDIR=${XDDESTDIR} : : From owner-freebsd-arch@FreeBSD.ORG Wed Mar 18 19:54:47 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 007341065678; Wed, 18 Mar 2009 19:54:47 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from gigi.cs.uoguelph.ca (gigi.cs.uoguelph.ca [131.104.94.210]) by mx1.freebsd.org (Postfix) with ESMTP id 9EF808FC18; Wed, 18 Mar 2009 19:54:46 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by gigi.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id n2IJsjUl016828; Wed, 18 Mar 2009 15:54:45 -0400 Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n2IK09114993; Wed, 18 Mar 2009 16:00:09 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Wed, 18 Mar 2009 16:00:09 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: freebsd-arch@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanned-By: MIMEDefang 2.63 on 131.104.94.210 Cc: kientzle@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Mar 2009 19:54:47 -0000 > Q: How does this relate to the new NFS lockd > recently committed? (by Doug Rabson? I can't > remember now.) My nfsv4 server only does VOP_ADVLOCK() with the F_GETLK and F_SETLK options, that are non-blocking. I checked with Doug R. and he doesn't think that there will be a problem. So, it looks like it should be ok, rick From owner-freebsd-arch@FreeBSD.ORG Wed Mar 18 22:55:21 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 576521065673 for ; Wed, 18 Mar 2009 22:55:21 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id D57AE8FC19 for ; Wed, 18 Mar 2009 22:55:20 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1Lk4fL-0003s9-DR for freebsd-arch@freebsd.org; Wed, 18 Mar 2009 22:55:16 +0000 Received: from 93-141-116-240.adsl.net.t-com.hr ([93.141.116.240]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 18 Mar 2009 22:55:15 +0000 Received: from ivoras by 93-141-116-240.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 18 Mar 2009 22:55:15 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Wed, 18 Mar 2009 23:54:43 +0100 Lines: 53 Message-ID: <49C17BB3.1010702@freebsd.org> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigF2732FF0C0492FAC97E24ED0" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 93-141-116-240.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.19 (Windows/20081209) In-Reply-To: X-Enigmail-Version: 0.95.7 Sender: news Subject: Re: Magic symlinks redux X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Mar 2009 22:55:21 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigF2732FF0C0492FAC97E24ED0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, I started doing something that would be nicely solved by magic symlinks so I remembered this thread. It started some 6+ months ago here: http://lists.freebsd.org/pipermail/freebsd-arch/2008-August/008473.html Is there any new development? Ivan Voras wrote: > I was reading about new things in NetBSD, and one thing caught my > attention: per-user /tmp. See > http://www.feyrer.de/NetBSD/bx/blosxom.cgi/nb_20080714_0251.html for > example. >=20 > Google says that a discussion about magic symlinks happens every now an= d > then in FreeBSD but nothing really gets done. I found this > implementation which looks like it's for 7.0: >=20 > http://butcher.heavennet.ru/patches/kernel/magiclinks/ >=20 > As far as I understand the VFS (which isn't much...) this looks like an= > trivial patch, and it's compatible with NetBSD. Since I'm interested in= > this (specifically for the per-user /tmp and maybe similar gadgetry), > I'd like to nurse this patch into the tree, if there are no objections > (of course, I'll bug anyone I can find who knows VFS to review it :) ).= >=20 --------------enigF2732FF0C0492FAC97E24ED0 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAknBe8AACgkQldnAQVacBci3EACdFymoSjzQ6cbbbCvG3l632WdZ nmoAoK1FziIPKZEjWf832/LM6QGUycDu =vrUf -----END PGP SIGNATURE----- --------------enigF2732FF0C0492FAC97E24ED0-- From owner-freebsd-arch@FreeBSD.ORG Wed Mar 18 23:40:04 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14DDC106566B; Wed, 18 Mar 2009 23:40:04 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.9.129]) by mx1.freebsd.org (Postfix) with ESMTP id 8748A8FC12; Wed, 18 Mar 2009 23:40:02 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id A805C730A1; Thu, 19 Mar 2009 00:44:47 +0100 (CET) Date: Thu, 19 Mar 2009 00:44:47 +0100 From: Luigi Rizzo To: Ivan Voras Message-ID: <20090318234447.GA21963@onelab2.iet.unipi.it> References: <49C17BB3.1010702@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="fdj2RfSjLxBAspz7" Content-Disposition: inline In-Reply-To: <49C17BB3.1010702@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-arch@freebsd.org Subject: Re: Magic symlinks redux X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Mar 2009 23:40:04 -0000 --fdj2RfSjLxBAspz7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Mar 18, 2009 at 11:54:43PM +0100, Ivan Voras wrote: > Hi, > > I started doing something that would be nicely solved by magic symlinks > so I remembered this thread. It started some 6+ months ago here: > > http://lists.freebsd.org/pipermail/freebsd-arch/2008-August/008473.html > > Is there any new development? at the time i did some quick tests on probably the same code as you reference below, and it is really trivial (attached). I do remember a discussion of a more complete implementation but was many times more complex and more intrusive and I am afraid that such a thing will never happen. BTW today i hit a problem where some form of magic symlinks would be useful. Say you are preparing a disk image in a subtree of your filesystem, and your image contains symlinks with absolute paths: to do things right, you should navigate the subtree in a chroot'ed environment, but chroot() requires root privs. If we had some form of magic symlinks that support a per-process remapping, one could e.g. set a variable that is prepended to the target of absolute symlinks, and achieve the same effect of a chroot without the need for root privs... cheers luigi > Ivan Voras wrote: > > I was reading about new things in NetBSD, and one thing caught my > > attention: per-user /tmp. See > > http://www.feyrer.de/NetBSD/bx/blosxom.cgi/nb_20080714_0251.html for > > example. > > > > Google says that a discussion about magic symlinks happens every now and > > then in FreeBSD but nothing really gets done. I found this > > implementation which looks like it's for 7.0: > > > > http://butcher.heavennet.ru/patches/kernel/magiclinks/ > > > > As far as I understand the VFS (which isn't much...) this looks like an > > trivial patch, and it's compatible with NetBSD. Since I'm interested in > > this (specifically for the per-user /tmp and maybe similar gadgetry), > > I'd like to nurse this patch into the tree, if there are no objections > > (of course, I'll bug anyone I can find who knows VFS to review it :) ). > > --fdj2RfSjLxBAspz7 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="20090211-symlinks.diff" Index: sys/kern/vfs_lookup.c =================================================================== --- sys/kern/vfs_lookup.c (revision 189113) +++ sys/kern/vfs_lookup.c (working copy) @@ -45,6 +45,7 @@ #include #include #include +#include // XXX symlinks #include #include #include @@ -87,6 +88,123 @@ } SYSINIT(vfs, SI_SUB_VFS, SI_ORDER_SECOND, nameiinit, NULL); +#ifdef MAGICLINKS +static int vfs_magiclinks = 1; +#else +static int vfs_magiclinks = 1; +#endif +SYSCTL_INT(_vfs, OID_AUTO, magiclinks, CTLFLAG_RW, &vfs_magiclinks, 0, + "Whether \"magic\" symlinks are expanded"); + +/* looks up a string returns the match len or 0 */ +static int +s_match(const char *key, int keylen, const char *haystack, const char *end) +{ + if (haystack + keylen >= end || haystack[keylen] != '}') + return 0; + if (strncmp(key, haystack, keylen)) + return 0; + return keylen; +} +#define MATCH(str) s_match(str, sizeof(str) - 1, src, end) + +static char * +s_subst(char *dst, const char *max, const char *value, int len) +{ + if (value == dst) { /* already copied, locate end of string */ + while (*dst) + dst++; + return dst; + } + /* check size, copy and replace */ + if (dst + len > max) /* overflow */ + return NULL; + bcopy(value, dst, len); + dst += len; + return dst; +} + +/* + * Substitute replacement text for 'magic' strings in symlinks. + * Looks for "@{string}", where is a + * recognized 'magic' string. Replaces the original with the + * appropriate replacement text. (Note that in some cases the + * replacement text may have zero length.) + * Assume *len is at least 3. + */ +static void +symlink_magic(struct thread *td, char *cp, int *len) +{ + char *src, *dst, *tmp, *end = cp + *len, *max; + int change = 0; + + /* quick return if nothing to replace */ + for (src = cp; src < end - 1; src++) { + if (src[0] == '@' && src[1] == '{') + break; + } + if (src == end - 1) /* no replacement */ + return; + + /* allocate a buffer for the replacement */ + dst = tmp = uma_zalloc(namei_zone, M_WAITOK); + if (dst == NULL) { /* no space for replacement */ + printf("zalloc fail in %s\n", __FUNCTION__); + return; + } + max = dst + MAXPATHLEN - 1; + for (src = cp; src < end - 1 && dst < max - 1;) { + int l; + if (src[0] != '@' || src[1] != '{') { + *dst++ = *src++; /* copy current char */ + continue; + } + src += 2; /* skip @{ */ + +printf("replace magic at %s\n", src); + /* + * The following checks should be ordered according + * to frequency of use. + */ + if ( (l = MATCH("machine_arch")) ) { + dst = s_subst(dst, max, MACHINE_ARCH, sizeof(MACHINE_ARCH) - 1); + } else if ( (l= MATCH("machine")) ) { + dst = s_subst(dst, max, MACHINE_ARCH, sizeof(MACHINE_ARCH) - 1); + } else if ( (l= MATCH("hostname")) ) { + getcredhostname(td->td_ucred, dst, max - dst); + dst = s_subst(dst, max, dst, 0); + } else if ( (l= MATCH("osrelease")) ) { + dst = s_subst(dst, max, osrelease, strlen(osrelease)); + } else if ( (l= MATCH("kernel_ident")) ) { + dst = s_subst(dst, max, kern_ident, strlen(kern_ident)); + } else if ( (l= MATCH("domainname")) ) { + dst = s_subst(dst, max, domainname, strlen(domainname)); + } else if ( (l= MATCH("ostype")) ) { + dst = s_subst(dst, max, ostype, strlen(ostype)); + } + if (dst == NULL) /* overflow */ + break; + if (l == 0) { /* no match, restore original */ + *dst++ = '@'; + *dst++ = '{'; + continue; + } + /* otherwise skip original name and } */ + src += l + 1; + change = 1; + } + if (change && dst) { + if (src < end) /* copy last char */ + *dst++ = *src; + *dst = '\0'; + printf("translating into %s\n", tmp); + *len = dst - tmp; + bcopy(tmp, cp, *len); + } + uma_zfree(namei_zone, tmp); +} +#undef MATCH + static int lookup_shared = 0; SYSCTL_INT(_vfs, OID_AUTO, lookup_shared, CTLFLAG_RW, &lookup_shared, 0, "Enables/Disables shared locks for path name translation"); @@ -280,6 +398,8 @@ error = ENOENT; break; } + if (vfs_magiclinks && linklen >3) /* at least @{} in the symlink */ + symlink_magic(td, cp, &linklen); if (linklen + ndp->ni_pathlen >= MAXPATHLEN) { if (ndp->ni_pathlen > 1) uma_zfree(namei_zone, cp); --fdj2RfSjLxBAspz7-- From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 00:37:42 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 161BE1065670 for ; Thu, 19 Mar 2009 00:37:42 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id B57478FC15 for ; Thu, 19 Mar 2009 00:37:41 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.2/8.14.1) with ESMTP id n2J0aEUF059276; Wed, 18 Mar 2009 18:36:15 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Wed, 18 Mar 2009 18:36:46 -0600 (MDT) Message-Id: <20090318.183646.-593221015.imp@bsdimp.com> To: jhein@timing.com From: "M. Warner Losh" In-Reply-To: <18877.57878.136116.691250@gromit.timing.com> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: ed@80386.nl, arch@freebsd.org Subject: Re: Final sanity pass: xdev X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 00:37:42 -0000 In message: <18877.57878.136116.691250@gromit.timing.com> John Hein writes: : M. Warner Losh wrote at 08:08 +0900 on Mar 15, 2009: : > In message: <18875.60334.947446.966085@gromit.timing.com> : > John Hein writes: : > : M. Warner Losh wrote at 09:00 -0600 on Mar 13, 2009: : > : > In message: <20090313125115.GU31961@hoeg.nl> : > : > Ed Schouten writes: : > : > : Hi Warner, : > : > : : > : > : The last couple of days/weeks I've been working a lot with LLVM+Clang. : > : > : I'm currently writing some BSD makefiles to see how hard it is to add : > : > : Clang to our base system, as a proof of concept. : > : > : : > : > : A nice thing about Clang is that it seems to be very easy to compile in : > : > : multiple backends into the same Clang binary. If we ever switch to Clang : > : > : in the base system (who knows), it would be interesting to see whether : > : > : it's useful to just compile in all backends by default. : > : > : : > : > : Anyway, I took a quick look at the xdev patch and it looks okay. :-) : > : > : > : > Woot! Please let me know if it causes problems. : > : : > : I meant to mention this earlier - OSREL (referenced in the _xi-links: : > : target) is not defined. So ports looking for arm-freebsd8.0-cc (for : > : instance) won't find it. : > : : > : : > : An earlier patch of yours had: : > : : > : .if !defined(OSREL) : > : OSREL!= uname -r | sed -e 's/[-(].*//' : > : .endif : > : : > : But the latest (and the one committed) does not. : > : > You are right... Dang. I must have comitted the wrong thing... There is a subtle point here that we've missed. This gives us the version on the host, not the version of the tree we're building. In practice, these need to be the same. However, if we support building a 6.x cross compiler on a 7.x system, we may have to revisit. : One more thing... Now that we've switched to bsd ar, installing : gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built : ports can't find arm-freebsd8.0-ar Doh! Thanks for the patch. I'll run it through my QA cycle. Warner : Adding usr.bin/ar to the cross tools does the job... : : Index: Makefile.inc1 : =================================================================== : RCS file: /base/FreeBSD-CVS/src/Makefile.inc1,v : retrieving revision 1.620 : diff -u -p -r1.620 Makefile.inc1 : --- Makefile.inc1 13 Mar 2009 10:40:38 -0000 1.620 : +++ Makefile.inc1 16 Mar 2009 05:11:12 -0000 : @@ -1035,6 +1035,7 @@ cross-tools: : .for _tool in \ : gnu/usr.bin/binutils \ : gnu/usr.bin/cc \ : + usr.bin/ar \ : usr.bin/sed \ : usr.bin/xlint/lint1 usr.bin/xlint/lint2 usr.bin/xlint/xlint \ : ${_btxld} \ : @@ -1370,7 +1371,8 @@ _xb-build-tools: : _xb-cross-tools: : .for _tool in \ : gnu/usr.bin/binutils \ : - gnu/usr.bin/cc : + gnu/usr.bin/cc \ : + usr.bin/ar : ${_+_}@${ECHODIR} "===> xdev ${_tool} (obj,depend,all)"; \ : cd ${.CURDIR}/${_tool}; \ : ${CDMAKE} DIRPRFX=${_tool}/ obj; \ : @@ -1395,7 +1397,8 @@ _xi-cross-tools: : @echo "_xi-cross-tools" : .for _tool in \ : gnu/usr.bin/binutils \ : - gnu/usr.bin/cc : + gnu/usr.bin/cc \ : + usr.bin/ar : ${_+_}@${ECHODIR} "===> xdev ${_tool} (install)"; \ : cd ${.CURDIR}/${_tool}; \ : ${CDMAKE} DIRPRFX=${_tool}/ install DESTDIR=${XDDESTDIR} : : From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 00:57:08 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8F48106566C for ; Thu, 19 Mar 2009 00:57:08 +0000 (UTC) (envelope-from jhein@timing.com) Received: from Daffy.timing.com (smtp.timing.com [206.168.13.218]) by mx1.freebsd.org (Postfix) with ESMTP id 740F88FC1F for ; Thu, 19 Mar 2009 00:57:08 +0000 (UTC) (envelope-from jhein@timing.com) Received: from gromit.timing.com (gromit.timing.com [206.168.13.209]) by Daffy.timing.com (8.13.1/8.13.1) with ESMTP id n2J0ufqL041061; Wed, 18 Mar 2009 18:56:43 -0600 (MDT) (envelope-from jhein@timing.com) Received: from gromit.timing.com (localhost [127.0.0.1]) by gromit.timing.com (8.14.3/8.14.3) with ESMTP id n2J0ueV8012195; Wed, 18 Mar 2009 18:56:40 -0600 (MDT) (envelope-from jhein@gromit.timing.com) Received: (from jhein@localhost) by gromit.timing.com (8.14.3/8.14.3/Submit) id n2J0ueij012192; Wed, 18 Mar 2009 18:56:40 -0600 (MDT) (envelope-from jhein) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18881.38984.133668.539997@gromit.timing.com> Date: Wed, 18 Mar 2009 18:56:40 -0600 From: John Hein To: "M. Warner Losh" In-Reply-To: <20090318.183646.-593221015.imp@bsdimp.com> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> X-Mailer: VM 7.19 under Emacs 22.3.1 X-Virus-Scanned: ClamAV version 0.91.2, clamav-milter version 0.91.2 on Daffy.timing.com X-Virus-Status: Clean Cc: ed@80386.nl, arch@freebsd.org Subject: Re: Final sanity pass: xdev X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 00:57:09 -0000 M. Warner Losh wrote at 18:36 -0600 on Mar 18, 2009: > In message: <18877.57878.136116.691250@gromit.timing.com> > John Hein writes: > : M. Warner Losh wrote at 08:08 +0900 on Mar 15, 2009: > : > In message: <18875.60334.947446.966085@gromit.timing.com> > : > John Hein writes: > : > : An earlier patch of yours had: > : > : > : > : .if !defined(OSREL) > : > : OSREL!= uname -r | sed -e 's/[-(].*//' > : > : .endif > : > : > : > : But the latest (and the one committed) does not. > : > > : > You are right... Dang. I must have comitted the wrong thing... > > There is a subtle point here that we've missed. This gives us the > version on the host, not the version of the tree we're building. In > practice, these need to be the same. However, if we support building > a 6.x cross compiler on a 7.x system, we may have to revisit. In our build env (I'm sure you recall), it works out fine in practice (for most ports [*]). Building 8.x targets on 7.x makes links like "arm-freebsd7.1-cc" but the cross built ports also look for that name - likely because the config-guess machinery uses uname also. I'm not so sure we need to support building the other direction (7 on 8) since we generally don't support that direction of compatibilty, but if it can be done with minimal effort, that'd be good. So, if it's just a matter of finding the appropriate cross version of cc or nm or whatever, this is okay. [*] But if there is some dependency in a particular port that actually looks into the freebsd version and does something different based on version, it may be an issue. Then on top of that, there's kernel version and userland version that may matter to certain ports. Perhaps we should consider setting UNAME_r in the environment when building across major OS levels (possibly outside the scope of /usr/src/Makefile*). At a certain point in the cross-arch / cross-major-OS-version building dance, we could also just say it's not supported and let people work it out with a native "same major" OS level build machine (possibly a virtual machine). > : One more thing... Now that we've switched to bsd ar, installing > : gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built > : ports can't find arm-freebsd8.0-ar > > Doh! Thanks for the patch. I'll run it through my QA cycle. Okay. Thanks for getting the 1000 monkeys to give it a test run ;) From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 01:04:52 2009 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6FCF21065670 for ; Thu, 19 Mar 2009 01:04:52 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 311708FC19 for ; Thu, 19 Mar 2009 01:04:52 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.2/8.14.1) with ESMTP id n2J119Si059532; Wed, 18 Mar 2009 19:01:09 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Wed, 18 Mar 2009 19:01:41 -0600 (MDT) Message-Id: <20090318.190141.2040345308.imp@bsdimp.com> To: jhein@timing.com From: "M. Warner Losh" In-Reply-To: <18881.38984.133668.539997@gromit.timing.com> References: <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> <18881.38984.133668.539997@gromit.timing.com> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: ed@80386.nl, arch@FreeBSD.org Subject: Re: Final sanity pass: xdev X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 01:04:52 -0000 In message: <18881.38984.133668.539997@gromit.timing.com> John Hein writes: : > : One more thing... Now that we've switched to bsd ar, installing : > : gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built : > : ports can't find arm-freebsd8.0-ar : > : > Doh! Thanks for the patch. I'll run it through my QA cycle. : : Okay. Thanks for getting the 1000 monkeys to give it a test run ;) Sadly, I'm an old-time C++ programmer, so I have a virtual base class that causes 1000 virtual monkeys to be created and shoot my foot at random intervals. Not the most useful thing in the world, but it is C++, so what can you do :) Warner From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 01:25:54 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8EF471065673 for ; Thu, 19 Mar 2009 01:25:54 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outD.internet-mail-service.net (outd.internet-mail-service.net [216.240.47.227]) by mx1.freebsd.org (Postfix) with ESMTP id 6EF4E8FC2B for ; Thu, 19 Mar 2009 01:25:54 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 1FC32B98F; Wed, 18 Mar 2009 18:25:54 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 6661B2D6011; Wed, 18 Mar 2009 18:25:53 -0700 (PDT) Message-ID: <49C19F2A.10406@elischer.org> Date: Wed, 18 Mar 2009 18:26:02 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.19 (Macintosh/20081209) MIME-Version: 1.0 To: John Hein References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> <18881.38984.133668.539997@gromit.timing.com> In-Reply-To: <18881.38984.133668.539997@gromit.timing.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: ed@80386.nl, arch@freebsd.org Subject: Re: Final sanity pass: xdev X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 01:25:55 -0000 John Hein wrote: > M. Warner Losh wrote at 18:36 -0600 on Mar 18, 2009: > > In message: <18877.57878.136116.691250@gromit.timing.com> > > John Hein writes: > > : M. Warner Losh wrote at 08:08 +0900 on Mar 15, 2009: > > : > In message: <18875.60334.947446.966085@gromit.timing.com> > > : > John Hein writes: > > : > : An earlier patch of yours had: > > : > : > > : > : .if !defined(OSREL) > > : > : OSREL!= uname -r | sed -e 's/[-(].*//' > > : > : .endif > > : > : > > : > : But the latest (and the one committed) does not. > > : > > > : > You are right... Dang. I must have comitted the wrong thing... > > > > There is a subtle point here that we've missed. This gives us the > > version on the host, not the version of the tree we're building. In > > practice, these need to be the same. However, if we support building > > a 6.x cross compiler on a 7.x system, we may have to revisit. > > In our build env (I'm sure you recall), it works out fine in practice > (for most ports [*]). > > Building 8.x targets on 7.x makes links like "arm-freebsd7.1-cc" but > the cross built ports also look for that name - likely because the > config-guess machinery uses uname also. > > I'm not so sure we need to support building the other direction (7 on 8) > since we generally don't support that direction of compatibilty, but > if it can be done with minimal effort, that'd be good. > > So, if it's just a matter of finding the appropriate cross version of > cc or nm or whatever, this is okay. > > [*] But if there is some dependency in a particular port that actually > looks into the freebsd version and does something different based on > version, it may be an issue. Then on top of that, there's kernel > version and userland version that may matter to certain ports. > > Perhaps we should consider setting UNAME_r in the environment when > building across major OS levels (possibly outside the scope of > /usr/src/Makefile*). > > At a certain point in the cross-arch / cross-major-OS-version building > dance, we could also just say it's not supported and let people work > it out with a native "same major" OS level build machine (possibly a > virtual machine). we also have this problem when running a different revision of software in a Jail. Possibly it should be possible to set the value that uname responds with for a process and its' children, (even more so that the environment variables) I'd like to be able to label a jail as a 7.1 jail on an 8.0 machine.. The problem in using teh single environment variable UNAME_r or friends is that the intermediate tools ARE running on a different environment than the final tools so one value may not be correct everywhere. we sort of need one value of UNAME_r up until teh tools are built, and then another value for teh binary build. But even that may be not general enough. > > > > : One more thing... Now that we've switched to bsd ar, installing > > : gnu/usr.bin/binutils installs the gnu ar as gnu-ar. Cross-built > > : ports can't find arm-freebsd8.0-ar > > > > Doh! Thanks for the patch. I'll run it through my QA cycle. > > Okay. Thanks for getting the 1000 monkeys to give it a test run ;) > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 01:56:07 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C8C91065672 for ; Thu, 19 Mar 2009 01:56:07 +0000 (UTC) (envelope-from jhein@timing.com) Received: from Daffy.timing.com (mx2.timing.com [206.168.13.218]) by mx1.freebsd.org (Postfix) with ESMTP id 37D158FC18 for ; Thu, 19 Mar 2009 01:56:07 +0000 (UTC) (envelope-from jhein@timing.com) Received: from gromit.timing.com (gromit.timing.com [206.168.13.209]) by Daffy.timing.com (8.13.1/8.13.1) with ESMTP id n2J1u4KM046024; Wed, 18 Mar 2009 19:56:05 -0600 (MDT) (envelope-from jhein@timing.com) Received: from gromit.timing.com (localhost [127.0.0.1]) by gromit.timing.com (8.14.3/8.14.3) with ESMTP id n2J1u25N015003; Wed, 18 Mar 2009 19:56:02 -0600 (MDT) (envelope-from jhein@gromit.timing.com) Received: (from jhein@localhost) by gromit.timing.com (8.14.3/8.14.3/Submit) id n2J1u2Hn015000; Wed, 18 Mar 2009 19:56:02 -0600 (MDT) (envelope-from jhein) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18881.42546.640583.971867@gromit.timing.com> Date: Wed, 18 Mar 2009 19:56:02 -0600 From: John Hein To: Julian Elischer In-Reply-To: <49C19F2A.10406@elischer.org> References: <18875.60334.947446.966085@gromit.timing.com> <20090315.080814.669286040.imp@bsdimp.com> <18877.57878.136116.691250@gromit.timing.com> <20090318.183646.-593221015.imp@bsdimp.com> <18881.38984.133668.539997@gromit.timing.com> <49C19F2A.10406@elischer.org> X-Mailer: VM 7.19 under Emacs 22.3.1 X-Virus-Scanned: ClamAV version 0.91.2, clamav-milter version 0.91.2 on Daffy.timing.com X-Virus-Status: Clean Cc: arch@freebsd.org Subject: Re: Final sanity pass: xdev X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 01:56:07 -0000 Julian Elischer wrote at 18:26 -0700 on Mar 18, 2009: > John Hein wrote: > > Perhaps we should consider setting UNAME_r in the environment when > > building across major OS levels (possibly outside the scope of > > /usr/src/Makefile*). > > > > At a certain point in the cross-arch / cross-major-OS-version building > > dance, we could also just say it's not supported and let people work > > it out with a native "same major" OS level build machine (possibly a > > virtual machine). > > we also have this problem when running a different revision of > software in a Jail. Possibly it should be possible to set the value > that uname responds with for a process and its' children, (even more > so that the environment variables) > I'd like to be able to label a jail as a 7.1 jail on an 8.0 machine.. > > The problem in using teh single environment variable UNAME_r > or friends is that the intermediate tools ARE running on a > different environment than the final tools so one value may not be > correct everywhere. > we sort of need one value of UNAME_r up until teh tools are built, and > then another value for teh binary build. But even that may be not > general enough. In our locally grown build env, that's pretty easy to do (pass different UNAME_r at different build stages). Perhaps you could even put it into the default env for a build chroot/jail (via login.conf). As I said, it's probably not in the scope of /usr/src/Makefile*, but because we can pass in UNAME_r from a higher level having Makefile.inc1 use 'uname -r' for OSREL (as it is currently named) in cross builds seems okay to me. From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 08:14:52 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6C192106566C for ; Thu, 19 Mar 2009 08:14:52 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.9.129]) by mx1.freebsd.org (Postfix) with ESMTP id D424F8FC21 for ; Thu, 19 Mar 2009 08:14:51 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id AFE3873098; Thu, 19 Mar 2009 09:19:36 +0100 (CET) Date: Thu, 19 Mar 2009 09:19:36 +0100 From: Luigi Rizzo To: geom@freebsd.org, phk@freebsd.org, ivan@freebsd.org, fabio@gandalf.sssup.it Message-ID: <20090319081936.GA32750@onelab2.iet.unipi.it> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="J/dobhs11T7y2rNN" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Cc: arch@freebsd.org Subject: RFC: adding 'proxy' nodes to provider ports (with patch) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 08:14:52 -0000 --J/dobhs11T7y2rNN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, Fabio Checconi and I have been thinking on how to implement "proxy" geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer port, do not do any data transformation, and can be transparently inserted or removed on top of a provider port while the tree is actively moving data. Our immediate need was to add/remove a scheduler while a disk is mounted, but there are possibly other uses e.g. if one wants to "sniff" the traffic through a disk, or do other ops that are transparent for the data stream. We would like opinion on the following solution, which seems extremely simple in terms of implementation. The idea is to intercept requests coming on a provider port, pp, and redirect them to a geom node acting as a proxy if the port is configured in this way: +=====...===...======+ H H H H H H +=====...====== cp ==+ | +---------------+ V | V +=====.....==== pp ==+ | +======= proxy_pp ==+ H 'ad0s1' H | H H H ------->--+ H H H gp -------<--+ H proxy_node H H H | H H +=======....===...===+ | +======= proxy_cp ==+ | V +---------------+ Normally the proxy does not exist, and the geom tree works as it does now. When we create a 'proxy' node, with something like geom my_proxy_class proxy ad0s1 we do something very similar to a 'create', but: - the proxy node is marked specially in gp->flags, so the core will not generate a g_new_provider_event when the provider port is created (this means there is no taste() call and nobody should be able to attach to the port). - the provider port we attach to is linked, with two pointers, to the provider and consumer ports of the proxy_node. In this situation, g_io_request() finds that port pp has a proxy attached to it, and immediately redirects the requests to the proxy, which does everything a geom node does (cloning requests, etc). When the proxy wants to pass the request down, it sends it again to pp, but now there is no redirection because the source can be identified as the proxy. The pointers in the bio insure a correct flow of the requests on the reverse path. Disconnecting a proxy is almost trivial: apart from handling possible races on the data path, we just need to clear pp->proxy_pp and pp->proxy_cp. After that, we can just send the regular destroy events to the proxy node, who will have to take care of flushing any pending bio's (e.g. see our geom_sched node that already does this). Overall the change is very small (see attached patch): a couple of lines in g_io_request, two extra fields in the g_provider, and the addition of a flag to gp->flags to control the generation of g_new_provider_event. There is basically no overhead on regular operation, and only a couple of extra pointers in struct g_provider (we use a spare bit in gp->flags to mark G_GEOM_PROXY nodes). The only things missing in the patch should be: - a check to avoid races on creation&destruction of a proxy. I am not so sure on how to achieve this, but creation and destruction are rare and can normally wait, so we could just piggyback the small critical section (manipulating pp->proxy_cp and pp->proxy_cp) into some other piece of code that is guaranteed to be race-free. - a check to prevent attaching to a provider port of a proxy (not a problem, i believe); - a check to prevent attaching a proxy to a provider port that already has one. Of course you can attach a proxy to another proxy, and if you want to change the order it is as simple as removing the existing proxy and reattaching it after the new one. Feedback welcome. cheers luigi and fabio --J/dobhs11T7y2rNN Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="geom-proxy.patch" Index: geom/geom_io.c =================================================================== --- geom/geom_io.c (revision 189933) +++ geom/geom_io.c (working copy) @@ -273,6 +273,10 @@ cp = bp->bio_from; pp = bp->bio_to; + /* Proxies are transparent, errors are caught by regular checks. */ + if (cp->geom->flags & G_GEOM_PROXY) + return (0); + /* Fail if access counters dont allow the operation */ switch(bp->bio_cmd) { case BIO_READ: @@ -343,6 +347,9 @@ bp->_bio_cflags = bp->bio_cflags; #endif + if (pp->proxy_provider != NULL && cp != pp->proxy_consumer) + pp = pp->proxy_provider; + if (bp->bio_cmd & (BIO_READ|BIO_WRITE|BIO_GETATTR)) { KASSERT(bp->bio_data != NULL, ("NULL bp->data in g_io_request(cmd=%hhu)", bp->bio_cmd)); Index: geom/geom_subr.c =================================================================== --- geom/geom_subr.c (revision 189933) +++ geom/geom_subr.c (working copy) @@ -365,6 +365,8 @@ g_cancel_event(gp); LIST_REMOVE(gp, geom); TAILQ_REMOVE(&geoms, gp, geoms); + if (gp->flags & G_GEOM_PROXY) + printf("g_destroy_geom(): destroying proxy %s", gp->name); g_free(gp->name); g_free(gp); } @@ -380,6 +382,11 @@ g_topology_assert(); G_VALID_GEOM(gp); g_trace(G_T_TOPOLOGY, "g_wither_geom(%p(%s))", gp, gp->name); + if (gp->flags & G_GEOM_PROXY) { + pp = LIST_FIRST(&gp->consumer)->provider; + pp->proxy_provider = NULL; + pp->proxy_consumer = NULL; + } if (!(gp->flags & G_GEOM_WITHER)) { gp->flags |= G_GEOM_WITHER; LIST_FOREACH(pp, &gp->provider, provider) @@ -581,7 +588,8 @@ pp->stat = devstat_new_entry(pp, -1, 0, DEVSTAT_ALL_SUPPORTED, DEVSTAT_TYPE_DIRECT, DEVSTAT_PRIORITY_MAX); LIST_INSERT_HEAD(&gp->provider, pp, provider); - g_post_event(g_new_provider_event, pp, M_WAITOK, pp, gp, NULL); + if (!(gp->flags & G_GEOM_PROXY)) + g_post_event(g_new_provider_event, pp, M_WAITOK, pp, gp, NULL); return (pp); } @@ -721,6 +729,20 @@ return (error); } +int +g_attach_proxy(struct g_consumer *cp, struct g_provider *pp, + struct g_provider *newpp) +{ + int error; + + error = g_attach(cp, pp); + if (!error) { + pp->proxy_provider = newpp; + pp->proxy_consumer = cp; + } + return (error); +} + void g_detach(struct g_consumer *cp) { Index: geom/geom.h =================================================================== --- geom/geom.h (revision 189933) +++ geom/geom.h (working copy) @@ -134,6 +134,7 @@ void *softc; unsigned flags; #define G_GEOM_WITHER 1 +#define G_GEOM_PROXY 2 }; /* @@ -190,6 +191,9 @@ #define G_PF_WITHER 0x2 #define G_PF_ORPHAN 0x4 + struct g_provider *proxy_provider; + struct g_consumer *proxy_consumer; + /* Two fields for the implementing class to use */ void *private; u_int index; @@ -219,6 +223,8 @@ /* geom_subr.c */ int g_access(struct g_consumer *cp, int nread, int nwrite, int nexcl); int g_attach(struct g_consumer *cp, struct g_provider *pp); +int g_attach_proxy(struct g_consumer *cp, struct g_provider *pp, + struct g_provider *newpp); void g_destroy_consumer(struct g_consumer *cp); void g_destroy_geom(struct g_geom *pp); void g_destroy_provider(struct g_provider *pp); --J/dobhs11T7y2rNN-- From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 10:12:00 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65EFD1065672; Thu, 19 Mar 2009 10:12:00 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206045082.chello.pl [87.206.45.82]) by mx1.freebsd.org (Postfix) with ESMTP id C56DB8FC24; Thu, 19 Mar 2009 10:11:59 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id BC16D45C98; Thu, 19 Mar 2009 10:44:30 +0100 (CET) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 4A34D45B26; Thu, 19 Mar 2009 10:44:25 +0100 (CET) Date: Thu, 19 Mar 2009 10:45:05 +0100 From: Pawel Jakub Dawidek To: Luigi Rizzo Message-ID: <20090319094505.GA1539@garage.freebsd.pl> References: <20090319081936.GA32750@onelab2.iet.unipi.it> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Dxnq1zWXvFF0Q93v" Content-Disposition: inline In-Reply-To: <20090319081936.GA32750@onelab2.iet.unipi.it> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: fabio@gandalf.sssup.it, geom@freebsd.org, arch@freebsd.org, ivan@freebsd.org Subject: Re: RFC: adding 'proxy' nodes to provider ports (with patch) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 10:12:00 -0000 --Dxnq1zWXvFF0Q93v Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 19, 2009 at 09:19:36AM +0100, Luigi Rizzo wrote: > Hi, >=20 > Fabio Checconi and I have been thinking on how to implement "proxy" > geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer > port, do not do any data transformation, and can be transparently > inserted or removed on top of a provider port while the tree is > actively moving data. >=20 > Our immediate need was to add/remove a scheduler while a disk is > mounted, but there are possibly other uses e.g. if one wants to > "sniff" the traffic through a disk, or do other ops that are > transparent for the data stream. >=20 > We would like opinion on the following solution, which seems > extremely simple in terms of implementation. >=20 > The idea is to intercept requests coming on a provider port, pp, and > redirect them to a geom node acting as a proxy if the port > is configured in this way: >=20 > +=3D=3D=3D=3D=3D...=3D=3D=3D...=3D=3D=3D=3D=3D=3D+ > H H > H H > H H > +=3D=3D=3D=3D=3D...=3D=3D=3D=3D=3D=3D cp =3D=3D+ > | +---------------+ > V | V > +=3D=3D=3D=3D=3D.....=3D=3D=3D=3D pp =3D=3D+ | +=3D=3D=3D=3D= =3D=3D=3D proxy_pp =3D=3D+ > H 'ad0s1' H | H H > H ------->--+ H H > H gp -------<--+ H proxy_node H > H H | H H > +=3D=3D=3D=3D=3D=3D=3D....=3D=3D=3D...=3D=3D=3D+ | +=3D=3D=3D= =3D=3D=3D=3D proxy_cp =3D=3D+ > | V > +---------------+ >=20 > Normally the proxy does not exist, and the geom tree works as it does now. >=20 > When we create a 'proxy' node, with something like >=20 > geom my_proxy_class proxy ad0s1 >=20 > we do something very similar to a 'create', but: >=20 > - the proxy node is marked specially in gp->flags, so the core will > not generate a g_new_provider_event when the provider port is created > (this means there is no taste() call and nobody should be able > to attach to the port). >=20 > - the provider port we attach to is linked, with two pointers, > to the provider and consumer ports of the proxy_node. >=20 > In this situation, g_io_request() finds that port pp has a proxy attached > to it, and immediately redirects the requests to the proxy, which > does everything a geom node does (cloning requests, etc). > When the proxy wants to pass the request down, it sends it again to pp, > but now there is no redirection because the source can be identified > as the proxy. The pointers in the bio insure a correct flow of the > requests on the reverse path. The one advantage I see for this over using regular GEOM rules is that new consumers go through proxy automatically. When I was working on similar functionality I more wanted to do something like this: consumer1 consumer2 \ / \ / provider Insert the proxy in the middle of any provider-consumer pair: consumer1 consumer2 | | proxy_provider | | / proxy_consumer / \ / provider This can be done (almost I think) atomically: /* First attach to the destination provider. */ g_attach(proxy_consumer, provider); /* Then switch original consumer to use proxy_provider (should be almost at= omic). */ consumer1->provider =3D proxy_provider; /* handle access counts */ In-flight I/O requests know how to go back, because they have source and destination stored in bio_from and bio_to fields, so no races here. > Disconnecting a proxy is almost trivial: apart from handling possible > races on the data path, we just need to clear pp->proxy_pp and pp->proxy_= cp. > After that, we can just send the regular destroy events to the proxy > node, who will have to take care of flushing any pending bio's (e.g. > see our geom_sched node that already does this). >=20 > Overall the change is very small (see attached patch): > a couple of lines in g_io_request, two extra fields in the g_provider, > and the addition of a flag to gp->flags to control the generation > of g_new_provider_event. > There is basically no overhead on regular operation, and only > a couple of extra pointers in struct g_provider (we use a spare > bit in gp->flags to mark G_GEOM_PROXY nodes). >=20 > The only things missing in the patch should be: >=20 > - a check to avoid races on creation&destruction of a proxy. > I am not so sure on how to achieve this, but creation and destruction > are rare and can normally wait, so we could just piggyback the > small critical section (manipulating pp->proxy_cp and pp->proxy_cp) > into some other piece of code that is guaranteed to be race-free. >=20 > - a check to prevent attaching to a provider port of a proxy > (not a problem, i believe); >=20 > - a check to prevent attaching a proxy to a provider port that already > has one. Of course you can attach a proxy to another proxy, and > if you want to change the order it is as simple as removing the > existing proxy and reattaching it after the new one. Could you provide link for the patch, as it was removed from your e-mail? --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --Dxnq1zWXvFF0Q93v Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFJwhQhForvXbEpPzQRAqtkAJ9zME7wMe4RZMsBYdAURm/9voIr/QCgwAim NK2iPFV6J/jPGqpLNhH3Cl8= =Fzxm -----END PGP SIGNATURE----- --Dxnq1zWXvFF0Q93v-- From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 11:08:53 2009 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 500691065670; Thu, 19 Mar 2009 11:08:53 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.9.129]) by mx1.freebsd.org (Postfix) with ESMTP id 110AD8FC23; Thu, 19 Mar 2009 11:08:52 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 1FECC73098; Thu, 19 Mar 2009 12:13:39 +0100 (CET) Date: Thu, 19 Mar 2009 12:13:39 +0100 From: Luigi Rizzo To: Pawel Jakub Dawidek Message-ID: <20090319111339.GA38075@onelab2.iet.unipi.it> References: <20090319081936.GA32750@onelab2.iet.unipi.it> <20090319094505.GA1539@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090319094505.GA1539@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i Cc: fabio@gandalf.sssup.it, geom@FreeBSD.org, phk@FreeBSD.org, arch@FreeBSD.org, ivan@FreeBSD.org Subject: Re: RFC: adding 'proxy' nodes to provider ports (with patch) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 11:08:53 -0000 On Thu, Mar 19, 2009 at 10:45:05AM +0100, Pawel Jakub Dawidek wrote: > > Hi, > > > > Fabio Checconi and I have been thinking on how to implement "proxy" > > geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer > > port, do not do any data transformation, and can be transparently > > inserted or removed on top of a provider port while the tree is > > actively moving data. ... > > Overall the change is very small (see attached patch): > > a couple of lines in g_io_request, two extra fields in the g_provider, > > and the addition of a flag to gp->flags to control the generation > > of g_new_provider_event. It seems that the attachment was removed so here it is http://info.iet.unipi.it/~luigi/FreeBSD/20090319-geom-proxy.patch cheers luigi From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 11:17:07 2009 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BE42A106566B; Thu, 19 Mar 2009 11:17:07 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.9.129]) by mx1.freebsd.org (Postfix) with ESMTP id 7A85F8FC0A; Thu, 19 Mar 2009 11:17:07 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id A686973098; Thu, 19 Mar 2009 12:21:52 +0100 (CET) Date: Thu, 19 Mar 2009 12:21:52 +0100 From: Luigi Rizzo To: Pawel Jakub Dawidek Message-ID: <20090319112152.GB38075@onelab2.iet.unipi.it> References: <20090319081936.GA32750@onelab2.iet.unipi.it> <20090319094505.GA1539@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090319094505.GA1539@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i Cc: fabio@gandalf.sssup.it, geom@FreeBSD.org, phk@FreeBSD.org, arch@FreeBSD.org, ivan@FreeBSD.org Subject: Re: RFC: adding 'proxy' nodes to provider ports (with patch) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 11:17:08 -0000 On Thu, Mar 19, 2009 at 10:45:05AM +0100, Pawel Jakub Dawidek wrote: > On Thu, Mar 19, 2009 at 09:19:36AM +0100, Luigi Rizzo wrote: ... > The one advantage I see for this over using regular GEOM rules is that > new consumers go through proxy automatically. When I was working on > similar functionality I more wanted to do something like this: > > consumer1 consumer2 > \ / > \ / > provider > > Insert the proxy in the middle of any provider-consumer pair: > > consumer1 consumer2 > | | > proxy_provider | > | / > proxy_consumer / > \ / > provider ok this is slightly different from what we have implemented, as we hook into the provider whereas you hook into the consumer. In our case we really need the hook to be in the provider so it intercepts accesses from all consumers, e.g. from /dev/ad0 and from the filesystems mounted on top of it. Given that the geom_disk node does not have a consumer on the bottom, we cannot do it differently. I can imagine that one might want to attach a proxy to a consumer port, but I cannot make a specific case where this would be needed. Also, I wonder how do i name a consumer port in the geom model ?? cheers luigi From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 12:01:37 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F26E81065670 for ; Thu, 19 Mar 2009 12:01:36 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id 6D9848FC13 for ; Thu, 19 Mar 2009 12:01:36 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: by fg-out-1718.google.com with SMTP id 13so37596fge.12 for ; Thu, 19 Mar 2009 05:01:35 -0700 (PDT) MIME-Version: 1.0 Received: by 10.86.96.1 with SMTP id t1mr243342fgb.46.1237462873065; Thu, 19 Mar 2009 04:41:13 -0700 (PDT) In-Reply-To: <20090319081936.GA32750@onelab2.iet.unipi.it> References: <20090319081936.GA32750@onelab2.iet.unipi.it> Date: Thu, 19 Mar 2009 12:41:13 +0100 Message-ID: From: =?ISO-8859-1?Q?Marius_N=FCnnerich?= To: Luigi Rizzo Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: fabio@gandalf.sssup.it, geom@freebsd.org, arch@freebsd.org, ivan@freebsd.org Subject: Re: RFC: adding 'proxy' nodes to provider ports (with patch) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 12:01:39 -0000 2009/3/19 Luigi Rizzo : > Hi, > > Fabio Checconi and I have been thinking on how to implement "proxy" > geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer > port, do not do any data transformation, and can be transparently > inserted or removed on top of a provider port while the tree is > actively moving data. > > Our immediate need was to add/remove a scheduler while a disk is > mounted, but there are possibly other uses e.g. if one wants to > "sniff" the traffic through a disk, or do other ops that are > transparent for the data stream. > > We would like opinion on the following solution, which seems > extremely simple in terms of implementation. > > The idea is to intercept requests coming on a provider port, pp, and > redirect them to a geom node acting as a proxy if the port > is configured in this way: > > =A0 =A0 +=3D=3D=3D=3D=3D...=3D=3D=3D...=3D=3D=3D=3D=3D=3D+ > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0H > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0H > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0H > =A0 =A0 +=3D=3D=3D=3D=3D...=3D=3D=3D=3D=3D=3D cp =3D=3D+ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | =A0 =A0 =A0 =A0 =A0+-----------= ----+ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 V =A0 =A0 =A0 =A0 =A0| =A0 =A0 = =A0 =A0 =A0 =A0 =A0 V > =A0 =A0 +=3D=3D=3D=3D=3D.....=3D=3D=3D=3D pp =3D=3D+ =A0 =A0 | =A0 =A0+= =3D=3D=3D=3D=3D=3D=3D proxy_pp =3D=3D+ > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 'ad0s1' =A0H =A0 =A0 | =A0 =A0H =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 H > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0------->--+ =A0 =A0H =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 H > =A0 =A0 H =A0 =A0 =A0 =A0gp =A0 =A0 =A0-------<--+ =A0 =A0H =A0 =A0proxy_= node =A0 =A0 H > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0H =A0 =A0 | =A0 =A0H =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 H > =A0 =A0 +=3D=3D=3D=3D=3D=3D=3D....=3D=3D=3D...=3D=3D=3D+ =A0 =A0 | =A0 = =A0+=3D=3D=3D=3D=3D=3D=3D proxy_cp =3D=3D+ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 = =A0 =A0 =A0 =A0 =A0 V > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0+---------= ------+ > > Normally the proxy does not exist, and the geom tree works as it does now= . > > When we create a 'proxy' node, with something like > > =A0 =A0 =A0 =A0geom my_proxy_class proxy ad0s1 > > we do something very similar to a 'create', but: > > - the proxy node is marked specially in gp->flags, so the core will > =A0not generate a g_new_provider_event when the provider port is created > =A0(this means there is no taste() call and nobody should be able > =A0to attach to the port). > > - the provider port we attach to is linked, with two pointers, > =A0to the provider and consumer ports of the proxy_node. > > In this situation, g_io_request() finds that port pp has a proxy attached > to it, and immediately redirects the requests to the proxy, which > does everything a geom node does (cloning requests, etc). > =A0 When the proxy wants to pass the request down, it sends it again to p= p, > but now there is no redirection because the source can be identified > as the proxy. =A0The pointers in the bio insure a correct flow of the > requests on the reverse path. > > Disconnecting a proxy is almost trivial: apart from handling possible > races on the data path, we just need to clear pp->proxy_pp and pp->proxy_= cp. > After that, we can just send the regular destroy events to the proxy > node, who will have to take care of flushing any pending bio's (e.g. > see our geom_sched node that already does this). > > Overall the change is very small (see attached patch): > a couple of lines in g_io_request, two extra fields in the g_provider, > and the addition of a flag to gp->flags to control the generation > of g_new_provider_event. > There is basically no overhead on regular operation, and only > a couple of extra pointers in struct g_provider (we use a spare > bit in gp->flags to mark G_GEOM_PROXY nodes). > > The only things missing in the patch should be: > > - a check to avoid races on creation&destruction of a proxy. > =A0I am not so sure on how to achieve this, but creation and destruction > =A0are rare and can normally wait, so we could just piggyback the > =A0small critical section (manipulating pp->proxy_cp and pp->proxy_cp) > =A0into some other piece of code that is guaranteed to be race-free. > > - a check to prevent attaching to a provider port of a proxy > =A0(not a problem, i believe); > > - a check to prevent attaching a proxy to a provider port that already > =A0has one. Of course you can attach a proxy to another proxy, and > =A0if you want to change the order it is as simple as removing the > =A0existing proxy and reattaching it after the new one. > > Feedback welcome. I wonder if it's really necessary to alter the GEOM infrastructure or if it is possible to do this with what's there already. Just an idea: Lock g_topology, put g_down and g_up to sleep, alter the consumer and provider pointers where you need it so the everything is routed through your proxy class (which isn't special in any way) and restart g_down and g_up. From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 12:56:51 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA7A3106566C; Thu, 19 Mar 2009 12:56:51 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.9.129]) by mx1.freebsd.org (Postfix) with ESMTP id 8A8398FC12; Thu, 19 Mar 2009 12:56:51 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id E49C6730A4; Thu, 19 Mar 2009 14:01:37 +0100 (CET) Date: Thu, 19 Mar 2009 14:01:37 +0100 From: Luigi Rizzo To: Marius N?nnerich Message-ID: <20090319130137.GB40489@onelab2.iet.unipi.it> References: <20090319081936.GA32750@onelab2.iet.unipi.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: fabio@gandalf.sssup.it, geom@freebsd.org, arch@freebsd.org, ivan@freebsd.org Subject: Re: RFC: adding 'proxy' nodes to provider ports (with patch) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 12:56:52 -0000 On Thu, Mar 19, 2009 at 12:41:13PM +0100, Marius N?nnerich wrote: > 2009/3/19 Luigi Rizzo : > > Hi, > > > > Fabio Checconi and I have been thinking on how to implement "proxy" > > geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer > > port, do not do any data transformation, and can be transparently > > inserted or removed on top of a provider port while the tree is > > actively moving data. ... > > The idea is to intercept requests coming on a provider port, pp, and > > redirect them to a geom node acting as a proxy if the port > > is configured in this way: ... > I wonder if it's really necessary to alter the GEOM infrastructure or > if it is possible to do this with what's there already. Just an idea: > Lock g_topology, put g_down and g_up to sleep, alter the consumer and > provider pointers where you need it so the everything is routed > through your proxy class (which isn't special in any way) and restart > g_down and g_up. we'll look into this, thanks. If we can spare the extra fields in the g_provider, the thing is even easier to do. I just don't know how your suggestion interferes with the naming: if I change the pointers, the name of a provider will not be anymore a prefix of the name of the node attached above. But maybe that is not an architectural requirements but just a convenient convention. cheers luigi From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 16:54:53 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2642F1065673 for ; Thu, 19 Mar 2009 16:54:53 +0000 (UTC) (envelope-from zachary.loafman@isilon.com) Received: from seaxch09.isilon.com (seaxch09.isilon.com [74.85.160.25]) by mx1.freebsd.org (Postfix) with ESMTP id 0016D8FC17 for ; Thu, 19 Mar 2009 16:54:52 +0000 (UTC) (envelope-from zachary.loafman@isilon.com) Received: from zloafman.west.isilon.com ([10.54.190.57]) by seaxch09.isilon.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 19 Mar 2009 09:42:52 -0700 Received: from zloafman.west.isilon.com (localhost.localdomain [127.0.0.1]) by zloafman.west.isilon.com (8.13.1/8.13.1) with ESMTP id n2JGgpH6005203; Thu, 19 Mar 2009 09:42:51 -0700 Received: (from zloafman@localhost) by zloafman.west.isilon.com (8.13.1/8.13.1/Submit) id n2JGgpfA005202; Thu, 19 Mar 2009 09:42:51 -0700 X-Authentication-Warning: zloafman.west.isilon.com: zloafman set sender to zachary.loafman@isilon.com using -f Date: Thu, 19 Mar 2009 09:42:51 -0700 From: Zachary Loafman To: Rick Macklem Message-ID: <20090319164251.GA13081@zloafman.west.isilon.com> References: <20090315205229.GV55200@elvis.mu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-OriginalArrivalTime: 19 Mar 2009 16:42:52.0195 (UTC) FILETIME=[BEE24F30:01C9A8B1] Cc: Alfred Perlstein , freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 16:54:53 -0000 First off, I wanted to start by saying something that may interest the community at large: We (Isilon) recently staffed a small NFS group. Our intention is to use and extend Rick's awesome effort. We will have three full-time employees working on producitizing it for us "soon" - by mid-May all three employees should be working on v4. It is our intention to give the work back, but we're still trying to work out our branching/upstreaming model. I don't know if that affects the timing on this being merged to CURRENT or not. It might be nice if we had an opportunity to review some things prior to APIs/VOPs being set in stone, but it would also be nice to get wider exposure for Rick's code. On Sun, Mar 15, 2009 at 05:20:20PM -0400, Rick Macklem wrote: > On Sun, 15 Mar 2009, Alfred Perlstein wrote: > > > >I think it wise to look at 4.1 and scoping that out before taking > >the time to integrate this to gain an understanding of: > NFSv4.1 is still way out there. It hasn't reached RFC stage yet and > vendors are only testing bits and pieces of it. (The current draft > of the "minor" revision is over 500 pages.) > > All the code vendors are currently shipping is running 4.0. I think v4.1 is closer than you might think. We've received numerous requests for pNFS, and I think many vendors will ship basic 4.1 stacks this year. > >1) what it would take to get to 4.1? > A lot. A required feature is something for handling RPC transport > called sessions. One guy has been looking at doing sessions for > FreeBSD (hopefully integrated with Doug Rabson's new RPC code), > but I have no idea if he has made any progress. Can you put us in contact? I'd like to avoid duplication of effort here. > >2) how we would interoperate with other machines until we > >get 4.1 (is everyone doing 4.0 or 4.1?). When will 4.1 become > >the defacto standard (is it already?)? > Systems should still support 4.0 for a long time. I have no idea > when 4.1 will become a defacto standard, but I'd guess years. We've idly been considering going 4.1-only given the relatively slow adoption of 4.0. 4.1 has created a fair amount of buzz and may raise adoption of 4.x. I can't really say for sure. Nor can I say for sure what we'd eventually settle on, since the relative cost of 4.0 once you have 4.1 is fairly small. > I've tried reading the drafts and got swamped. Honestly, I think a > 4.1 implementation would take man years of effort and is beyond > what I am capable of. I hope we can help. :) -- Zach Loafman | Staff Engineer | Isilon Systems From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 18:04:35 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F132F106566B; Thu, 19 Mar 2009 18:04:35 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id B5C798FC14; Thu, 19 Mar 2009 18:04:35 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 6F40146B55; Thu, 19 Mar 2009 14:04:35 -0400 (EDT) Date: Thu, 19 Mar 2009 18:04:35 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Zachary Loafman In-Reply-To: <20090319164251.GA13081@zloafman.west.isilon.com> Message-ID: References: <20090315205229.GV55200@elvis.mu.org> <20090319164251.GA13081@zloafman.west.isilon.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Rick Macklem , Alfred Perlstein , freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 18:04:36 -0000 On Thu, 19 Mar 2009, Zachary Loafman wrote: > First off, I wanted to start by saying something that may interest the > community at large: We (Isilon) recently staffed a small NFS group. Our > intention is to use and extend Rick's awesome effort. We will have three > full-time employees working on producitizing it for us "soon" - by mid-May > all three employees should be working on v4. It is our intention to give the > work back, but we're still trying to work out our branching/upstreaming > model. > > I don't know if that affects the timing on this being merged to CURRENT or > not. It might be nice if we had an opportunity to review some things prior > to APIs/VOPs being set in stone, but it would also be nice to get wider > exposure for Rick's code. If we adopt Rick's code as a "second" experimental stack, and use the existing stack as the default, then I think we can reasonably claim that the binary interfaces specific to it can be changed, especially if we do a good job of tagging which interfaces those are. I have proposed Rick for a commit bit with the hopes of getting the NFS code in the tree sooner rather than later so that it can get exposure, etc, however. BTW, this news from Isilon sounds excellent, and is something that the community as a whole will appreciate a great deal. Something I would like to see happen in the quite short term is get more hands looking at Edward's NFSv4 ACL implementation. Rick's code supports it, but since the NFSv4 ACL code requires rolling our ACL ABI, writes things to disk, etc, I think it's actually much more sensitive to binary compatibility concerns. I'd like to get as many hands as possible doing a review of the semantics (especially UFS, POSIX.1e, Solaris/ZFS, NFSv4, Samba, smbfs, Mac OS X, etc compatability concerns) in the next month or so as possible. Having done the POSIX.1e ACL support almost ten years ago now, I can say that you get stuck with a lot of things you regret if you're not careful, and that the most important concern is semantic interoperability with other systems. The plan right now is to have a semi-formal NFSv4 on FreeBSD session (mini-summit) at the developer summit, FYI, with a goal of working out a number of things relating to NFSv4, the future of NFS on FreeBSD, and the 8.x release. Among other things, I hope we can come out with a clear set of "stuff that needs to be done", maybe even a "who's doing it". Robert N M Watson Computer Laboratory University of Cambridge > > On Sun, Mar 15, 2009 at 05:20:20PM -0400, Rick Macklem wrote: >> On Sun, 15 Mar 2009, Alfred Perlstein wrote: >>> >>> I think it wise to look at 4.1 and scoping that out before taking >>> the time to integrate this to gain an understanding of: >> NFSv4.1 is still way out there. It hasn't reached RFC stage yet and >> vendors are only testing bits and pieces of it. (The current draft >> of the "minor" revision is over 500 pages.) >> >> All the code vendors are currently shipping is running 4.0. > > I think v4.1 is closer than you might think. We've received numerous > requests for pNFS, and I think many vendors will ship basic 4.1 stacks > this year. > >>> 1) what it would take to get to 4.1? >> A lot. A required feature is something for handling RPC transport >> called sessions. One guy has been looking at doing sessions for >> FreeBSD (hopefully integrated with Doug Rabson's new RPC code), >> but I have no idea if he has made any progress. > > Can you put us in contact? I'd like to avoid duplication of effort here. > >>> 2) how we would interoperate with other machines until we >>> get 4.1 (is everyone doing 4.0 or 4.1?). When will 4.1 become >>> the defacto standard (is it already?)? >> Systems should still support 4.0 for a long time. I have no idea >> when 4.1 will become a defacto standard, but I'd guess years. > > We've idly been considering going 4.1-only given the relatively slow > adoption of 4.0. 4.1 has created a fair amount of buzz and may raise > adoption of 4.x. I can't really say for sure. Nor can I say for sure > what we'd eventually settle on, since the relative cost of 4.0 once you > have 4.1 is fairly small. > >> I've tried reading the drafts and got swamped. Honestly, I think a >> 4.1 implementation would take man years of effort and is beyond >> what I am capable of. > > I hope we can help. :) > > -- > Zach Loafman | Staff Engineer | Isilon Systems > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Thu Mar 19 18:40:49 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDFEF10656D8; Thu, 19 Mar 2009 18:40:49 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from gigi.cs.uoguelph.ca (gigi.cs.uoguelph.ca [131.104.94.210]) by mx1.freebsd.org (Postfix) with ESMTP id 8C5158FC08; Thu, 19 Mar 2009 18:40:49 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by gigi.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id n2JIemOk002077; Thu, 19 Mar 2009 14:40:48 -0400 Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n2JIkDH11502; Thu, 19 Mar 2009 14:46:13 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Thu, 19 Mar 2009 14:46:13 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Zachary Loafman In-Reply-To: <20090319164251.GA13081@zloafman.west.isilon.com> Message-ID: References: <20090315205229.GV55200@elvis.mu.org> <20090319164251.GA13081@zloafman.west.isilon.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanned-By: MIMEDefang 2.63 on 131.104.94.210 Cc: Alfred Perlstein , freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 18:40:51 -0000 On Thu, 19 Mar 2009, Zachary Loafman wrote: > First off, I wanted to start by saying something that may interest the > community at large: We (Isilon) recently staffed a small NFS group. Our > intention is to use and extend Rick's awesome effort. We will have three > full-time employees working on producitizing it for us "soon" - by > mid-May all three employees should be working on v4. It is our intention > to give the work back, but we're still trying to work out our > branching/upstreaming model. > Sounds like good news to me. > I don't know if that affects the timing on this being merged to CURRENT > or not. It might be nice if we had an opportunity to review some things > prior to APIs/VOPs being set in stone, but it would also be nice to get > wider exposure for Rick's code. > The only VFS change I've done is redefining the va_filerev/i_modrev attribute/i-node field such that it satisfies the requirements of the nfsv4 Change attribute. (Way back when, i_modrev was put in for not quite nfs by me, I think? I don't think anything else uses it, but I can't be sure?) The changes are: 1 - move it from the in memory i-node to spare space at the end of the on-disk i-node, so that it will survive a server crash. 2 - incrementing it for metadata changes as well as data changes (I can't think how having it survive a crash could break any app. that might be using it. The fact that it's value would change for metadata changes as well as data changes might affect something?) Does anyone know of an app. that uses the va_filerev attribute? [stuff snipped] > I think v4.1 is closer than you might think. We've received numerous > requests for pNFS, and I think many vendors will ship basic 4.1 stacks > this year. > I'll email on nfsv4@linux-nfs.org and ask. (I haven't been to a testing event, but I'd be under an NDA if I had been. My impression was that no one had SSV working yet and, although there had been a fair amount of pNFS testing, sessions were lagging behind. No sessions (and SSV is the front end part of sessions) and no 4.1. (I wouldn't be surprised if some server vendors are getting close, but without a Linux client...) > > Can you put us in contact? I'd like to avoid duplication of effort here. > I'll ping him via email and then contact you. [good stuff snipped] rick From owner-freebsd-arch@FreeBSD.ORG Fri Mar 20 15:58:33 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36992106566C; Fri, 20 Mar 2009 15:58:33 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from skerryvore.cs.uoguelph.ca (skerryvore.cs.uoguelph.ca [131.104.94.204]) by mx1.freebsd.org (Postfix) with ESMTP id 9BCB68FC0C; Fri, 20 Mar 2009 15:58:32 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by skerryvore.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id n2KFwVfV007833; Fri, 20 Mar 2009 11:58:31 -0400 Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n2KG3vv25648; Fri, 20 Mar 2009 12:03:57 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Fri, 20 Mar 2009 12:03:57 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Zachary Loafman In-Reply-To: <20090319164251.GA13081@zloafman.west.isilon.com> Message-ID: References: <20090315205229.GV55200@elvis.mu.org> <20090319164251.GA13081@zloafman.west.isilon.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanned-By: MIMEDefang 2.63 on 131.104.94.204 Cc: Alfred Perlstein , freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2009 15:58:33 -0000 On Thu, 19 Mar 2009, Zachary Loafman wrote: [good stuff snipped] > > I don't know if that affects the timing on this being merged to CURRENT > or not. It might be nice if we had an opportunity to review some things > prior to APIs/VOPs being set in stone, but it would also be nice to get > wider exposure for Rick's code. > Other than the va_filerev/i_modrev issue already mentioned (btw, I grep'd /usr/src and nothing uses va_filerev outside the kernel, it seems), I realized there is another API/VOP issue (kinda in the trivial category, but I should mention it). FreeBSD-CURRENT is out of bits for mnt_flag (the last spare one is used by Edward's ACL code) and it would be nice to add a couple of MNT_EXxxx flags for things like enabling/disabling delegations. Two obvious possible solutions: 1 - bump mnt_flag up to 64 bits 2 - create a separate mnt_exflag field [more stuff snipped] > > I think v4.1 is closer than you might think. We've received numerous > requests for pNFS, and I think many vendors will ship basic 4.1 stacks > this year. > I emailed a request for "predictions" about this on nfsv4@linux-nfs.org and I've only gotten one response sofar: [Tom Haynes wrote] Subject: Re: nfsv4.1 timeframe Rick Macklem wrote: > Anyone feel like making a prediction? (Ideally somewhat realistic:-) > Next year sounds more realistic based on what I saw at Connectathon. I'll post more responses if I get them, rick From owner-freebsd-arch@FreeBSD.ORG Fri Mar 20 19:46:07 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 054CA1065676; Fri, 20 Mar 2009 19:46:07 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id C6A9A8FC17; Fri, 20 Mar 2009 19:46:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (pool-98-109-39-197.nwrknj.fios.verizon.net [98.109.39.197]) by cyrus.watson.org (Postfix) with ESMTPSA id 63BE546B6C; Fri, 20 Mar 2009 15:46:06 -0400 (EDT) Received: from localhost (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id n2KJk0Xp028411; Fri, 20 Mar 2009 15:46:00 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-arch@freebsd.org Date: Fri, 20 Mar 2009 15:38:44 -0400 User-Agent: KMail/1.9.7 References: <20090319164251.GA13081@zloafman.west.isilon.com> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200903201538.45268.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Fri, 20 Mar 2009 15:46:00 -0400 (EDT) X-Virus-Scanned: ClamAV 0.94.2/9145/Fri Mar 20 10:59:16 2009 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Rick Macklem , Alfred Perlstein , Zachary Loafman Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2009 19:46:07 -0000 On Friday 20 March 2009 12:03:57 pm Rick Macklem wrote: > > On Thu, 19 Mar 2009, Zachary Loafman wrote: > > [good stuff snipped] > > > > I don't know if that affects the timing on this being merged to CURRENT > > or not. It might be nice if we had an opportunity to review some things > > prior to APIs/VOPs being set in stone, but it would also be nice to get > > wider exposure for Rick's code. > > > Other than the va_filerev/i_modrev issue already mentioned (btw, I grep'd > /usr/src and nothing uses va_filerev outside the kernel, it seems), I > realized there is another API/VOP issue (kinda in the trivial category, > but I should mention it). FreeBSD-CURRENT is out of bits for mnt_flag (the > last spare one is used by Edward's ACL code) and it would be nice to add a > couple of MNT_EXxxx flags for things like enabling/disabling delegations. > > Two obvious possible solutions: > 1 - bump mnt_flag up to 64 bits > 2 - create a separate mnt_exflag field FreeBSD in general is moving away from new flags in mnt_flags I believe. At least if you purely need NFS-specific flags you can just add new string options via nmount(2) and set appropriate settings in private flags field in your NFS-specific mount data. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Mar 20 19:55:11 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC9D5106566C; Fri, 20 Mar 2009 19:55:11 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from gigi.cs.uoguelph.ca (gigi.cs.uoguelph.ca [131.104.94.210]) by mx1.freebsd.org (Postfix) with ESMTP id 777988FC13; Fri, 20 Mar 2009 19:55:11 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by gigi.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id n2KJt98L000715; Fri, 20 Mar 2009 15:55:09 -0400 Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id n2KK0cq26444; Fri, 20 Mar 2009 16:00:38 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Fri, 20 Mar 2009 16:00:38 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: John Baldwin In-Reply-To: <200903201538.45268.jhb@freebsd.org> Message-ID: References: <20090319164251.GA13081@zloafman.west.isilon.com> <200903201538.45268.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanned-By: MIMEDefang 2.63 on 131.104.94.210 Cc: Zachary Loafman , Alfred Perlstein , freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2009 19:55:12 -0000 On Fri, 20 Mar 2009, John Baldwin wrote: > > FreeBSD in general is moving away from new flags in mnt_flags I believe. At > least if you purely need NFS-specific flags you can just add new string > options via nmount(2) and set appropriate settings in private flags field in > your NFS-specific mount data. > Ok, thanks. I vaguely knew about nmount(2), but didn't realize it was being used for server exports. (The MNT_EXxxx flags are/were used by the NFS server and are relevant when set on an exported local fs like UFS/FFS. As such, they are either generic mount stuff or UFS/FFS specific. I think ZFS keeps its own export stuff, so making it UFS/FFS specific might make sense?) Anyhow, it's not urgent, since the server works ok without any additional flags. rick From owner-freebsd-arch@FreeBSD.ORG Fri Mar 20 21:38:17 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5D90106564A; Fri, 20 Mar 2009 21:38:17 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id B4DA78FC18; Fri, 20 Mar 2009 21:38:17 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (pool-98-109-39-197.nwrknj.fios.verizon.net [98.109.39.197]) by cyrus.watson.org (Postfix) with ESMTPSA id 49DCA46B65; Fri, 20 Mar 2009 17:38:17 -0400 (EDT) Received: from localhost (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id n2KLcBFn029100; Fri, 20 Mar 2009 17:38:11 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: Rick Macklem Date: Fri, 20 Mar 2009 16:57:57 -0400 User-Agent: KMail/1.9.7 References: <200903201538.45268.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200903201657.58470.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Fri, 20 Mar 2009 17:38:11 -0400 (EDT) X-Virus-Scanned: ClamAV 0.94.2/9145/Fri Mar 20 10:59:16 2009 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Zachary Loafman , Alfred Perlstein , freebsd-arch@freebsd.org Subject: Re: NFS version 4.0 for FreeBSD-CURRENT X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2009 21:38:18 -0000 On Friday 20 March 2009 4:00:38 pm Rick Macklem wrote: > > On Fri, 20 Mar 2009, John Baldwin wrote: > > > > > FreeBSD in general is moving away from new flags in mnt_flags I believe. At > > least if you purely need NFS-specific flags you can just add new string > > options via nmount(2) and set appropriate settings in private flags field in > > your NFS-specific mount data. > > > Ok, thanks. I vaguely knew about nmount(2), but didn't realize it was > being used for server exports. (The MNT_EXxxx flags are/were used by the > NFS server and are relevant when set on an exported local fs like UFS/FFS. > As such, they are either generic mount stuff or UFS/FFS specific. I > think ZFS keeps its own export stuff, so making it UFS/FFS specific > might make sense?) > > Anyhow, it's not urgent, since the server works ok without any additional > flags. Ahhh, ok, that is different then. I must have misunderstood. If it is flags the NFS server sets on arbitrary filesystems to manage exporting then a 'mnt_exflags' field might be best. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Sat Mar 21 12:46:17 2009 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB490106566B for ; Sat, 21 Mar 2009 12:46:17 +0000 (UTC) (envelope-from robert@fledge.watson.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 98FB68FC0C for ; Sat, 21 Mar 2009 12:46:17 +0000 (UTC) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 54E1246B0D for ; Sat, 21 Mar 2009 08:46:17 -0400 (EDT) X-Return-Path: X-Received: from cyrus.watson.org ([unix socket]) by cyrus.watson.org (Cyrus v2.3.13) with LMTPA; Fri, 20 Mar 2009 18:20:36 -0400 X-Sieve: CMU Sieve 2.3 X-Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by cyrus.watson.org (Postfix) with ESMTP id 29E8746B0D for ; Fri, 20 Mar 2009 18:20:36 -0400 (EDT) X-Received: from hub.freebsd.org (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 3AAC020450B; Fri, 20 Mar 2009 22:20:18 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) X-Received: from hub.freebsd.org (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id BFB551065738; Fri, 20 Mar 2009 22:20:16 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) X-Delivered-To: current@FreeBSD.org X-Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E19FC106564A; Fri, 20 Mar 2009 22:20:04 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) X-Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id BCB2E8FC17; Fri, 20 Mar 2009 22:20:04 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) X-Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 60A3446B0D; Fri, 20 Mar 2009 18:20:04 -0400 (EDT) Date: Fri, 20 Mar 2009 22:20:04 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: current@FreeBSD.org, net@FreeBSD.org In-Reply-To: Message-ID: References: <20080526110543.J26343@fledge.watson.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Sender: owner-freebsd-current@freebsd.org Errors-To: owner-freebsd-current@freebsd.org ReSent-Date: Sat, 21 Mar 2009 12:46:14 +0000 (GMT) ReSent-From: robert ReSent-To: arch@FreeBSD.org ReSent-Subject: Re: HEADS UP: IFF_NEEDSGIANT consumers to be disabled, removed (was: Re: Wiki page for non-MPSAFE network stack de-orbit scheduling) ReSent-Message-ID: ReSent-User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) Cc: Subject: Re: HEADS UP: IFF_NEEDSGIANT consumers to be disabled, removed (was: Re: Wiki page for non-MPSAFE network stack de-orbit scheduling) X-BeenThere: freebsd-arch@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Mar 2009 12:46:18 -0000 On Mon, 16 Feb 2009, Robert Watson wrote: > The following schedule is proposed, assuming nothing goes horribly wrong > with the new USB code in the next few weeks, and remaining nits relating to > USB network and 802.11 drivers are handled: > > 16 February 2009 HEADS UP to lists (this e-mail) > 01 March 2009 Disable build of all IFF_NEEDSGIANT drivers in 8.x > 01 April 2009 Remove all IFF_NEEDSGIANT drivers from 8.x Just a teminder that 1 April is gradually approaching. At that point I will remove from the tree the various non-MPSAFE network device drivers that are already disconnected from the build, incuding if_sl, if_ppp, if_ar, and the old USB ethernet drivers. Robert N M Watson Computer Laboratory University of Cambridge _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"