From owner-freebsd-cluster@FreeBSD.ORG Tue Jun 17 11:49:40 2003 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C564F37B401 for ; Tue, 17 Jun 2003 11:49:40 -0700 (PDT) Received: from grant.org (grant.org [64.56.118.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 183B943FAF for ; Tue, 17 Jun 2003 11:49:40 -0700 (PDT) (envelope-from mgrant@grant.org) Received: from grant.org (localhost [127.0.0.1]) by grant.org (8.12.8p1/8.12.8) with ESMTP id h5HIncNm001990 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 17 Jun 2003 14:49:39 -0400 (EDT) (envelope-from mgrant@grant.org) Received: (from root@localhost) by grant.org (8.12.8p1/8.12.8/Submit) id h5HIncAs001989 for freebsd-cluster@freebsd.org; Tue, 17 Jun 2003 14:49:38 -0400 (EDT) Date: Tue, 17 Jun 2003 14:49:38 -0400 From: Michael Grant To: freebsd-cluster@freebsd.org Message-ID: <20030617184938.GA1078@grant.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i Subject: Re: iSCSI and clustering with FreeBSD X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2003 18:49:41 -0000 Every time I look into clustering my box, I run into the same thing that Linux is apparently much father ahead in this respect. It sends shudders through me to think that I might have to migrate to Linux. I could do it but it would take a lot of time. It would be great if some better clustering tools appeared for Freebsd. The main thing that is blocking me from building something is a file system that replicates across multiple boxes. Is there still nothing today that I could use on freebsd? I looked into coda and intermezzo but they're not really the right thing. I need something much more transparent. OpenGFS and Lustre were mentioned. Has anyone actually tried to simply compile OpenGFS for freebsd just fixing the compilation problems, or is it really a porting job with lots of rewrite? Would it solve my problems? (see below) One thing that seems like it would help me a lot would be a way to know when a file was modified. Anyone know how to do this? Again, this is possible in Linux. Maybe a simple file replication daemon could be written on top of this. Furthermore, I'm looking at clustering where the other boxes in the cluster are not at the same ISP. This adds additional headache for sure. If anyone has any ideas along these lines, let me know. I want to do this for reliability and load sharing, not for supercomputing. However, it sure would be cool to be able to migrate a process to a different box at a different location... It looks like no matter what I do, I need a second box next to the first one to redirect packets to the other box if one of the boxes goes down. I'd probably do this with NAT or an ip tunnel. This second box almost makes it seem not worthwhile to put the other boxes in different ISPs. Anyone have better ideas? I'd be interested in people's experience who have clustered boxes in this way. I use the term cluster somewhat loosely. More like making a set of boxes which back each other up and share the load. Michael Grant From owner-freebsd-cluster@FreeBSD.ORG Tue Jun 17 12:05:52 2003 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3080537B401 for ; Tue, 17 Jun 2003 12:05:52 -0700 (PDT) Received: from walnut.he.net (walnut.he.net [64.71.137.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id B328643FB1 for ; Tue, 17 Jun 2003 12:05:51 -0700 (PDT) (envelope-from kmacy@fsmware.com) Received: from localhost (kmacy@localhost) by walnut.he.net (8.8.6p2003-03-31/8.8.2) with ESMTP id MAA02915; Tue, 17 Jun 2003 12:05:53 -0700 Date: Tue, 17 Jun 2003 12:05:53 -0700 (PDT) From: Kip Macy X-Sender: kmacy@walnut.he.net To: Michael Grant In-Reply-To: <20030617184938.GA1078@grant.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-cluster@freebsd.org Subject: Re: iSCSI and clustering with FreeBSD X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2003 19:05:52 -0000 > OpenGFS and Lustre were mentioned. Has anyone actually tried to > simply compile OpenGFS for freebsd just fixing the compilation > problems, or is it really a porting job with lots of rewrite? Would > it solve my problems? (see below) I track the OpenGFS development list and it is clear that it has a ways to go with respect to robustness of lock management. I don't think that porting would be that major an undertaking. However, It almost seems that adding clustering to an existing journalled file system would be easier than stabilizing OpenGFS. Lustre is designed for extremely large local clusters. > Furthermore, I'm looking at clustering where the other boxes in the > cluster are not at the same ISP. This adds additional headache for > sure. If anyone has any ideas along these lines, let me know. I want > to do this for reliability and load sharing, not for supercomputing. > However, it sure would be cool to be able to migrate a process to a > different box at a different location... This is not at all what OpenGFS is designed for. That is something that AFS would be perfect for. However, Transarc botched it with respect to the market. At least in the medium term NFSv4 is probably the closest you're going to get. > > It looks like no matter what I do, I need a second box next to the > first one to redirect packets to the other box if one of the boxes > goes down. I'd probably do this with NAT or an ip tunnel. This > second box almost makes it seem not worthwhile to put the other boxes > in different ISPs. Anyone have better ideas? I'm sure some of the ASPs must have solved that problem. From owner-freebsd-cluster@FreeBSD.ORG Tue Jun 17 12:11:09 2003 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 957A637B401 for ; Tue, 17 Jun 2003 12:11:09 -0700 (PDT) Received: from iota.root-servers.ch (iota.root-servers.ch [193.41.193.195]) by mx1.FreeBSD.org (Postfix) with SMTP id 5E10C43F3F for ; Tue, 17 Jun 2003 12:11:07 -0700 (PDT) (envelope-from gabriel_ambuehl@buz.ch) Received: (qmail 45451 invoked from network); 17 Jun 2003 19:11:06 -0000 Received: from dclient217-162-135-22.hispeed.ch (HELO gaxp1800.root.li) (217.162.135.22) by 0 with SMTP; 17 Jun 2003 19:11:06 -0000 Date: Tue, 17 Jun 2003 21:13:49 +0200 From: Gabriel Ambuehl X-Mailer: The Bat! (v1.62i) Educational Organization: BUZ Internet Services X-Priority: 3 (Normal) Message-ID: <69186061515.20030617211349@buz.ch> To: Kip Macy In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: freebsd-cluster@freebsd.org cc: Michael Grant Subject: Re[2]: iSCSI and clustering with FreeBSD X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: gabriel_ambuehl@buz.ch List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2003 19:11:09 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hello Kip, Tuesday, June 17, 2003, 9:05:53 PM, you wrote: >> goes down. I'd probably do this with NAT or an ip tunnel. This >> second box almost makes it seem not worthwhile to put the other boxes >> in different ISPs. Anyone have better ideas? > I'm sure some of the ASPs must have solved that problem. The ASPs usually run their own, DB backed code and thus can count on the DB to replicate itself, no? Or Akamai have mostly static stuff they can just rsync or so... Best regards, Gabriel -----BEGIN PGP SIGNATURE----- Version: PGP 6.0.2i iQEVAwUBPu9aY8Za2WpymlDxAQGejQgAhzRtXCNM8QQGDfPBefaTlw0ksqA1x+yU 3BOyTchUuUM/k3P8nPw0Gu0Y9lv7BECJ0Qgh7NMXM6XmNby2m4s75jOP+X/Q7V9e BCVWaGxOss4qmtFzt5afsdpB+ADivFJoUJt0QXNohR0EHBDeEmAIaWhQOwqYxQFk Peu3fNbS51mhyBtbsMlMeAV26jYcqyaySATn0Rd7IF5kJwHfylNAAaIXDsDGqEXT IGsDzm5wiw+Ppu0NG7x4XcAZHuRC3QddlP6X+megPKWagQCbtSNBeCjk5ULysjGc hFdcI5Gcw8fUGGwt50WBT3BuUCBDn3lRPd20BrWksn1EYiYy3Y66NQ== =1R0V -----END PGP SIGNATURE----- From owner-freebsd-cluster@FreeBSD.ORG Tue Jun 17 12:18:13 2003 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 08E5837B401 for ; Tue, 17 Jun 2003 12:18:13 -0700 (PDT) Received: from walnut.he.net (walnut.he.net [64.71.137.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9137143F85 for ; Tue, 17 Jun 2003 12:18:12 -0700 (PDT) (envelope-from kmacy@fsmware.com) Received: from localhost (kmacy@localhost) by walnut.he.net (8.8.6p2003-03-31/8.8.2) with ESMTP id MAA10326; Tue, 17 Jun 2003 12:18:14 -0700 Date: Tue, 17 Jun 2003 12:18:14 -0700 (PDT) From: Kip Macy X-Sender: kmacy@walnut.he.net To: Gabriel Ambuehl In-Reply-To: <69186061515.20030617211349@buz.ch> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-cluster@freebsd.org Subject: Re: Re[2]: iSCSI and clustering with FreeBSD X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2003 19:18:13 -0000 > > The ASPs usually run their own, DB backed code and thus can count on > the DB to replicate itself, no? Or have gone to the trouble to write their own replication layer. I know of at least on ASP that wrote one on top of MySQL before MySQL provided its own replication support. > Or Akamai have mostly static stuff they can just rsync or so... As a cache you can just refetch the original data. That is the beauty of caching, as data loss is a major concern with file systems. -Kip From owner-freebsd-cluster@FreeBSD.ORG Tue Jun 17 19:42:39 2003 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC1DE37B401 for ; Tue, 17 Jun 2003 19:42:39 -0700 (PDT) Received: from mail007.syd.optusnet.com.au (mail007.syd.optusnet.com.au [210.49.20.180]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4534F43FAF for ; Tue, 17 Jun 2003 19:42:38 -0700 (PDT) (envelope-from Peter.Ross@alumni.tu-berlin.de) Received: from guckloch.optushome.com.au (melax2-188.dialup.optusnet.com.au [210.49.64.188])h5I2gJg31765; Wed, 18 Jun 2003 12:42:20 +1000 Received: from guckloch.optushome.com.au (localhost.optushome.com.au [127.0.0.1])h5I2eHiJ001402; Wed, 18 Jun 2003 12:40:20 +1000 (EST) (envelope-from Peter.Ross@alumni.tu-berlin.de) Received: from localhost (petros@localhost)h5I2eAGF001399; Wed, 18 Jun 2003 12:40:14 +1000 (EST) X-Authentication-Warning: guckloch.optushome.com.au: petros owned process doing -bs Date: Wed, 18 Jun 2003 12:40:08 +1000 (EST) From: Peter Ross X-X-Sender: petros@guckloch.optushome.com.au To: Kip Macy In-Reply-To: Message-ID: <20030618122633.R807@guckloch.optushome.com.au> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-cluster@freebsd.org cc: Michael Grant Subject: Re: iSCSI and clustering with FreeBSD X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2003 02:42:40 -0000 On Tue, 17 Jun 2003, Kip Macy wrote: > > Furthermore, I'm looking at clustering where the other boxes in the > > cluster are not at the same ISP. This adds additional headache for > > sure. If anyone has any ideas along these lines, let me know. I want > > to do this for reliability and load sharing, not for supercomputing. > > However, it sure would be cool to be able to migrate a process to a > > different box at a different location... > > This is not at all what OpenGFS is designed for. That is something that > AFS would be perfect for. I read about bigger projects in Germany running AFS successfully. I 've got a really enthusiastic mail from an administrator maintaining one of these sites. But I read some rumours that AFS isn't running well on FreeBSD (gossip on the German FreeBSD mailing list). Is it true? BTW: I wrote two weeks ago that I like to start writing an iSCSI initiator. I spent the last days reading documents and sources and installing my programming and test environment. So I didn't start writing yet but I'm on the way.. Peter Ross From owner-freebsd-cluster@FreeBSD.ORG Tue Jun 17 20:33:27 2003 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5AB3437B401 for ; Tue, 17 Jun 2003 20:33:27 -0700 (PDT) Received: from walnut.he.net (walnut.he.net [64.71.137.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id DEF9443F75 for ; Tue, 17 Jun 2003 20:33:26 -0700 (PDT) (envelope-from kmacy@fsmware.com) Received: from localhost (kmacy@localhost) by walnut.he.net (8.8.6p2003-03-31/8.8.2) with ESMTP id UAA22412; Tue, 17 Jun 2003 20:33:31 -0700 Date: Tue, 17 Jun 2003 20:33:31 -0700 (PDT) From: Kip Macy X-Sender: kmacy@walnut.he.net To: Peter Ross In-Reply-To: <20030618122633.R807@guckloch.optushome.com.au> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-cluster@freebsd.org Subject: Re: iSCSI and clustering with FreeBSD X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2003 03:33:27 -0000 > I read about bigger projects in Germany running AFS successfully. I 've > got a really enthusiastic mail from an administrator maintaining one of > these sites. AFS continues to be successfully deployed by many universities, CMU and Stanford come to mind. > But I read some rumours that AFS isn't running well on FreeBSD (gossip on > the German FreeBSD mailing list). Is it true? I've never worked with AFS. The closest I've come to it is knowing one of the main developers. > BTW: I wrote two weeks ago that I like to start writing an iSCSI > initiator. I spent the last days reading documents and sources and > installing my programming and test environment. > > So I didn't start writing yet but I'm on the way.. There is a fair amount to come up to speed with. A number of aspects of the protocol don't make sense until you try implementing it. -Kip From owner-freebsd-cluster@FreeBSD.ORG Wed Jun 18 00:16:48 2003 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C28A737B401 for ; Wed, 18 Jun 2003 00:16:48 -0700 (PDT) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9A9BE43FAF for ; Wed, 18 Jun 2003 00:16:47 -0700 (PDT) (envelope-from sporner@nentec.de) Received: from [212.227.126.155] (helo=mrelayng.kundenserver.de) by moutng.kundenserver.de with esmtp (Exim 3.35 #1) id 19SXAH-0003o3-00; Wed, 18 Jun 2003 09:15:29 +0200 Received: from [80.131.138.80] (helo=gate.nentec.de) (TLSv1:EDH-RSA-DES-CBC3-SHA:168) (Exim 3.35 #1) id 19SXAH-0005RH-00; Wed, 18 Jun 2003 09:15:29 +0200 Received: from nenny.nentec.de (nenny.nentec.de [153.92.64.1]) by gate.nentec.de (8.11.3/) with ESMTP id h5I7FQi27642; Wed, 18 Jun 2003 09:15:26 +0200 Received: from nentec.de (andromeda.nentec.de [153.92.64.34]) by nenny.nentec.de (8.11.3/8.11.3) with ESMTP id h5I7FK023331; Wed, 18 Jun 2003 09:15:20 +0200 Message-ID: <3EF01187.7010709@nentec.de> Date: Wed, 18 Jun 2003 09:15:19 +0200 From: Andy Sporner User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2a) Gecko/20020910 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Michael Grant References: <20030617184938.GA1078@grant.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS-perl11-milter (http://amavis.org/) cc: freebsd-cluster@freebsd.org Subject: Re: iSCSI and clustering with FreeBSD X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2003 07:16:49 -0000 Michael Grant wrote: >It looks like no matter what I do, I need a second box next to the >first one to redirect packets to the other box if one of the boxes >goes down. I'd probably do this with NAT or an ip tunnel. This >second box almost makes it seem not worthwhile to put the other boxes >in different ISPs. Anyone have better ideas? > > > This is only a problem with migrating processes. A long time ago (and if you do a search in the IBM "linux" knowledgebase you might find it (ca 1996)) that a resolver protocol would be a good idea that is application based. Kind of like a DNS for applications. This would solve the problem. I had the idea to actually make a patch that would put this into the "connect" primitive in the socket layer so that it would be transparent to the user. HTTP redirect does most of it, but not every thing. For process Migration it is a problem however, since each machine at a minimum has a unique IP address. Therefore some sort of translation is needed to balance the traffic. If there be risk seeking beta testers there, I have somewhere handy a version of my software that has a load balancing NAT in the kernel. It is limited to TCP in the moment. There are some other minor limitations as well which are in the process of being fixed. But if anyone wishes to try it, send me a mail and I will send a small (~200K) tar file with code. In short, I think as long as you wish to redirect applications you are sort of stuck with a middle box. Either that or get a special kind of ethernet switch that does this (ahem ahem ahem... ;-) in hardware and then you don't waste a computer. Andy From owner-freebsd-cluster@FreeBSD.ORG Wed Jun 18 01:44:14 2003 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DD09837B401 for ; Wed, 18 Jun 2003 01:44:14 -0700 (PDT) Received: from iota.root-servers.ch (iota.root-servers.ch [193.41.193.195]) by mx1.FreeBSD.org (Postfix) with SMTP id 11A3743FBD for ; Wed, 18 Jun 2003 01:44:13 -0700 (PDT) (envelope-from gabriel_ambuehl@buz.ch) Received: (qmail 72171 invoked from network); 18 Jun 2003 08:44:12 -0000 Received: from dclient217-162-135-22.hispeed.ch (HELO gaxp1800.root.li) (217.162.135.22) by 0 with SMTP; 18 Jun 2003 08:44:12 -0000 Date: Wed, 18 Jun 2003 10:46:18 +0200 From: Gabriel Ambuehl X-Mailer: The Bat! (v1.62i) Educational Organization: BUZ Internet Services X-Priority: 3 (Normal) Message-ID: <1711841578.20030618104618@buz.ch> To: Peter Ross In-Reply-To: <20030618122633.R807@guckloch.optushome.com.au> References: <20030618122633.R807@guckloch.optushome.com.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: freebsd-cluster@freebsd.org Subject: Re[2]: iSCSI and clustering with FreeBSD X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: gabriel_ambuehl@buz.ch List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2003 08:44:15 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hello Peter, > But I read some rumours that AFS isn't running well on FreeBSD (gossip on > the German FreeBSD mailing list). Is it true? ETZH.h (my former U til I quit) runs a giant, campus wide AFS network. Most people say it kinda sucks but when its down, it seems to be the SANs fault, not AFS. > BTW: I wrote two weeks ago that I like to start writing an iSCSI > initiator. I spent the last days reading documents and sources and > installing my programming and test environment. > So I didn't start writing yet but I'm on the way.. You maybe want some help with that? Best regards, Gabriel -----BEGIN PGP SIGNATURE----- Version: PGP 6.0.2i iQEVAwUBPvAYzsZa2WpymlDxAQG30Qf+Iq6mEsHb4V1gbT5VorimiP6sTRwvlCp/ 1GMT0iwqGExBUv0+Livq6wU5psdOr8wfs+bpGkjGkAaavgZPjTZB+6qq519kBxdo 9qH3cUPNP/NgZq12TzSQplMeEMTUo7+jJUhZFxYEW+MRJF3K38w8eWsKkB/gm+7q HwEHmacUihfSzGrQ/+89CfngAanrTZXAExiOsll11Glyzv3JCyPDUQ/ae3vxSkc7 mcxqkzBOX+HAYSIbplQt83lxs+Q/byMMJkJ6vNNvqGp78/7SFbTnU12xhDP4bYSM 0XoKN1ztuiOAf9/b9V8UNlnYVG68YzHcl0Ts5H2twUJ+1JbTCUTfbA== =W00g -----END PGP SIGNATURE----- From owner-freebsd-cluster@FreeBSD.ORG Wed Jun 18 01:44:15 2003 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E1DDD37B404 for ; Wed, 18 Jun 2003 01:44:14 -0700 (PDT) Received: from iota.root-servers.ch (iota.root-servers.ch [193.41.193.195]) by mx1.FreeBSD.org (Postfix) with SMTP id 1188043F3F for ; Wed, 18 Jun 2003 01:44:13 -0700 (PDT) (envelope-from gabriel_ambuehl@buz.ch) Received: (qmail 72176 invoked from network); 18 Jun 2003 08:44:12 -0000 Received: from dclient217-162-135-22.hispeed.ch (HELO gaxp1800.root.li) (217.162.135.22) by 0 with SMTP; 18 Jun 2003 08:44:12 -0000 Date: Wed, 18 Jun 2003 10:46:26 +0200 From: Gabriel Ambuehl X-Mailer: The Bat! (v1.62i) Educational Organization: BUZ Internet Services X-Priority: 3 (Normal) Message-ID: <731849421.20030618104626@buz.ch> To: Andy Sporner In-Reply-To: <3EF01187.7010709@nentec.de> References: <20030617184938.GA1078@grant.org> <3EF01187.7010709@nentec.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: freebsd-cluster@freebsd.org Subject: Re[2]: iSCSI and clustering with FreeBSD X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: gabriel_ambuehl@buz.ch List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2003 08:44:15 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hello Andy, Wednesday, June 18, 2003, 9:15:19 AM, you wrote: > In short, I think as long as you wish to redirect applications you are > sort of stuck > with a middle box. Either that or get a special kind of ethernet switch > that does this > (ahem ahem ahem... ;-) in hardware and then you don't waste a computer. If can live with NAT, ipf/ipnat will can do this out of the box with some code that detects switches and changes rules accordingly. And with divert sockets of ipfw, you could even write you're own userland daemon to do mangle the packets... Best regards, Gabriel -----BEGIN PGP SIGNATURE----- Version: PGP 6.0.2i iQEVAwUBPvAY1sZa2WpymlDxAQH/6wf/VOyGl5vc05/WRdnPpMFqWcGdVQrBkqVd ZlREzM+q+OJWYpgYbDLfy49HCLCVuMm329GohwN9PZGD3fnjdNZNGQ+SQS9apQxt Mhd1Q5Nsd8pAB+M5g/Wja6sgXGCFho7s0+L8naUk1vKxm00XuDltyhyN1V3/7Jcf dd9mdf6C1JVQiVoh6Z4LuMlblFj4mCqJVnLPctcjsg601IzPNXOsFyJHFrUDfk7D 6rY2xGlatP52J7uK9HOFmL71xWZlRLxP2YyLhpGw1iPracrxdnYCVKUqBTR7hAw2 aJchp8FZ9G65YKYEy24cLqGULaG1ZD/CVqBnARfIDWZrPZ/AQ6Htww== =86X0 -----END PGP SIGNATURE----- From owner-freebsd-cluster@FreeBSD.ORG Wed Jun 18 01:58:52 2003 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 76F4B37B401 for ; Wed, 18 Jun 2003 01:58:52 -0700 (PDT) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.186]) by mx1.FreeBSD.org (Postfix) with ESMTP id B0F8D43FAF for ; Wed, 18 Jun 2003 01:58:51 -0700 (PDT) (envelope-from sporner@nentec.de) Received: from [212.227.126.162] (helo=mrelayng.kundenserver.de) by moutng.kundenserver.de with esmtp (Exim 3.35 #1) id 19SYmH-0001Ry-00; Wed, 18 Jun 2003 10:58:49 +0200 Received: from [80.131.138.80] (helo=gate.nentec.de) (TLSv1:EDH-RSA-DES-CBC3-SHA:168) (Exim 3.35 #1) id 19SYmH-00077i-00; Wed, 18 Jun 2003 10:58:49 +0200 Received: from nenny.nentec.de (nenny.nentec.de [153.92.64.1]) by gate.nentec.de (8.11.3/) with ESMTP id h5I8whi11842; Wed, 18 Jun 2003 10:58:43 +0200 Received: from nentec.de (andromeda.nentec.de [153.92.64.34]) by nenny.nentec.de (8.11.3/8.11.3) with ESMTP id h5I8wa028668; Wed, 18 Jun 2003 10:58:37 +0200 Message-ID: <3EF029BC.70707@nentec.de> Date: Wed, 18 Jun 2003 10:58:36 +0200 From: Andy Sporner User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2a) Gecko/20020910 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gabriel_ambuehl@buz.ch References: <20030617184938.GA1078@grant.org> <3EF01187.7010709@nentec.de> <731849421.20030618104626@buz.ch> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS-perl11-milter (http://amavis.org/) cc: freebsd-cluster@freebsd.org Subject: Re: iSCSI and clustering with FreeBSD X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2003 08:58:52 -0000 Gabriel Ambuehl wrote: >If can live with NAT, ipf/ipnat will can do this out of the box with >some >code that detects switches and changes rules accordingly. > >And with divert sockets of ipfw, you could even write you're own >userland daemon to do mangle the packets... > > I thought this too. but it only does ICMP redirects and that isn't sufficient for what I will need. Not only that some people might want to hide networks behind the load balancer. Also with the user space stuff there is a slow-down because of the context switch. I went to a presentation by Guido (??) at the BSDcon-2000 about this and he was even talking about loadable kernel modules to do this too. But the more I looked at the code the more I had the impression that what people thought about NAT was the other direction (hiding computers behind a firewall). There was some other limitation that I cannot recall at the moment. (I had originally written an interface to create files to the front end of IPFW but there was some kind of problem that basically was too deep to try to fix). Andy