From owner-freebsd-cluster Sun Jan 12 7:44: 9 2003 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3D27837B401 for ; Sun, 12 Jan 2003 07:44:08 -0800 (PST) Received: from iota.root-servers.ch (iota.root-servers.ch [193.41.193.195]) by mx1.FreeBSD.org (Postfix) with SMTP id AB8C443F75 for ; Sun, 12 Jan 2003 07:44:06 -0800 (PST) (envelope-from gabriel_ambuehl@buz.ch) Received: (qmail 72010 invoked from network); 12 Jan 2003 15:44:04 -0000 Received: from dclient80-218-105-86.hispeed.ch (HELO gaxp1800.root.li) (80.218.105.86) by 0 with SMTP; 12 Jan 2003 15:44:04 -0000 Date: Sun, 12 Jan 2003 16:44:24 +0100 From: Gabriel Ambuehl X-Mailer: The Bat! (v1.60q) Educational Reply-To: gabriel_ambuehl@buz.ch Organization: BUZ Internet Services X-Priority: 3 (Normal) Message-ID: <193699876562.20030112164424@buz.ch> To: owner-freebsd-cluster@FreeBSD.ORG, Clifton Royston Cc: "Rouzer, Charles A (Chuck)" , freebsd-cluster@FreeBSD.ORG Subject: Re[2]: freebsd cluster target market In-Reply-To: <20030110132550.A18143@lava.net> References: <000901c2b8ef$57e4eb70$0201000a@LAPTOP> <20030110132550.A18143@lava.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG -----BEGIN PGP SIGNED MESSAGE----- Hello Clifton, Saturday, January 11, 2003, 12:25:50 AM, you wrote: > Hosting of static content is easy by comparison, because you can > simply replicate the data. What's really a bitch to solve is those stupid users that use the filesystem as a database. It's easy enough to replicate databases, they come with the required tools (MySQL does so, PostGRE can be forced to do it and the commercial ones can do it anyway). Fail over for the DB servers is relatively easy, either patch the DB lib to have an array of servers it will cycle through on failure, do IP takeover on the servers (which is relatively hard to get right without either managed switches or hardware reboot facilities in place as there's always a state where a server claims its IP but doesn't do anything else anymore) or just use DNS with very low caching to route the requests to another back end server. It's a major PITA to get FS replication right when there's steady writing to that FS. Something like http://www.complang.tuwien.ac.at/reisner/drbd/ could be a solution (not infinitely scalable but it could help) but it would have to run on FreeBSD. Or maybe it would be possible to implement networked RAID within RAIDframe in 5.0? Best regards, Gabriel -----BEGIN PGP SIGNATURE----- Version: PGP 6.0.2i iQEVAwUBPiF/TMZa2WpymlDxAQFT9Qf+J93oYmZLFeHoWeRQN5J4TGwn0pCf8TPV c10jJet95blDCG9YxagjchJ6E3qm/mGiEUnSNVexf7RFCS1oeyDox6+IuKw9hieY L6Pyf8UEXjYiMLG9H79Kxxp0IYCjHgbGiPQ2UEh9ngfKicbnt1Jt+bn/cw6iUrnX Jh66hUdyWFvthnqBGr7/F+CLWW3Ao8rcVCCH4Lxfehq7d5IWHI9492uC5bOUgFYM 3vPI8sMgRxk8RzcyeCQ6gMXHgln58CbTjZgsuBnq2FGRWxyzu7iJWTZ/oBu9jYCZ PrHERLH18V4Erj3yKPIrCsBiMRewPV2y0cVvIVJxnExMRENc+IlDEw== =jy2j -----END PGP SIGNATURE----- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Sun Jan 12 13:49:34 2003 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ED55937B401 for ; Sun, 12 Jan 2003 13:49:33 -0800 (PST) Received: from vps.vitalit.com (vps.vitalit.com [64.105.194.69]) by mx1.FreeBSD.org (Postfix) with ESMTP id 349B143F3F for ; Sun, 12 Jan 2003 13:49:33 -0800 (PST) (envelope-from car@vitalit.com) Received: from LAPTOP (gso167-138-145.triad.rr.com [24.167.138.145]) by vps.vitalit.com (8.12.3/8.12.3) with SMTP id h0CLnPhQ017749 for ; Sun, 12 Jan 2003 16:49:27 -0500 (EST) (envelope-from car@vitalit.com) Message-ID: <001701c2ba84$6f307010$0201000a@LAPTOP> From: "Rouzer, Charles A (Chuck)" To: References: <000901c2b8ef$57e4eb70$0201000a@LAPTOP> <20030110132550.A18143@lava.net> <193699876562.20030112164424@buz.ch> Subject: Re: freebsd cluster target market Date: Sun, 12 Jan 2003 16:49:01 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2720.3000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In their own words, everyone seems to have similar request. I've mentioned in the past that once an iSCSI driver (nbd replacement) is available for FreeBSD we should be able to combine it with vinum to create a failover capable mirrored file system (drdb replacement solution). Check the archives for recent specifics regarding this type of failover. Chuck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Sun Jan 12 13:59:43 2003 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2011B37B401 for ; Sun, 12 Jan 2003 13:59:42 -0800 (PST) Received: from iota.root-servers.ch (iota.root-servers.ch [193.41.193.195]) by mx1.FreeBSD.org (Postfix) with SMTP id 5836343EB2 for ; Sun, 12 Jan 2003 13:59:40 -0800 (PST) (envelope-from gabriel_ambuehl@buz.ch) Received: (qmail 87966 invoked from network); 12 Jan 2003 21:59:32 -0000 Received: from dclient80-218-105-86.hispeed.ch (HELO gaxp1800.root.li) (80.218.105.86) by 0 with SMTP; 12 Jan 2003 21:59:32 -0000 Date: Sun, 12 Jan 2003 23:00:29 +0100 From: Gabriel Ambuehl X-Mailer: The Bat! (v1.60q) Educational Reply-To: gabriel_ambuehl@buz.ch Organization: BUZ Internet Services X-Priority: 3 (Normal) Message-ID: <37722441765.20030112230029@buz.ch> To: "Rouzer, Charles A (Chuck)" Cc: freebsd-cluster@FreeBSD.ORG Subject: Re[2]: freebsd cluster target market In-Reply-To: <001701c2ba84$6f307010$0201000a@LAPTOP> References: <000901c2b8ef$57e4eb70$0201000a@LAPTOP> <20030110132550.A18143@lava.net> <193699876562.20030112164424@buz.ch> <001701c2ba84$6f307010$0201000a@LAPTOP> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG -----BEGIN PGP SIGNED MESSAGE----- Hello Charles, Sunday, January 12, 2003, 10:49:01 PM, you wrote: > In their own words, everyone seems to have similar request. > I've mentioned in the past that once an iSCSI driver (nbd replacement) > is available for FreeBSD we should be able to combine it with vinum to > create a failover capable mirrored file system (drdb replacement solution). > Check the archives for recent specifics regarding this type of failover. Is there any work being done on iSCSI for FreeBSD? And more importantly, we need both frontend AND backend for this to work. I'm no fan of vinum myself but combined with iSCSI it could pose a solution. I actually once HAD vinum doing network RAID. It's an awful hack and not of any practical use, but enough for proof of concept: Mount foreign filesystem over NFS. Use vn(4) and vnconfig to create a FS backed "partition", use disklabel to assign the correct partition type to it and you're set. It's just horrible unstable, slow and prone to error. Best regards, Gabriel -----BEGIN PGP SIGNATURE----- Version: PGP 6.0.2i iQEVAwUBPiHXccZa2WpymlDxAQH+JQf9FWFtzAF64/pghNZG+Rn0Su3IU1gHqasE 2ZfdKQ6UDUAi+NeoX2uB+6+ZbEaUo6CHt16YW1D6au4jdkOmvqiz3POwPRiuklx8 c5pJNCgSFLveVTVAEId+BD3yav9+/CPTff7tfCZGRwv2pF3A3+pMfk62raJeRAm4 roWPm0gcW6y+w1w8SEeo3XVxVpkutqN4gs97MtK8he8dZzoglmUp2Y7eE+jxL0Ma KIt8uXrp7uRlNGpa8xpdF3zSupAH3NJBs536wvny3Qi6LoEJiHZ8FJvUirMT8Kp5 SS+wXoEsiFjxE7c3j2wOKXFE6PhYtKmzeANWfxa2e9DLtxV7c+1+Wg== =YNYB -----END PGP SIGNATURE----- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Sun Jan 12 14:20: 9 2003 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3EBA437B401 for ; Sun, 12 Jan 2003 14:20:07 -0800 (PST) Received: from vps.vitalit.com (vps.vitalit.com [64.105.194.69]) by mx1.FreeBSD.org (Postfix) with ESMTP id 71B3B43EB2 for ; Sun, 12 Jan 2003 14:20:06 -0800 (PST) (envelope-from car@vitalit.com) Received: from LAPTOP (gso167-138-145.triad.rr.com [24.167.138.145]) by vps.vitalit.com (8.12.3/8.12.3) with SMTP id h0CMK5hQ018849 for ; Sun, 12 Jan 2003 17:20:05 -0500 (EST) (envelope-from car@vitalit.com) Message-ID: <003a01c2ba88$b6f20c70$0201000a@LAPTOP> From: "Rouzer, Charles A (Chuck)" To: References: <000901c2b8ef$57e4eb70$0201000a@LAPTOP> <20030110132550.A18143@lava.net> <193699876562.20030112164424@buz.ch> <001701c2ba84$6f307010$0201000a@LAPTOP> <37722441765.20030112230029@buz.ch> Subject: Re: freebsd cluster target market Date: Sun, 12 Jan 2003 17:19:46 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2720.3000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > Is there any work being done on iSCSI for FreeBSD? And more > importantly, we need both frontend AND backend for this to work. I've come across some posts using google search in regards to iSCSI for FreeBSD, but I'm not aware of any work in progress. I imagine its just a matter of time as iSCSI hopes to displace fiber channel SANs. Can you be more specific about frontend and backend? I am assuming that if someone is writing an iSCSI driver that they would implement full functionality: mounting (frontend) and exporting (backend). Otherwise its not much use. > I'm no fan of vinum myself but combined with iSCSI it could pose a > solution. Recently I have found vinum useful as a "volume manager" (not RAID though that works too). I can create a concat plex/volume and when needed increase the diskspace by adding a subdisk of any available or needed size and using growfs. This allows many more partitions than normally available which is why many use volume managers for large systems. Haven't tried it in a production environment yet, maybe someone else is using this method in production? Now all I need is process migration and process migration capable network file system. ;-) Chuck P.S. Please do not cc me replies. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Sun Jan 12 14:27: 9 2003 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 944DB37B401 for ; Sun, 12 Jan 2003 14:27:07 -0800 (PST) Received: from iota.root-servers.ch (iota.root-servers.ch [193.41.193.195]) by mx1.FreeBSD.org (Postfix) with SMTP id 3660043F5B for ; Sun, 12 Jan 2003 14:27:06 -0800 (PST) (envelope-from gabriel_ambuehl@buz.ch) Received: (qmail 89217 invoked from network); 12 Jan 2003 22:27:04 -0000 Received: from dclient80-218-105-86.hispeed.ch (HELO gaxp1800.root.li) (80.218.105.86) by 0 with SMTP; 12 Jan 2003 22:27:04 -0000 Date: Sun, 12 Jan 2003 23:27:40 +0100 From: Gabriel Ambuehl X-Mailer: The Bat! (v1.60q) Educational Reply-To: gabriel_ambuehl@buz.ch Organization: BUZ Internet Services X-Priority: 3 (Normal) Message-ID: <31724072031.20030112232740@buz.ch> Cc: freebsd-cluster@FreeBSD.ORG Subject: Re[2]: freebsd cluster target market In-Reply-To: <003a01c2ba88$b6f20c70$0201000a@LAPTOP> References: <000901c2b8ef$57e4eb70$0201000a@LAPTOP> <20030110132550.A18143@lava.net> <193699876562.20030112164424@buz.ch> <001701c2ba84$6f307010$0201000a@LAPTOP> <37722441765.20030112230029@buz.ch> <003a01c2ba88$b6f20c70$0201000a@LAPTOP> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG -----BEGIN PGP SIGNED MESSAGE----- Hello Charles, Sunday, January 12, 2003, 11:19:46 PM, you wrote: >> Is there any work being done on iSCSI for FreeBSD? And more >> importantly, we need both frontend AND backend for this to work. > I've come across some posts using google search in regards to iSCSI for > FreeBSD, but I'm not aware of any work in progress. I imagine its just a > matter of time as iSCSI hopes to displace fiber channel SANs. One would hope so. I think there's a Linux implementation for iSCSI that could probably be ported to FreeBSD. There's also a commercial NetBSD based iSCSI appliance of which I don't know whether the iSCSI part is open. I doubt it, though. If it were, that'd be the place to start. > Can you be more specific about frontend and backend? I am assuming that > if someone is writing an iSCSI driver that they would implement full > functionality: mounting (frontend) and exporting (backend). Otherwise its > not much use. That meant exactly. > Now all I need is process migration and process migration capable network > file system. ;-) Try porting MOSIX to FreeBSD... Or use Linux and get both problems solved for "free" (not entirely, cause you get bugged by Linux ;-). Best regards, Gabriel -----BEGIN PGP SIGNATURE----- Version: PGP 6.0.2i iQEVAwUBPiHd1MZa2WpymlDxAQGtPQgAhg0DruuTDBzaduXr3P2yIZZGCX+k6ygw 4aG9JAnXcBIba1EQmVrUXx6sBUxzGAHEk6Lt2BETrrjHjvo2TZXUcSE97KeGHh8e Let6M9z4+v5t9cxu3ZqtV2bR3lYEAACR2VHs1iC364vbccVT54/TG1FW3UpLY2fI X/Lb5xAFwcu/OSwmgykn/XoCUePm4E14SkWBdZcNhryOzC+DYzwgiIIrtZJUE+gP yB294qOhyd5aout5uwT0icSlNEyG4AlObs/jB0foaMl/LVt4tvD3l0RLgcljz+dM Id5XXMDVbqMxI1zVu7zIh/KrweUuRXNydr3/xAwnaZG4p66uRSNrPg== =8gvx -----END PGP SIGNATURE----- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Sun Jan 12 14:35: 2 2003 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 10EDA37B401 for ; Sun, 12 Jan 2003 14:35:02 -0800 (PST) Received: from vps.vitalit.com (vps.vitalit.com [64.105.194.69]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2E9B043F18 for ; Sun, 12 Jan 2003 14:35:01 -0800 (PST) (envelope-from car@vitalit.com) Received: from LAPTOP (gso167-138-145.triad.rr.com [24.167.138.145]) by vps.vitalit.com (8.12.3/8.12.3) with SMTP id h0CMZ0hQ019323 for ; Sun, 12 Jan 2003 17:35:00 -0500 (EST) (envelope-from car@vitalit.com) Message-ID: <004901c2ba8a$cc16ba90$0201000a@LAPTOP> From: "Rouzer, Charles A (Chuck)" To: References: <000901c2b8ef$57e4eb70$0201000a@LAPTOP> <20030110132550.A18143@lava.net> <193699876562.20030112164424@buz.ch> <001701c2ba84$6f307010$0201000a@LAPTOP> <37722441765.20030112230029@buz.ch> <003a01c2ba88$b6f20c70$0201000a@LAPTOP> <31724072031.20030112232740@buz.ch> Subject: Re: freebsd cluster target market Date: Sun, 12 Jan 2003 17:34:40 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2720.3000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > Now all I need is process migration and process migration capable > > network file system. ;-) > > Try porting MOSIX to FreeBSD... Or use Linux and get both problems > solved for "free" (not entirely, cause you get bugged by Linux ;-). Past threads seemed to indicate bproc as the best candidate. The V9FS is also interesting. And hopefully soon LinuxBIOS should be able to boot FreeBSD. http://www.acl.lanl.gov/cluster/projects/index.html#v9fs Chuck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Sun Jan 12 14:48:55 2003 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7C46637B401 for ; Sun, 12 Jan 2003 14:48:54 -0800 (PST) Received: from iota.root-servers.ch (iota.root-servers.ch [193.41.193.195]) by mx1.FreeBSD.org (Postfix) with SMTP id CB30443F3F for ; Sun, 12 Jan 2003 14:48:52 -0800 (PST) (envelope-from gabriel_ambuehl@buz.ch) Received: (qmail 90106 invoked from network); 12 Jan 2003 22:48:50 -0000 Received: from dclient80-218-105-86.hispeed.ch (HELO gaxp1800.root.li) (80.218.105.86) by 0 with SMTP; 12 Jan 2003 22:48:50 -0000 Date: Sun, 12 Jan 2003 23:48:53 +0100 From: Gabriel Ambuehl X-Mailer: The Bat! (v1.60q) Educational Reply-To: gabriel_ambuehl@buz.ch Organization: BUZ Internet Services X-Priority: 3 (Normal) Message-ID: <152725345062.20030112234853@buz.ch> To: freebsd-cluster@FreeBSD.ORG Subject: Re[2]: freebsd cluster target market In-Reply-To: <004901c2ba8a$cc16ba90$0201000a@LAPTOP> References: <000901c2b8ef$57e4eb70$0201000a@LAPTOP> <20030110132550.A18143@lava.net> <193699876562.20030112164424@buz.ch> <001701c2ba84$6f307010$0201000a@LAPTOP> <37722441765.20030112230029@buz.ch> <003a01c2ba88$b6f20c70$0201000a@LAPTOP> <31724072031.20030112232740@buz.ch> <004901c2ba8a$cc16ba90$0201000a@LAPTOP> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG -----BEGIN PGP SIGNED MESSAGE----- Hello Charles, > Past threads seemed to indicate bproc as the best candidate. The V9FS > is also interesting. And hopefully soon LinuxBIOS should be able to boot > FreeBSD. I don't really need distributed FS for my clusters. Realtime mirroring is what I'm after. But if we get that for free while we're at it, I'll also use a distributed FS, of course. Best regards, Gabriel -----BEGIN PGP SIGNATURE----- Version: PGP 6.0.2i iQEVAwUBPiHiz8Za2WpymlDxAQHRjQgAzVNAl5vIRvkrmoZNszE6OFwj/5lZsZiZ wEbP/szcdfRQeKoOl2qlwMF+oTgFTZu+9FLIPAFQqAHqe1hbHKOkKQsjXxQe6PH4 xPUIr+oOBOhrAXjjuhuxAYlQsdvvaf8Z9h0ln9wS7AiQ59sGBe3iI9mGS+Q9NHTA R/ySbYkwQhRedH9n9Yh2kPp7AulJeEemyV//Pt7qxfnAaqYvdCOBiDau4jUnIxlt Wa8wYjfvfzu/s4Xnr7DmHmvbkoIOCw6BMKibprd1ArHhYVrCYWz7Yqx0tUBZ7oIY FRyrqv0ScWRURaqT8P/FL/Y1KvCBjw+K+ubJeFjMGUfmKH6lWIsf7w== =An2j -----END PGP SIGNATURE----- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Mon Jan 13 1:29:25 2003 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9D7CD37B401 for ; Mon, 13 Jan 2003 01:29:22 -0800 (PST) Received: from gate.nentec.de (gate2.nentec.de [194.25.215.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id BC8C743EB2 for ; Mon, 13 Jan 2003 01:29:20 -0800 (PST) (envelope-from sporner@nentec.de) Received: from nenny.nentec.de (root@nenny.nentec.de [153.92.64.1]) by gate.nentec.de (8.11.3/8.9.3) with ESMTP id h0D9THG16557; Mon, 13 Jan 2003 10:29:18 +0100 Received: from nentec.de (andromeda.nentec.de [153.92.64.34]) by nenny.nentec.de (8.11.3/8.11.3) with ESMTP id h0D9T9c31994; Mon, 13 Jan 2003 10:29:10 +0100 Message-ID: <3E2286E5.20609@nentec.de> Date: Mon, 13 Jan 2003 10:29:09 +0100 From: Andy Sporner User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2a) Gecko/20020910 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Rouzer, Charles A (Chuck)" Cc: freebsd-cluster@FreeBSD.ORG Subject: Re: freebsd cluster target market References: <000901c2b8ef$57e4eb70$0201000a@LAPTOP> <20030110132550.A18143@lava.net> <193699876562.20030112164424@buz.ch> <001701c2ba84$6f307010$0201000a@LAPTOP> <37722441765.20030112230029@buz.ch> <003a01c2ba88$b6f20c70$0201000a@LAPTOP> <31724072031.20030112232740@buz.ch> <004901c2ba8a$cc16ba90$0201000a@LAPTOP> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS-perl11-milter (http://amavis.org/) Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This seems to be a good of time as any to post my status on the bproc port. I have been at this since Dec-23 and I can only say that it is not as easy as I had imagined. It has been relegated to a lower priority because there are many things that I garnered from it that make sense in my phase 2 plans. However I have a fundamental problem with some of the architectural choices that were made. If somebody has more cycles, I can make ready my initial work on porting the BPROC stuff in the form of a tar file. What is missing is the actual mechanics of the process extraction and the hooks in the OS. Where I differ with some of the ideas is that the communications should be done in a user process, not in the kernel. Call me a traditionalist, but I don't see that much savings in this particular context to load more stuff there. It is more attractive to make a series of IOCTL functions to insert and remove processes leaving a bare minimum of things in the kernel. I have searched the web and found a project called "Maui" that is BPROC API compatible and also exists for FreeBSD. I have not looked much more into it. But it might be a point on where some urgent needs for this kind of thing can be resolved. As for filesystems and the like. I had also time to make a survey of all of the items in consideration: - DRBD - IScsi - The problems of NFS. I have come to the conclusion that, while a low layer solution is fine, there still is a problem coordinating access to filesystems built on top. Iscsi would make failover of the devices easier, but you still have to syncronize access. As for NFS, problems are noted with regard to changing things under the cover--due largely to the opaque nature of the file handles. I am still studying this, but I think it is possible to convert the filehandles into a physical reference (such as filesystem_id:Inode) so that this reference can float around and be stateless within a cluster. In this way this application does not even know which server he got the information from. This takes care of the top level syncronization. The main point I have not come to a conclusion on yet, is to either patch NFS to handle things this way, or to just do a new FS from scratch. Adding NFS functionality would be great, but there are some new directions I also want to pursue as well. Such as a registration service so that a Filesystem can be found on the network by request. That way when a failover occurs, the I/O can continue. Additionally I would like this to be TCP based so that very large buffers can be transmitted ( > 64K). This has the additional benefit of being ready for WAN. This is really just a small snip from a document I will be publishing on the web (as soon as Deutsche Telekom get re-establish my DSL!!!). What is coming in the same time frame is release 215 of my software suite, which now includes a network NAT to do load balancing as well as regular failover. This would allow for highly available virtual servers. The next step after 215 is to focus on the filesystem that I described above. Hopefully 2nd week of February this is realizable. It would be good if I-scsi is available because that just would make the failover stuff more flexible. I plan initially to use two servers with shared SCSI busses. It would be great to throw this away in favor of I-SCSI. Bye for now (more work to do..... ;-) Andy To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message