From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 00:11:28 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C18F416A4CE for ; Tue, 23 Nov 2004 00:11:28 +0000 (GMT) Received: from bsdhosting.net (bsdhosting.net [65.39.221.113]) by mx1.FreeBSD.org (Postfix) with SMTP id 3BF2743D31 for ; Tue, 23 Nov 2004 00:11:28 +0000 (GMT) (envelope-from jhopper@bsdhosting.net) Received: (qmail 71883 invoked from network); 23 Nov 2004 00:09:36 -0000 Received: from unknown (HELO ?192.168.1.2?) (jhopper@bsdhosting.net@65.39.221.113) by bsdhosting.net with SMTP; 23 Nov 2004 00:09:36 -0000 From: Justin Hopper To: freebsd-cluster@freebsd.org Content-Type: text/plain Message-Id: <1101168686.3370.210.camel@work.gusalmighty.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Mon, 22 Nov 2004 16:11:26 -0800 Content-Transfer-Encoding: 7bit Subject: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 00:11:28 -0000 I was waiting for some activity on this list so that I wouldn't just join and blurt out a question, but so far I haven't seen any messages on this list. My question is pretty general: what hardware clustering options are there for FreeBSD? I was looking at the Appro Clustering product that they have, but they list only Linux and Windoze as supported OSes. I scoured their site looking for documentation on exactly how their clustering appliance works, like how it actually presents itself to the OS as a unified piece of hardware, but I could find no documentation at this level. The idea of using a blade-style system seems appealing, if it worked in FreeBSD, which is the only operating system that we use for production servers. Any thoughts? -- Justin Hopper UNIX Systems Engineer BSDHosting.net Hosting Division of Digital Oasys Inc. http://www.bsdhosting.net From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 00:29:46 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 40B2616A4CE for ; Tue, 23 Nov 2004 00:29:46 +0000 (GMT) Received: from knight.ixsystems.net (afg.ixsystems.net [206.40.55.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 103A143D5C for ; Tue, 23 Nov 2004 00:29:46 +0000 (GMT) (envelope-from matto@knight.ixsystems.net) Received: from knight.ixsystems.net (localhost [127.0.0.1]) by knight.ixsystems.net (8.12.10/8.11.6) with ESMTP id iAN09CZU058339; Mon, 22 Nov 2004 16:09:13 -0800 (PST) (envelope-from matto@knight.ixsystems.net) Received: (from matto@localhost) by knight.ixsystems.net (8.12.10/8.12.9/Submit) id iAN09CWj058338; Mon, 22 Nov 2004 16:09:12 -0800 (PST) (envelope-from matto) Date: Mon, 22 Nov 2004 16:09:12 -0800 From: Matt Olander To: Justin Hopper Message-ID: <20041122160912.L31380@knight.ixsystems.net> References: <1101168686.3370.210.camel@work.gusalmighty.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <1101168686.3370.210.camel@work.gusalmighty.com>; from jhopper@bsdhosting.net on Mon, Nov 22, 2004 at 04:11:26PM -0800 cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 00:29:46 -0000 On Mon, Nov 22, 2004 at 04:11:26PM -0800, Justin Hopper wrote: > I was waiting for some activity on this list so that I wouldn't just > join and blurt out a question, but so far I haven't seen any messages on > this list. yeah, it's pretty quiet on here ;) > My question is pretty general: what hardware clustering options are > there for FreeBSD? I was looking at the Appro Clustering product that > they have, but they list only Linux and Windoze as supported OSes. I > scoured their site looking for documentation on exactly how their > clustering appliance works, like how it actually presents itself to the > OS as a unified piece of hardware, but I could find no documentation at > this level. The idea of using a blade-style system seems appealing, if > it worked in FreeBSD, which is the only operating system that we use for > production servers. hmmm...my company resells Intel's blade solution, which is the one that was co-developed with IBM. but it's not really a 'hardware clustering' option. what are you trying to get it to do? are you looking for computational clustering, or high availability? cheers, -matt -- Matt Olander (408)943-4100 Phone (408)943-4101 Fax www.offmyserver.com -- "Those who don't read have no advantage over those who can't" -Mark Twain From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 00:42:42 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D416A16A4CE for ; Tue, 23 Nov 2004 00:42:42 +0000 (GMT) Received: from bsdhosting.net (bsdhosting.net [65.39.221.113]) by mx1.FreeBSD.org (Postfix) with SMTP id 6163B43D1D for ; Tue, 23 Nov 2004 00:42:42 +0000 (GMT) (envelope-from jhopper@bsdhosting.net) Received: (qmail 76295 invoked from network); 23 Nov 2004 00:40:50 -0000 Received: from unknown (HELO ?192.168.1.2?) (jhopper@bsdhosting.net@65.39.221.113) by bsdhosting.net with SMTP; 23 Nov 2004 00:40:50 -0000 From: Justin Hopper To: Matt Olander In-Reply-To: <20041122160912.L31380@knight.ixsystems.net> References: <1101168686.3370.210.camel@work.gusalmighty.com> <20041122160912.L31380@knight.ixsystems.net> Content-Type: text/plain Message-Id: <1101170559.3370.223.camel@work.gusalmighty.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Mon, 22 Nov 2004 16:42:39 -0800 Content-Transfer-Encoding: 7bit cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 00:42:42 -0000 On Mon, 2004-11-22 at 16:09, Matt Olander wrote: > On Mon, Nov 22, 2004 at 04:11:26PM -0800, Justin Hopper wrote: > > I was waiting for some activity on this list so that I wouldn't just > > join and blurt out a question, but so far I haven't seen any messages on > > this list. > > yeah, it's pretty quiet on here ;) > > > My question is pretty general: what hardware clustering options are > > there for FreeBSD? I was looking at the Appro Clustering product that > > they have, but they list only Linux and Windoze as supported OSes. I > > scoured their site looking for documentation on exactly how their > > clustering appliance works, like how it actually presents itself to the > > OS as a unified piece of hardware, but I could find no documentation at > > this level. The idea of using a blade-style system seems appealing, if > > it worked in FreeBSD, which is the only operating system that we use for > > production servers. > > hmmm...my company resells Intel's blade solution, which is the one that > was co-developed with IBM. but it's not really a 'hardware clustering' > option. what are you trying to get it to do? are you looking for > computational clustering, or high availability? My term of "hardware clustering" might have been incorrect. I'm looking more for high availability, but a large pool of resources would be beneficial as well. It would be ideal to have a system where you can add new blades as more resources become necessary, instead of adding individual servers which each run their own OS and have their own pool of resources. -- Justin Hopper UNIX Systems Engineer BSDHosting.net Hosting Division of Digital Oasys Inc. http://www.bsdhosting.net From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 00:53:18 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB22016A4CE for ; Tue, 23 Nov 2004 00:53:18 +0000 (GMT) Received: from knight.ixsystems.net (afg.ixsystems.net [206.40.55.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id A061A43D4C for ; Tue, 23 Nov 2004 00:53:18 +0000 (GMT) (envelope-from matto@knight.ixsystems.net) Received: from knight.ixsystems.net (localhost [127.0.0.1]) by knight.ixsystems.net (8.12.10/8.11.6) with ESMTP id iAN0WjZU058589; Mon, 22 Nov 2004 16:32:45 -0800 (PST) (envelope-from matto@knight.ixsystems.net) Received: (from matto@localhost) by knight.ixsystems.net (8.12.10/8.12.9/Submit) id iAN0WjMc058588; Mon, 22 Nov 2004 16:32:45 -0800 (PST) (envelope-from matto) Date: Mon, 22 Nov 2004 16:32:45 -0800 From: Matt Olander To: Justin Hopper Message-ID: <20041122163244.M31380@knight.ixsystems.net> References: <1101168686.3370.210.camel@work.gusalmighty.com> <20041122160912.L31380@knight.ixsystems.net> <1101170559.3370.223.camel@work.gusalmighty.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <1101170559.3370.223.camel@work.gusalmighty.com>; from jhopper@bsdhosting.net on Mon, Nov 22, 2004 at 04:42:39PM -0800 cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 00:53:19 -0000 On Mon, Nov 22, 2004 at 04:42:39PM -0800, Justin Hopper wrote: > My term of "hardware clustering" might have been incorrect. I'm looking > more for high availability, but a large pool of resources would be > beneficial as well. It would be ideal to have a system where you can > add new blades as more resources become necessary, instead of adding > individual servers which each run their own OS and have their own pool > of resources. hey Justin, There's a ton of stuff to help you. A few that come to mind for fail over are pound, plb, and freevrrpd. They all might do something of what you are looking for. For most blade setups, they typically come with some software that allows you to setup an image server with various server images and auto-deploy them to various blades. For instance, the Intel blades come with Veritas OpForce to control image deployment. It's mostly for restoring from a bare-metal state though. As in, a blade goes down and you have a rule setup that slot3 is a webserver. So, you call the data-center tech and tell him to swap out the blade in slot3 (in which you turn on a red indicator light on the front so he knows which one) with a fresh one, preferrably from the few extra that you had the foresight to purchase ;) The system detects that a new blade is in slot3 and deploys the webserver image, as per your rule. If anything goes wrong, you remote console in via the built-in management module. That's all in a perfect world, of course. In reality, Veritas doesn't officially support FreeBSD as an operating system for their OpForce software, but I'm talking to them and we'll see if it goes anywhere :P regards, -matt > > -- > Justin Hopper > UNIX Systems Engineer > BSDHosting.net > Hosting Division of Digital Oasys Inc. > http://www.bsdhosting.net -- Matt Olander (408)943-4100 Phone (408)943-4101 Fax www.offmyserver.com -- "Those who don't read have no advantage over those who can't" -Mark Twain From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 01:20:32 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8226D16A4CE for ; Tue, 23 Nov 2004 01:20:32 +0000 (GMT) Received: from bsdhosting.net (bsdhosting.net [65.39.221.113]) by mx1.FreeBSD.org (Postfix) with SMTP id EF4CD43D4C for ; Tue, 23 Nov 2004 01:20:31 +0000 (GMT) (envelope-from jhopper@bsdhosting.net) Received: (qmail 81787 invoked from network); 23 Nov 2004 01:18:40 -0000 Received: from unknown (HELO ?192.168.1.2?) (jhopper@bsdhosting.net@65.39.221.113) by bsdhosting.net with SMTP; 23 Nov 2004 01:18:40 -0000 From: Justin Hopper To: Matt Olander In-Reply-To: <20041122163244.M31380@knight.ixsystems.net> References: <1101168686.3370.210.camel@work.gusalmighty.com> <20041122160912.L31380@knight.ixsystems.net> <1101170559.3370.223.camel@work.gusalmighty.com> <20041122163244.M31380@knight.ixsystems.net> Content-Type: text/plain Message-Id: <1101172829.15634.5.camel@work.gusalmighty.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Mon, 22 Nov 2004 17:20:30 -0800 Content-Transfer-Encoding: 7bit cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 01:20:32 -0000 On Mon, 2004-11-22 at 16:32, Matt Olander wrote: > On Mon, Nov 22, 2004 at 04:42:39PM -0800, Justin Hopper wrote: > > My term of "hardware clustering" might have been incorrect. I'm looking > > more for high availability, but a large pool of resources would be > > beneficial as well. It would be ideal to have a system where you can > > add new blades as more resources become necessary, instead of adding > > individual servers which each run their own OS and have their own pool > > of resources. > > hey Justin, > > There's a ton of stuff to help you. A few that come to mind for fail > over are pound, plb, and freevrrpd. They all might do something of what you > are looking for. > > For most blade setups, they typically come with some software that > allows you to setup an image server with various server images and > auto-deploy them to various blades. For instance, the Intel blades come > with Veritas OpForce to control image deployment. > > It's mostly for restoring from a bare-metal state though. As in, a blade > goes down and you have a rule setup that slot3 is a webserver. So, you > call the data-center tech and tell him to swap out the blade in slot3 > (in which you turn on a red indicator light on the front so he knows > which one) with a fresh one, preferrably from the few extra that you had > the foresight to purchase ;) > > The system detects that a new blade is in slot3 and deploys the > webserver image, as per your rule. If anything goes wrong, you remote > console in via the built-in management module. That's all in a perfect > world, of course. In reality, Veritas doesn't officially support FreeBSD > as an operating system for their OpForce software, but I'm talking to > them and we'll see if it goes anywhere :P Interesting. So most blade servers allow for each node in the cluster to run as it's own system, for example as a webserver, right? Is there no appliance that allows for the details of the hardware to be hidden from the OS and instead present to the OS a unified architecture, like it's just one machine, but with the ability to add more nodes to expand CPU, RAM, and disk? I guess this was my misunderstanding, as this is what I assumed the blade systems did. I assume it would be incredibly tricky to manage dynamically configurable hardware in the operating system, but I also assumed that somebody had pulled it off, but maybe not? -- Justin Hopper UNIX Systems Engineer BSDHosting.net Hosting Division of Digital Oasys Inc. http://www.bsdhosting.net From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 01:29:35 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ECD8D16A4CE for ; Tue, 23 Nov 2004 01:29:34 +0000 (GMT) Received: from knight.ixsystems.net (afg.ixsystems.net [206.40.55.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6BF3643D1F for ; Tue, 23 Nov 2004 01:29:34 +0000 (GMT) (envelope-from matto@knight.ixsystems.net) Received: from knight.ixsystems.net (localhost [127.0.0.1]) by knight.ixsystems.net (8.12.10/8.11.6) with ESMTP id iAN191ZU058897; Mon, 22 Nov 2004 17:09:01 -0800 (PST) (envelope-from matto@knight.ixsystems.net) Received: (from matto@localhost) by knight.ixsystems.net (8.12.10/8.12.9/Submit) id iAN191sI058896; Mon, 22 Nov 2004 17:09:01 -0800 (PST) (envelope-from matto) Date: Mon, 22 Nov 2004 17:09:01 -0800 From: Matt Olander To: Justin Hopper Message-ID: <20041122170900.N31380@knight.ixsystems.net> References: <1101168686.3370.210.camel@work.gusalmighty.com> <20041122160912.L31380@knight.ixsystems.net> <1101170559.3370.223.camel@work.gusalmighty.com> <20041122163244.M31380@knight.ixsystems.net> <1101172829.15634.5.camel@work.gusalmighty.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <1101172829.15634.5.camel@work.gusalmighty.com>; from jhopper@bsdhosting.net on Mon, Nov 22, 2004 at 05:20:30PM -0800 cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 01:29:35 -0000 On Mon, Nov 22, 2004 at 05:20:30PM -0800, Justin Hopper wrote: > Interesting. So most blade servers allow for each node in the cluster > to run as it's own system, for example as a webserver, right? Yes. The blades are all independent systems that share common redundant infrastructure, such as switch, powersupply, etc. > Is there no appliance that allows for the details of the hardware to be > hidden from the OS and instead present to the OS a unified architecture, > like it's just one machine, but with the ability to add more nodes to > expand CPU, RAM, and disk? I guess this was my misunderstanding, as > this is what I assumed the blade systems did. I assume it would be > incredibly tricky to manage dynamically configurable hardware in the > operating system, but I also assumed that somebody had pulled it off, > but maybe not? Sure. You can buy a Sun Micro Fire 12k 36-way for about 1.3 million ;) An x86 based blade system combined with some opensource fail-over software will definitely do what you need though, and is quite a bit easier on the wallet! Regards, -matt > -- > Justin Hopper > UNIX Systems Engineer > BSDHosting.net > Hosting Division of Digital Oasys Inc. > http://www.bsdhosting.net -- Matt Olander (408)943-4100 Phone (408)943-4101 Fax www.offmyserver.com -- "Those who don't read have no advantage over those who can't" -Mark Twain From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 02:13:25 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 345AE16A4CE for ; Tue, 23 Nov 2004 02:13:25 +0000 (GMT) Received: from mail2.zrz.tu-berlin.de (mail2.zrz.TU-Berlin.DE [130.149.4.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0DE6643D1D for ; Tue, 23 Nov 2004 02:13:24 +0000 (GMT) (envelope-from Peter.Ross@alumni.tu-berlin.de) Received: from localhost ([127.0.0.1] helo=mail2.zrz.TU-Berlin.DE) by mail2.zrz.tu-berlin.de with esmtp (exim-4.43) id 1CWQBF-0000MX-Ct; Tue, 23 Nov 2004 03:13:22 +0100 Received: from mail2.zrz.TU-Berlin.DE ([130.149.4.14]) by mail2.zrz.TU-Berlin.DE (MailMonitor for SMTP v1.2.2 ) ; Tue, 23 Nov 2004 03:13:21 +0100 (CET) Received: from mailbox.tu-berlin.de ([130.149.4.18]) by mail2.zrz.tu-berlin.de with esmtp (exim-4.43) id 1CWQBF-0000MM-Bb; Tue, 23 Nov 2004 03:13:21 +0100 Received: from apache by mailbox.tu-berlin.de with local (exim-4.43) id 1CWQBF-0007fj-HT; Tue, 23 Nov 2004 03:13:21 +0100 Received: from 211.26.240.17 (SquirrelMail authenticated user rossicbb) by mailbox.TU-Berlin.DE with HTTP; Tue, 23 Nov 2004 13:13:21 +1100 (EST) Message-ID: <41102.211.26.240.17.1101176001.squirrel@mailbox.TU-Berlin.DE> Date: Tue, 23 Nov 2004 13:13:21 +1100 (EST) From: "Peter Ross" To: In-Reply-To: <20041122163244.M31380@knight.ixsystems.net> References: <1101168686.3370.210.camel@work.gusalmighty.com> <20041122160912.L31380@knight.ixsystems.net> <1101170559.3370.223.camel@work.gusalmighty.com> <20041122163244.M31380@knight.ixsystems.net> X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.8) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Scanned: Sophos MailMonitor on mail2.zrz.tu-berlin.de; Tue, 23 Nov 2004 03:13:21 +0100 cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 02:13:25 -0000 Matt Olander wrote: > The system detects that a new blade is in slot3 and deploys the > webserver image, as per your rule. If I do a first installation I write a script documenting everything I change after deploying a base system (e.g. changes in /etc, installing of packages, creation of directories or symlinks etc.) Later I do a network boot, installing the base system, running the script and restore data. Advantages: The script documents my machine and I always know what a machine does. A new installation is always clean and does not contain traces of former try and error procedures, reconfigurations etc. I can use it with newer OS and software versions. (OK, I have to check it once). It is easy to convert a machine from running service A (e.g. FTP) to a machine serving B (e.g. a mail server). If I am in need of faster recovery I have a bunch of harddisks or even blades ready to go. Imaging is a tool created by whimps to copy Windows systems (where installation does black magic and usually needs interaction;-). And imaging is a source of wealth for manufacturers. See the license fee for the IBM Director Remote Deployment Module. As A UNIX admin familiar with a routine described above I do not see many advantages. Regards Peter From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 03:15:24 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EC1C816A4CF for ; Tue, 23 Nov 2004 03:15:24 +0000 (GMT) Received: from bsdhosting.net (bsdhosting.net [65.39.221.113]) by mx1.FreeBSD.org (Postfix) with SMTP id 6F13F43D45 for ; Tue, 23 Nov 2004 03:15:24 +0000 (GMT) (envelope-from jhopper@bsdhosting.net) Received: (qmail 98105 invoked from network); 23 Nov 2004 03:13:32 -0000 Received: from unknown (HELO ?192.168.1.2?) (jhopper@bsdhosting.net@65.39.221.113) by bsdhosting.net with SMTP; 23 Nov 2004 03:13:32 -0000 From: Justin Hopper To: Matt Olander In-Reply-To: <20041122170900.N31380@knight.ixsystems.net> References: <1101168686.3370.210.camel@work.gusalmighty.com> <20041122160912.L31380@knight.ixsystems.net> <1101170559.3370.223.camel@work.gusalmighty.com> <20041122163244.M31380@knight.ixsystems.net> <1101172829.15634.5.camel@work.gusalmighty.com> <20041122170900.N31380@knight.ixsystems.net> Content-Type: text/plain Message-Id: <1101179722.15634.12.camel@work.gusalmighty.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Mon, 22 Nov 2004 19:15:22 -0800 Content-Transfer-Encoding: 7bit cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 03:15:25 -0000 On Mon, 2004-11-22 at 17:09, Matt Olander wrote: > On Mon, Nov 22, 2004 at 05:20:30PM -0800, Justin Hopper wrote: > > Interesting. So most blade servers allow for each node in the cluster > > to run as it's own system, for example as a webserver, right? > > Yes. The blades are all independent systems that share common > redundant infrastructure, such as switch, powersupply, etc. > > > Is there no appliance that allows for the details of the hardware to be > > hidden from the OS and instead present to the OS a unified architecture, > > like it's just one machine, but with the ability to add more nodes to > > expand CPU, RAM, and disk? I guess this was my misunderstanding, as > > this is what I assumed the blade systems did. I assume it would be > > incredibly tricky to manage dynamically configurable hardware in the > > operating system, but I also assumed that somebody had pulled it off, > > but maybe not? > > Sure. You can buy a Sun Micro Fire 12k 36-way for about 1.3 million ;) > An x86 based blade system combined with some opensource fail-over > software will definitely do what you need though, and is quite a bit > easier on the wallet! Thanks for the info, Matt. This is indeed what I wanted to know about blade servers. Unfortunate that they are not what I had thought they were though. I'll take a look at the docs for freevrrpd, that sounds pretty interesting. -- Justin Hopper UNIX Systems Engineer BSDHosting.net Hosting Division of Digital Oasys Inc. http://www.bsdhosting.net From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 03:40:44 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F042A16A4CE for ; Tue, 23 Nov 2004 03:40:44 +0000 (GMT) Received: from mail2.zrz.tu-berlin.de (mail2.zrz.TU-Berlin.DE [130.149.4.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 23B1043D41 for ; Tue, 23 Nov 2004 03:40:44 +0000 (GMT) (envelope-from Peter.Ross@alumni.tu-berlin.de) Received: from localhost ([127.0.0.1] helo=mail2.zrz.TU-Berlin.DE) by mail2.zrz.tu-berlin.de with esmtp (exim-4.43) id 1CWRXn-0001OP-9v; Tue, 23 Nov 2004 04:40:43 +0100 Received: from mail2.zrz.TU-Berlin.DE ([130.149.4.14]) by mail2.zrz.TU-Berlin.DE (MailMonitor for SMTP v1.2.2 ) ; Tue, 23 Nov 2004 04:40:42 +0100 (CET) Received: from mailbox.tu-berlin.de ([130.149.4.18]) by mail2.zrz.tu-berlin.de with esmtp (exim-4.43) id 1CWRXm-0001OE-Br; Tue, 23 Nov 2004 04:40:42 +0100 Received: from apache by mailbox.tu-berlin.de with local (exim-4.43) id 1CWRXm-0008VH-Jd; Tue, 23 Nov 2004 04:40:42 +0100 Received: from 211.26.240.17 (SquirrelMail authenticated user rossicbb) by mailbox.TU-Berlin.DE with HTTP; Tue, 23 Nov 2004 14:40:42 +1100 (EST) Message-ID: <58496.211.26.240.17.1101181242.squirrel@mailbox.TU-Berlin.DE> Date: Tue, 23 Nov 2004 14:40:42 +1100 (EST) From: "Peter Ross" To: In-Reply-To: <1101179722.15634.12.camel@work.gusalmighty.com> References: <1101168686.3370.210.camel@work.gusalmighty.com> <20041122160912.L31380@knight.ixsystems.net> <1101170559.3370.223.camel@work.gusalmighty.com> <20041122163244.M31380@knight.ixsystems.net> <1101172829.15634.5.camel@work.gusalmighty.com> <20041122170900.N31380@knight.ixsystems.net> <1101179722.15634.12.camel@work.gusalmighty.com> X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.8) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Scanned: Sophos MailMonitor on mail2.zrz.tu-berlin.de; Tue, 23 Nov 2004 04:40:42 +0100 cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 03:40:45 -0000 Justin Hopper wrote: > I'll take a look at the docs for freevrrpd, that sounds pretty > interesting. I used VRRP on Linux machines. It worked very well. I used ipfilter rules on a firewall infront of servers to setup TCP connections by random. The rule is only needed for a setup packet because the firewall keeps track of established connections. (BTW: I did the same with the random netfilter module on Linux). Besides of that I used pound to loadbalance HTTP requests to multiple web servers. AFS seems to be generally a good idea to mirror filesystems but I do not have working experience with it. And it seems that FreeBSD has OpenAFS and an own implementation? (And I heard rumours that both are highly experimental?) Maybe it is good to collect all the small programs you can use for clustering and higher availibility together on a website. I do not know one yet. Regards Peter From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 05:44:40 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C2D6516A4CE for ; Tue, 23 Nov 2004 05:44:40 +0000 (GMT) Received: from tinker.exit.com (tinker.exit.com [206.223.0.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6A28F43D48 for ; Tue, 23 Nov 2004 05:44:40 +0000 (GMT) (envelope-from frank@exit.com) Received: from realtime.exit.com (realtime [206.223.0.5]) by tinker.exit.com (8.13.1/8.12.9) with ESMTP id iAN5rCQN097579; Mon, 22 Nov 2004 21:53:21 -0800 (PST) (envelope-from frank@exit.com) Received: from realtime.exit.com (localhost [127.0.0.1]) by realtime.exit.com (8.13.1/8.12.9) with ESMTP id iAN5iUJ0072092; Mon, 22 Nov 2004 21:44:30 -0800 (PST) (envelope-from frank@realtime.exit.com) Received: (from frank@localhost) by realtime.exit.com (8.13.1/8.13.1/Submit) id iAN5iUUQ072091; Mon, 22 Nov 2004 21:44:30 -0800 (PST) (envelope-from frank) From: Frank Mayhar Message-Id: <200411230544.iAN5iUUQ072091@realtime.exit.com> In-Reply-To: <1101172829.15634.5.camel@work.gusalmighty.com> To: Justin Hopper Date: Mon, 22 Nov 2004 21:44:30 -0800 (PST) X-Copyright0: Copyright 2004 Frank Mayhar. All Rights Reserved. X-Copyright1: Permission granted for electronic reproduction as Usenet News or email only. X-Mailer: ELM [version 2.4ME+ PL119 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: frank@exit.com List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 05:44:40 -0000 Justin Hopper wrote: > Is there no appliance that allows for the details of the hardware to be > hidden from the OS and instead present to the OS a unified architecture, > like it's just one machine, but with the ability to add more nodes to > expand CPU, RAM, and disk? I guess this was my misunderstanding, as > this is what I assumed the blade systems did. I assume it would be > incredibly tricky to manage dynamically configurable hardware in the > operating system, but I also assumed that somebody had pulled it off, > but maybe not? Well, there's software that does that (or something like it), but not in BSD-land, at least not yet. Although I agitated hard to port what is now OpenSSI (see http://www.openssi.org/) to FreeBSD back in 2000, they ported it to Linux instead, sigh. I would really love it if someone would actually pay me to do the port to FreeBSD (or, better, to reimplement it). But that's pretty unlikely. So no, clustering support is thin on the ground for BSD. Maybe someday, when Matt manages to start work on his SSI stuff for Dragonfly. -- Frank Mayhar frank@exit.com http://www.exit.com/ Exit Consulting http://www.gpsclock.com/ http://www.exit.com/blog/frank/ From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 16:12:41 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8BF5D16A4CE for ; Tue, 23 Nov 2004 16:12:41 +0000 (GMT) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D4DB243D49 for ; Tue, 23 Nov 2004 16:12:40 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id iANGCZum031802; Tue, 23 Nov 2004 10:12:35 -0600 (CST) (envelope-from anderson@centtech.com) Message-ID: <41A3616F.20502@centtech.com> Date: Tue, 23 Nov 2004 10:12:31 -0600 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.3) Gecko/20041110 X-Accept-Language: en-us, en MIME-Version: 1.0 To: frank@exit.com References: <200411230544.iAN5iUUQ072091@realtime.exit.com> In-Reply-To: <200411230544.iAN5iUUQ072091@realtime.exit.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/598/Sat Nov 20 16:30:09 2004 clamav-milter version 0.80j on mh1.centtech.com X-Virus-Status: Clean cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 16:12:41 -0000 Frank Mayhar wrote: > Well, there's software that does that (or something like it), but not in BSD-land, at > least not yet. Although I agitated hard to port what is now OpenSSI (see > http://www.openssi.org/) to FreeBSD back in 2000, they ported it to Linux instead, > sigh. I would really love it if someone would actually pay me to do the port to > FreeBSD (or, better, to reimplement it). But that's pretty unlikely. Maybe you should take up a collection like others have done. I bet if you are a qualified coder for the project, you could raise enough to do it. There are a lot of people looking for this. I'm interested in clustered filesystems to be used for high-availability scalable NFS servers. Eric -- ------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Talk sense to a fool and he calls you foolish. ------------------------------------------------------------------ From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 16:38:59 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 27EAF16A4CE for ; Tue, 23 Nov 2004 16:38:59 +0000 (GMT) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6258543D48 for ; Tue, 23 Nov 2004 16:38:58 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (nkdqte@localhost [127.0.0.1]) by lurza.secnetix.de (8.12.11/8.12.11) with ESMTP id iANGctuw037933 for ; Tue, 23 Nov 2004 17:38:56 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.12.11/8.12.11/Submit) id iANGctsC037932; Tue, 23 Nov 2004 17:38:55 +0100 (CET) (envelope-from olli) Date: Tue, 23 Nov 2004 17:38:55 +0100 (CET) Message-Id: <200411231638.iANGctsC037932@lurza.secnetix.de> From: Oliver Fromme To: freebsd-cluster@FreeBSD.ORG In-Reply-To: <41A3616F.20502@centtech.com> X-Newsgroups: list.freebsd-cluster User-Agent: tin/1.5.4-20000523 ("1959") (UNIX) (FreeBSD/4.10-RELEASE (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: freebsd-cluster@FreeBSD.ORG List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 16:38:59 -0000 Eric Anderson wrote: > I'm interested in clustered filesystems to be used for high-availability > scalable NFS servers. For that purpose I can recommend NetApp filers (which can be clustered to achieve HA configurations). Not exactly a FreeBSD-based solution (although their firm- ware _is_ based on some BSD, I have heard), and not a cheap one, though. But it works very nicely. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co KG, Oettingenstr. 2, 80538 München Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "To this day, many C programmers believe that 'strong typing' just means pounding extra hard on the keyboard." -- Peter van der Linden From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 16:48:22 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5C5DE16A4CE for ; Tue, 23 Nov 2004 16:48:22 +0000 (GMT) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id A9F0F43D5E for ; Tue, 23 Nov 2004 16:48:21 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id iANGmKeE032417 for ; Tue, 23 Nov 2004 10:48:21 -0600 (CST) (envelope-from anderson@centtech.com) Message-ID: <41A369D1.6000303@centtech.com> Date: Tue, 23 Nov 2004 10:48:17 -0600 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.3) Gecko/20041110 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-cluster@freebsd.org References: <200411231638.iANGctsC037932@lurza.secnetix.de> In-Reply-To: <200411231638.iANGctsC037932@lurza.secnetix.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.80/598/Sat Nov 20 16:30:09 2004 clamav-milter version 0.80j on mh1.centtech.com X-Virus-Status: Clean Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 16:48:22 -0000 Oliver Fromme wrote: > Eric Anderson wrote: > > I'm interested in clustered filesystems to be used for high-availability > > scalable NFS servers. > > For that purpose I can recommend NetApp filers (which can > be clustered to achieve HA configurations). > > Not exactly a FreeBSD-based solution (although their firm- > ware _is_ based on some BSD, I have heard), and not a cheap > one, though. But it works very nicely. While NetApps are nice, they are *WAY* too costly for most uses. They have priced themselves out of a very large market. We currently are using some clustering software from Polyserve, but it only runs on linux (Suse and Redhat). It's great stuff, and I can honestly say the support and all is pretty good - but doesn't run with FreeBSD. I've nagged them about it, but they claim there's no market for them.. I see projects like lustre, and opengfs, and wonder 'why not for FreeBSD?'.. If I were a good coder, I'd work on porting it.. Eric -- ------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Talk sense to a fool and he calls you foolish. ------------------------------------------------------------------ From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 21:14:32 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C754416A4CE for ; Tue, 23 Nov 2004 21:14:32 +0000 (GMT) Received: from priv-edtnes40.telusplanet.net (outbound05.telus.net [199.185.220.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3DCC743D46 for ; Tue, 23 Nov 2004 21:14:32 +0000 (GMT) (envelope-from alex@taskforce-1.com) Received: from node2.grid0 ([206.116.226.103]) by priv-edtnes40.telusplanet.netESMTP <20041123211431.WUYI27713.priv-edtnes40.telusplanet.net@node2.grid0> for ; Tue, 23 Nov 2004 14:14:31 -0700 From: Alex Pavlovic Organization: TF-1 To: freebsd-cluster@freebsd.org Date: Tue, 23 Nov 2004 13:16:53 -0800 User-Agent: KMail/1.6.2 References: <1101168686.3370.210.camel@work.gusalmighty.com> <20041122163244.M31380@knight.ixsystems.net> <1101172829.15634.5.camel@work.gusalmighty.com> In-Reply-To: <1101172829.15634.5.camel@work.gusalmighty.com> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="big5" Content-Transfer-Encoding: 7bit Message-Id: <200411231316.53793.alex@taskforce-1.com> Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 21:14:32 -0000 On November 22, 2004 05:20 pm, Justin Hopper wrote: > Is there no appliance that allows for the details of the hardware to be > hidden from the OS and instead present to the OS a unified architecture, > like it's just one machine, but with the ability to add more nodes to > expand CPU, RAM, and disk? I guess this was my misunderstanding, as > this is what I assumed the blade systems did. I assume it would be > incredibly tricky to manage dynamically configurable hardware in the > operating system, but I also assumed that somebody had pulled it off, > but maybe not? There is. It's called SSI or single system image. Basically it provides you with single root, init and process space. Currently I know of two open source products that implement this ( OpenSSI and openMosix ). Unfortantely they are both targeted toward linux. openMosix seems to be geared toward computational aspects ( HPC ), while OpenSSI project is trying to unify various cluster factions and provide a "one size fits all" solution. There are some other papers on FreeBSD clusters that people have designed, my favourite is the one on a very nice general computing cluster published by Brooks Davis ( Aerospace Corporation ), look here: http://people.freebsd.org/~brooks/papers/bsdcon2003/fbsdcluster.pdf There is also some information on the grid computing available here: http://people.freebsd.org/~brooks/pubs/usebsd2004/fbsdgrids.pdf Just something on the side, Manex Visual Effects actually used a 32 node FreeBSD cluster as the core rendering farm to make some of the special effects for the "Matrix" movie. You can read the story if you haven't already here: http://www.freebsd.org/news/press-rel-1.html Cheers. -- Alex Pavlovic TF-1 From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 22:45:41 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5557D16A4CE for ; Tue, 23 Nov 2004 22:45:41 +0000 (GMT) Received: from bsdhosting.net (bsdhosting.net [65.39.221.113]) by mx1.FreeBSD.org (Postfix) with SMTP id 8801A43D5D for ; Tue, 23 Nov 2004 22:45:40 +0000 (GMT) (envelope-from jhopper@bsdhosting.net) Received: (qmail 64682 invoked from network); 23 Nov 2004 22:43:48 -0000 Received: from unknown (HELO ?192.168.1.2?) (jhopper@bsdhosting.net@65.39.221.113) by bsdhosting.net with SMTP; 23 Nov 2004 22:43:48 -0000 From: Justin Hopper To: freebsd-cluster@freebsd.org In-Reply-To: <200411231316.53793.alex@taskforce-1.com> References: <1101168686.3370.210.camel@work.gusalmighty.com> <20041122163244.M31380@knight.ixsystems.net> <1101172829.15634.5.camel@work.gusalmighty.com> <200411231316.53793.alex@taskforce-1.com> Content-Type: text/plain Message-Id: <1101249938.15634.87.camel@work.gusalmighty.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Tue, 23 Nov 2004 14:45:39 -0800 Content-Transfer-Encoding: 7bit Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 22:45:41 -0000 On Tue, 2004-11-23 at 13:16, Alex Pavlovic wrote: > On November 22, 2004 05:20 pm, Justin Hopper wrote: > > Is there no appliance that allows for the details of the hardware to be > > hidden from the OS and instead present to the OS a unified architecture, > > like it's just one machine, but with the ability to add more nodes to > > expand CPU, RAM, and disk? I guess this was my misunderstanding, as > > this is what I assumed the blade systems did. I assume it would be > > incredibly tricky to manage dynamically configurable hardware in the > > operating system, but I also assumed that somebody had pulled it off, > > but maybe not? > > There is. It's called SSI or single system image. Basically it provides you > with single root, init and process space. Currently I know of two open > source products that implement this ( OpenSSI and openMosix ). Unfortantely > they are both targeted toward linux. openMosix seems to be geared toward > computational aspects ( HPC ), while OpenSSI project is trying to unify > various cluster factions and provide a "one size fits all" solution. > > There are some other papers on FreeBSD clusters that people have designed, > my favourite is the one on a very nice general computing cluster published by > Brooks Davis ( Aerospace Corporation ), look here: > http://people.freebsd.org/~brooks/papers/bsdcon2003/fbsdcluster.pdf > There is also some information on the grid computing available here: > http://people.freebsd.org/~brooks/pubs/usebsd2004/fbsdgrids.pdf > > Just something on the side, Manex Visual Effects actually used a 32 node > FreeBSD cluster as the core rendering farm to make some of the special effects > for the "Matrix" movie. You can read the story if you haven't already here: > http://www.freebsd.org/news/press-rel-1.html Thank you, this was much more of what I was looking for. Very interesting articles, especially the one on the Aerospace Corporation. At least I have the answer that I was looking for, that the blade systems are not at all what I was thinking they were. I wonder what the Aerospace Corporation is doing now with the cluster? The paper had mentioned that they might look into the Opteron processor and amd64 / freebsd 5.x. It would be interesting to know what they were able to do with that technology and how their efforts on committing some of their research back to FreeBSD is going. -- Justin Hopper UNIX Systems Engineer BSDHosting.net Hosting Division of Digital Oasys Inc. http://www.bsdhosting.net From owner-freebsd-cluster@FreeBSD.ORG Tue Nov 23 23:01:31 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A506B16A4CE for ; Tue, 23 Nov 2004 23:01:31 +0000 (GMT) Received: from knight.ixsystems.net (afg.ixsystems.net [206.40.55.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 706E143D2F for ; Tue, 23 Nov 2004 23:01:31 +0000 (GMT) (envelope-from matto@knight.ixsystems.net) Received: from knight.ixsystems.net (localhost [127.0.0.1]) by knight.ixsystems.net (8.12.10/8.11.6) with ESMTP id iANMerZU067894; Tue, 23 Nov 2004 14:40:54 -0800 (PST) (envelope-from matto@knight.ixsystems.net) Received: (from matto@localhost) by knight.ixsystems.net (8.12.10/8.12.9/Submit) id iANMerXk067893; Tue, 23 Nov 2004 14:40:53 -0800 (PST) (envelope-from matto) Date: Tue, 23 Nov 2004 14:40:53 -0800 From: Matt Olander To: Justin Hopper Message-ID: <20041123144053.S31380@knight.ixsystems.net> References: <1101168686.3370.210.camel@work.gusalmighty.com> <20041122163244.M31380@knight.ixsystems.net> <1101172829.15634.5.camel@work.gusalmighty.com> <200411231316.53793.alex@taskforce-1.com> <1101249938.15634.87.camel@work.gusalmighty.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <1101249938.15634.87.camel@work.gusalmighty.com>; from jhopper@bsdhosting.net on Tue, Nov 23, 2004 at 02:45:39PM -0800 cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2004 23:01:31 -0000 On Tue, Nov 23, 2004 at 02:45:39PM -0800, Justin Hopper wrote: > Thank you, this was much more of what I was looking for. Very > interesting articles, especially the one on the Aerospace Corporation. > > At least I have the answer that I was looking for, that the blade > systems are not at all what I was thinking they were. > > I wonder what the Aerospace Corporation is doing now with the cluster? > The paper had mentioned that they might look into the Opteron processor > and amd64 / freebsd 5.x. It would be interesting to know what they were > able to do with that technology and how their efforts on committing some > of their research back to FreeBSD is going. hi Justin, we just had Brooks down to assist us in presenting FreeBSD clustering to a few opensource groups last week in the SF Bay Area. he did a somewhat updated version of his presentation that he gave at BSDCon. Aerospace uses their cluster for parallel processing to do things such as simulations. This type of cluster wouldn't help you unless you had software designed to take advantage of parallel processing via submitting jobs to the cluster. it isn't designed for application fail-over and instead uses LAM/MPI to pass messages across the cluster. there should be a short video segment of Brooks and I talking about a cluster like this on TechTV at this press release: http://www.offmyserver.com/cgi-bin/store/news/techtv_121803.html I didn't mention OpenMosix since it's a linux kernel implementation and doesn't work with FreeBSD :/ cheers, -matt -- Matt Olander (408)943-4100 Phone (408)943-4101 Fax www.offmyserver.com -- "Those who don't read have no advantage over those who can't" -Mark Twain From owner-freebsd-cluster@FreeBSD.ORG Wed Nov 24 07:22:57 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6038216A4CF for ; Wed, 24 Nov 2004 07:22:57 +0000 (GMT) Received: from mail.uk.psi.com (mail.uk.psi.com [154.8.2.142]) by mx1.FreeBSD.org (Postfix) with ESMTP id 911D543D49 for ; Wed, 24 Nov 2004 07:22:56 +0000 (GMT) (envelope-from alan.barrow@psinet.telstra.co.uk) Received: from ip10.vpn.uk.psi.com ([154.8.4.10] helo=psinet.telstra.co.uk) by mail.uk.psi.com with esmtp (Exim 4.41) id 1CWrUG-000081-II; Wed, 24 Nov 2004 07:22:49 +0000 Message-ID: <41A436A7.9080704@psinet.telstra.co.uk> Date: Wed, 24 Nov 2004 07:22:15 +0000 From: alan barrow Organization: PSINet a telstra company User-Agent: Mozilla Thunderbird 0.5 (Macintosh/20040208) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Eric Anderson References: <200411230544.iAN5iUUQ072091@realtime.exit.com> <41A3616F.20502@centtech.com> In-Reply-To: <41A3616F.20502@centtech.com> Content-Type: multipart/mixed; boundary="------------050806020803060305070200" X-Content-Filtered-By: Mailman/MimeDel 2.1.1 cc: freebsd-cluster@freebsd.org Subject: Re: Clustering options X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: alan.barrow@psinet.telstra.co.uk List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Nov 2004 07:22:57 -0000 This is a multi-part message in MIME format. --------------050806020803060305070200 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Set up a collection :) We are short of these tools, and if you need it tested in real life scenarios, you have my support. yours a.r.b. Eric Anderson wrote: > Frank Mayhar wrote: > >> Well, there's software that does that (or something like it), but not >> in BSD-land, at >> least not yet. Although I agitated hard to port what is now OpenSSI >> (see >> http://www.openssi.org/) to FreeBSD back in 2000, they ported it to >> Linux instead, >> sigh. I would really love it if someone would actually pay me to do >> the port to >> FreeBSD (or, better, to reimplement it). But that's pretty unlikely. > > > Maybe you should take up a collection like others have done. I bet if > you are a qualified coder for the project, you could raise enough to > do it. There are a lot of people looking for this. > > I'm interested in clustered filesystems to be used for > high-availability scalable NFS servers. > > Eric > > --------------050806020803060305070200--