From owner-freebsd-cluster Mon Jun 24 1:45:39 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from gate.nentec.de (gate2.nentec.de [194.25.215.66]) by hub.freebsd.org (Postfix) with ESMTP id 0A3D937BAFD for ; Mon, 24 Jun 2002 01:43:27 -0700 (PDT) Received: from nenny.nentec.de (root@nenny.nentec.de [153.92.64.1]) by gate.nentec.de (8.11.3/8.9.3) with ESMTP id g5O8hPA01012; Mon, 24 Jun 2002 10:43:25 +0200 Received: from nentec.de (andromeda.nentec.de [153.92.64.34]) by nenny.nentec.de (8.11.3/8.11.3/SuSE Linux 8.11.1-0.5) with ESMTP id g5O8hDZ28087; Mon, 24 Jun 2002 10:43:14 +0200 Message-ID: <3D16DBA1.5090703@nentec.de> Date: Mon, 24 Jun 2002 10:43:13 +0200 From: Andy Sporner User-Agent: Mozilla/5.0 (X11; U; Linux i686; de-AT; rv:0.9.8) Gecko/20020204 X-Accept-Language: de-at, de, en, en-us MIME-Version: 1.0 To: Derek Barrett Cc: freebsd-cluster@FreeBSD.ORG Subject: Re: Another long post sorry! References: <20020621191111.18718.qmail@graffiti.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS-perl11-milter (http://amavis.org/) Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Derek Barrett wrote: >Andy, I like your style, and I agree with most of what you >have said. You seem determined to make a difference, > More out of sheer frustration. This is something I *can* change. > and that's what is needed on this project, since as you say, getting the approval of the higher ups just isn't possible. One thing I've learned quickly in my >young life is sometimes it's better not to ask, and just do it. Seems >like that's where you are going on this one. And what boggles me is that >this is a feature that alot of people would really kill for. I am >so surprised there is nothing in place to date! > So am I. > >I've been hoping to help out on freebsd for some time now, >maybe this can be a good project. I'm especially interested in the >documentation side. > Good, then I will pass you the documentation tommorrow (25.06) It is skimpy and perhaps somewhat out of date. These are the reasons that this never went anywhere because of the sheer magnitude of work and I couldn't see putting something out that was incomplete. Andy To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Mon Jun 24 2:26:25 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from postoffice.aims.com.au (eth0.lnk.aims.com.au [203.31.73.253]) by hub.freebsd.org (Postfix) with ESMTP id 3A03637B403 for ; Mon, 24 Jun 2002 02:26:17 -0700 (PDT) Received: from postoffice.aims.com.au (nts-ts1.aims.private [192.168.10.2]) by postoffice.aims.com.au with ESMTP id g5O9QGm41318 for ; Mon, 24 Jun 2002 19:26:16 +1000 (EST) (envelope-from chris@aims.com.au) Received: from ntsts1 by aims.com.au with SMTP (MDaemon.v3.5.3.R) for ; Mon, 24 Jun 2002 19:26:13 +1000 Reply-To: From: "Chris Knight" To: Cc: Subject: Update to Andy's Clustering Code Date: Mon, 24 Jun 2002 19:26:12 +1000 Message-ID: <001a01c21b61$2f0f8880$020aa8c0@aims.private> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <3D16DD43.3050209@nentec.de> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Importance: Normal X-Return-Path: chris@aims.com.au X-MDaemon-Deliver-To: freebsd-cluster@freebsd.org Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Howdy, OK, this is now available at http://www.aims.com.au/chris/cluster/. Didn't get a chance to play with it on the weekend - too busy updating all the Apache installations we have :-( Regards, Chris Knight Systems Administrator AIMS Independent Computer Professionals Tel: +61 3 6334 6664 Fax: +61 3 6331 7032 Mob: +61 419 528 795 Web: http://www.aims.com.au > -----Original Message----- > From: Andy Sporner [mailto:sporner@nentec.de] > Sent: Monday, 24 June 2002 18:50 > To: chris@aims.com.au > Subject: Re: Potential Code Host (Here is the semi-final release) > > > Hey Chris, > > I fixed a lot of bugs and accomplished some reorganization to better > benefit the build process. Attached you will find the tar file. > > There are no-known errattas for this one... > > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Mon Jun 24 12: 9:32 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from mg.hk5.outblaze.com (202-77-181-23.outblaze.com [202.77.181.23]) by hub.freebsd.org (Postfix) with ESMTP id D1EE337B400 for ; Mon, 24 Jun 2002 12:09:26 -0700 (PDT) Received: from ws2.hk5.outblaze.com (202-77-181-84.outblaze.com [202.77.181.84]) by mg.hk5.outblaze.com (8.11.2/8.11.2) with SMTP id g5OJ9I609337 for ; Mon, 24 Jun 2002 19:09:19 GMT Received: (qmail 6985 invoked by uid 1001); 24 Jun 2002 19:09:18 -0000 Message-ID: <20020624190918.6984.qmail@graffiti.net> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit Mime-Version: 1.0 X-Mailer: MIME-tools 4.104 (Entity 4.117) Received: from [66.51.217.108] by ws2.hk5.outblaze.com with http for derekbarrett@graffiti.net; Tue, 25 Jun 2002 03:09:18 +0800 From: derekbarrett@graffiti.net To: Andy Sporner Cc: freebsd-cluster@FreeBSD.ORG Date: Tue, 25 Jun 2002 03:09:18 +0800 Subject: Re: Another long post sorry! X-Originating-Ip: 66.51.217.108 X-Originating-Server: ws2.hk5.outblaze.com Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Sounds good Andy, go ahead and throw that documentation my way! ----- Original Message ----- From: Andy Sporner Date: Mon, 24 Jun 2002 10:43:13 +0200 To: Derek Barrett Subject: Re: Another long post sorry! > Derek Barrett wrote: > > >Andy, I like your style, and I agree with most of what you > >have said. You seem determined to make a difference, > > > More out of sheer frustration. This is something I *can* change. > > > and that's what is needed on this project, since as you say, getting the approval of the higher ups just isn't possible. One thing I've learned quickly in my > >young life is sometimes it's better not to ask, and just do it. Seems > >like that's where you are going on this one. And what boggles me is that > >this is a feature that alot of people would really kill for. I am > >so surprised there is nothing in place to date! > > > So am I. > > > > >I've been hoping to help out on freebsd for some time now, > >maybe this can be a good project. I'm especially interested in the > >documentation side. > > > > Good, then I will pass you the documentation tommorrow (25.06) It is skimpy > and perhaps somewhat out of date. > > These are the reasons that this never went anywhere because of the sheer > magnitude > of work and I couldn't see putting something out that was incomplete. > > Andy > > > > -- _______________________________________________ Get your free email from http://www.graffiti.net Powered by Outblaze To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Mon Jun 24 14:41:22 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from testdcc.outblaze.com (202-77-223-23.outblaze.com [202.77.223.23]) by hub.freebsd.org (Postfix) with ESMTP id 7BCA237B40D for ; Mon, 24 Jun 2002 14:41:04 -0700 (PDT) Received: from ws3.hk5.outblaze.com (202-77-181-90.outblaze.com [202.77.181.90]) by testdcc.outblaze.com (8.12.2/8.12.2) with SMTP id g5OLev04025135 for ; Mon, 24 Jun 2002 21:40:57 GMT Received: (qmail 32042 invoked by uid 1001); 24 Jun 2002 21:40:57 -0000 Message-ID: <20020624214057.32041.qmail@graffiti.net> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Mailer: MIME-tools 5.41 (Entity 5.404) Received: from [66.51.217.108] by ws3.hk5.outblaze.com with http for derekbarrett@graffiti.net; Tue, 25 Jun 2002 05:40:57 +0800 From: derekbarrett@graffiti.net To: Cc: Date: Tue, 25 Jun 2002 05:40:57 +0800 Subject: FreeBSD forums has offered help X-Originating-Ip: 66.51.217.108 X-Originating-Server: ws3.hk5.outblaze.com X-DCC-Outblaze-Metrics: testdcc.outblaze.com 100; Body=1 Fuz1=1 Fuz2=1 Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi Andy, The admin over at http://www.freebsdforums.org/ has offered help if you need it in the future, in terms of offering a mirror resource. His server is over in the Rocky Mountains region of USA. I posted a sabre rattling thread over there in the bulletin board, which will hopefully generate some interest in this project. Some people didn't like what I had to say (I was abrasive in making my point), but others agreed and checked out the page. Andy, you should read that thread and if there is anything you would like me to say differently, (or not make posts like that in the future) please let me know, I figured it was the best way to draw attention. I post as Ted over there. I usually get my technical emails to a that email address, and keep the personal ones on this one. http://www.freebsdforums.org/forums/showthread.php?s=&threadid=1447 Thanks, Derek aka Ted Striker -- _______________________________________________ Get your free email from http://www.graffiti.net Powered by Outblaze To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Tue Jun 25 4:39:47 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from gate.nentec.de (gate2.nentec.de [194.25.215.66]) by hub.freebsd.org (Postfix) with ESMTP id C946D37B401 for ; Tue, 25 Jun 2002 04:39:39 -0700 (PDT) Received: from nenny.nentec.de (root@nenny.nentec.de [153.92.64.1]) by gate.nentec.de (8.11.3/8.9.3) with ESMTP id g5PBdaA17857 for ; Tue, 25 Jun 2002 13:39:36 +0200 Received: from nentec.de (andromeda.nentec.de [153.92.64.34]) by nenny.nentec.de (8.11.3/8.11.3/SuSE Linux 8.11.1-0.5) with ESMTP id g5PBdPZ04560 for ; Tue, 25 Jun 2002 13:39:26 +0200 Message-ID: <3D18566D.9000207@nentec.de> Date: Tue, 25 Jun 2002 13:39:25 +0200 From: Andy Sporner User-Agent: Mozilla/5.0 (X11; U; Linux i686; de-AT; rv:0.9.8) Gecko/20020204 X-Accept-Language: de-at, de, en, en-us MIME-Version: 1.0 To: freebsd-cluster Subject: Early documentation- Content-Type: multipart/mixed; boundary="------------080804050703040804070006" X-Virus-Scanned: by AMaViS-perl11-milter (http://amavis.org/) Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. --------------080804050703040804070006 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Here is some very poor documentation for those early adoptors among us. Derek is going to be working on better documentation, so this is only interrim... Andy --------------080804050703040804070006 Content-Type: text/plain; name="docset.txt" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="docset.txt" Installing basic cluster software: 1. Obtain and install the CSE patch if your system is FreeBSD. 2. Download the latest cluster source code. Place in current directory 3. unzip the archive with the 'gunzip' command. 4. Un-TAR the archive with 'tar xvf phase1-???-src.tar' 5. Change directories to phase1. 6. Start the build of the software with 'make'. 7. The resultant software is located in /tmp/buildpkg. Change directories there. 8. Start the install process with ./inst.cluster During the build process, a tar file is created in ~phase1/binaries. For machines with the same machine architecture (and OS version) this file be used instead of going through a complete build process on each machine. Un-TAR this file and start from step 8 above. It is up to the system administrator to integrate the startup of the cluster software into the system startup. Installing CSE Patch: For best results, the source code should be available for libkvm, ps, and top. 1. Unpack the patch file. 2. Enter the directory and type 'make'. During this process, various source files are modified within 'ps'. It will be necessary to rebuild libkvm, top and ps. Note that PS now displays the app id during it's default output in the second column. II CONFIGURATION: Once the cluster software is loaded, start the daemon by the following command: clusterd This will cause the load of the cluster modules and start all other subtasts. Before this it might be necesarry to tune the cluster daemon. This is done by modifying the file '/usr/local/cluster/etc/cluster.conf'. There are various tunable options there. Now the configuration of the cluster can begin. In order to do this, start the cluster configuration GUI. This is done by invoking 'cl_admin'. NOTE: it is necesary that 'clusterd' is running on the local node where cl_admin is running. Two areas are present, 'resources' and 'nodes'. First create the names of all of the nodes of the cluster. This is done by right-clicking on the heading 'nodes' and selecting 'NEW'. The name should either match the hostname of the machine or the override name specified in 'cluster.conf' for that machine. Then for each node under the "LLI" heading, right click to add each network interface IP address that specifies the new node (Really this *MUST* be an IP addresss--not a dns name). When all have been added, the resources need to be added. Right click on the 'resources' heading and select new. Enter a name for the resource and it's weight (that is how much of the machine will it utilize). The weight is used to schedule failover of applications. The monitor will never schedule applications where their weight cannot be allocated. It a resource becomes too large, it can be rescheduled to another node. Autostart allows a resource to start right away when the cluster starts initially. The name of the resource should be a legal filename since it will have a corresponding script in /usr/local/cluster/etc/rc.d. The script will recieve two possible parameters: 'start' and 'stop', which cause the resource to be started or stopped, respectively. For IP address failover configure a 'zero' weight and for the start option the ifconfig option to bring up the address and for the stop option the command to bring down the interface. For other things model the script like an standard 'rc' script. There is an example script in /usr/local/cluster/lib directory. To add nodes to the resources, drag either the resource over the node, or the node over the resource. Control of any of the objects on the configuration screen can be done with a right-click. To start the cluster software into active mode: (IE: this is what should be in the RC script) type 'cluster start'. The current running status can be returned by 'cluster status'. To stop the cluster, enter 'cluster stop'. --------------080804050703040804070006-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Tue Jun 25 10:54:35 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from babylon.chem.psu.edu (babylon.chem.psu.edu [146.186.189.194]) by hub.freebsd.org (Postfix) with ESMTP id 22A7E37B403 for ; Tue, 25 Jun 2002 10:54:27 -0700 (PDT) Received: from mentat.chem.psu.edu [128.118.172.82] by babylon.chem.psu.edu with ESMTP (SMTPD32-7.10) id AE41BD031C; Tue, 25 Jun 2002 13:54:09 -0400 Date: Tue, 25 Jun 2002 13:54:16 -0400 Mime-Version: 1.0 (Apple Message framework v482) Content-Type: text/plain; charset=US-ASCII; format=flowed Subject: CPU-intensive Kernel Optimizations From: Lance M.Westerhoff To: freebsd-cluster@FreeBSD.ORG Content-Transfer-Encoding: 7bit Message-Id: <91142972-8864-11D6-AD49-00039357F10C@psu.edu> X-Mailer: Apple Mail (2.482) Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello All- Forgive me as I am relatively new to freeBSD. This may not be the right forum to discuss this issue, so please feel free to point me in the right direction. First, a little background (and perhaps an introduction). I am a grad student and the primary sys admin in the Merz Research Group here at PennState University (http://merz.chem.psu.edu/). Up until recently, we had only really used IRIX-based SGIs to run our high performance, computational biochemistry code. In the past few months, we have been researching and buying new equipment to take the place of our aging SGIs. We ended up buying 20 dual processor AMDs (plus a 2TB RAID). Now that I am in the midst of preparing and clustering these new boxes together for production use, I am having trouble getting the processor performance that I would expect from these machines. So far these 1.67 GHz Athlons are only moderately faster, per processor, then a 1.2 GHz Athlon I benchmarked before making the purchase. Since as far as I can tell, other then the processor speeds, the rest of the hardware between the two machines is the same (ie: DDR memory, SCSI disk, etc). The only obvious difference I can see is that I am trying to use FreeBSD instead of the Linux that was installed on the benchmark machine. After researching the two OSs for a couple of articles I wrote for a Darwin/Mac OS X site (see http://www.applelust.com/alust/terminal/terminal_index.shtml for more info), I have decided that for this new cluster, I would like to give FreeBSD a try instead of Linux. In fact, I am happy to report that I have switched all of our x86 workstations from Linux to FreeBSD with complete success...now it would be nice for our new AMD cluster to go the same route. Ultimately, though, I need to go with performance. So, without further ado, here is my question. I assume that since this is a clustering list, there are probably at least a few people out there that could give me a few pointers on system/kernel tuning (or at least point me in the right direction). Understand that most of the system tuning I have had to do in the past has been for Linux and SysV derived OSs such as IRIX. Does anyone know of some good sources on the web? So far everything I have found has been for tuning the kernel for web performance. Unfortunately, the kind of performance we need is very different from the kind of performance required for web applications. Does anyone know of any good kernel options I should be compiling in (or out)? etc? Thanks in advance! -Lance To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Tue Jun 25 11:53:24 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from mailout10.sul.t-online.com (mailout10.sul.t-online.com [194.25.134.21]) by hub.freebsd.org (Postfix) with ESMTP id 78EC337B4FC for ; Tue, 25 Jun 2002 11:52:28 -0700 (PDT) Received: from fwd03.sul.t-online.de by mailout10.sul.t-online.com with smtp id 17MvQP-0006gX-09; Tue, 25 Jun 2002 20:52:25 +0200 Received: from Andro-Beta.Leidinger.net (520065502893-0001@[217.83.31.126]) by fmrl03.sul.t-online.com with esmtp id 17MvQf-28tIPYC; Tue, 25 Jun 2002 20:52:41 +0200 Received: from Magelan.Leidinger.net (Magelan [192.168.1.1]) by Andro-Beta.Leidinger.net (8.11.6/8.11.6) with ESMTP id g5PIqGx13351; Tue, 25 Jun 2002 20:52:16 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from Leidinger.net (netchild@localhost [127.0.0.1]) by Magelan.Leidinger.net (8.12.3/8.12.3) with ESMTP id g5PIq9X4002484; Tue, 25 Jun 2002 20:52:13 +0200 (CEST) (envelope-from netchild@Leidinger.net) Message-Id: <200206251852.g5PIq9X4002484@Magelan.Leidinger.net> Date: Tue, 25 Jun 2002 20:52:08 +0200 (CEST) From: Alexander Leidinger Subject: Re: CPU-intensive Kernel Optimizations To: lancew@psu.edu Cc: freebsd-cluster@FreeBSD.ORG In-Reply-To: <91142972-8864-11D6-AD49-00039357F10C@psu.edu> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii X-Sender: 520065502893-0001@t-dialin.net Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 25 Jun, Lance M.Westerhoff wrote: > required for web applications. Does anyone know of any good kernel > options I should be compiling in (or out)? etc? Have a look at /sys/i386/conf/LINT: - only use the "cpu *_CPU" you need (I686_CPU in your case) - have a look at "options CPU_*" /etc/make.conf: - set CFLAGS, COPTFLAGS and CPUTYPE to fit your needs (you better recompile everything after that, don'T forget a "make clean" in your kernel directory), e.g.: * CPUTYPE=i686 * COPTFLAGS= -Os -pipe -mcpu=i686 * CFLAGS= -Os -pipe -mcpu=i686 And perhaps you want to patch the kernel, see http://www.leidinger.net/FreeBSD/misc-patches/kern_exec.c.patch And then there is ths icc port, Intels C compiler for linux. It is able to produce native FreeBSD executables. You can't compile the kernel (yet) with it, but it speeds up your app upto 20%. Bye, Alexander. -- Give a man a fish and you feed him for a day; teach him to use the Net and he won't bother you for weeks. http://www.Leidinger.net Alexander @ Leidinger.net GPG fingerprint = C518 BC70 E67F 143F BE91 3365 79E2 9C60 B006 3FE7 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Tue Jun 25 21:33: 4 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from mg.hk5.outblaze.com (202-77-181-23.outblaze.com [202.77.181.23]) by hub.freebsd.org (Postfix) with ESMTP id 868FD37B430 for ; Tue, 25 Jun 2002 21:32:27 -0700 (PDT) Received: from ws2.hk5.outblaze.com (202-77-181-84.outblaze.com [202.77.181.84]) by mg.hk5.outblaze.com (8.11.2/8.11.2) with SMTP id g5Q4WDD27742 for ; Wed, 26 Jun 2002 04:32:13 GMT Received: (qmail 30517 invoked by uid 1001); 26 Jun 2002 04:32:13 -0000 Message-ID: <20020626043213.30513.qmail@graffiti.net> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit Mime-Version: 1.0 X-Mailer: MIME-tools 4.104 (Entity 4.117) Received: from [66.51.217.108] by ws2.hk5.outblaze.com with http for derekbarrett@graffiti.net; Wed, 26 Jun 2002 12:32:13 +0800 From: derekbarrett@graffiti.net To: Andy Sporner , freebsd-cluster Date: Wed, 26 Jun 2002 12:32:13 +0800 Subject: Re: Early documentation- X-Originating-Ip: 66.51.217.108 X-Originating-Server: ws2.hk5.outblaze.com Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This documentation seems pretty strong at this point. Here are some areas where I think I can contribute: 1. Overall description of the cluster architecture, a general overview of how its mechanisms work 2. A link or brief instructions on how to make sure source code is available for libkvm, ps, and top. 3. Examples of an rc startup script 4. HTML formatting of the instructions, for public posting (with the maroon and yellow FreeBSD "style") I'm sure I will have more to add after running the installation, but at this point it seems to have all the major elements needed. Derek ----- Original Message ----- From: Andy Sporner Date: Tue, 25 Jun 2002 13:39:25 +0200 To: freebsd-cluster Subject: Early documentation- > Here is some very poor documentation for those early adoptors among us. > Derek is going to > be working on better documentation, so this is only interrim... > > Andy > > Content-Type: text/plain; name="docset.txt" > Installing basic cluster software: > > 1. Obtain and install the CSE patch if your system is FreeBSD. > 2. Download the latest cluster source code. > Place in current directory > 3. unzip the archive with the 'gunzip' command. > 4. Un-TAR the archive with 'tar xvf phase1-???-src.tar' > 5. Change directories to phase1. > 6. Start the build of the software with 'make'. > 7. The resultant software is located in /tmp/buildpkg. > Change directories there. > 8. Start the install process with ./inst.cluster > > During the build process, a tar file is created in ~phase1/binaries. For > machines with the same machine architecture (and OS version) this file > be used instead of going through a complete build process on each machine. > Un-TAR this file and start from step 8 above. > > It is up to the system administrator to integrate the startup of the > cluster software into the system startup. > > Installing CSE Patch: > > For best results, the source code should be available for libkvm, ps, and top. > > 1. Unpack the patch file. > 2. Enter the directory and type 'make'. > > During this process, various source files are modified within 'ps'. It will > be necessary to rebuild libkvm, top and ps. Note that PS now displays the > app id during it's default output in the second column. > > II CONFIGURATION: > > Once the cluster software is loaded, start the daemon by the following > command: > > clusterd > > This will cause the load of the cluster modules and start all other > subtasts. > > Before this it might be necesarry to tune the cluster daemon. This is > done by modifying the file '/usr/local/cluster/etc/cluster.conf'. There > are various tunable options there. > > Now the configuration of the cluster can begin. In order to do this, > start the cluster configuration GUI. This is done by invoking 'cl_admin'. > NOTE: it is necesary that 'clusterd' is running on the local node where > cl_admin is running. > > Two areas are present, 'resources' and 'nodes'. > > First create the names of all of the nodes of the cluster. This is > done by right-clicking on the heading 'nodes' and selecting 'NEW'. The > name should either match the hostname of the machine or the override > name specified in 'cluster.conf' for that machine. > > Then for each node under the "LLI" heading, right click to add each > network interface IP address that specifies the new node (Really this > *MUST* be an IP addresss--not a dns name). When all have been added, > the resources need to be added. > > Right click on the 'resources' heading and select new. Enter a name > for the resource and it's weight (that is how much of the machine > will it utilize). The weight is used to schedule failover of applications. > The monitor will never schedule applications where their weight cannot > be allocated. It a resource becomes too large, it can be rescheduled > to another node. Autostart allows a resource to start right away when > the cluster starts initially. > > The name of the resource should be a legal filename since it will > have a corresponding script in /usr/local/cluster/etc/rc.d. The script > will recieve two possible parameters: 'start' and 'stop', which cause > the resource to be started or stopped, respectively. For IP address > failover configure a 'zero' weight and for the start option the ifconfig > option to bring up the address and for the stop option the command > to bring down the interface. For other things model the script like > an standard 'rc' script. There is an example script in > /usr/local/cluster/lib directory. > > To add nodes to the resources, drag either the resource over the node, > or the node over the resource. > > Control of any of the objects on the configuration screen can be done > with a right-click. > > To start the cluster software into active mode: (IE: this is what should > be in the RC script) type 'cluster start'. The current running status > can be returned by 'cluster status'. To stop the cluster, enter > 'cluster stop'. > > > > > > > > > > > > > -- _______________________________________________ Get your free email from http://www.graffiti.net Powered by Outblaze To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Thu Jun 27 4:30:40 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from gate.nentec.de (gate2.nentec.de [194.25.215.66]) by hub.freebsd.org (Postfix) with ESMTP id BCADE37B400 for ; Thu, 27 Jun 2002 04:30:33 -0700 (PDT) Received: from nenny.nentec.de (root@nenny.nentec.de [153.92.64.1]) by gate.nentec.de (8.11.3/8.9.3) with ESMTP id g5RBUSA21885; Thu, 27 Jun 2002 13:30:28 +0200 Received: from nentec.de (andromeda.nentec.de [153.92.64.34]) by nenny.nentec.de (8.11.3/8.11.3/SuSE Linux 8.11.1-0.5) with ESMTP id g5RBUFZ24802; Thu, 27 Jun 2002 13:30:16 +0200 Message-ID: <3D1AF747.6030803@nentec.de> Date: Thu, 27 Jun 2002 13:30:15 +0200 From: Andy Sporner User-Agent: Mozilla/5.0 (X11; U; Linux i686; de-AT; rv:0.9.8) Gecko/20020204 X-Accept-Language: de-at, de, en, en-us MIME-Version: 1.0 To: Amar Takhar , freebsd-cluster Subject: Re: host (cvs or otherwise) (about what phase1 means...) References: <20020621210549.GA41195@drunkmonk.net> <3D16DDB3.1010202@nentec.de> <20020626024357.GA79555@drunkmonk.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS-perl11-milter (http://amavis.org/) Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi Amar, I would suggest 'freebsdcluster' as a name--since there has never been much thought about others. The only point is that somebody might (rightly??) complain that 'failover' is only one kind of cluster and not for instance 'beowulf' that is so common in usage. My belief is that if you can make a reliable computing platform, then you can scalability is really only about scheduling. We are making a network switch that does this. So I would suggest taking the newest (212) version and putting it into the CVS in the following format: phase1/ patches/ Where 212 is put into phase1 and patches contains the CSE patch. I hope soon I will have my internet access again where I can directly access such things. We should also decide who has commit access-- which should probably be those who are doing to most to update and maintain the code base. BTW: For those curious, "Phase 1" is failover, Phase 2, will provide NUMA (Non-Uniform Memory Access) like functionality where processes can migrate between nodes. Some aspects are network migratable sockets. I have realized this by an allocation patch (that assures conversational port numbers are unique across the cluster-- It isn't available yet, but hopefully soon) and a feature of this switch device to keep track of movement of the process in the cluster. (This can also be realized by NAT, but not as efficiently). The idea is that unlike NUMA, where the OS is the single point of failure, several instances of an OS provide a solution to this and work cooperatively, sharing memory pages to allows processes to move (sort of like swapping to a remote machine). The advantage is that it should be theoretically possible to construct a cluster that achieves near perpetual availability of an application. (I am not sure what the standard is for calculating available--is it that at least a certain percentage of users can access the application or that it has to at least be available). So if there are 10 nodes making up an application cluster, all load sharing, if one crashses, the processes there die as well as any processes that had any kind of context there, but the other machines go on and those affected can immediately re-connect and be able to get to a live machine that survives, while a Numa type machine would still be booting or the cluster software would be waiting to see if it has in fact died. Andy Amar Takhar wrote: >Well, the machine is up, and working good, does the program have an actual >name?, if so i can get .stanford.edu as the host for the machine.. or >freebsd-cluster.stanford.edu, either way, the host will be used for the mailing >lists, cvs, web etc... > >amar. > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Thu Jun 27 6:17: 0 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from gate.nentec.de (gate2.nentec.de [194.25.215.66]) by hub.freebsd.org (Postfix) with ESMTP id 72F9F37B401 for ; Thu, 27 Jun 2002 06:16:52 -0700 (PDT) Received: from nenny.nentec.de (root@nenny.nentec.de [153.92.64.1]) by gate.nentec.de (8.11.3/8.9.3) with ESMTP id g5RDGoA08255; Thu, 27 Jun 2002 15:16:50 +0200 Received: from nentec.de (andromeda.nentec.de [153.92.64.34]) by nenny.nentec.de (8.11.3/8.11.3/SuSE Linux 8.11.1-0.5) with ESMTP id g5RDGcZ29419; Thu, 27 Jun 2002 15:16:38 +0200 Message-ID: <3D1B1036.3010505@nentec.de> Date: Thu, 27 Jun 2002 15:16:38 +0200 From: Andy Sporner User-Agent: Mozilla/5.0 (X11; U; Linux i686; de-AT; rv:0.9.8) Gecko/20020204 X-Accept-Language: de-at, de, en, en-us MIME-Version: 1.0 To: derekbarrett@graffiti.net Cc: chris@aims.com.au, freebsd-cluster Subject: Re: ERRATA to post for 211 code References: <20020627131103.14441.qmail@graffiti.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS-perl11-milter (http://amavis.org/) Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi, This erratta is correctly for 212 code ;-). The 211 code will not build at all! :-( because of some organizational problems that were resolved in 212. Andy derekbarrett@graffiti.net wrote: >Hi Chris, > >Running 'make' to build this software was giving the error: > >cc -o clusterd cl_support.o cl_cluster.o cl_conf.o cl_module.o -ldl >/usr/libexec/elf/ld: cannot find -ldl >*** Error code 1 > >Stop in ~/phase1/clusterd. >*** Error code 1 > >Stop in ~/phase1. > >To solve this problem: > ># vi ~/phase1/clusterd/Makefile > >Edit this line: > > $(CC) -o clusterd $(OBJS) -ldl > >To read: > > $(CC) -o clusterd $(OBJS) > >Re-rerun make, and the software should build properly. This worked >for me on FreeBSD 4.5 > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Thu Jun 27 14: 7:11 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from postoffice.aims.com.au (eth0.lnk.aims.com.au [203.31.73.253]) by hub.freebsd.org (Postfix) with ESMTP id 4BDF437B401 for ; Thu, 27 Jun 2002 14:07:01 -0700 (PDT) Received: from postoffice.aims.com.au (nts-ts1.aims.private [192.168.10.2]) by postoffice.aims.com.au with ESMTP id g5RL6xm29081 for ; Fri, 28 Jun 2002 07:06:59 +1000 (EST) (envelope-from chris@aims.com.au) Received: from ntsts1 by aims.com.au with SMTP (MDaemon.v3.5.3.R) for ; Fri, 28 Jun 2002 07:06:16 +1000 Reply-To: From: "Chris Knight" To: , Cc: "'freebsd-cluster'" Subject: RE: ERRATA to post for 211 code Date: Fri, 28 Jun 2002 07:06:14 +1000 Message-ID: <018b01c21e1e$79cee6d0$020aa8c0@aims.private> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <3D1B1036.3010505@nentec.de> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Importance: Normal X-Return-Path: chris@aims.com.au X-MDaemon-Deliver-To: freebsd-cluster@FreeBSD.ORG Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Howdy, Thanks. I've put the errata up on the site. Regards, Chris Knight Systems Administrator AIMS Independent Computer Professionals Tel: +61 3 6334 6664 Fax: +61 3 6331 7032 Mob: +61 419 528 795 Web: http://www.aims.com.au > -----Original Message----- > From: Andy Sporner [mailto:sporner@nentec.de] > Sent: Thursday, 27 June 2002 23:17 > To: derekbarrett@graffiti.net > Cc: chris@aims.com.au; freebsd-cluster > Subject: Re: ERRATA to post for 211 code > > > Hi, > > This erratta is correctly for 212 code ;-). The 211 code > will not build > at all! :-( > because of some organizational problems that were resolved in 212. > > Andy > > > derekbarrett@graffiti.net wrote: > > >Hi Chris, > > > >Running 'make' to build this software was giving the error: > > > >cc -o clusterd cl_support.o cl_cluster.o cl_conf.o cl_module.o -ldl > >/usr/libexec/elf/ld: cannot find -ldl > >*** Error code 1 > > > >Stop in ~/phase1/clusterd. > >*** Error code 1 > > > >Stop in ~/phase1. > > > >To solve this problem: > > > ># vi ~/phase1/clusterd/Makefile > > > >Edit this line: > > > > $(CC) -o clusterd $(OBJS) -ldl > > > >To read: > > > > $(CC) -o clusterd $(OBJS) > > > >Re-rerun make, and the software should build properly. This worked > >for me on FreeBSD 4.5 > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Sat Jun 29 16: 5: 5 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 43D9037B400 for ; Sat, 29 Jun 2002 16:05:04 -0700 (PDT) Received: from jolly.drunkmonk.net (jolly.drunkmonk.net [66.37.140.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id BF89B43E06 for ; Sat, 29 Jun 2002 16:05:03 -0700 (PDT) (envelope-from verm@jolly.drunkmonk.net) Received: from jolly.drunkmonk.net (localhost [127.0.0.1]) by jolly.drunkmonk.net (8.12.3/8.12.3) with ESMTP id g5TN53f4008241 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for ; Sat, 29 Jun 2002 17:05:03 -0600 (MDT) (envelope-from verm@jolly.drunkmonk.net) Received: (from verm@localhost) by jolly.drunkmonk.net (8.12.3/8.12.1/Submit) id g5TN53Q0008240 for freebsd-cluster@FreeBSD.ORG; Sat, 29 Jun 2002 17:05:03 -0600 (MDT) Date: Sat, 29 Jun 2002 17:05:02 -0600 From: Amar Takhar To: freebsd-cluster@FreeBSD.ORG Subject: FreeBSD Cluster project. Message-ID: <20020629230502.GA8106@drunkmonk.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hey, i just want to make a quick head-count on people interested in working on ths FreeBSD Cluster project with Andy. This includes people willing to help with the Programming/Doc aspects of the project. Thus far, i only know of Andy and myself. We have a dedicated machine for the project, kindly donated and hosted by stanford.edu. That's about it, just send me an email either here or in private so we can get this all sorted :) Thanks! Amar. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Sat Jun 29 19:20:40 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2AEDE37B400 for ; Sat, 29 Jun 2002 19:20:38 -0700 (PDT) Received: from ws2.hk5.outblaze.com (202-77-181-84.outblaze.com [202.77.181.84]) by mx1.FreeBSD.org (Postfix) with SMTP id 34D3243E26 for ; Sat, 29 Jun 2002 19:20:36 -0700 (PDT) (envelope-from derekbarrett@graffiti.net) Received: (qmail 24991 invoked by uid 1001); 30 Jun 2002 02:20:29 -0000 Message-ID: <20020630022029.24990.qmail@graffiti.net> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit Mime-Version: 1.0 X-Mailer: MIME-tools 4.104 (Entity 4.117) Received: from [66.51.217.108] by ws2.hk5.outblaze.com with http for derekbarrett@graffiti.net; Sun, 30 Jun 2002 10:20:29 +0800 From: derekbarrett@graffiti.net To: Amar Takhar , freebsd-cluster@FreeBSD.ORG Date: Sun, 30 Jun 2002 10:20:29 +0800 Subject: Re: FreeBSD Cluster project. X-Originating-Ip: 66.51.217.108 X-Originating-Server: ws2.hk5.outblaze.com Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Count me in! ----- Original Message ----- From: Amar Takhar Date: Sat, 29 Jun 2002 17:05:02 -0600 To: freebsd-cluster@FreeBSD.ORG Subject: FreeBSD Cluster project. > Hey, i just want to make a quick head-count on people interested in > working on ths FreeBSD Cluster project with Andy. This includes people willing > to help with the Programming/Doc aspects of the project. Thus far, i only know > of Andy and myself. We have a dedicated machine for the project, kindly donated > and hosted by stanford.edu. That's about it, just send me an email either here > or in private so we can get this all sorted :) > > Thanks! > > Amar. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-cluster" in the body of the message > > -- _______________________________________________ Get your free email from http://www.graffiti.net Powered by Outblaze To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message From owner-freebsd-cluster Sat Jun 29 20:13:17 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E425937B400 for ; Sat, 29 Jun 2002 20:13:14 -0700 (PDT) Received: from basit.cc (wireless.cs.twsu.edu [156.26.10.125]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9D23643E0A for ; Sat, 29 Jun 2002 20:13:14 -0700 (PDT) (envelope-from basit@basit.cc) Received: from localhost ([127.0.0.1]) by basit.cc with esmtp (Exim 4.05) id 17OV8d-0007s4-00 for freebsd-cluster@FreeBSD.ORG; Sat, 29 Jun 2002 22:12:35 -0500 Date: Sat, 29 Jun 2002 22:12:35 -0500 (CDT) From: Abdul Basit X-X-Sender: basit@wireless.cs.twsu.edu To: freebsd-cluster@FreeBSD.ORG Subject: Re: FreeBSD Cluster project. In-Reply-To: <20020630022029.24990.qmail@graffiti.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG count me too thanks -basit On Sun, 30 Jun 2002 derekbarrett@graffiti.net wrote: > Count me in! > ----- Original Message ----- > From: Amar Takhar > Date: Sat, 29 Jun 2002 17:05:02 -0600 > To: freebsd-cluster@FreeBSD.ORG > Subject: FreeBSD Cluster project. > > > > Hey, i just want to make a quick head-count on people interested in > > working on ths FreeBSD Cluster project with Andy. This includes people willing > > to help with the Programming/Doc aspects of the project. Thus far, i only know > > of Andy and myself. We have a dedicated machine for the project, kindly donated > > and hosted by stanford.edu. That's about it, just send me an email either here > > or in private so we can get this all sorted :) > > > > Thanks! > > > > Amar. > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-cluster" in the body of the message > > > > > > -- > _______________________________________________ > Get your free email from http://www.graffiti.net > > Powered by Outblaze > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-cluster" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message