From owner-freebsd-cluster@FreeBSD.ORG Fri Dec 16 15:05:04 2011 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79389106575A for ; Fri, 16 Dec 2011 15:05:04 +0000 (UTC) (envelope-from filip@valder.cz) Received: from smtp.vsb.cz (smtp.vsb.cz [IPv6:2001:718:1001:149::155]) by mx1.freebsd.org (Postfix) with ESMTP id 2178C8FC15 for ; Fri, 16 Dec 2011 15:05:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp.vsb.cz (Postfix) with ESMTP id 58BDE1C1F53 for ; Fri, 16 Dec 2011 16:05:00 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at smtp.vsb.cz Received: from smtp.vsb.cz ([127.0.0.1]) by localhost (smtp-ha1.vsb.cz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3UoggU79hjdu for ; Fri, 16 Dec 2011 16:04:57 +0100 (CET) Received: from [IPv6:2001:718:1001:158:224:1dff:fe85:f221] (unknown [IPv6:2001:718:1001:158:224:1dff:fe85:f221]) by smtp.vsb.cz (Postfix) with ESMTPSA id C8FC31C1F12 for ; Fri, 16 Dec 2011 16:04:57 +0100 (CET) Message-ID: <4EEB5E19.5020105@valder.cz> Date: Fri, 16 Dec 2011 16:04:57 +0100 From: Filip Valder User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111003 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-cluster@freebsd.org Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms040201080505040002080807" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: HA storage, diskless booting, clustering X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 15:05:04 -0000 Toto je elektronicky podepsaná zpráva ve formátu MIME. --------------ms040201080505040002080807 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: quoted-printable Hi everyone! It seems that I brought diskless booting back to life in FreeBSD 7.2 and now I'm about to try it in 8.2 together with HAST. It seems that nothing relevant has changed in diskless since 7.2. The official docs also seem outdated, so it's a good playground for my experiments and I'm very happy about my quick and dirty hacks. Or clean...? Who knows? :-) First of all prepare your HAST environment with this guide: http://www.freebsd.org/doc/handbook/disks-hast.html - SSD disks and gigE are fast enough, so don't be afraid of it! Just use attached ifstated.conf instead of the devd.conf in order to avoid split-brain situations. - Configure all that HAST, pfsync, CARP and other stuff to get this "Gemini" system working. When your HA storage is alive, go ahead and: - prepare your PXE environment (incl. DHCP) and tftpboot, like: /hast/pxe/tftpboot/ /hast/pxe/tftpboot/boot /hast/pxe/tftpboot/boot/defaults =2E.. /hast/pxe/tftpboot/boot/firmware =2E.. /hast/pxe/tftpboot/boot/kernel =2E.. /hast/pxe/tftpboot/boot/modules =2E.. /hast/pxe/tftpboot/boot/zfs =2E.. /hast/pxe/tftpboot/boot/loader.conf /hast/pxe/tftpboot/boot/mbr /hast/pxe/tftpboot/boot/boot0sio /hast/pxe/tftpboot/boot/boot2 /hast/pxe/tftpboot/boot/boot0 /hast/pxe/tftpboot/boot/boot1 /hast/pxe/tftpboot/boot/cdboot /hast/pxe/tftpboot/boot/loader.4th /hast/pxe/tftpboot/boot/loader.help /hast/pxe/tftpboot/boot/loader.rc /hast/pxe/tftpboot/boot/support.4th /hast/pxe/tftpboot/boot/gptboot /hast/pxe/tftpboot/boot/boot /hast/pxe/tftpboot/boot/pmbr /hast/pxe/tftpboot/boot/screen.4th /hast/pxe/tftpboot/boot/frames.4th /hast/pxe/tftpboot/boot/device.hints /hast/pxe/tftpboot/boot/loader /hast/pxe/tftpboot/boot/beastie.4th /hast/pxe/tftpboot/boot/pxeboot /hast/pxe/tftpboot/pxeboot -> /hast/pxe/tftpboot/boot/pxeboot - sysinstall your diskless machine into /hast/pxe/diskless-box, and separate /, /usr and /var according to the fstab (below) - configure NFS in order to export that HAST wonder - pay attention that CARP and UDP are not friends. You have to configure rpcbind, nfsd a mountd to listen on the CARP interface instead of a wildcard address. - hack some files in the /hast/pxe/diskless-box/root/etc: Add to rc.conf: background_fsck=3D"NO" dumpdev=3D"NO" netfs_types=3D"nfs4:NFS4 smbfs:SMB portalfs:PORTAL nwfs:NWFS" Comment out theese lines in rc: dlv=3D`/sbin/sysctl -n vfs.nfs.diskless_valid 2> /dev/null` if [ ${dlv:=3D0} -ne 0 -o -f /etc/diskless ]; then sh /etc/rc.initdiskless fi Comment out rc.d/mountcritremote from this line (incl.) until the end of file: =2E /etc/rc.subr fstab: my-hast:/hast/pxe/diskless-box/root / nfs -L,noauto,rw = 0 0 my-hast:/hast/pxe/diskless-box/usr /usr nfs -L,rw 0 = 0 my-hast:/hast/pxe/diskless-box/var /var nfs -L,rw 0 = 0 When it all seems ready, launch your diskless box and see what happens! :-) TODOs: I will try it with 8.2, will introduce a template system, so that diskless-boxes will have / and /usr in common, with separated /etc/rc.conf and /var. If it all works you will get some low- or mid-end clustering solution. Hint: export PATH_FSTAB via /etc/rc could help with individual fstabs according to hostnames e.g. ******** NEWS: I've tried it already. It works great! ******** Some detailed docs should come from me sooner or later. This is just a nightly draft... Cheers! Filip --=20 filip@valder.cz | +420 608 123 966 | http://www.valder.cz/ --------------ms040201080505040002080807--