From owner-freebsd-xen@FreeBSD.ORG Mon Oct 20 07:19:44 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D7D07F4 for ; Mon, 20 Oct 2014 07:19:44 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BAF236C5 for ; Mon, 20 Oct 2014 07:19:43 +0000 (UTC) Received: from [172.16.2.46] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id CE5D39DFA11; Mon, 20 Oct 2014 09:09:43 +0200 (CEST) Mime-Version: 1.0 (Mac OS X Mail 8.0 \(1990.1\)) Subject: Re: disk loss From: Egoitz Aurrekoetxea In-Reply-To: <000101cfe3f1$91407da0$b3c178e0$@ezwind.net> Date: Mon, 20 Oct 2014 09:09:41 +0200 Message-Id: <65CC3330-E22F-4253-918E-72CA9B004A81@sarenet.es> References: <000001cfe3ca$8d242950$a76c7bf0$@ezwind.net> <5436CF13.4080509@citrix.com> <000101cfe3f1$91407da0$b3c178e0$@ezwind.net> To: Jay West X-Mailer: Apple Mail (2.1990.1) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Oct 2014 07:19:44 -0000 Good morning, I would recommend you using NFS instead of iSCSI. It=E2=80=99s far more = better to handle the connection to disk arrays (the FreeNAS in this = situation) through a mature and stable protocol like NFS and not something manipulating blocks directly. I would advise you to = rely the responsibility of serving the SR to NFS.=20 Best regards, > El 9/10/2014, a las 20:48, Jay West escribi=C3=B3: >=20 > Only the FreeBSD guests > are affected. From owner-freebsd-xen@FreeBSD.ORG Mon Oct 20 12:34:25 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 28AD6250 for ; Mon, 20 Oct 2014 12:34:25 +0000 (UTC) Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EE451DF8 for ; Mon, 20 Oct 2014 12:34:24 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by gateway2.nyi.internal (Postfix) with ESMTP id 4E7CB1A7B for ; Mon, 20 Oct 2014 08:34:17 -0400 (EDT) Received: from web3 ([10.202.2.213]) by compute2.internal (MEProxy); Mon, 20 Oct 2014 08:34:17 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:x-sasl-enc:from:to :mime-version:content-transfer-encoding:content-type:subject :date:in-reply-to:references; s=smtpout; bh=ac1XY/U67vTaLqyceHDp frvUsAg=; b=H3PgW/bbNqLlkMP8JeNHPprb8kUqwjmssdrwYtBu8t0H7efteuMw n8Df07j1yjNT6Xn24vD/ZJ8LY7lcNH49kS/FHMRH4CoMGt/yBQU6Z6jU+WQDszz/ pnayvQLjyygqnYdSnUXpvHEaTpiZboWMMn9TWogfiVJQbUZEXeEdDFA= Received: by web3.nyi.internal (Postfix, from userid 99) id 102651144F9; Mon, 20 Oct 2014 08:34:17 -0400 (EDT) Message-Id: <1413808457.2828604.181041145.4121AB54@webmail.messagingengine.com> X-Sasl-Enc: It2VP+zdJiIchCshNECgC2KsyrClfjA/PbxDn3LY2r3D 1413808457 From: Mark Felder To: freebsd-xen@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-e69fc525 Subject: Re: disk loss Date: Mon, 20 Oct 2014 07:34:17 -0500 In-Reply-To: <65CC3330-E22F-4253-918E-72CA9B004A81@sarenet.es> References: <000001cfe3ca$8d242950$a76c7bf0$@ezwind.net> <5436CF13.4080509@citrix.com> <000101cfe3f1$91407da0$b3c178e0$@ezwind.net> <65CC3330-E22F-4253-918E-72CA9B004A81@sarenet.es> X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Oct 2014 12:34:25 -0000 On Mon, Oct 20, 2014, at 02:09, Egoitz Aurrekoetxea wrote: > Good morning, >=20 > I would recommend you using NFS instead of iSCSI. It=E2=80=99s far more b= etter to > handle the connection to disk arrays (the FreeNAS in this situation) > through a mature and stable protocol like NFS > and not something manipulating blocks directly. I would advise you to > rely the responsibility of serving the SR to NFS.=20 >=20 You can't have redundant paths with NFS (in FreeBSD), though. I'm not so sure everyone would agree that NFS is mature and stable, either :-) My personal experience with building a Xen+FreeBSD cluster concluded that NFS was far too slow and unreliable, and a properly configured iSCSI with multiple paths and proper alignment was extremely fast. From owner-freebsd-xen@FreeBSD.ORG Mon Oct 20 13:54:29 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 44C61B4C; Mon, 20 Oct 2014 13:54:29 +0000 (UTC) Received: from ezwind.net (bobby.ezwind.net [199.188.211.146]) by mx1.freebsd.org (Postfix) with ESMTP id 0BE5999E; Mon, 20 Oct 2014 13:54:28 +0000 (UTC) Received: from jayPC by ezwind.net (MDaemon PRO v9.6.5) with ESMTP id 20-md50000166867.msg; Mon, 20 Oct 2014 08:54:09 -0500 X-Spam-Processed: ezwind.net, Mon, 20 Oct 2014 08:54:09 -0500 (not processed: spam filter heuristic analysis disabled) X-Authenticated-Sender: jwest@ezwind.net X-MDRemoteIP: 97.91.122.42 X-Return-Path: prvs=1370823ff3=jwest@ezwind.net X-Envelope-From: jwest@ezwind.net From: "Jay West" To: "'Mark Felder'" , References: <000001cfe3ca$8d242950$a76c7bf0$@ezwind.net> <5436CF13.4080509@citrix.com> <000101cfe3f1$91407da0$b3c178e0$@ezwind.net> <65CC3330-E22F-4253-918E-72CA9B004A81@sarenet.es> <1413808457.2828604.181041145.4121AB54@webmail.messagingengine.com> In-Reply-To: <1413808457.2828604.181041145.4121AB54@webmail.messagingengine.com> Subject: RE: disk loss Date: Mon, 20 Oct 2014 09:01:30 -0500 Message-ID: <000901cfec6e$5a15f5a0$0e41e0e0$@ezwind.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQLZQaRIeWHWrqmNu5N4SPLKldtZ2QLBV7crAp5Pg98Cig4q9QHtqe+omdec8uA= Content-Language: en-us X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Oct 2014 13:54:29 -0000 Egoitz wrote... > I would recommend you using NFS instead of iSCSI. It=E2=80=99s far = more better=20 > to handle the connection to disk arrays (the FreeNAS in this=20 > situation) through a mature and stable protocol like NFS and not=20 > something manipulating blocks directly. I would advise you to rely the = > responsibility of serving the SR to NFS. To which Mark Felder replied: You can't have redundant paths with NFS (in FreeBSD), though. I'm not so = sure everyone would agree that NFS is mature and stable, either :-) My personal experience with building a Xen+FreeBSD cluster concluded = that NFS was far too slow and unreliable, and a properly configured = iSCSI with multiple paths and proper alignment was extremely fast. --------------- NFS mature & stable (?? Subjective), but more importantly - it's not the = right choice for a SAN from a speed nor technology perspective. Mark, along with probably most of the production infrastructure = implementors - is (subjectively) correct :) That all being said, I'd wager that other than specific use cases (ex. = Shared content for a webserver farm, which on freebsd pretty much HAS to = be NFS because FreeBSD as of yet does not support any cluster aware = filesystems)... most people are using iSCSI for that type of common use = case in a large environment. It'd behoove freebsd to see why there is an = issue (where there is none with Windows or Linux Guests). J From owner-freebsd-xen@FreeBSD.ORG Mon Oct 20 14:05:58 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 03EE6261; Mon, 20 Oct 2014 14:05:58 +0000 (UTC) Received: from ezwind.net (bobby.ezwind.net [199.188.211.146]) by mx1.freebsd.org (Postfix) with ESMTP id BEEC7D04; Mon, 20 Oct 2014 14:05:57 +0000 (UTC) Received: from jayPC by ezwind.net (MDaemon PRO v9.6.5) with ESMTP id 15-md50000166894.msg; Mon, 20 Oct 2014 09:05:43 -0500 X-Spam-Processed: ezwind.net, Mon, 20 Oct 2014 09:05:43 -0500 (not processed: spam filter heuristic analysis disabled) X-Authenticated-Sender: jwest@ezwind.net X-MDRemoteIP: 97.91.122.42 X-Return-Path: prvs=1370823ff3=jwest@ezwind.net X-Envelope-From: jwest@ezwind.net From: "Jay West" To: "'Mark Felder'" , References: <000001cfe3ca$8d242950$a76c7bf0$@ezwind.net> <5436CF13.4080509@citrix.com> <000101cfe3f1$91407da0$b3c178e0$@ezwind.net> <65CC3330-E22F-4253-918E-72CA9B004A81@sarenet.es> <1413808457.2828604.181041145.4121AB54@webmail.messagingengine.com> <000901cfec6e$5a15f5a0$0e41e0e0$@ezwind.net> In-Reply-To: <000901cfec6e$5a15f5a0$0e41e0e0$@ezwind.net> Subject: RE: disk loss Date: Mon, 20 Oct 2014 09:13:04 -0500 Message-ID: <000f01cfec6f$f7ab7e10$e7027a30$@ezwind.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQLZQaRIeWHWrqmNu5N4SPLKldtZ2QLBV7crAp5Pg98Cig4q9QHtqe+oAiJfBOaZxo12UA== Content-Language: en-us X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Oct 2014 14:05:58 -0000 I had written.... >It'd behoove freebsd to see why there is an issue (where there is none = with Windows or Linux Guests). One thing I should clarify... in a Xenserver environment with iSCSI = backed SR's and (mostly) Freebsd guests... In our setup we specifically = set up iSCSI to be the link to the Xenserver Hypervisor pool's SR's. We = do NOT have any of the guest OS's talking iSCSI. So long story short, = freebsd will never see iSCSI in this case - only the pool of hypervisors = does. Again, I'd wager this is the most common type of setup for = Xenserver/iSCSInas/FreeBSDguests. Long story short, if for some reason the iSCSI connection is interrupted = (someone trips over a cable, or the san crashes and reboots, etc.)... = the windows and linux guests recover just fine. FreeBSD on the other = hand, will generally still have the OS disk but any additional disks are = unreadable. Thus, I'd suspect it has something to do with how freebsd = handles disk I/O in a Xenserver environment (and has nothing to do with = iscsi inside FreeBSD). J From owner-freebsd-xen@FreeBSD.ORG Mon Oct 20 15:01:27 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ECC1AA3D for ; Mon, 20 Oct 2014 15:01:27 +0000 (UTC) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B9E1771 for ; Mon, 20 Oct 2014 15:01:26 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.04,756,1406592000"; d="scan'208";a="182961046" Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.3.181.6; Mon, 20 Oct 2014 11:00:46 -0400 Message-ID: <5445239D.60401@citrix.com> Date: Mon, 20 Oct 2014 17:00:45 +0200 From: =?windows-1252?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Jay West , Subject: Re: disk loss References: <000001cfe3ca$8d242950$a76c7bf0$@ezwind.net> <5436CF13.4080509@citrix.com> <000101cfe3f1$91407da0$b3c178e0$@ezwind.net> In-Reply-To: <000101cfe3f1$91407da0$b3c178e0$@ezwind.net> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-DLP: MIA2 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Oct 2014 15:01:28 -0000 El 09/10/14 a les 20.48, Jay West ha escrit: > > Roger wrote.... > ------ >> When this happens, can you log into the affected VM (with the disk > "supposedly" attached) and paste the output of `xenstore-ls -fp > /local/domain/` (replace with the ID of the domain) here. > Will do that later today, no problem. > >> You will need the xen-tools package installed in order to do that. > Yep, xen-tools exists for Freebsd and it's always installed. Ping? Could you look into this, it would probably help figure out if XenServer is even trying to attach the disks to the guest. Roger. From owner-freebsd-xen@FreeBSD.ORG Tue Oct 21 07:24:01 2014 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 66207E37; Tue, 21 Oct 2014 07:24:01 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F26F8E46; Tue, 21 Oct 2014 07:24:00 +0000 (UTC) Received: from [172.16.2.46] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 0D4299DE39C; Tue, 21 Oct 2014 09:23:51 +0200 (CEST) Mime-Version: 1.0 (Mac OS X Mail 8.0 \(1990.1\)) Subject: Re: disk loss From: Egoitz Aurrekoetxea In-Reply-To: <1413808457.2828604.181041145.4121AB54@webmail.messagingengine.com> Date: Tue, 21 Oct 2014 09:23:49 +0200 Message-Id: <0084B089-9F31-43F0-8ED4-77CB45AE424D@sarenet.es> References: <000001cfe3ca$8d242950$a76c7bf0$@ezwind.net> <5436CF13.4080509@citrix.com> <000101cfe3f1$91407da0$b3c178e0$@ezwind.net> <65CC3330-E22F-4253-918E-72CA9B004A81@sarenet.es> <1413808457.2828604.181041145.4121AB54@webmail.messagingengine.com> To: Mark Felder X-Mailer: Apple Mail (2.1990.1) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Oct 2014 07:24:01 -0000 Egoitz Aurrekoetxea Departamento de sistemas 944 209 470 Parque Tecnol=C3=B3gico. Edificio 103 48170 Zamudio (Bizkaia) egoitz@sarenet.es www.sarenet.es Antes de imprimir este correo electr=C3=B3nico piense si es necesario = hacerlo. > El 20/10/2014, a las 14:34, Mark Felder escribi=C3=B3= : >=20 >=20 >=20 > On Mon, Oct 20, 2014, at 02:09, Egoitz Aurrekoetxea wrote: >> Good morning, >>=20 >> I would recommend you using NFS instead of iSCSI. It=E2=80=99s far = more better to >> handle the connection to disk arrays (the FreeNAS in this situation) >> through a mature and stable protocol like NFS >> and not something manipulating blocks directly. I would advise you to >> rely the responsibility of serving the SR to NFS.=20 >>=20 >=20 > You can't have redundant paths with NFS (in FreeBSD), though. I'm not = so > sure everyone would agree that NFS is mature and stable, either :-) Sure you can have redundant paths with a proper configuration with = Spanning tree=E2=80=A6 and everything not touching directly blocks is IMHO always safer=E2=80=A6 because the part you rely the disks = integrity is all in the same place=E2=80=A6 and takes care of committing = changes properly having less probabilities to leave the vdi corrupt=E2=80=A6. Faster??=E2=80=A6 how have you written to disk?? the file system you = have used to which NFS was doing I/O how did it manage the write = commits?. Did=20 you use Jumbo frames?=E2=80=A6 there is pretty relative all that you = have said=E2=80=A6 And IMHO yes, NFS is tested and tried (mainly in some = versions and configs)=20 very extensively and with very high loads=E2=80=A6 apart nowadays 10gbps = switches exist... >=20 > My personal experience with building a Xen+FreeBSD cluster concluded > that NFS was far too slow and unreliable, and a properly configured > iSCSI with multiple paths and proper alignment was extremely fast. > _______________________________________________ > freebsd-xen@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-xen > To unsubscribe, send any mail to "freebsd-xen-unsubscribe@freebsd.org"