From owner-freebsd-fs@FreeBSD.ORG Fri Feb 19 23:41:42 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A30D106568D; Fri, 19 Feb 2010 23:41:42 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-px0-f180.google.com (mail-px0-f180.google.com [209.85.216.180]) by mx1.freebsd.org (Postfix) with ESMTP id 5ABCA8FC14; Fri, 19 Feb 2010 23:41:42 +0000 (UTC) Received: by pxi10 with SMTP id 10so401208pxi.13 for ; Fri, 19 Feb 2010 15:41:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=9XTAyw02DJicC3DXxRSMjkkuMMFqhla6sPbwi4OYaKg=; b=ZoAUz8QkxpaqFps+z33KNCKMtez2s6sROz1lYZ9V1YQcoiV41PhOxecgwKlCcviW4d XFq5WyUFvVari+0FeoJ19IscaGSUPvJ6q3GMnvwBAiayKjjZkGHmQXTdnqcSc621iJ0z CjbgWP88CRyoduM3NMUnMowN0O/srTiY+I624= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=MMNIioJFm+TAAfOg7bG3sl3qkiFNiX7iFSoMp38TlUnHWHoedkJnvrYSp38w18xYb2 HgNsyR2kaTWEnACVbP2fknkDfliJTWqsvt1tIep0Uy+iKlakiv3JxBNnXaiD/0MTbhcJ Nsrs3Z5F7xIf+NCchrWCsUaE1ge+tP9KrBrRU= MIME-Version: 1.0 Received: by 10.142.249.16 with SMTP id w16mr3142851wfh.346.1266622901949; Fri, 19 Feb 2010 15:41:41 -0800 (PST) In-Reply-To: <20100219233744.GG1617@garage.freebsd.pl> References: <20100219200725.GA1617@garage.freebsd.pl> <7d6fde3d1002191511h4caac149tf39dcc37cf750afe@mail.gmail.com> <20100219233744.GG1617@garage.freebsd.pl> Date: Fri, 19 Feb 2010 15:41:41 -0800 Message-ID: <7d6fde3d1002191541k51ab8526ub18a7c05484112f5@mail.gmail.com> From: Garrett Cooper To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: HAST (Highly Available Storage) now in HEAD. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 23:41:42 -0000 On Fri, Feb 19, 2010 at 3:37 PM, Pawel Jakub Dawidek wrot= e: > On Fri, Feb 19, 2010 at 03:11:44PM -0800, Garrett Cooper wrote: >> =A0 =A0 Very cool stuff. How many nodes max are you targeting for this >> service [...] > > Currently HAST is intended for use only with High Availability clusters, > not for performance clusters and is limited to exactly two nodes - one > primary node, which has access to shared storage and one secondary node, > which just receives updates from primary. User's applications should > only work on primary node. I was looking forward a bit more than what's setup today TBH, but that's ok= ... >> [...] and what are some of the performance numbers for syncing >> across the network (say with a 1GigE or 10GigE connection)? > > HAST should be able to saturate 1GigE link if you have fast enough > storage. I've patches in the works to save data copying between userland > and kernel. Currently, eg. write I/O request comes from the kernel, it > is copied to hastd userland daemon, hastd copies it back to the kernel > when writing to local component and then copies it again to the kernel > when sending over network to secondary node. In other words a lot of > copying. I prototyped a model where data is not copied at all between > userland and kernel, but it needs a bit more work. Ouch. Lots of CoW / copyout...? Is the data being checksummed / verified somewhere to ensure integrity at all (I would think so / hope so...)? Cheers! -Garrett