From owner-freebsd-geom@FreeBSD.ORG Mon Dec 8 00:20:56 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BFB7D1065677 for ; Mon, 8 Dec 2008 00:20:56 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: from rn-out-0910.google.com (rn-out-0910.google.com [64.233.170.187]) by mx1.freebsd.org (Postfix) with ESMTP id 7CE658FC1B for ; Mon, 8 Dec 2008 00:20:56 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: by rn-out-0910.google.com with SMTP id j71so991667rne.12 for ; Sun, 07 Dec 2008 16:20:55 -0800 (PST) Received: by 10.90.83.2 with SMTP id g2mr978365agb.79.1228695655466; Sun, 07 Dec 2008 16:20:55 -0800 (PST) Received: by 10.90.73.15 with HTTP; Sun, 7 Dec 2008 16:20:55 -0800 (PST) Message-ID: Date: Mon, 8 Dec 2008 01:20:55 +0100 From: "=?ISO-8859-1?Q?Marius_N=FCnnerich?=" To: "Poul-Henning Kamp" In-Reply-To: <31114.1228597207@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <31114.1228597207@critter.freebsd.dk> Cc: freebsd-geom@freebsd.org Subject: Re: Trivial(?) reorganization of topology lock in geom_event X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Dec 2008 00:20:56 -0000 On Sat, Dec 6, 2008 at 10:00 PM, Poul-Henning Kamp wrote: > In message , "=?ISO- > 8859-1?Q?Marius_N=FCnnerich?=" writes: >>Hi, >> >>while working on the DTrace probes for geom I noticed that >>g_topology_lock() is called 20 times per second from the g_event >>thread, even though the thread only runs 10 times per second when >>idle. Maybe it is possible to change the locking like in this patch? I >>also changed the position of one unlocking of g_eventlock. > > In theory the timeout is not necessary, it was added as a stopgap > because there were synchronisation issues long time ago. > > Try dropping the timeout and see if you can provoke problems, > if not, kill it. Did so, please take a look at this patch: http://nuenneri.ch/freebsd/geom_tl2.patch I am running a version of this with the DTrace probes included, I hope the patch is complete. I did a few buildkernels and some of the geom regression tests, so far no problems. I changed the position of the loop to better match how it's like in the up and down threads. What do you think of it? Thanks Marius From owner-freebsd-geom@FreeBSD.ORG Mon Dec 8 11:06:56 2008 Return-Path: Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FBC1106567B for ; Mon, 8 Dec 2008 11:06:56 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 1D0088FC17 for ; Mon, 8 Dec 2008 11:06:56 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id mB8B6uDi014262 for ; Mon, 8 Dec 2008 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id mB8B6tnc014258 for freebsd-geom@FreeBSD.org; Mon, 8 Dec 2008 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 8 Dec 2008 11:06:55 GMT Message-Id: <200812081106.mB8B6tnc014258@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-geom@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-geom@FreeBSD.org X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Dec 2008 11:06:56 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/129245 geom [geom] gcache is more suitable for suffix based provid o kern/128398 geom [PATCH] glabel(8): teach geom_label to recognise gpt l f kern/128276 geom [gmirror] machine lock up when gmirror module is used o kern/126902 geom [geom] [geom_label] Kernel panic during install boot o kern/124973 geom [gjournal] [patch] boot order affects geom_journal con o kern/124969 geom gvinum(8): gvinum raid5 plex does not detect missing s o kern/124294 geom [geom] gmirror(8) have inappropriate logic when workin o kern/124130 geom [gmirror][usb] gmirror fails to start usb devices that o kern/123962 geom [panic] [gjournal] gjournal (455Gb data, 8Gb journal), o kern/123630 geom [patch] [gmirror] gmirror doesnt allow the original dr o kern/123122 geom [geom] GEOM / gjournal kernel lock f kern/122415 geom [geom] UFS labels are being constantly created and rem o kern/122067 geom [geom] [panic] Geom crashed during boot o kern/121559 geom [patch] [geom] geom label class allows to create inacc o kern/121364 geom [gmirror] Removing all providers create a "zombie" mir o kern/120231 geom [geom] GEOM_CONCAT error adding second drive o kern/120044 geom [msdosfs] [geom] incorrect MSDOSFS label fries adminis o kern/120021 geom [geom] [panic] net-p2p/qbittorrent crashes system when o kern/119743 geom [geom] geom label for cds is keeped after dismount and f kern/115547 geom [geom] [patch] [request] let GEOM Eli get password fro o kern/114532 geom [geom] GEOM_MIRROR shows up in kldstat even if compile o kern/113957 geom [gmirror] gmirror is intermittently reporting a degrad o kern/113837 geom [geom] unable to access 1024 sector size storage o kern/113419 geom [geom] geom fox multipathing not failing back p bin/110705 geom gmirror(8) control utility does not exit with correct o kern/107707 geom [geom] [patch] [request] add new class geom_xbox360 to o kern/104389 geom [geom] [patch] sys/geom/geom_dump.c doesn't encode XML o kern/98034 geom [geom] dereference of NULL pointer in acd_geom_detach o kern/94632 geom [geom] Kernel output resets input while GELI asks for o kern/90582 geom [geom] [panic] Restore cause panic string (ffs_blkfree o bin/90093 geom fdisk(8) incapable of altering in-core geometry a kern/89660 geom [vinum] [patch] [panic] due to g_malloc returning null o kern/89546 geom [geom] GEOM error s kern/89102 geom [geom] [panic] panic when forced unmount FS from unplu o kern/87544 geom [gbde] mmaping large files on a gbde filesystem deadlo o kern/84556 geom [geom] GBDE-encrypted swap causes panic at shutdown o kern/79251 geom [2TB] newfs fails on 2.6TB gbde device o kern/79035 geom [vinum] gvinum unable to create a striped set of mirro o bin/78131 geom gbde(8) "destroy" not working. s kern/73177 geom kldload geom_* causes panic due to memory exhaustion 40 problems total. From owner-freebsd-geom@FreeBSD.ORG Mon Dec 8 12:20:58 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD14D1065673 for ; Mon, 8 Dec 2008 12:20:58 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 840BE8FC19 for ; Mon, 8 Dec 2008 12:20:58 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1L9f6f-0005Wv-6E for freebsd-geom@freebsd.org; Mon, 08 Dec 2008 12:20:57 +0000 Received: from 195.208.174.178 ([195.208.174.178]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 08 Dec 2008 12:20:57 +0000 Received: from vadim_nuclight by 195.208.174.178 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 08 Dec 2008 12:20:57 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Vadim Goncharov Date: Mon, 8 Dec 2008 12:20:49 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 24 Message-ID: References: <6AEFCB5D-BF7F-49EE-9EDC-E5CD63920508@clamothe.com> <20081205144806.GA3284@garage.freebsd.pl> <8C75F72F-A68C-4E42-97F6-FA4BD4B2F57A@clamothe.com> <20081205204515.GA2303@garage.freebsd.pl> X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 195.208.174.178 X-Comment-To: Pawel Jakub Dawidek User-Agent: slrn/0.9.8.1 (FreeBSD) Sender: news Subject: Re: gmirror insert error: Synchronization request failed (error=1) X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Dec 2008 12:20:58 -0000 Hi Pawel Jakub Dawidek! On Fri, 5 Dec 2008 21:45:15 +0100; Pawel Jakub Dawidek wrote about 'Re: gmirror insert error: Synchronization request failed (error=1)': >> What offset should I use, then? Does this only apply to the first =20 >> labels in each slice, or should there be an offset between each label? > First 16 sectors is where bsdlabel keeps its metadata. bsdlabel(8) > correctly skips those, but not sysinstall, which is lame on our > (FreeBSD) side, I know. What? bsdlabel occupies only two sectors, in fact, only one, because first sector (0) is occupied by boot1, if any. Both UFS and swap are teached not to use first 8K of space, to be that able to contain boot2. In fact, the only problem that could arise from an offset-0 UFS partition is that a glabel(8) will incorrectly detect /dev/ufs/label on a slice itself, not partition, but I always have swap ('b' partition) starting from offset 0 and never had any problems (my 'a' UFS'es start from 16 when there no swap). So what metadata do you really mean? -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] From owner-freebsd-geom@FreeBSD.ORG Mon Dec 8 12:30:22 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7870D106564A for ; Mon, 8 Dec 2008 12:30:22 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 036348FC08 for ; Mon, 8 Dec 2008 12:30:21 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1L9fFk-0005rP-PC for freebsd-geom@freebsd.org; Mon, 08 Dec 2008 12:30:20 +0000 Received: from 195.208.174.178 ([195.208.174.178]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 08 Dec 2008 12:30:20 +0000 Received: from vadim_nuclight by 195.208.174.178 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 08 Dec 2008 12:30:20 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Vadim Goncharov Followup-To: gmane.os.freebsd.devel.geom Date: Mon, 8 Dec 2008 12:30:12 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 55 Message-ID: References: <4939287C.3020208@icyb.net.ua> X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 195.208.174.178 X-Comment-To: Andriy Gapon User-Agent: slrn/0.9.8.1 (FreeBSD) Sender: news Subject: Re: partition covering the whole slice [repost] X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Dec 2008 12:30:22 -0000 Hi Andriy Gapon! On Fri, 05 Dec 2008 15:11:24 +0200; Andriy Gapon wrote about 'partition covering the whole slice [repost]': > I have a disk with two slices and each slices has a single real > partition covering the whole slice, sector-to-sector. > I don't remember how I managed to configure the disk this way, is this > even possible? :-) > You can immediately spot another oddity - I never used glabel on this > disk, but I did use tunefs -L to label the UFS filesystems within the > partitions. > Now it seems that the label of filesystems is also somehow recognized as > a label for the whole slice. E.g. "ufs/extbackup" is exatcly the same as > "ad12s1". Weird. > Here's some additional data: > $ ls -1 /dev/ad12* > /dev/ad12 > /dev/ad12s1 > /dev/ad12s1a > /dev/ad12s2 > /dev/ad12s2a > Looks usual. > $ ls -1 /dev/ufs/ > extbackup > extbackupa > extstuff > extstuffa > So there is one "normal" label for each filesystem and the second label > for it as a filesystem in partition "a" of a labeled slice. > There is nothing in /dev/label though. > Ultimately I would like to fix this so that I don't see labels on the > slices. Yes, of course. You should not intermix using glabel(8) utilizing /dev/ufs (via tunefs) and bsdlabel partition starting from offset 0. This is because glabel can't distinguish is that slice or partition - with offset 0 superblock will be at the same position. You can try to erase bsdlabel completely (if this is not your boot partition) from the slice and use filesystem directly from the slice. This will not affect mount as you're already using labels. The other way will require shrinking-then-moving partition on the disk and editing disklabel, better done with newfs(8). -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] From owner-freebsd-geom@FreeBSD.ORG Wed Dec 10 22:15:45 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E4071065675; Wed, 10 Dec 2008 22:15:45 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: from rn-out-0910.google.com (rn-out-0910.google.com [64.233.170.190]) by mx1.freebsd.org (Postfix) with ESMTP id C132E8FC14; Wed, 10 Dec 2008 22:15:44 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: by rn-out-0910.google.com with SMTP id j71so792741rne.12 for ; Wed, 10 Dec 2008 14:15:43 -0800 (PST) Received: by 10.90.28.20 with SMTP id b20mr1181978agb.60.1228947343685; Wed, 10 Dec 2008 14:15:43 -0800 (PST) Received: by 10.90.73.15 with HTTP; Wed, 10 Dec 2008 14:15:43 -0800 (PST) Message-ID: Date: Wed, 10 Dec 2008 23:15:43 +0100 From: "=?ISO-8859-1?Q?Marius_N=FCnnerich?=" To: freebsd-geom@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: Cc: Alexander@leidinger.net, jb@freebsd.org, FreeBSD Current , phk@freebsd.org Subject: Re: DTrace probes for geom_kern, geom_io and geom_event X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Dec 2008 22:15:45 -0000 current CC'ed, maybe there are some people interested in DTrace that don't read geom. On Thu, Dec 4, 2008 at 9:41 PM, Marius N=FCnnerich wro= te: > Hi, > > I wrote a bunch of DTrace probes for the core geom files mentioned in > the subject. The patch for current is available at > http://nuenneri.ch/freebsd/geom_probes.patch > > Anyone interested in testing them? Just apply the patch, add options > KDTRACE_HOOKS to your kernel and build it like this: > # make WITH_CTF=3D1 KERNCONF=3DYOURKERNEL buildkernel installkernel > > After reboot you can > # kldload dtraceall > and see the new probes with > # dtrace -lP geom > > A sample script: > #!/usr/sbin/dtrace -s > #pragma D option quiet > > geom::: > { > @geom[execname, probemod, probefunc, probename] =3D count(); > @geom_all[execname, probemod, probefunc, probename] =3D count(); > } > > tick-10sec > { > normalize(@geom, 10) > printa("%@8u %@8u %12s %s:%s:%s\n", @geom_all, @geom); > printf("\n"); > clear(@geom); > } > > This is hand copied. You can chmod 755 and run it. > > I'm not sure how to handle the opt_kdtrace.h case in geom.h, see patch li= ne 842. > Any comments on the patch? > > @phk: Are you interested in committing this when there are no > complaints? Are you interested in more probes? After some tips from Alexander Leidinger I updated the patch, new version h= ere: http://nuenneri.ch/freebsd/geom_probes2.patch There are some questions I'd like to discuss: 1. I wrote the SDT_PROBE_DEFINEs right before the function definition, I know this violates the usual style as that stuff would normally belong to the top of the file. I think in this case it would be worthwhile to break with this tradition 2. Should I use the full function name for the probes (with the g_ prefix) even though it's defined under the provider geom 3. Should there be a probe for every switch case in g_io_check? I think this won't work with the fall-through that is used right now 4. Alexander proposed to change the module name kern to core. I'm not sure about this as kern refers to the filename, like io and event do 5. I'm thinking about defining a G_TRACE macro for SDT_PROBE(geom, ...) 6. Does anybody know of a way to probe functions with varargs properly? Like g_trace 7. What about g_bioq_(un)lock functions, I just added one probe for it, I do not really see a point in adding entry and return probes (they are there with FBT anyway). This is consistent with g_topology_lock etc. What about making macros of the two functions like g_topology_lock 8. What about adding macros for (un)locking other locks like g_eventlock, so one could add probes and trace them 9. Writing hundreds of entry and return probes is boring, especially as there is the FBT provider. Maybe it's possible to give the FBT probes better names like geom:io:g_io_schedule_down:entry instead of fbt:kernel:g_io_schedule:entry. Every FBT probe has a provider of fbt und module of kernel right now. One also has to define the argument types which I think FBT figures out by itself. If this would work we could concentrate on adding SDT probes to interesting places inside of functions or macros Bye Marius From owner-freebsd-geom@FreeBSD.ORG Thu Dec 11 12:08:08 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF3F61065678 for ; Thu, 11 Dec 2008 12:08:08 +0000 (UTC) (envelope-from oxy@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) by mx1.freebsd.org (Postfix) with ESMTP id 6CA7A8FC17 for ; Thu, 11 Dec 2008 12:08:08 +0000 (UTC) (envelope-from oxy@field.hu) Received: from localhost (green.field.hu [217.20.130.28]) by green.field.hu (Postfix) with ESMTP id 3B7E1B24E5 for ; Thu, 11 Dec 2008 12:52:52 +0100 (CET) X-Virus-Scanned: by Amavisd-new (Spamassassin+Razor2+Pyzor+DCC+Bayes db, Clamd Antivirus) at field.hu Received: from green.field.hu ([217.20.130.28]) by localhost (green.field.hu [217.20.130.28]) (amavisd-new, port 10024) with ESMTP id usDwAyi1yQPz for ; Thu, 11 Dec 2008 12:52:48 +0100 (CET) Received: from [192.168.1.2] (catv4E5CB4D6.pool.t-online.hu [78.92.180.214]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by green.field.hu (Postfix) with ESMTPSA id 56896B24EA for ; Thu, 11 Dec 2008 12:52:48 +0100 (CET) Message-ID: <4940FF0F.2020404@field.hu> Date: Thu, 11 Dec 2008 12:52:47 +0100 From: oxy User-Agent: Thunderbird 2.0.0.18 (Windows/20081105) MIME-Version: 1.0 To: freebsd-geom@freebsd.org Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Subject: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Dec 2008 12:08:08 -0000 Is there any method to use encrypted raid5 volumes? i created the raid5 volume with gvinum, works fine, but when i try to encrypt it: geli init -P -K /root/enc.key /dev/gvinum/raid5 it says: Cannot store Metadata....Operation not permitted any ideas? thank you! From owner-freebsd-geom@FreeBSD.ORG Thu Dec 11 13:11:54 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F9C11065675; Thu, 11 Dec 2008 13:11:54 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id DF7B48FC12; Thu, 11 Dec 2008 13:11:53 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (pD9E2DE31.dip.t-dialin.net [217.226.222.49]) by redbull.bpaserver.net (Postfix) with ESMTP id C24072E15C; Thu, 11 Dec 2008 13:54:43 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id E326E18DD8C; Thu, 11 Dec 2008 13:54:39 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1229000080; bh=C+xXCrh5QqVWSJp3IOxCqYnWayngc0aB9 DaFmHkQ7NE=; h=Message-ID:Date:From:To:Cc:Subject:References: In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=rBN0t1fuCG5Hyf8ZLD3R5ppm+H60gYS4Q1jDVI5FXNfTaRQjt9SvrBvOHhWN1oR6U Wei65UFjgtmSAeu5GJiU9Wd54WGbOorrddMu6SwfUMjUuxX0s8qhmgp/+U7aA2KOeqJ IMBGOABiECDm8roPTQuL17WGd3E9X6rVqlngSODU5WPr9RSS9Hzlvu1icR8k1pP3XB9 OyExSo2dCfw8T1Kt/p08Mn2l5Y+AlNayuumz8uZ9on5Pio++rgUT9PULThf2FohWgeu KDkIZmXPzIJ+hsc9Km5sqIqJ1rc9LhHDInPyVPjPHZdo6V6Zlr9ftskWwvW/DmAifMJ ek82hh9sA== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id mBBCsdgU018136; Thu, 11 Dec 2008 13:54:39 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Thu, 11 Dec 2008 13:54:38 +0100 Message-ID: <20081211135438.52433nmj45ia112c@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Thu, 11 Dec 2008 13:54:38 +0100 From: Alexander Leidinger To: Marius =?utf-8?b?TsO8bm5lcmljaA==?= References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.3) / FreeBSD-8.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-MailScanner-ID: C24072E15C.1B2A6 X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, ORDB-RBL, SpamAssassin (not cached, score=-14.6, required 6, BAYES_00 -15.00, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, MIME_8BIT_HEADER 0.30, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: jb@freebsd.org, FreeBSD Current , phk@freebsd.org, freebsd-geom@freebsd.org Subject: Re: DTrace probes for geom_kern, geom_io and geom_event X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Dec 2008 13:11:54 -0000 Quoting Marius N=C3=BCnnerich (from Wed, 10 Dec 2008 = =20 23:15:43 +0100): > After some tips from Alexander Leidinger I updated the patch, new =20 > version here: > http://nuenneri.ch/freebsd/geom_probes2.patch Again: I just reviewed the patch, so I don't have the complete context =20 of the functions, just what I see in the patch (-> high level dtrace =20 review, not geom specific probe review). Still inconsistent locking probes. Lock is fired without the lock =20 held, unlock is fired with the lock held. Both should IMHO be fired =20 either with the lock held or without the lock held, but not with the =20 current mix. g_new_bio/g_io_schedule_up: the return probe has the name "entry" in =20 your patch. A msleep probe could give some more info (sometimes there are even =20 zero arguments, but there are 3-5 things which could be interesting to =20 know). Similar for tsleep (the time should be provided in the probe =20 arguments too). I don't think we need "loop" probes. Given that g_trace is a debugging function and that dtrace is =20 superior, I don't think you need to instrument g_trace with dtrace =20 probes. g_topology_lock/unlock should provide the lock in the probe arguments. =20 Again, see above, either both probes firing with the lock, or without =20 the lock, but not mixed as it is. > There are some questions I'd like to discuss: > 2. Should I use the full function name for the probes (with the g_ > prefix) even though it's defined under the provider geom IMHO yes, it's more easy for the person grepping around, as "bioq" can =20 be found in a lot of unrelated places, but "g_bioq_init" only in =20 places where you want to know about. > 3. Should there be a probe for every switch case in g_io_check? I > think this won't work with the fall-through that is used right now IMHO at least in every code block which is doing something sensible. =20 Dtrace is not expensive, having a lot of probes does not hurt (except =20 maybe in a critical path). You could fire an read_not_permitted probe, =20 or a write_not_permitted probe or whatever. This can be done =20 additionally to the return probe. This way it's very easy to see if =20 there's a permission problem, without the need to write errno checks =20 in dtrace. If you have a lot of returns but only a handful of =20 permission errors, it's better to have some specific probes which can =20 be fired. Keep in mind dtrace is designed to be used to debug problems =20 on production systems. > 4. Alexander proposed to change the module name kern to core. I'm not > sure about this as kern refers to the filename, like io and event do - core for kern - core_io for io (maybe) - core_event for event (maybe) This way you can use gmirror, graid3, ... later as module names and =20 people/sysadmins without much GEOM knowledge don't have a problem to =20 see distinguish with real GEOM core stuff and stuff in GEOM providers. > 7. What about g_bioq_(un)lock functions, I just added one probe for > it, I do not really see a point in adding entry and return probes > (they are there with FBT anyway). This is consistent with > g_topology_lock etc. What about making macros of the two functions > like g_topology_lock Regarding FBT: the advantage of the geom dtrace-provider is that you =20 can tell to give everything for geom, while with with the fbt =20 dtrace-provider you need to know the naming conventions in the kernel. =20 So while you have in fbt the possibility to get access to the probes, =20 the sysadmin which does not know much about GEOM can get a meaningful =20 overview in case entry and return probes available in the geom =20 dtrace-provider. A lot of places in the kernel do not have a naming =20 convention like GEOM, so when we handle them (e.g. the linuxulator), =20 we need to add entry/return probes so that sysadmins without knowledge =20 about kernel internals can search for solutions of their problems. We =20 should be consistent in our probe naming, else it's not easy to use =20 dtrace. > 9. Writing hundreds of entry and return probes is boring, especially > as there is the FBT provider. Maybe it's possible to give the FBT > probes better names like geom:io:g_io_schedule_down:entry instead of > fbt:kernel:g_io_schedule:entry. Every FBT probe has a provider of fbt > und module of kernel right now. One also has to define the argument > types which I think FBT figures out by itself. If this would work we > could concentrate on adding SDT probes to interesting places inside of > functions or macros Both ways have good and bad parts. One argument against this is to =20 stay synchronized with vendor code. Another one is complexity to =20 handle this (currently the fbt part is automatic, I don't see a way to =20 handle related stuff which is spread into several places to within the =20 same namespace without introducing hints into different places which =20 tells what belongs where). HTH, Alexander. --=20 "They shot him five times. But he's though." =09=09-- Santino Corleone, "Chapter 2", page 79 http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-geom@FreeBSD.ORG Thu Dec 11 20:57:01 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3EC8A106564A for ; Thu, 11 Dec 2008 20:57:01 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id 069D28FC14 for ; Thu, 11 Dec 2008 20:57:00 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 72542 invoked by uid 2001); 11 Dec 2008 20:56:59 -0000 Date: Thu, 11 Dec 2008 14:56:59 -0600 From: "Rick C. Petty" To: oxy Message-ID: <20081211205659.GA72478@keira.kiwi-computer.com> References: <4940FF0F.2020404@field.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4940FF0F.2020404@field.hu> User-Agent: Mutt/1.4.2.3i Cc: freebsd-geom@freebsd.org Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd2008@kiwi-computer.com List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Dec 2008 20:57:01 -0000 On Thu, Dec 11, 2008 at 12:52:47PM +0100, oxy wrote: > Is there any method to use encrypted raid5 volumes? > i created the raid5 volume with gvinum, works fine, but when i try to > encrypt it: > geli init -P -K /root/enc.key /dev/gvinum/raid5 > it says: > Cannot store Metadata....Operation not permitted > any ideas? > thank you! What's the output of "gvinum l"? -- Rick C. Petty From owner-freebsd-geom@FreeBSD.ORG Thu Dec 11 22:39:23 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 799D41065670 for ; Thu, 11 Dec 2008 22:39:23 +0000 (UTC) (envelope-from oxy@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) by mx1.freebsd.org (Postfix) with ESMTP id 325768FC18 for ; Thu, 11 Dec 2008 22:39:22 +0000 (UTC) (envelope-from oxy@field.hu) Received: from localhost (green.field.hu [217.20.130.28]) by green.field.hu (Postfix) with ESMTP id DD97AB25AC for ; Thu, 11 Dec 2008 23:39:19 +0100 (CET) X-Virus-Scanned: by Amavisd-new (Spamassassin+Razor2+Pyzor+DCC+Bayes db, Clamd Antivirus) at field.hu Received: from green.field.hu ([217.20.130.28]) by localhost (green.field.hu [217.20.130.28]) (amavisd-new, port 10024) with ESMTP id MV4JDAYBgfpZ for ; Thu, 11 Dec 2008 23:39:15 +0100 (CET) Received: from [192.168.1.2] (catv4E5CB4D6.pool.t-online.hu [78.92.180.214]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by green.field.hu (Postfix) with ESMTPSA id D91D0B25AB for ; Thu, 11 Dec 2008 23:39:15 +0100 (CET) Message-ID: <49419691.4020403@field.hu> Date: Thu, 11 Dec 2008 23:39:13 +0100 From: oxy User-Agent: Thunderbird 2.0.0.18 (Windows/20081105) MIME-Version: 1.0 CC: freebsd-geom@freebsd.org References: <4940FF0F.2020404@field.hu> <20081211205659.GA72478@keira.kiwi-computer.com> In-Reply-To: <20081211205659.GA72478@keira.kiwi-computer.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Dec 2008 22:39:23 -0000 here it is: [root@test /]# gvinum l 4 drives: D disk_2 State: up /dev/ad9 A: 0/238475 MB (0%) D disk_1 State: up /dev/ad8 A: 0/238475 MB (0%) D disk_4 State: up /dev/ad5 A: 0/238475 MB (0%) D disk_3 State: up /dev/ad4 A: 0/238475 MB (0%) 1 volume: V raid5 State: down Plexes: 1 Size: 698 GB 1 plex: P raid5.p0 R5 State: down Subdisks: 4 Size: 698 GB 4 subdisks: S raid5.p0.s0 State: stale D: disk_1 Size: 232 GB S raid5.p0.s1 State: stale D: disk_2 Size: 232 GB S raid5.p0.s2 State: stale D: disk_3 Size: 232 GB S raid5.p0.s3 State: stale D: disk_4 Size: 232 GB [root@test /]# geli init -P -K /root/raid5.key /dev/gvinum/raid5 geli: Cannot store metadata on /dev/gvinum/raid5: Device not configured. Rick C. Petty írta: > On Thu, Dec 11, 2008 at 12:52:47PM +0100, oxy wrote: > >> Is there any method to use encrypted raid5 volumes? >> i created the raid5 volume with gvinum, works fine, but when i try to >> encrypt it: >> geli init -P -K /root/enc.key /dev/gvinum/raid5 >> it says: >> Cannot store Metadata....Operation not permitted >> any ideas? >> thank you! >> > > What's the output of "gvinum l"? > > -- Rick C. Petty > _______________________________________________ > freebsd-geom@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-geom > To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org" > From owner-freebsd-geom@FreeBSD.ORG Fri Dec 12 04:01:38 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A6E51065672 for ; Fri, 12 Dec 2008 04:01:38 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id E48868FC17 for ; Fri, 12 Dec 2008 04:01:37 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 76503 invoked by uid 2001); 12 Dec 2008 04:01:37 -0000 Date: Thu, 11 Dec 2008 22:01:37 -0600 From: "Rick C. Petty" To: oxy Message-ID: <20081212040137.GA76422@keira.kiwi-computer.com> References: <4940FF0F.2020404@field.hu> <20081211205659.GA72478@keira.kiwi-computer.com> <49419680.4010003@field.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <49419680.4010003@field.hu> User-Agent: Mutt/1.4.2.3i Cc: freebsd-geom@freebsd.org Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd2008@kiwi-computer.com List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2008 04:01:38 -0000 On Thu, Dec 11, 2008 at 11:38:56PM +0100, oxy wrote: > here it is: > > [root@test /]# gvinum l > 4 drives: > D disk_2 State: up /dev/ad9 A: 0/238475 MB (0%) > D disk_1 State: up /dev/ad8 A: 0/238475 MB (0%) > D disk_4 State: up /dev/ad5 A: 0/238475 MB (0%) > D disk_3 State: up /dev/ad4 A: 0/238475 MB (0%) > > 1 volume: > V raid5 State: down Plexes: 1 Size: 698 GB > > 1 plex: > P raid5.p0 R5 State: down Subdisks: 4 Size: 698 GB > > 4 subdisks: > S raid5.p0.s0 State: stale D: disk_1 Size: 232 GB > S raid5.p0.s1 State: stale D: disk_2 Size: 232 GB > S raid5.p0.s2 State: stale D: disk_3 Size: 232 GB > S raid5.p0.s3 State: stale D: disk_4 Size: 232 GB > > [root@test /]# geli init -P -K /root/raid5.key /dev/gvinum/raid5 > geli: Cannot store metadata on /dev/gvinum/raid5: Device not configured. The error message is quite accurate-- the raid5 volume is down because the plex is stale. You need to run a "gvinum start raid5.p0" and let it complete before the volume will be "up". This operation will sync the four plexes and write out the parity info. There are a set of patches that lulf@ has which I believe put the volume in "up" state initially instead of "down", but maybe it only works for mirrors. The code in current and RELENG_7 does initially put the volume in "down" state. -- Rick C. Petty From owner-freebsd-geom@FreeBSD.ORG Fri Dec 12 08:36:41 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 050BE1065670 for ; Fri, 12 Dec 2008 08:36:41 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) Received: from bgo1smout1.broadpark.no (bgo1smout1.broadpark.no [217.13.4.94]) by mx1.freebsd.org (Postfix) with ESMTP id B37928FC17 for ; Fri, 12 Dec 2008 08:36:40 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-disposition: inline Content-type: text/plain; charset=us-ascii Received: from bgo1sminn1.broadpark.no ([217.13.4.93]) by bgo1smout1.broadpark.no (Sun Java(tm) System Messaging Server 6.3-3.01 (built Jul 12 2007; 32bit)) with ESMTP id <0KBR00G2Y6GEZE20@bgo1smout1.broadpark.no> for freebsd-geom@freebsd.org; Fri, 12 Dec 2008 08:36:14 +0100 (CET) Received: from carrot.studby.ntnu.no ([80.203.120.105]) by bgo1sminn1.broadpark.no (Sun Java(tm) System Messaging Server 6.3-3.01 (built Jul 12 2007; 32bit)) with ESMTP id <0KBR00MJX6GECA20@bgo1sminn1.broadpark.no> for freebsd-geom@freebsd.org; Fri, 12 Dec 2008 08:36:14 +0100 (CET) Date: Fri, 12 Dec 2008 09:37:09 +0100 From: Ulf Lilleengen To: "Rick C. Petty" Message-id: <20081212083708.GA1455@carrot.studby.ntnu.no> References: <4940FF0F.2020404@field.hu> <20081211205659.GA72478@keira.kiwi-computer.com> <49419680.4010003@field.hu> <20081212040137.GA76422@keira.kiwi-computer.com> In-reply-to: <20081212040137.GA76422@keira.kiwi-computer.com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: oxy , freebsd-geom@freebsd.org Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2008 08:36:41 -0000 On tor, des 11, 2008 at 10:01:37pm -0600, Rick C. Petty wrote: *snip* > There are a set of patches that lulf@ has which I believe put the volume in > "up" state initially instead of "down", but maybe it only works for > mirrors. The code in current and RELENG_7 does initially put the volume in > "down" state. > Yes, it only works for mirrors, since I thought it doesn't really matter if a mirror is properly initialized, since the user need to put data into the mirror for it to be useful anyway. The same goes for RAID-5 I guess, but I was not sure if it might trigger some weird behaviour since parity would not match if reading the volume. I will test out a small modification I made, which removes the need to run 'gvinum start' on the raid5 plexes. -- Ulf Lilleengen From owner-freebsd-geom@FreeBSD.ORG Fri Dec 12 10:03:40 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40F59106564A for ; Fri, 12 Dec 2008 10:03:40 +0000 (UTC) (envelope-from oxy@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) by mx1.freebsd.org (Postfix) with ESMTP id EB8048FC08 for ; Fri, 12 Dec 2008 10:03:39 +0000 (UTC) (envelope-from oxy@field.hu) Received: from localhost (green.field.hu [217.20.130.28]) by green.field.hu (Postfix) with ESMTP id D59E7B24C0; Fri, 12 Dec 2008 11:03:36 +0100 (CET) X-Virus-Scanned: by Amavisd-new (Spamassassin+Razor2+Pyzor+DCC+Bayes db, Clamd Antivirus) at field.hu Received: from green.field.hu ([217.20.130.28]) by localhost (green.field.hu [217.20.130.28]) (amavisd-new, port 10024) with ESMTP id fqGQgaif3lZO; Fri, 12 Dec 2008 11:03:32 +0100 (CET) Received: from [192.168.1.2] (catv4E5CB4D6.pool.t-online.hu [78.92.180.214]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by green.field.hu (Postfix) with ESMTPSA id B913FB24B5; Fri, 12 Dec 2008 11:03:32 +0100 (CET) Message-ID: <494236F2.5040803@field.hu> Date: Fri, 12 Dec 2008 11:03:30 +0100 From: oxy User-Agent: Thunderbird 2.0.0.18 (Windows/20081105) MIME-Version: 1.0 To: Ulf Lilleengen References: <4940FF0F.2020404@field.hu> <20081211205659.GA72478@keira.kiwi-computer.com> <49419680.4010003@field.hu> <20081212040137.GA76422@keira.kiwi-computer.com> <20081212083708.GA1455@carrot.studby.ntnu.no> In-Reply-To: <20081212083708.GA1455@carrot.studby.ntnu.no> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: "Rick C. Petty" , freebsd-geom@freebsd.org Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2008 10:03:40 -0000 You mean that after every reboot i have to start the raid5 manually and re-sync? Ulf Lilleengen írta: > On tor, des 11, 2008 at 10:01:37pm -0600, Rick C. Petty wrote: > *snip* > >> There are a set of patches that lulf@ has which I believe put the volume in >> "up" state initially instead of "down", but maybe it only works for >> mirrors. The code in current and RELENG_7 does initially put the volume in >> "down" state. >> >> > Yes, it only works for mirrors, since I thought it doesn't really matter if a > mirror is properly initialized, since the user need to put data into the > mirror for it to be useful anyway. The same goes for RAID-5 I guess, but I > was not sure if it might trigger some weird behaviour since parity would not > match if reading the volume. I will test out a small modification I made, > which removes the need to run 'gvinum start' on the raid5 plexes. > > From owner-freebsd-geom@FreeBSD.ORG Fri Dec 12 10:07:55 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 140B11065675 for ; Fri, 12 Dec 2008 10:07:55 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 8B6DB8FC29 for ; Fri, 12 Dec 2008 10:07:54 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1LB4w0-0006dC-DI for freebsd-geom@freebsd.org; Fri, 12 Dec 2008 10:07:48 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 12 Dec 2008 10:07:48 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 12 Dec 2008 10:07:48 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Fri, 12 Dec 2008 11:07:40 +0100 Lines: 51 Message-ID: References: <4940FF0F.2020404@field.hu> <20081211205659.GA72478@keira.kiwi-computer.com> <49419680.4010003@field.hu> <20081212040137.GA76422@keira.kiwi-computer.com> <20081212083708.GA1455@carrot.studby.ntnu.no> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigA98048F3A1A0F141F5ECCCBA" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.18 (X11/20081125) In-Reply-To: <20081212083708.GA1455@carrot.studby.ntnu.no> X-Enigmail-Version: 0.95.0 Sender: news Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2008 10:07:55 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigA98048F3A1A0F141F5ECCCBA Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Ulf Lilleengen wrote: > On tor, des 11, 2008 at 10:01:37pm -0600, Rick C. Petty wrote: > *snip* >> There are a set of patches that lulf@ has which I believe put the volu= me in >> "up" state initially instead of "down", but maybe it only works for >> mirrors. The code in current and RELENG_7 does initially put the volu= me in >> "down" state. >> > Yes, it only works for mirrors, since I thought it doesn't really matte= r if a > mirror is properly initialized, since the user need to put data into th= e > mirror for it to be useful anyway. The same goes for RAID-5 I guess, bu= t I > was not sure if it might trigger some weird behaviour since parity woul= d not > match if reading the volume. I will test out a small modification I mad= e, > which removes the need to run 'gvinum start' on the raid5 plexes. It doesn't have to be "weird" behaviour, depending on whether gvinum checks parity on reads (does it?). If it does, it will only have to ignore checksum errors in this case. I suppose people will want to run utilities like diskinfo -vt on the volume with invalid parities so it's not a theoretical scenario :) --------------enigA98048F3A1A0F141F5ECCCBA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJQjfsldnAQVacBcgRAmAkAKDcZeY7tVtgbxu4bpFH/m1m50zj3gCeKL6H PzbmxYNjZHER22QRw/AvMYE= =YTrG -----END PGP SIGNATURE----- --------------enigA98048F3A1A0F141F5ECCCBA-- From owner-freebsd-geom@FreeBSD.ORG Fri Dec 12 11:58:28 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C69B41065670 for ; Fri, 12 Dec 2008 11:58:28 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) Received: from bgo1smout1.broadpark.no (bgo1smout1.broadpark.no [217.13.4.94]) by mx1.freebsd.org (Postfix) with ESMTP id 8341A8FC16 for ; Fri, 12 Dec 2008 11:58:28 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-disposition: inline Content-type: text/plain; charset=us-ascii Received: from bgo1sminn1.broadpark.no ([217.13.4.93]) by bgo1smout1.broadpark.no (Sun Java(tm) System Messaging Server 6.3-3.01 (built Jul 12 2007; 32bit)) with ESMTP id <0KBR00J9WIKYWB60@bgo1smout1.broadpark.no> for freebsd-geom@freebsd.org; Fri, 12 Dec 2008 12:58:10 +0100 (CET) Received: from carrot.studby.ntnu.no ([80.203.120.105]) by bgo1sminn1.broadpark.no (Sun Java(tm) System Messaging Server 6.3-3.01 (built Jul 12 2007; 32bit)) with ESMTP id <0KBR0003EIKX5N01@bgo1sminn1.broadpark.no> for freebsd-geom@freebsd.org; Fri, 12 Dec 2008 12:58:10 +0100 (CET) Date: Fri, 12 Dec 2008 13:59:05 +0100 From: Ulf Lilleengen To: oxy Message-id: <20081212125905.GA39875@carrot.studby.ntnu.no> References: <4940FF0F.2020404@field.hu> <20081211205659.GA72478@keira.kiwi-computer.com> <49419680.4010003@field.hu> <20081212040137.GA76422@keira.kiwi-computer.com> <20081212083708.GA1455@carrot.studby.ntnu.no> <494236F2.5040803@field.hu> In-reply-to: <494236F2.5040803@field.hu> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: "Rick C. Petty" , freebsd-geom@freebsd.org Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2008 11:58:28 -0000 On Fri, Dec 12, 2008 at 11:03:30AM +0100, oxy wrote: > You mean that after every reboot i have to start the raid5 manually and > re-sync? > No, just initially after creating it, and if one of the disks fails, in which case you will probably put in a new one and re-sync it. -- Ulf Lilleengen From owner-freebsd-geom@FreeBSD.ORG Fri Dec 12 12:08:27 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EF261065670 for ; Fri, 12 Dec 2008 12:08:27 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) Received: from bgo1smout1.broadpark.no (bgo1smout1.broadpark.no [217.13.4.94]) by mx1.freebsd.org (Postfix) with ESMTP id 59E018FC19 for ; Fri, 12 Dec 2008 12:08:27 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-disposition: inline Content-type: text/plain; charset=us-ascii Received: from bgo1sminn1.broadpark.no ([217.13.4.93]) by bgo1smout1.broadpark.no (Sun Java(tm) System Messaging Server 6.3-3.01 (built Jul 12 2007; 32bit)) with ESMTP id <0KBR00JGTJ16WBA0@bgo1smout1.broadpark.no>; Fri, 12 Dec 2008 13:07:54 +0100 (CET) Received: from carrot.studby.ntnu.no ([80.203.120.105]) by bgo1sminn1.broadpark.no (Sun Java(tm) System Messaging Server 6.3-3.01 (built Jul 12 2007; 32bit)) with ESMTP id <0KBR00NZ1J155431@bgo1sminn1.broadpark.no>; Fri, 12 Dec 2008 13:07:54 +0100 (CET) Date: Fri, 12 Dec 2008 14:08:49 +0100 From: Ulf Lilleengen To: Ivan Voras Message-id: <20081212130848.GB39875@carrot.studby.ntnu.no> References: <4940FF0F.2020404@field.hu> <20081211205659.GA72478@keira.kiwi-computer.com> <49419680.4010003@field.hu> <20081212040137.GA76422@keira.kiwi-computer.com> <20081212083708.GA1455@carrot.studby.ntnu.no> In-reply-to: User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-geom@freebsd.org Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2008 12:08:27 -0000 On Fri, Dec 12, 2008 at 11:07:40AM +0100, Ivan Voras wrote: > Ulf Lilleengen wrote: > > On tor, des 11, 2008 at 10:01:37pm -0600, Rick C. Petty wrote: > > *snip* > >> There are a set of patches that lulf@ has which I believe put the volume in > >> "up" state initially instead of "down", but maybe it only works for > >> mirrors. The code in current and RELENG_7 does initially put the volume in > >> "down" state. > >> > > Yes, it only works for mirrors, since I thought it doesn't really matter if a > > mirror is properly initialized, since the user need to put data into the > > mirror for it to be useful anyway. The same goes for RAID-5 I guess, but I > > was not sure if it might trigger some weird behaviour since parity would not > > match if reading the volume. I will test out a small modification I made, > > which removes the need to run 'gvinum start' on the raid5 plexes. > > It doesn't have to be "weird" behaviour, depending on whether gvinum > checks parity on reads (does it?). If it does, it will only have to > ignore checksum errors in this case. It does check parity on reads. But I think it doesn't matter, since no sane data has been written in that block anyway. But as you say, one way to handle it is to ignore the checksums if the data is known to not be initialized, but then wouldn't one have to keep track of which blocks have a valid parity and which who does not? > I suppose people will want to run utilities like diskinfo -vt on the > volume with invalid parities so it's not a theoretical scenario :) > I guess, but I then one can just initialize the volume anyway. -- Ulf Lilleengen From owner-freebsd-geom@FreeBSD.ORG Fri Dec 12 12:29:39 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 400D21065676 for ; Fri, 12 Dec 2008 12:29:39 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id B61A58FC24 for ; Fri, 12 Dec 2008 12:29:38 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1LB79B-00038V-6P for freebsd-geom@freebsd.org; Fri, 12 Dec 2008 12:29:33 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 12 Dec 2008 12:29:33 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 12 Dec 2008 12:29:33 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Fri, 12 Dec 2008 13:29:24 +0100 Lines: 69 Message-ID: References: <4940FF0F.2020404@field.hu> <20081211205659.GA72478@keira.kiwi-computer.com> <49419680.4010003@field.hu> <20081212040137.GA76422@keira.kiwi-computer.com> <20081212083708.GA1455@carrot.studby.ntnu.no> <20081212130848.GB39875@carrot.studby.ntnu.no> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigB04A2D7738FD0766A66FE88C" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.18 (X11/20081125) In-Reply-To: <20081212130848.GB39875@carrot.studby.ntnu.no> X-Enigmail-Version: 0.95.0 Sender: news Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2008 12:29:39 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigB04A2D7738FD0766A66FE88C Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Ulf Lilleengen wrote: > On Fri, Dec 12, 2008 at 11:07:40AM +0100, Ivan Voras wrote: >> Ulf Lilleengen wrote: >>> On tor, des 11, 2008 at 10:01:37pm -0600, Rick C. Petty wrote: >>> *snip* >>>> There are a set of patches that lulf@ has which I believe put the vo= lume in >>>> "up" state initially instead of "down", but maybe it only works for >>>> mirrors. The code in current and RELENG_7 does initially put the vo= lume in >>>> "down" state. >>>> >>> Yes, it only works for mirrors, since I thought it doesn't really mat= ter if a >>> mirror is properly initialized, since the user need to put data into = the >>> mirror for it to be useful anyway. The same goes for RAID-5 I guess, = but I >>> was not sure if it might trigger some weird behaviour since parity wo= uld not >>> match if reading the volume. I will test out a small modification I m= ade, >>> which removes the need to run 'gvinum start' on the raid5 plexes. >> It doesn't have to be "weird" behaviour, depending on whether gvinum >> checks parity on reads (does it?). If it does, it will only have to >> ignore checksum errors in this case. > It does check parity on reads. But I think it doesn't matter, since no = sane data > has been written in that block anyway.=20 >=20 > But as you say, one way to handle it is to ignore the checksums if the = data > is known to not be initialized, but then wouldn't one have to keep trac= k of > which blocks have a valid parity and which who does not? >=20 >> I suppose people will want to run utilities like diskinfo -vt on the >> volume with invalid parities so it's not a theoretical scenario :) >> > I guess, but I then one can just initialize the volume anyway. In the interest of simplicity, maybe a single, well documented flag that says "don't check checksums on reads" will do best. It will also probably help read performance. Of course, it should be off by default and its implications explained in the man page :) --------------enigB04A2D7738FD0766A66FE88C Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJQlkkldnAQVacBcgRApUQAKD0eupyquvLWzYn/CbczuHqL+w3RACbBhPG acftybWfOZLfIIq+YtpUzPQ= =vFAl -----END PGP SIGNATURE----- --------------enigB04A2D7738FD0766A66FE88C-- From owner-freebsd-geom@FreeBSD.ORG Fri Dec 12 15:50:27 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 342D410656A9 for ; Fri, 12 Dec 2008 15:50:27 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id BA5C58FC45 for ; Fri, 12 Dec 2008 15:50:26 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 82749 invoked by uid 2001); 12 Dec 2008 15:50:23 -0000 Date: Fri, 12 Dec 2008 09:50:23 -0600 From: "Rick C. Petty" To: Ulf Lilleengen Message-ID: <20081212155023.GA82667@keira.kiwi-computer.com> References: <4940FF0F.2020404@field.hu> <20081211205659.GA72478@keira.kiwi-computer.com> <49419680.4010003@field.hu> <20081212040137.GA76422@keira.kiwi-computer.com> <20081212083708.GA1455@carrot.studby.ntnu.no> <20081212130848.GB39875@carrot.studby.ntnu.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20081212130848.GB39875@carrot.studby.ntnu.no> User-Agent: Mutt/1.4.2.3i Cc: Ivan Voras , freebsd-geom@freebsd.org Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd2008@kiwi-computer.com List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2008 15:50:27 -0000 On Fri, Dec 12, 2008 at 02:08:49PM +0100, Ulf Lilleengen wrote: > > > > It doesn't have to be "weird" behaviour, depending on whether gvinum > > checks parity on reads (does it?). If it does, it will only have to > > ignore checksum errors in this case. > It does check parity on reads. But I think it doesn't matter, since no sane data > has been written in that block anyway. > > But as you say, one way to handle it is to ignore the checksums if the data > is known to not be initialized, but then wouldn't one have to keep track of > which blocks have a valid parity and which who does not? IMO, trying to read a block that hasn't been initialized is perfectly acceptable as an "error". I would just mark the volume as up. Chances are a sane admin would be writing to the blocks before reading the same blocks (except in the disk test scenario, in which case why are they bothering with a raid5?). If a read happens on a block that hasn't been written to, it is a parity error. The real question is what happens when parity errors happen. I guess I suggest allowing you to force the plex up (via setstate) if you are pretty confident reads will only happen after writes (which is the case after newfs-ing the volume). In either case, always mark the volume as up.. the plex can be in a degraded state, meaning the parity has failed but reads can still happen. It sounds perfect to me; the states reflect the actual state of things. -- Rick C. Petty From owner-freebsd-geom@FreeBSD.ORG Fri Dec 12 18:06:23 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49C6D106564A for ; Fri, 12 Dec 2008 18:06:23 +0000 (UTC) (envelope-from mikej@paymentallianceintl.com) Received: from mx2.confluenttech.com (mx2.confluentasp.com [216.26.153.14]) by mx1.freebsd.org (Postfix) with ESMTP id 0491C8FC08 for ; Fri, 12 Dec 2008 18:06:22 +0000 (UTC) (envelope-from mikej@paymentallianceintl.com) Received: from calvin.pai.local (calvin.pai.local [10.0.6.33]) by mx2.confluenttech.com (8.14.1/8.13.8) with ESMTP id mBCG092f041490 for ; Fri, 12 Dec 2008 11:00:10 -0500 (EST) (envelope-from mikej@paymentallianceintl.com) Authentication-Results: mx2.confluenttech.com from=mikej@paymentallianceintl.com; sender-id=neutral; spf=neutral X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4325 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Date: Fri, 12 Dec 2008 11:00:09 -0500 Message-ID: In-Reply-To: <20081212155023.GA82667@keira.kiwi-computer.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Importance: normal Priority: normal Thread-Topic: Encrypting raid5 volume with geli Thread-Index: AclccYw46n9leSsVQWK1YfxBh6BaCwAAG/OA From: "Michael Jung" To: Subject: RE: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2008 18:06:23 -0000 FreeBSD charon.confluentasp.com 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #2: Thu Sep 4 12:06:08 EDT 2008 In the interest of this thread I tried to duplicate the problem. I created: 10 drives: D d9 State: up /dev/da9 A: 0/17366 MB (0%) D d8 State: up /dev/da8 A: 0/17366 MB (0%) D d7 State: up /dev/da7 A: 0/17366 MB (0%) D d6 State: up /dev/da6 A: 0/17366 MB (0%) D d5 State: up /dev/da5 A: 0/17366 MB (0%) D d4 State: up /dev/da4 A: 0/17366 MB (0%) D d3 State: up /dev/da3 A: 0/17366 MB (0%) D d2 State: up /dev/da2 A: 0/17366 MB (0%) D d1 State: up /dev/da1 A: 0/17366 MB (0%) D d0 State: up /dev/da0 A: 0/17366 MB (0%) 1 volume: V test State: up Plexes: 1 Size: 152 GB 1 plex: P test.p0 R5 State: up Subdisks: 10 Size: 152 GB 10 subdisks: S test.p0.s9 State: up D: d9 Size: 16 GB S test.p0.s8 State: up D: d8 Size: 16 GB S test.p0.s7 State: up D: d7 Size: 16 GB S test.p0.s6 State: up D: d6 Size: 16 GB S test.p0.s5 State: up D: d5 Size: 16 GB S test.p0.s4 State: up D: d4 Size: 16 GB S test.p0.s3 State: up D: d3 Size: 16 GB S test.p0.s2 State: up D: d2 Size: 16 GB S test.p0.s1 State: up D: d1 Size: 16 GB S test.p0.s0 State: up D: d0 Size: 16 GB Which I can newfs and mount (root@charon) /etc# mount /dev/gvinum/test /mnt (root@charon) /etc# df -h Filesystem Size Used Avail Capacity Mounted on /dev/ad4s1a 357G 119G 209G 36% / devfs 1.0K 1.0K 0B 100% /dev 172.0.255.28:/data/unix 1.3T 643G 559G 54% /nas1 /dev/gvinum/test 148G 4.0K 136G 0% /mnt But with /dev/gvinum/test unmounted if I try: (root@charon) /etc# geli init -P -K /root/test.key /dev/gvinum/test geli: Cannot store metadata on /dev/gvinum/test: Operation not permitted. (root@charon) /etc# My random file was created like dd if=/dev/random of=/root/test.key bs=64 count=1 I use GELI at home with no trouble, although not with a gvinum volume. --mikej CONFIDENTIALITY NOTE: This message is intended only for the use of the individual or entity to whom it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, please notify us by telephone at (502) 212-4001 or notify us at PAI , Dept. 99, 11857 Commonwealth Drive, Louisville, KY 40299. Thank you. From owner-freebsd-geom@FreeBSD.ORG Sat Dec 13 14:17:05 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAE71106564A for ; Sat, 13 Dec 2008 14:17:05 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.247]) by mx1.freebsd.org (Postfix) with ESMTP id 7D8978FC0C for ; Sat, 13 Dec 2008 14:17:05 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) Received: by an-out-0708.google.com with SMTP id c2so686539anc.13 for ; Sat, 13 Dec 2008 06:17:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type:references; bh=T+8eqtt2AXqdIZgBRUOYLYLhL2zBtHqCmArHg/CEU3c=; b=JLT6vFGbbRArk6TXfK0/rbOtIwTjKhTLTRreZ61cx+JccvzxBM4kAgCKAtFNRrTk2m SoiqjwVJrAxIr05WahTZALb3qUGmasyFv3RGP6SPdYJ4BxOPkyitg5BB1AwzV6MUsFy8 53jtBSTKzdRjeMmtKc39hJ13sirV8+DLjgi0c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:references; b=xFT8zj1nKz0zHQJEUVooezILatG4ZGYI82kPuTrLlaMJsSGaGNlqAk6Nj2zSvl+qZw RhRtqKMGTvxLDMgjmbomfdW2HlEfJkvZ9UafXjuTYI1L9XJwQiEBlVTW4hn0IzWlpRd8 /WRrw7+C9N1HRafJJ+6k5SFZcjWCknW58Znz8= Received: by 10.100.164.20 with SMTP id m20mr3560115ane.121.1229177824838; Sat, 13 Dec 2008 06:17:04 -0800 (PST) Received: by 10.100.210.20 with HTTP; Sat, 13 Dec 2008 06:17:04 -0800 (PST) Message-ID: <917871cf0812130617s2c321612m1497bc2de8aa8501@mail.gmail.com> Date: Sat, 13 Dec 2008 15:17:04 +0100 From: "Ulf Lilleengen" To: "Michael Jung" In-Reply-To: <917871cf0812130559r6d423688q57287dd765d6edf4@mail.gmail.com> MIME-Version: 1.0 References: <20081212155023.GA82667@keira.kiwi-computer.com> <917871cf0812130559r6d423688q57287dd765d6edf4@mail.gmail.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-geom@freebsd.org Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Dec 2008 14:17:05 -0000 On Sat, Dec 13, 2008 at 2:59 PM, Ulf Lilleengen wrote: > > > On Fri, Dec 12, 2008 at 5:00 PM, Michael Jung < > mikej@paymentallianceintl.com> wrote: > >> FreeBSD charon.confluentasp.com 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE >> #2: Thu Sep 4 12:06:08 EDT 2008 >> > *snip* > >> > I hope to commit the attached change in the near future. > > -- > Ulf Lilleengen > Done, rev 186038 -- Ulf Lilleengen From owner-freebsd-geom@FreeBSD.ORG Sat Dec 13 14:30:22 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC2791065670 for ; Sat, 13 Dec 2008 14:30:22 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.250]) by mx1.freebsd.org (Postfix) with ESMTP id 79F768FC16 for ; Sat, 13 Dec 2008 14:30:22 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) Received: by an-out-0708.google.com with SMTP id c2so687674anc.13 for ; Sat, 13 Dec 2008 06:30:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type:references; bh=nVZyvGXrz6zfUhPLaMcB8YzTcJRzzWJ6SwlmyD+qn1Q=; b=L1UhLAGhw/FcHWFhXqR+h6ueQ/12ab3PJlQuiufgqPcM7I7OJcYnmX8XcDcbJ1+Os5 XYkd9RtitUVuHdrzCzwNkZjd/ulXnnBsslXJMn5sK2h+BpqRYmpl28mLiwHrZXzmBQOG IYiFP+dJvgTHdGvD92EoduQrdofvJ68S65iBM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:references; b=MDQgojSTyTGHKTQyxXhv3Wcm3W9lRjhnjzwp9s51poPq94GrWsjfzf9mo230Kn5HkK kfJGZLqcHwI6LardV/MAPqwTouLQhSU9I/4+OUVcA2FoNCNzIryd9hqzQtZGG43QysaQ MVtJtvsgx8Ihzce9gth7IR/Bl+hzg8/IuqfdE= Received: by 10.100.131.13 with SMTP id e13mr3555718and.57.1229176741252; Sat, 13 Dec 2008 05:59:01 -0800 (PST) Received: by 10.100.210.20 with HTTP; Sat, 13 Dec 2008 05:59:01 -0800 (PST) Message-ID: <917871cf0812130559r6d423688q57287dd765d6edf4@mail.gmail.com> Date: Sat, 13 Dec 2008 14:59:01 +0100 From: "Ulf Lilleengen" To: "Michael Jung" In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_11901_3716848.1229176741247" References: <20081212155023.GA82667@keira.kiwi-computer.com> X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-geom@freebsd.org Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Dec 2008 14:30:22 -0000 ------=_Part_11901_3716848.1229176741247 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Fri, Dec 12, 2008 at 5:00 PM, Michael Jung wrote: > FreeBSD charon.confluentasp.com 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE > #2: Thu Sep 4 12:06:08 EDT 2008 > > In the interest of this thread I tried to duplicate the problem. I > created: > > 10 drives: > D d9 State: up /dev/da9 A: 0/17366 MB > (0%) > D d8 State: up /dev/da8 A: 0/17366 MB > (0%) > D d7 State: up /dev/da7 A: 0/17366 MB > (0%) > D d6 State: up /dev/da6 A: 0/17366 MB > (0%) > D d5 State: up /dev/da5 A: 0/17366 MB > (0%) > D d4 State: up /dev/da4 A: 0/17366 MB > (0%) > D d3 State: up /dev/da3 A: 0/17366 MB > (0%) > D d2 State: up /dev/da2 A: 0/17366 MB > (0%) > D d1 State: up /dev/da1 A: 0/17366 MB > (0%) > D d0 State: up /dev/da0 A: 0/17366 MB > (0%) > > 1 volume: > V test State: up Plexes: 1 Size: 152 > GB > > 1 plex: > P test.p0 R5 State: up Subdisks: 10 Size: 152 > GB > > 10 subdisks: > S test.p0.s9 State: up D: d9 Size: 16 > GB > S test.p0.s8 State: up D: d8 Size: 16 > GB > S test.p0.s7 State: up D: d7 Size: 16 > GB > S test.p0.s6 State: up D: d6 Size: 16 > GB > S test.p0.s5 State: up D: d5 Size: 16 > GB > S test.p0.s4 State: up D: d4 Size: 16 > GB > S test.p0.s3 State: up D: d3 Size: 16 > GB > S test.p0.s2 State: up D: d2 Size: 16 > GB > S test.p0.s1 State: up D: d1 Size: 16 > GB > S test.p0.s0 State: up D: d0 Size: 16 > GB > > Which I can newfs and mount > > (root@charon) /etc# mount /dev/gvinum/test /mnt > (root@charon) /etc# df -h > Filesystem Size Used Avail Capacity Mounted on > /dev/ad4s1a 357G 119G 209G 36% / > devfs 1.0K 1.0K 0B 100% /dev > 172.0.255.28:/data/unix 1.3T 643G 559G 54% /nas1 > /dev/gvinum/test 148G 4.0K 136G 0% /mnt > > But with /dev/gvinum/test unmounted if I try: > > (root@charon) /etc# geli init -P -K /root/test.key /dev/gvinum/test > geli: Cannot store metadata on /dev/gvinum/test: Operation not > permitted. > (root@charon) /etc# > > My random file was created like > > dd if=/dev/random of=/root/test.key bs=64 count=1 > > I use GELI at home with no trouble, although not with a gvinum volume. > Hello, When I tried this myself, I also got the EPERM error in return. I though this was very strange. I went through the gvinum code today, and put debugging prints everywhere, but everything looked fine, and it was only raid5 volumes that failed. Then I saw that the EPERM error came from the underlying providers of geom (more specifially from the read requests to the parity stripes etc), so I was starting to suspect that it was not a gvinum error. But still, I was able to write/read from the disks from outside of gvinum! Then, I discovered in geom userland code that it opens the disk where metadata should be written in write only mode. Then I discovered the reason: gvinum tries to write to the stripe in question, but has to read back the parity data from one of the other stripes. But, they are opened O_WRONLY, so the request fails. I tried opening the device as O_RDWR, and everything is find. Phew :) You can bet I was frustrated I hope to commit the attached change in the near future. -- Ulf Lilleengen ------=_Part_11901_3716848.1229176741247 Content-Type: application/octet-stream; name=geomfix.diff Content-Transfer-Encoding: base64 X-Attachment-Id: f_fooe7yb30 Content-Disposition: attachment; filename=geomfix.diff SW5kZXg6IHN1YnIuYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzdWJyLmMJKHJldmlzaW9uIDE4NTkzMCkKKysr IHN1YnIuYwkod29ya2luZyBjb3B5KQpAQCAtMjExLDcgKzIxMSw3IEBACiAJc2VjdG9yID0gTlVM TDsKIAllcnJvciA9IDA7CiAKLQlmZCA9IG9wZW4ocGF0aCwgT19XUk9OTFkpOworCWZkID0gb3Bl bihwYXRoLCBPX1JEV1IpOwogCWlmIChmZCA9PSAtMSkKIAkJcmV0dXJuIChlcnJubyk7CiAJbWVk aWFzaXplID0gZ19nZXRfbWVkaWFzaXplKG5hbWUpOwo= ------=_Part_11901_3716848.1229176741247-- From owner-freebsd-geom@FreeBSD.ORG Sat Dec 13 20:22:36 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36A831065676 for ; Sat, 13 Dec 2008 20:22:36 +0000 (UTC) (envelope-from oxy@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) by mx1.freebsd.org (Postfix) with ESMTP id CB9FB8FC16 for ; Sat, 13 Dec 2008 20:22:35 +0000 (UTC) (envelope-from oxy@field.hu) Received: from localhost (green.field.hu [217.20.130.28]) by green.field.hu (Postfix) with ESMTP id 376CAB2530; Sat, 13 Dec 2008 21:22:34 +0100 (CET) X-Virus-Scanned: by Amavisd-new (Spamassassin+Razor2+Pyzor+DCC+Bayes db, Clamd Antivirus) at field.hu Received: from green.field.hu ([217.20.130.28]) by localhost (green.field.hu [217.20.130.28]) (amavisd-new, port 10024) with ESMTP id AbLevQWgFPiQ; Sat, 13 Dec 2008 21:22:27 +0100 (CET) Received: from webmail.field.hu (green.field.hu [217.20.130.28]) by green.field.hu (Postfix) with ESMTP id 0DD9FB252E; Sat, 13 Dec 2008 21:22:27 +0100 (CET) Received: from 79.122.6.53 (SquirrelMail authenticated user oxy) by webmail.field.hu with HTTP; Sat, 13 Dec 2008 21:22:27 +0100 (CET) Message-ID: <3934.79.122.6.53.1229199747.squirrel@webmail.field.hu> In-Reply-To: <917871cf0812130559r6d423688q57287dd765d6edf4@mail.gmail.com> References: <20081212155023.GA82667@keira.kiwi-computer.com> <917871cf0812130559r6d423688q57287dd765d6edf4@mail.gmail.com> Date: Sat, 13 Dec 2008 21:22:27 +0100 (CET) From: oxy@field.hu To: "Ulf Lilleengen" User-Agent: SquirrelMail/1.5.1 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-2 Content-Transfer-Encoding: 8bit Cc: Michael Jung , freebsd-geom@freebsd.org Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Dec 2008 20:22:36 -0000 as i read it seems that it's useless for sync the raid after initing, i have no chance to encrypt it with geli, am i right? On Szo, December 13, 2008 14:59, Ulf Lilleengen wrote: > On Fri, Dec 12, 2008 at 5:00 PM, Michael Jung > >> wrote: >> > >> FreeBSD charon.confluentasp.com 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE >> #2: Thu Sep 4 12:06:08 EDT 2008 >> >> >> In the interest of this thread I tried to duplicate the problem. I >> created: >> >> >> 10 drives: >> D d9 State: up /dev/da9 A: 0/17366 MB >> (0%) >> D d8 State: up /dev/da8 A: 0/17366 MB >> (0%) >> D d7 State: up /dev/da7 A: 0/17366 MB >> (0%) >> D d6 State: up /dev/da6 A: 0/17366 MB >> (0%) >> D d5 State: up /dev/da5 A: 0/17366 MB >> (0%) >> D d4 State: up /dev/da4 A: 0/17366 MB >> (0%) >> D d3 State: up /dev/da3 A: 0/17366 MB >> (0%) >> D d2 State: up /dev/da2 A: 0/17366 MB >> (0%) >> D d1 State: up /dev/da1 A: 0/17366 MB >> (0%) >> D d0 State: up /dev/da0 A: 0/17366 MB >> (0%) >> >> >> 1 volume: >> V test State: up Plexes: 1 Size: 152 >> GB >> >> >> 1 plex: >> P test.p0 R5 State: up Subdisks: 10 Size: 152 >> GB >> >> >> 10 subdisks: >> S test.p0.s9 State: up D: d9 Size: 16 >> GB >> S test.p0.s8 State: up D: d8 Size: 16 >> GB >> S test.p0.s7 State: up D: d7 Size: 16 >> GB >> S test.p0.s6 State: up D: d6 Size: 16 >> GB >> S test.p0.s5 State: up D: d5 Size: 16 >> GB >> S test.p0.s4 State: up D: d4 Size: 16 >> GB >> S test.p0.s3 State: up D: d3 Size: 16 >> GB >> S test.p0.s2 State: up D: d2 Size: 16 >> GB >> S test.p0.s1 State: up D: d1 Size: 16 >> GB >> S test.p0.s0 State: up D: d0 Size: 16 >> GB >> >> >> Which I can newfs and mount >> >> >> (root@charon) /etc# mount /dev/gvinum/test /mnt >> (root@charon) /etc# df -h >> Filesystem Size Used Avail Capacity Mounted on >> /dev/ad4s1a 357G 119G 209G 36% / >> devfs 1.0K 1.0K 0B 100% /dev >> 172.0.255.28:/data/unix 1.3T 643G 559G 54% /nas1 >> /dev/gvinum/test 148G 4.0K 136G 0% /mnt >> >> >> But with /dev/gvinum/test unmounted if I try: >> >> >> (root@charon) /etc# geli init -P -K /root/test.key /dev/gvinum/test >> geli: Cannot store metadata on /dev/gvinum/test: Operation not >> permitted. (root@charon) /etc# >> >> >> My random file was created like >> >> >> dd if=/dev/random of=/root/test.key bs=64 count=1 >> >> I use GELI at home with no trouble, although not with a gvinum volume. >> >> > > Hello, > > > When I tried this myself, I also got the EPERM error in return. I though > this was very strange. I went through the gvinum code today, and put > debugging prints everywhere, but everything looked fine, and it was only > raid5 volumes > > that failed. Then I saw that the EPERM error came from the underlying > providers of geom (more specifially from the read requests to the parity > stripes etc), so I was starting to suspect that it was not a gvinum error. > But still, I was > able to write/read from the disks from outside of gvinum! > > Then, I discovered in geom userland code that it opens the disk where > metadata should be written in write only mode. Then I discovered the > reason: > gvinum tries to write to the stripe in question, but has to read back the > parity data from one of the other stripes. But, they are opened O_WRONLY, > so the request fails. I tried opening the device as O_RDWR, and everything > is find. > > Phew :) You can bet I was frustrated > > > I hope to commit the attached change in the near future. > > > -- > Ulf Lilleengen > _______________________________________________ > freebsd-geom@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-geom > To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org" > > From owner-freebsd-geom@FreeBSD.ORG Sat Dec 13 21:28:36 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 519541065673 for ; Sat, 13 Dec 2008 21:28:36 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id CFAD98FC12 for ; Sat, 13 Dec 2008 21:28:35 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 99177 invoked by uid 2001); 13 Dec 2008 21:28:35 -0000 Date: Sat, 13 Dec 2008 15:28:35 -0600 From: "Rick C. Petty" To: oxy@field.hu Message-ID: <20081213212835.GA99136@keira.kiwi-computer.com> References: <20081212155023.GA82667@keira.kiwi-computer.com> <917871cf0812130559r6d423688q57287dd765d6edf4@mail.gmail.com> <3934.79.122.6.53.1229199747.squirrel@webmail.field.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3934.79.122.6.53.1229199747.squirrel@webmail.field.hu> User-Agent: Mutt/1.4.2.3i Cc: freebsd-geom@freebsd.org Subject: Re: Encrypting raid5 volume with geli X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd2008@kiwi-computer.com List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Dec 2008 21:28:36 -0000 On Sat, Dec 13, 2008 at 09:22:27PM +0100, oxy@field.hu wrote: > as i read it seems that it's useless for sync the raid after initing, i Not sure what you mean here. RAID5 sync after creation is pretty typical. I think it many cases it's unnecessary, but the current gvinum code does perform a sync-after-create for raid5. > have no chance to encrypt it with geli, am i right? With the patch lulf@ just committed, you should be able to geli a raid5 volume under gvinum. You may have to wait for it to be MFC'd if you're not using HEAD (FreeBSD-CURRENT). You could also apply his patch to your source code and rebuild from /usr/src/sbin/gvinum/ and it should work for you. -- Rick C. Petty