From owner-freebsd-geom@FreeBSD.ORG Sun Nov 8 05:14:36 2009 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81CB6106568B for ; Sun, 8 Nov 2009 05:14:36 +0000 (UTC) (envelope-from ross@grinz.com) Received: from mail.boomhaus.com (emerson.grinz.com [64.219.233.251]) by mx1.freebsd.org (Postfix) with ESMTP id 577108FC08 for ; Sun, 8 Nov 2009 05:14:36 +0000 (UTC) Received: from [192.168.2.2] (cpe-66-25-201-82.sw.res.rr.com [66.25.201.82]) by mail.boomhaus.com (Postfix) with ESMTPSA id 726B713504 for ; Sat, 7 Nov 2009 22:33:17 -0600 (CST) Message-ID: <4AF64F7D.8020802@grinz.com> Date: Sat, 07 Nov 2009 22:56:29 -0600 From: Ross Gohlke User-Agent: Mozilla-Thunderbird 2.0.0.22 (X11/20090707) MIME-Version: 1.0 To: freebsd-geom@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: gjournal questions and observations X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Nov 2009 05:14:36 -0000 KUDOS Congratulations to all GEOM contributors. While I am new to GEOM, so far I am very impressed with the way it is designed and the capabilities (both realized and anticipated) the design offers. QUESTIONS 1. What is the best way to journal whole disks whose slices (without partitions) are used by gconcat and gmirror? Does the same apply for gvinum? The ultimate scenario seems to be journaling another GEOM class such as gmirror because gjournal handles the synchronization of all mirror consumers. You can turn off autosync on the mirror, thus saving CPU cycles and improving disk access. (Am I right?) 2. How should gjournal and gmirror be configured when the journal is outside, instead of inside, the mirror? The above scenario only seems possible if a) you are willing to journal slices, which is not best practice [1] [2] or b) you use whole disks in your mirrors, which is not very realistic. Further I am on PowerPC and don't even have bsdlabel, so journaling slices and mirroring partitions is not an option anyway. My thought was to journal each disk separately, outside the mirror, and keep autosynchronization on for the mirror. [1] http://lists.freebsd.org/pipermail/freebsd-questions/2008-April/173501.html [2] http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2008-11/msg00247.html 3. What is the best way to completely remove a whole disk journal such that re-issuing % gjournal label /dev/ad0 does not require -f? I have tried gpart destroy/create and newfs -E. I have not tried blanking the whole disk with dd, nor have I tried newfs -E on the whole disk. 4. Does it matter whether gjournal is loaded when gjournal label is issued? Originally I was journaling slices, and I was unable to properly stop a particular slice. % gjournal stop ad0s6.journal % gjournal list Showed the slice still loaded, but under a different name: ie, ufsid/48x6x1bxc39394x7 I don't know if it was related to the originally issued gjournal label command or that gjournal was loaded before the slice was labeled. I believe what fixed it was: % dd if=/dev/zero of=/dev/ad0s6 OBSERVATIONS GJOURNAL These may seem obvious to you but were hard won by me. 1. Never try to store a whole disk's journal on a slice of the same disk. 2. When journaling a whole disk and keeping the journal on the same disk, not only must the very last sector of the disk be reserved, but also the last gig (or whatever the size of your journal). Just leave it as free space when you create slices with gpart. While gjournal man page states journaling an existing file system REQUIRES a separate device for storing the journal, it appears to work without specifying a second device. At least % gjournal label -f /dev/ad0 seems to work, using the end of /dev/ad0 to store the journal whether a slice occupies those sectors or not. (Consequently, trying to gmirror the last slice when it occupies journal sectors will fail.) Regards, Ross Gohlke From owner-freebsd-geom@FreeBSD.ORG Mon Nov 9 11:06:52 2009 Return-Path: Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7CA01065695 for ; Mon, 9 Nov 2009 11:06:52 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id C58258FC15 for ; Mon, 9 Nov 2009 11:06:52 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id nA9B6qZD078996 for ; Mon, 9 Nov 2009 11:06:52 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id nA9B6qci078994 for freebsd-geom@FreeBSD.org; Mon, 9 Nov 2009 11:06:52 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 9 Nov 2009 11:06:52 GMT Message-Id: <200911091106.nA9B6qci078994@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-geom@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-geom@FreeBSD.org X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2009 11:06:52 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/140352 geom [geom] gjournal + glabel not working o kern/139847 geom [geom_mbr] load/unload causes system to hang o kern/135898 geom [geom] Severe filesystem corruption - large files or l o kern/134922 geom [gmirror] [panic] kernel panic when use fdisk on disk o kern/134113 geom [geli] Problem setting secondary GELI key o kern/134044 geom [geom] gmirror(8) overwrites fs with stale data from r o kern/133931 geom [geli] [request] intentionally wrong password to destr o bin/132845 geom [geom] [patch] ggated(8) does not close files opened a o kern/132273 geom glabel(8): [patch] failing on journaled partition f kern/132242 geom [gmirror] gmirror.ko fails to fully initialize o kern/131353 geom [geom] gjournal(8) kernel lock p docs/130548 geom [patch] gjournal(8) man page is missing sysctls o kern/129674 geom [geom] gjournal root did not mount on boot o kern/129645 geom gjournal(8): GEOM_JOURNAL causes system to fail to boo o kern/129245 geom [geom] gcache is more suitable for suffix based provid f kern/128276 geom [gmirror] machine lock up when gmirror module is used f kern/126902 geom [geom] geom_label: kernel panic during install boot o kern/124973 geom [gjournal] [patch] boot order affects geom_journal con o kern/124969 geom gvinum(8): gvinum raid5 plex does not detect missing s f kern/124294 geom [geom] gmirror(8) have inappropriate logic when workin o kern/123962 geom [panic] [gjournal] gjournal (455Gb data, 8Gb journal), o kern/123122 geom [geom] GEOM / gjournal kernel lock o kern/122738 geom [geom] gmirror list "losts consumers" after gmirror de f kern/122415 geom [geom] UFS labels are being constantly created and rem o kern/122067 geom [geom] [panic] Geom crashed during boot o kern/121559 geom [patch] [geom] geom label class allows to create inacc o kern/121364 geom [gmirror] Removing all providers create a "zombie" mir o kern/120091 geom [geom] [geli] [gjournal] geli does not prompt for pass o kern/120021 geom [geom] [panic] net-p2p/qbittorrent crashes system when o kern/119743 geom [geom] geom label for cds is keeped after dismount and o kern/115856 geom [geli] ZFS thought it was degraded when it should have o kern/115547 geom [geom] [patch] [request] let GEOM Eli get password fro o kern/114532 geom [geom] GEOM_MIRROR shows up in kldstat even if compile o kern/113957 geom [gmirror] gmirror is intermittently reporting a degrad o kern/113885 geom [gmirror] [patch] improved gmirror balance algorithm o kern/113837 geom [geom] unable to access 1024 sector size storage o kern/113419 geom [geom] geom fox multipathing not failing back p bin/110705 geom gmirror(8) control utility does not exit with correct o kern/107707 geom [geom] [patch] [request] add new class geom_xbox360 to o kern/104389 geom [geom] [patch] sys/geom/geom_dump.c doesn't encode XML o kern/98034 geom [geom] dereference of NULL pointer in acd_geom_detach o kern/94632 geom [geom] Kernel output resets input while GELI asks for o kern/90582 geom [geom] [panic] Restore cause panic string (ffs_blkfree o bin/90093 geom fdisk(8) incapable of altering in-core geometry a kern/89660 geom [vinum] [patch] [panic] due to g_malloc returning null o kern/89546 geom [geom] GEOM error o kern/88601 geom [geli] geli cause kernel panic under heavy disk usage o kern/87544 geom [gbde] mmaping large files on a gbde filesystem deadlo o kern/84556 geom [geom] [panic] GBDE-encrypted swap causes panic at shu o kern/79251 geom [2TB] newfs fails on 2.6TB gbde device o kern/79035 geom [vinum] gvinum unable to create a striped set of mirro o bin/78131 geom gbde(8) "destroy" not working. s kern/73177 geom kldload geom_* causes panic due to memory exhaustion 53 problems total. From owner-freebsd-geom@FreeBSD.ORG Mon Nov 9 16:24:42 2009 Return-Path: Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7AF56106566C for ; Mon, 9 Nov 2009 16:24:42 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 361638FC15 for ; Mon, 9 Nov 2009 16:24:42 +0000 (UTC) Received: from localhost (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id F112E19E027 for ; Mon, 9 Nov 2009 17:24:40 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 6081419E023 for ; Mon, 9 Nov 2009 17:24:38 +0100 (CET) Message-ID: <4AF84245.7070108@quip.cz> Date: Mon, 09 Nov 2009 17:24:37 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.4) Gecko/20091017 SeaMonkey/2.0 MIME-Version: 1.0 To: freebsd-geom@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: gjournal and calculation of the size of journal provider X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2009 16:24:42 -0000 What is the right rule for journal size calculation? There are two sources stating different things. 1] journal size depends on disk write speed http://lists.freebsd.org/pipermail/freebsd-fs/2006-June/002016.html "For example your disk can write at 60MB/s. Journal switch time is 10 seconds. The journal provider has to have place to keep two journals (active and inactive). So bascially you need 60*10*2MB + gjournal headers." 2] journal size depends on RAM size http://www.freebsd.org/doc/en/articles/gjournal-desktop/article.html#UNDERSTANDING-JOURNALING "Your RAM size should fit in 30% of the journal provider's space. For example, if your system has 1 GB RAM, create an approximately 3.3 GB journal provider. (Multiply your RAM size with 3.3 to obtain the size of the journal)." What's the right size for journal on 143GB 15k rpm SAS disks on machine with 16GB of RAM? Based on second case, it will be more than 50 GB - one third of the size of disk. This is insane vasting. I have gjournal on few of our machines with size of journal set to 2GB on SATA disks in gmirror. Miroslav Lachman From owner-freebsd-geom@FreeBSD.ORG Tue Nov 10 10:24:20 2009 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34F70106568B for ; Tue, 10 Nov 2009 10:24:20 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id E5D098FC1D for ; Tue, 10 Nov 2009 10:24:19 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.50) id 1N7nta-0000qh-Co for freebsd-geom@freebsd.org; Tue, 10 Nov 2009 11:24:18 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 10 Nov 2009 11:24:18 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 10 Nov 2009 11:24:18 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Tue, 10 Nov 2009 11:24:12 +0100 Lines: 32 Message-ID: References: <4AF84245.7070108@quip.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.23 (X11/20090928) In-Reply-To: <4AF84245.7070108@quip.cz> Sender: news Subject: Re: gjournal and calculation of the size of journal provider X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2009 10:24:20 -0000 Miroslav Lachman wrote: > What is the right rule for journal size calculation? > There are two sources stating different things. > > 1] journal size depends on disk write speed > http://lists.freebsd.org/pipermail/freebsd-fs/2006-June/002016.html > > "For example your disk can write > at 60MB/s. Journal switch time is 10 seconds. The journal provider has > to have place to keep two journals (active and inactive). So bascially > you need 60*10*2MB + gjournal headers." > > > 2] journal size depends on RAM size > http://www.freebsd.org/doc/en/articles/gjournal-desktop/article.html#UNDERSTANDING-JOURNALING > > > "Your RAM size should fit in 30% of the journal provider's space. For > example, if your system has 1 GB RAM, create an approximately 3.3 GB > journal provider. (Multiply your RAM size with 3.3 to obtain the size of > the journal)." > > > What's the right size for journal on 143GB 15k rpm SAS disks on machine > with 16GB of RAM? Based on second case, it will be more than 50 GB - one > third of the size of disk. This is insane vasting. It really does depend on the speed of drives but it could be approximated by saying there will not be more data to write than the size of memory (which is probably wrong since you can write from /dev/zero indefinitely). The first advice is sufficient, but you should probably extend the result by 20% to be safer. From owner-freebsd-geom@FreeBSD.ORG Tue Nov 10 13:26:23 2009 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49A3D106566C for ; Tue, 10 Nov 2009 13:26:23 +0000 (UTC) (envelope-from ross@grinz.com) Received: from mail.boomhaus.com (emerson.grinz.com [64.219.233.251]) by mx1.freebsd.org (Postfix) with ESMTP id F35978FC15 for ; Tue, 10 Nov 2009 13:26:22 +0000 (UTC) Received: from [192.168.2.2] (cpe-66-25-201-82.sw.res.rr.com [66.25.201.82]) by mail.boomhaus.com (Postfix) with ESMTPSA id 922681526B for ; Tue, 10 Nov 2009 07:02:58 -0600 (CST) Message-ID: <4AF96A13.8080306@grinz.com> Date: Tue, 10 Nov 2009 07:26:43 -0600 From: Ross Gohlke User-Agent: Mozilla-Thunderbird 2.0.0.22 (X11/20090707) MIME-Version: 1.0 To: freebsd-geom@freebsd.org References: <4AF64F7D.8020802@grinz.com> In-Reply-To: <4AF64F7D.8020802@grinz.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: gjournal questions and observations X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2009 13:26:23 -0000 > QUESTIONS > 3. What is the best way to completely remove a whole disk journal such > that re-issuing > % gjournal label /dev/ad0 > does not require -f? Blanking the beginning of the drive did the trick. % dd if=/dev/zero of=/dev/ad0 bs=1m count=600 > 1. What is the best way to journal whole disks whose slices (without > partitions) are used by gconcat and gmirror? Does the same apply for > gvinum? % gjournal label /dev/ad0 % gjournal load (Don't newfs the journal; wait for concat) % gconcat load % gconcat label ports ad0.journals11 % newfs /dev/concat/ports % mount /dev/concat/ports /mnt/ports Add new slices to the journal, not the disk. % gpart add -i 12 ad0.journal Setting up a gmirrored gconcat did not work. Another guy tried gconcatted gmirrors and had a similar result [1]. He was able to fix the problem with fsck, which did not work for me: [1] http://docs.freebsd.org/cgi/getmsg.cgi?fetch=0+0+archive/2006/freebsd-geom/20060806.freebsd-geom This was the result when trying to rsync /usr/ports to /mnt/ports (/dev/mirror/ports). #/var/log/messages ... Nov 9 17:00:51 rorty kernel: GEOM_CONCAT: Device ports created (id=1362796578). Nov 9 17:00:51 rorty kernel: GEOM_CONCAT: Disk ad0s11 attached to ports. Nov 9 17:00:51 rorty kernel: GEOM_CONCAT: Device ports activated. Nov 9 17:00:51 rorty kernel: GEOM_MIRROR: Device ports: provider concat/ports marked as inactive, skipping. Nov 9 17:00:51 rorty kernel: GEOM_MIRROR: Device ports: provider ufsid/4af8442b031fbe48 marked as inactive, skipping. Nov 9 17:00:59 rorty kernel: GEOM_MIRROR: Device mirror/ports launched (1/1). Nov 9 17:01:57 rorty kernel: ad1: TIMEOUT - READ_MUL retrying (1 retry left) LBA=116413904 Nov 9 17:02:50 rorty kernel: ad1: TIMEOUT - READ_MUL retrying (1 retry left) LBA=115497232 Nov 9 17:02:50 rorty kernel: ad0: TIMEOUT - WRITE_MUL retrying (1 retry left) LBA=19936658 Nov 9 17:02:55 rorty kernel: ad1: TIMEOUT - READ_MUL retrying (1 retry left) LBA=115498352 Nov 9 17:03:10 rorty kernel: g_vfs_done():mirror/ports[WRITE(offset=1073758208, length=2048)]error = 5 Nov 9 17:03:10 rorty kernel: g_vfs_done():mirror/ports[WRITE(offset=1073760256, length=2048)]error = 5 Nov 9 17:03:10 rorty kernel: g_vfs_done():mirror/ports[WRITE(offset=1073762304, length=2048)]error = 5 Nov 9 17:03:41 rorty kernel: initiate_write_filepage: already started Nov 9 17:03:41 rorty kernel: initiga_tvef_sw_rdiotnee_(f)i:lmeiprargoer:/ paolrrtesa[dWyR IsTtEa(rotfefdset Nov 9 17:03:41 rorty kernel: =1i0n7i3t7i6a2t3e0_4w,r ilteen_gftihl=e2p0a4g8e):] earlrroera d=y 5st Nov 9 17:03:41 rorty kernel: artedg_v Nov 9 17:03:41 rorty kernel: fs_done():mirror/ports[WRITE(offset=1073760256, length=2048)]error = 5 Nov 9 17:03:41 rorty kernel: g_vfs_done():mirror/ports[WRITE(offset=1073758208, length=2048)]error = 5 Nov 9 17:03:42 rorty kernel: initiate_write_filepage: already started Nov 9 17:03:42 rorty kernel: initiate_write_filepage: already started Nov 9 17:03:42 rorty kernel: initiate_wgr_ivtfes__fdiolneep(a)g:em:i rarlorre/apdoyr tsst[aWrRtIeTdE(o Nov 9 17:03:42 rorty kernel: ffset=1073758208, length=2048)]error = 5 Nov 9 17:03:42 rorty kernel: g_vfs_done():mirror/ports[WRITE(offset=1073760256, length=2048)]error = 5 Nov 9 17:03:42 rorty kernel: g_vfs_done():mirror/ports[WRITE(offset=1073762304, length=2048)]error = 5 Nov 9 17:04:12 rorty kernel: initiate_write_filepage: already started Nov 9 17:04:12 rorty kernel: ingit_ivaftse__dworniet(e)_:fmiilrerpoarg/ep:o ratlsr[eWaRdIyT Es(toafrftseedt=1 Nov 9 17:04:12 rorty kernel: 073i7n6i2t3i0a4t,e _lwernigtteh_=fi2l0e4p8a)g]ee:r raolrr e=a d5y Nov 9 17:04:12 rorty kernel: startge_dvf Nov 9 17:04:12 rorty kernel: s_done():mirror/ports[WRIiTnEi(toifaftsee_tw=ri1t0e7_3f7i6l0e2p5a6g,e :l eanlgrteha=d2y0 4s8t)$ Nov 9 17:04:12 rorty kernel: = 5 Nov 9 17:04:12 rorty kernel: g_vfs_done():mirror/ports[WRIT ... I have ruled out the disk being bad because it works fine with gmirror OR gconcat, just not both. Regards, Ross From owner-freebsd-geom@FreeBSD.ORG Tue Nov 10 19:02:20 2009 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2185106568F for ; Tue, 10 Nov 2009 19:02:20 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077035009.chello.pl [89.77.35.9]) by mx1.freebsd.org (Postfix) with ESMTP id E14AE8FC17 for ; Tue, 10 Nov 2009 19:02:19 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 8B86845E8F; Tue, 10 Nov 2009 20:02:18 +0100 (CET) Received: from localhost (chello089077035009.chello.pl [89.77.35.9]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id D3EAB45E82; Tue, 10 Nov 2009 20:02:12 +0100 (CET) Date: Tue, 10 Nov 2009 20:02:13 +0100 From: Pawel Jakub Dawidek To: Ross Gohlke Message-ID: <20091110190213.GB3194@garage.freebsd.pl> References: <4AF64F7D.8020802@grinz.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Y7xTucakfITjPcLV" Content-Disposition: inline In-Reply-To: <4AF64F7D.8020802@grinz.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-geom@freebsd.org Subject: Re: gjournal questions and observations X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2009 19:02:20 -0000 --Y7xTucakfITjPcLV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Nov 07, 2009 at 10:56:29PM -0600, Ross Gohlke wrote: > KUDOS > Congratulations to all GEOM contributors. While I am new to GEOM, so far= =20 > I am very impressed with the way it is designed and the capabilities=20 > (both realized and anticipated) the design offers. >=20 > QUESTIONS > 1. What is the best way to journal whole disks whose slices (without=20 > partitions) are used by gconcat and gmirror? Does the same apply for gvin= um? > The ultimate scenario seems to be journaling another GEOM class such as= =20 > gmirror because gjournal handles the synchronization of all mirror=20 > consumers. You can turn off autosync on the mirror, thus saving CPU=20 > cycles and improving disk access. (Am I right?) You should always gjournal top-most provider, so you always put UFS on top of .journal provider. Don't do anything with .journal besides of file system configuration. > 2. How should gjournal and gmirror be configured when the journal is=20 > outside, instead of inside, the mirror? > The above scenario only seems possible if a) you are willing to journal= =20 > slices, which is not best practice [1] [2] or b) you use whole disks in= =20 > your mirrors, which is not very realistic. > Further I am on PowerPC and don't even have bsdlabel, so journaling=20 > slices and mirroring partitions is not an option anyway. > My thought was to journal each disk separately, outside the mirror, and= =20 > keep autosynchronization on for the mirror. >=20 > [1]=20 > http://lists.freebsd.org/pipermail/freebsd-questions/2008-April/173501.ht= ml > [2]=20 > http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2008-11/msg0024= 7.html See above. You can safely gmirror disks, slices or partition and put gjournal on top of gmirror and file system on top of gjournal. > 3. What is the best way to completely remove a whole disk journal such=20 > that re-issuing > % gjournal label /dev/ad0 > does not require -f? > I have tried gpart destroy/create and newfs -E. I have not tried=20 > blanking the whole disk with dd, nor have I tried newfs -E on the whole= =20 > disk. gjournal stop .journal (or ); gjournal clear > 4. Does it matter whether gjournal is loaded when gjournal label is issue= d? > Originally I was journaling slices, and I was unable to properly stop a= =20 > particular slice. > % gjournal stop ad0s6.journal > % gjournal list > Showed the slice still loaded, but under a different name: > ie, ufsid/48x6x1bxc39394x7 You provider is accessible by few different names. This ufsid/ thing (which I don't like) is one of them. Once you stop gjournal on one name it is recreated using another name. Besides of using -h option to gjournal label and hardcoding provider name there is not much we can do. > While gjournal man page states journaling an existing file system=20 > REQUIRES a separate device for storing the journal, it appears to work=20 > without specifying a second device. At least > % gjournal label -f /dev/ad0 > seems to work, using the end of /dev/ad0 to store the journal whether a= =20 > slice occupies those sectors or not. (Consequently, trying to gmirror=20 > the last slice when it occupies journal sectors will fail.) It will eventually work until your UFS will start to use space gjournal is using for journal. Absolutely don't do that. Its like creating 4GB file system on 3GB provider - at some point you will need the missing 1GB.. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --Y7xTucakfITjPcLV Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFK+bi1ForvXbEpPzQRAp5jAKDkRwmq4/XZpAnuMUH+P2I55i7HXgCg0qUh 8HZEy8nEoesLwXkwcuiKus4= =qo5a -----END PGP SIGNATURE----- --Y7xTucakfITjPcLV-- From owner-freebsd-geom@FreeBSD.ORG Wed Nov 11 15:59:10 2009 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7A1D1065676; Wed, 11 Nov 2009 15:59:10 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id A206E8FC2A; Wed, 11 Nov 2009 15:59:10 +0000 (UTC) Received: from localhost (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 5F0F219E02A; Wed, 11 Nov 2009 16:59:09 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id BAAF419E023; Wed, 11 Nov 2009 16:59:06 +0100 (CET) Message-ID: <4AFADF4A.80404@quip.cz> Date: Wed, 11 Nov 2009 16:59:06 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.4) Gecko/20091017 SeaMonkey/2.0 MIME-Version: 1.0 To: Ivan Voras References: <4AF84245.7070108@quip.cz> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-geom@freebsd.org Subject: Re: gjournal and calculation of the size of journal provider X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2009 15:59:11 -0000 Ivan Voras wrote: > Miroslav Lachman wrote: >> What is the right rule for journal size calculation? >> There are two sources stating different things. >> >> 1] journal size depends on disk write speed >> http://lists.freebsd.org/pipermail/freebsd-fs/2006-June/002016.html >> >> "For example your disk can write >> at 60MB/s. Journal switch time is 10 seconds. The journal provider has >> to have place to keep two journals (active and inactive). So bascially >> you need 60*10*2MB + gjournal headers." >> >> >> 2] journal size depends on RAM size >> http://www.freebsd.org/doc/en/articles/gjournal-desktop/article.html#UNDERSTANDING-JOURNALING >> >> >> "Your RAM size should fit in 30% of the journal provider's space. For >> example, if your system has 1 GB RAM, create an approximately 3.3 GB >> journal provider. (Multiply your RAM size with 3.3 to obtain the size >> of the journal)." >> >> >> What's the right size for journal on 143GB 15k rpm SAS disks on >> machine with 16GB of RAM? Based on second case, it will be more than >> 50 GB - one third of the size of disk. This is insane vasting. > > It really does depend on the speed of drives but it could be > approximated by saying there will not be more data to write than the > size of memory (which is probably wrong since you can write from > /dev/zero indefinitely). The first advice is sufficient, but you should > probably extend the result by 20% to be safer. So is it safe to use 4GB on PERC6 array, which is capable of 150MB/s write speed by dd test? dd if=/dev/zero of=/dev/mfid0s2e bs=1m count=10000 (150 * 10 * 2 * 1.2) = 3600 150 is write speed in MB/s 10 is journal switch time 2 is active + inactive journal 1.2 is +20% to base safer And next question about journal. I saw following message in log after reboot: GEOM_JOURNAL: Journal 1933335573: mfid0s2d contains journal. GEOM_JOURNAL: Journal 1933335573: mfid0s2e contains data. GEOM_JOURNAL: Journal mfid0s2e clean. GEOM_JOURNAL: BIO_FLUSH not supported by mfid0s2d. GEOM_JOURNAL: BIO_FLUSH not supported by mfid0s2e. "BIO_FLUSH not supported" - is it OK to use gjournal on top of the Dell PERC (LSI MegaRAID) with battery backup unit? I think so, but rather ask somebody... :) Miroslav Lachman From owner-freebsd-geom@FreeBSD.ORG Wed Nov 11 16:07:09 2009 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 007DD106566C for ; Wed, 11 Nov 2009 16:07:09 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id AE2B68FC1A for ; Wed, 11 Nov 2009 16:07:08 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.50) id 1N8Fis-0002z0-AW for freebsd-geom@freebsd.org; Wed, 11 Nov 2009 17:07:06 +0100 Received: from 78-1-155-113.adsl.net.t-com.hr ([78.1.155.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 11 Nov 2009 17:07:06 +0100 Received: from ivoras by 78-1-155-113.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 11 Nov 2009 17:07:06 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Wed, 11 Nov 2009 17:06:38 +0100 Lines: 64 Message-ID: References: <4AF84245.7070108@quip.cz> <4AFADF4A.80404@quip.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 78-1-155-113.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.21 (X11/20090612) In-Reply-To: <4AFADF4A.80404@quip.cz> Sender: news Subject: Re: gjournal and calculation of the size of journal provider X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2009 16:07:09 -0000 Miroslav Lachman wrote: > Ivan Voras wrote: >> Miroslav Lachman wrote: >>> What is the right rule for journal size calculation? >>> There are two sources stating different things. >>> >>> 1] journal size depends on disk write speed >>> http://lists.freebsd.org/pipermail/freebsd-fs/2006-June/002016.html >>> >>> "For example your disk can write >>> at 60MB/s. Journal switch time is 10 seconds. The journal provider has >>> to have place to keep two journals (active and inactive). So bascially >>> you need 60*10*2MB + gjournal headers." >>> >>> >>> 2] journal size depends on RAM size >>> http://www.freebsd.org/doc/en/articles/gjournal-desktop/article.html#UNDERSTANDING-JOURNALING >>> >>> >>> >>> "Your RAM size should fit in 30% of the journal provider's space. For >>> example, if your system has 1 GB RAM, create an approximately 3.3 GB >>> journal provider. (Multiply your RAM size with 3.3 to obtain the size >>> of the journal)." >>> >>> >>> What's the right size for journal on 143GB 15k rpm SAS disks on >>> machine with 16GB of RAM? Based on second case, it will be more than >>> 50 GB - one third of the size of disk. This is insane vasting. >> >> It really does depend on the speed of drives but it could be >> approximated by saying there will not be more data to write than the >> size of memory (which is probably wrong since you can write from >> /dev/zero indefinitely). The first advice is sufficient, but you should >> probably extend the result by 20% to be safer. > > > So is it safe to use 4GB on PERC6 array, which is capable of 150MB/s > write speed by dd test? > dd if=/dev/zero of=/dev/mfid0s2e bs=1m count=10000 > > (150 * 10 * 2 * 1.2) = 3600 > > 150 is write speed in MB/s > 10 is journal switch time > 2 is active + inactive journal > 1.2 is +20% to base safer This looks fine! > And next question about journal. I saw following message in log after > reboot: > > GEOM_JOURNAL: Journal 1933335573: mfid0s2d contains journal. > GEOM_JOURNAL: Journal 1933335573: mfid0s2e contains data. > GEOM_JOURNAL: Journal mfid0s2e clean. > GEOM_JOURNAL: BIO_FLUSH not supported by mfid0s2d. > GEOM_JOURNAL: BIO_FLUSH not supported by mfid0s2e. > > "BIO_FLUSH not supported" - is it OK to use gjournal on top of the Dell > PERC (LSI MegaRAID) with battery backup unit? I think so, but rather ask > somebody... :) I think you are safe if the controller has the BBU working and enabled. From owner-freebsd-geom@FreeBSD.ORG Thu Nov 12 15:17:25 2009 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A4441065734 for ; Thu, 12 Nov 2009 15:17:25 +0000 (UTC) (envelope-from tony@crosswinds.net) Received: from out-mx.crosswinds.net (out-mx1.crosswinds.net [8.21.33.38]) by mx1.freebsd.org (Postfix) with ESMTP id 329A88FC18 for ; Thu, 12 Nov 2009 15:17:24 +0000 (UTC) Received: from admin.crosswinds.net (admin.crosswinds.net [8.21.33.2]) by out-mx.crosswinds.net (Postfix) with ESMTP id 4D63E1E8E35 for ; Thu, 12 Nov 2009 10:17:24 -0500 (EST) Received: by admin.crosswinds.net (Postfix, from userid 1001) id 355FD86182; Thu, 12 Nov 2009 10:17:53 -0500 (EST) Date: Thu, 12 Nov 2009 10:17:53 -0500 From: Tony Holmes To: freebsd-geom@freebsd.org Message-ID: <20091112151753.GA20087@crosswinds.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.18 (2008-05-17) Subject: GJournal too Small? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 15:17:25 -0000 I have a 894GB gstripe that I've put gjournal on top of. Since it was an unused stripe I placed the data+journal into the same partition. The server is fairly heavily used and hung suddenly. On reboot I got this in the dmesg: GEOM_JOURNAL: Journal 3472355975: mirror/gm0s1e contains journal. GEOM_STRIPE: Device st0 created (id=2649322337). GEOM_STRIPE: Disk mirror/gm0s1f attached to st0. GEOM_MIRROR: Device mirror/gm1 launched (1/1). GEOM_STRIPE: Disk mirror/gm1s1f attached to st0. GEOM_STRIPE: Device st0 activated. GEOM_JOURNAL: Journal 2630378703: stripe/st0 contains data. GEOM_JOURNAL: Journal 2630378703: stripe/st0 contains journal. GEOM_JOURNAL: Journal stripe/st0 clean. GEOM_JOURNAL: Timeout. Journal gjournal 3472355975 cannot be completed. That last line worries me. >From a quick google, it appears that the gjournal is too small. Since I created it with the single partition, I would have expected the journal to be autosized correctly. I know the OS is a little out of date but has been working very well until the past couple months. I have 1 hang approximately every month. Information about system: FreeBSD fs.cwahi.net 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #0: Mon Dec 1 09:12:42 EST 2008 root@app.cwahi.com:/usr/obj/usr/src/sys/CWahi amd64 fs# gjournal list Geom name: gjournal 1493846988 ID: 1493846988 Providers: 1. Name: stripe/st0.journal Mediasize: 958334500352 (893G) Sectorsize: 512 Mode: r1w1e1 Consumers: 1. Name: stripe/st0 Mediasize: 959408242688 (894G) Sectorsize: 512 Mode: r1w1e1 Jend: 959408242176 Jstart: 958334500352 Role: Data,Journal Newfs'd with: newfs -J -b 16384 -f 2048 -m 4 /dev/stripe/st0.journal -- Tony Holmes Ph: (416) 993-1219 Founder and Senior Systems Architect Crosswinds Internet Communications Inc. From owner-freebsd-geom@FreeBSD.ORG Thu Nov 12 21:18:33 2009 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB8EE10656CE for ; Thu, 12 Nov 2009 21:18:33 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 3C8258FC16 for ; Thu, 12 Nov 2009 21:18:33 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.50) id 1N8h3n-00014p-JM for freebsd-geom@freebsd.org; Thu, 12 Nov 2009 22:18:31 +0100 Received: from 93-138-63-251.adsl.net.t-com.hr ([93.138.63.251]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 12 Nov 2009 22:18:31 +0100 Received: from ivoras by 93-138-63-251.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 12 Nov 2009 22:18:31 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Thu, 12 Nov 2009 22:18:02 +0100 Lines: 53 Message-ID: References: <20091112151753.GA20087@crosswinds.net> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 93-138-63-251.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.21 (X11/20090612) In-Reply-To: <20091112151753.GA20087@crosswinds.net> Sender: news Subject: Re: GJournal too Small? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 21:18:33 -0000 Tony Holmes wrote: > I have a 894GB gstripe that I've put gjournal on top of. Since it was > an unused stripe I placed the data+journal into the same partition. > The server is fairly heavily used and hung suddenly. On reboot I got > this in the dmesg: > > GEOM_JOURNAL: Journal 3472355975: mirror/gm0s1e contains journal. > GEOM_STRIPE: Device st0 created (id=2649322337). > GEOM_STRIPE: Disk mirror/gm0s1f attached to st0. > GEOM_MIRROR: Device mirror/gm1 launched (1/1). > GEOM_STRIPE: Disk mirror/gm1s1f attached to st0. > GEOM_STRIPE: Device st0 activated. > GEOM_JOURNAL: Journal 2630378703: stripe/st0 contains data. > GEOM_JOURNAL: Journal 2630378703: stripe/st0 contains journal. > GEOM_JOURNAL: Journal stripe/st0 clean. > GEOM_JOURNAL: Timeout. Journal gjournal 3472355975 cannot be completed. > > That last line worries me. > >>From a quick google, it appears that the gjournal is too small. Since > I created it with the single partition, I would have expected the journal > to be autosized correctly. The message "Timeout. Journal %s cannot be completed." is printed when the gjournal composite device is created with data and journal on separate providers. It means that a timeout occurred while gjournal waited for both providers to come online. Your message contains something that looks like a journal ID (3472355975) which isn't in the code in 7-stable and 8-stable/head. How did you get that line? Assuming magic has happened and the journal ID (3472355975) is correct then it means you have two gjournal devices, one of those created on the stripe st0 (2630378703). It could mean that somehow, there is still recognizable metadata on your drives and/or partitions which confuses gjournal. > I know the OS is a little out of date but has been working very well until > the past couple months. I have 1 hang approximately every month. > > Information about system: > > FreeBSD fs.cwahi.net 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #0: Mon Dec 1 09:12:42 EST 2008 root@app.cwahi.com:/usr/obj/usr/src/sys/CWahi amd64 > > fs# gjournal list > Geom name: gjournal 1493846988 > ID: 1493846988 This, on the other hand, is a third gjournal ID. Assuming that somehow all this information is correct, you should probably send the output of sysctl -b kern.geom.confxml before anyone can unravel what has happened :)