From owner-freebsd-fs@FreeBSD.ORG Sun Mar 22 16:09:29 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E1D711065673 for ; Sun, 22 Mar 2009 16:09:29 +0000 (UTC) (envelope-from gsutter@zer0.org) Received: from kyanos.zer0.org (kyanos.zer0.org [149.20.54.45]) by mx1.freebsd.org (Postfix) with ESMTP id C47368FC12 for ; Sun, 22 Mar 2009 16:09:29 +0000 (UTC) (envelope-from gsutter@zer0.org) Received: from localhost (localhost [127.0.0.1]) by kyanos.zer0.org (Postfix) with ESMTP id 3E567126366B; Sun, 22 Mar 2009 05:05:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at zer0.org Received: from kyanos.zer0.org ([127.0.0.1]) by localhost (kyanos.zer0.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id DeNSsehInk2A; Sun, 22 Mar 2009 05:05:22 -0700 (PDT) Received: by kyanos.zer0.org (Postfix, from userid 1001) id D7D341263667; Sun, 22 Mar 2009 05:05:21 -0700 (PDT) Date: Sun, 22 Mar 2009 05:05:21 -0700 From: Gregory Sutter To: Pawel Jakub Dawidek Message-ID: <20090322120521.GA5584@kyanos.zer0.org> References: <20090311215518.GA25410@gvr.gvr.org> <20090312195904.GA1786@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="yrj/dFKFPuw6o+aM" Content-Disposition: inline In-Reply-To: <20090312195904.GA1786@garage.freebsd.pl> Organization: Zer0 X-Purpose: For great justice! Mail-Copies-To: poster X-PGP-Fingerprint: D161 E4EA 4BFA 2427 F3F9 5B1F 2015 31D5 845D FEDD X-PGP-Key: http://zer0.org/~gsutter/gsutter.pgp User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-fs@freebsd.org Subject: Re: RFC: geli+gmirror problem + solution X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Mar 2009 16:09:30 -0000 --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2009-03-12 20:59 +0100, Pawel Jakub Dawidek wrote: > On Wed, Mar 11, 2009 at 10:55:18PM +0100, Guido van Rooij wrote: > > > > We have a setup where we have two disks in a gmirror with GELI on top > > of it. We boot from it (using a small unencrypted USB stick). > > We this have the G_ELI_FLAG_BOOT set. > >=20 > > Anyway, when rebooting, geli does not detach and this the gmirror is > > never destroyed resulting in a rebuild each time. >=20 > Maybe you could implement detaching on reboot > for geli providers? You can find example of how to do this in three last > functions in sys/geom/mirror/g_mirror.c. Has this been filed as a PR, and has any progress been made on a fix? I couldn't find anything exact in GNATS using the expected search terms, but I did find some resemblances: kern/119293: a similar-looking situation with GBDE on a gmirror kern/113957: a case in which gmirror sometimes reports degraded arrays As it happens, I just ran into this problem earlier today. My gmirror is still rebuilding from the incident. It was a disconcerting thing to have happen upon the first reboot after setting up the new storage. (Until I read this thread, I didn't know which layer had the problem; I've got quite the onion of geoms set up in /dev/mirror/gm0s1e.eli.journal.) It seems like a workaround could be to: - add the shutdown keyword to the geli rc script. The script already has a geli_stop function. and - use 'shutdown', send a signal to init, or make the 3-finger salute instead of using 'halt' or 'reboot', which don't use init and don't call rc.shutdown. --=20 Gregory S. Sutter A little nonsense now and then mailto:gsutter@zer0.org is relished by the wisest men. http://zer0.org/~gsutter/ --Roald Dahl (as Willy Wonka) --yrj/dFKFPuw6o+aM Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFJximBIBUx1YRd/t0RAm77AJ9GBw477mj4IR5jN+248ptZBYf4uACfbJ+2 U7nHNVM5oqrx3TBXIihG7kU= =U/07 -----END PGP SIGNATURE----- --yrj/dFKFPuw6o+aM-- From owner-freebsd-fs@FreeBSD.ORG Sun Mar 22 17:27:20 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 77E37106566C for ; Sun, 22 Mar 2009 17:27:20 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206045082.chello.pl [87.206.45.82]) by mx1.freebsd.org (Postfix) with ESMTP id A45388FC13 for ; Sun, 22 Mar 2009 17:27:19 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 81EC64569A; Sun, 22 Mar 2009 18:27:17 +0100 (CET) Received: from localhost (chello087206045082.chello.pl [87.206.45.82]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id C419A45683; Sun, 22 Mar 2009 18:27:11 +0100 (CET) Date: Sun, 22 Mar 2009 18:27:49 +0100 From: Pawel Jakub Dawidek To: Gregory Sutter Message-ID: <20090322172749.GJ3102@garage.freebsd.pl> References: <20090311215518.GA25410@gvr.gvr.org> <20090312195904.GA1786@garage.freebsd.pl> <20090322120521.GA5584@kyanos.zer0.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/Zw+/jwnNHcBRYYu" Content-Disposition: inline In-Reply-To: <20090322120521.GA5584@kyanos.zer0.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: RFC: geli+gmirror problem + solution X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Mar 2009 17:27:20 -0000 --/Zw+/jwnNHcBRYYu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Mar 22, 2009 at 05:05:21AM -0700, Gregory Sutter wrote: > On 2009-03-12 20:59 +0100, Pawel Jakub Dawidek wrote: > > On Wed, Mar 11, 2009 at 10:55:18PM +0100, Guido van Rooij wrote: > > > > > > We have a setup where we have two disks in a gmirror with GELI on top > > > of it. We boot from it (using a small unencrypted USB stick). > > > We this have the G_ELI_FLAG_BOOT set. > > >=20 > > > Anyway, when rebooting, geli does not detach and this the gmirror is > > > never destroyed resulting in a rebuild each time. > >=20 > > Maybe you could implement detaching on reboot > > for geli providers? You can find example of how to do this in three last > > functions in sys/geom/mirror/g_mirror.c. >=20 > Has this been filed as a PR, and has any progress been made on a fix? The rest of the discussion was made private (probably by accident), sorry about that. Fix is already committed to HEAD. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --/Zw+/jwnNHcBRYYu Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFJxnUVForvXbEpPzQRAjPnAKCxE2i7sLiEx33euhAelneP0LWWtQCfTP7Z OhQjoWqDYrKC7Cfivjb7+Wg= =7WdH -----END PGP SIGNATURE----- --/Zw+/jwnNHcBRYYu-- From owner-freebsd-fs@FreeBSD.ORG Sun Mar 22 19:43:36 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 273391065745; Sun, 22 Mar 2009 19:43:36 +0000 (UTC) (envelope-from gsutter@zer0.org) Received: from kyanos.zer0.org (kyanos.zer0.org [149.20.54.45]) by mx1.freebsd.org (Postfix) with ESMTP id 06E3F8FC24; Sun, 22 Mar 2009 19:43:35 +0000 (UTC) (envelope-from gsutter@zer0.org) Received: from localhost (localhost [127.0.0.1]) by kyanos.zer0.org (Postfix) with ESMTP id E0A19126364E; Sun, 22 Mar 2009 12:43:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at zer0.org Received: from kyanos.zer0.org ([127.0.0.1]) by localhost (kyanos.zer0.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id oHdOozrFKOeK; Sun, 22 Mar 2009 12:43:35 -0700 (PDT) Received: by kyanos.zer0.org (Postfix, from userid 1001) id 8221F1259C5D; Sun, 22 Mar 2009 12:43:35 -0700 (PDT) Date: Sun, 22 Mar 2009 12:43:35 -0700 From: Gregory Sutter To: Pawel Jakub Dawidek Message-ID: <20090322194335.GL2075@kyanos.zer0.org> References: <20090311215518.GA25410@gvr.gvr.org> <20090312195904.GA1786@garage.freebsd.pl> <20090322120521.GA5584@kyanos.zer0.org> <20090322172749.GJ3102@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/aHqdxxe3Vo3WJ36" Content-Disposition: inline In-Reply-To: <20090322172749.GJ3102@garage.freebsd.pl> Organization: Zer0 X-Purpose: For great justice! Mail-Copies-To: poster X-PGP-Fingerprint: D161 E4EA 4BFA 2427 F3F9 5B1F 2015 31D5 845D FEDD X-PGP-Key: http://zer0.org/~gsutter/gsutter.pgp User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-fs@freebsd.org Subject: Re: RFC: geli+gmirror problem + solution X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Mar 2009 19:43:38 -0000 --/aHqdxxe3Vo3WJ36 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2009-03-22 18:27 +0100, Pawel Jakub Dawidek wrote: > On Sun, Mar 22, 2009 at 05:05:21AM -0700, Gregory Sutter wrote: > > On 2009-03-12 20:59 +0100, Pawel Jakub Dawidek wrote: > > > On Wed, Mar 11, 2009 at 10:55:18PM +0100, Guido van Rooij wrote: > > > > > > > > Anyway, when rebooting, geli does not detach and this the gmirror is > > > > never destroyed resulting in a rebuild each time. > > >=20 > > > Maybe you could implement detaching on reboot > > > for geli providers? You can find example of how to do this in three l= ast > > > functions in sys/geom/mirror/g_mirror.c. > >=20 > > Has this been filed as a PR, and has any progress been made on a fix? >=20 > The rest of the discussion was made private (probably by accident), > sorry about that. Fix is already committed to HEAD. Woot. Thanks for committing that! This fix looks like a straightforward MFC: http://svnweb.freebsd.org/viewvc/base/head/sys/geom/eli/g_eli.c?r1=3D172506= &r2=3D189900 Do you agree, and if so, will you please MFC to 7-STABLE? That's where I'm encountering the problem. --=20 Gregory S. Sutter Mostly Harmless mailto:gsutter@zer0.org=20 http://zer0.org/~gsutter/=20 --/aHqdxxe3Vo3WJ36 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFJxpTnIBUx1YRd/t0RAsp1AJ42Fb8dT65dyoooTw45fkWeQIfSYgCeNK2S LMl9ZjK1DbVr5zuva91rI8g= =w2fw -----END PGP SIGNATURE----- --/aHqdxxe3Vo3WJ36-- From owner-freebsd-fs@FreeBSD.ORG Sun Mar 22 20:34:02 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2E1C106566C for ; Sun, 22 Mar 2009 20:34:02 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206045082.chello.pl [87.206.45.82]) by mx1.freebsd.org (Postfix) with ESMTP id 2A1278FC08 for ; Sun, 22 Mar 2009 20:34:01 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4D12B45685; Sun, 22 Mar 2009 21:33:59 +0100 (CET) Received: from localhost (chello087206045082.chello.pl [87.206.45.82]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 4FFEA45683; Sun, 22 Mar 2009 21:33:50 +0100 (CET) Date: Sun, 22 Mar 2009 21:34:27 +0100 From: Pawel Jakub Dawidek To: Gregory Sutter Message-ID: <20090322203427.GL3102@garage.freebsd.pl> References: <20090311215518.GA25410@gvr.gvr.org> <20090312195904.GA1786@garage.freebsd.pl> <20090322120521.GA5584@kyanos.zer0.org> <20090322172749.GJ3102@garage.freebsd.pl> <20090322194335.GL2075@kyanos.zer0.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="svExV93C05KqedWb" Content-Disposition: inline In-Reply-To: <20090322194335.GL2075@kyanos.zer0.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: RFC: geli+gmirror problem + solution X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Mar 2009 20:34:03 -0000 --svExV93C05KqedWb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Mar 22, 2009 at 12:43:35PM -0700, Gregory Sutter wrote: > On 2009-03-22 18:27 +0100, Pawel Jakub Dawidek wrote: > > On Sun, Mar 22, 2009 at 05:05:21AM -0700, Gregory Sutter wrote: > > > On 2009-03-12 20:59 +0100, Pawel Jakub Dawidek wrot= e: > > > > On Wed, Mar 11, 2009 at 10:55:18PM +0100, Guido van Rooij wrote: > > > > > > > > > > Anyway, when rebooting, geli does not detach and this the gmirror= is > > > > > never destroyed resulting in a rebuild each time. > > > >=20 > > > > Maybe you could implement detaching on reboot > > > > for geli providers? You can find example of how to do this in three= last > > > > functions in sys/geom/mirror/g_mirror.c. > > >=20 > > > Has this been filed as a PR, and has any progress been made on a fix? > >=20 > > The rest of the discussion was made private (probably by accident), > > sorry about that. Fix is already committed to HEAD. >=20 > Woot. Thanks for committing that! >=20 > This fix looks like a straightforward MFC: > http://svnweb.freebsd.org/viewvc/base/head/sys/geom/eli/g_eli.c?r1=3D1725= 06&r2=3D189900 >=20 > Do you agree, and if so, will you please MFC to 7-STABLE? That's > where I'm encountering the problem. Let it settle for a week in HEAD, then I'll do MFC. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --svExV93C05KqedWb Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFJxqDTForvXbEpPzQRAtpZAKCol+D6SxC/ztlPxmC10L4sMbIA7wCgw+l/ JlQeJ7OagIWTj88YdCabbuo= =n/7g -----END PGP SIGNATURE----- --svExV93C05KqedWb-- From owner-freebsd-fs@FreeBSD.ORG Mon Mar 23 07:59:22 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2BC61065673; Mon, 23 Mar 2009 07:59:22 +0000 (UTC) (envelope-from guido@gvr.org) Received: from gvr.gvr.org (gvr-gw.gvr.org [82.95.154.195]) by mx1.freebsd.org (Postfix) with ESMTP id A47EC8FC0A; Mon, 23 Mar 2009 07:59:22 +0000 (UTC) (envelope-from guido@gvr.org) Received: by gvr.gvr.org (Postfix, from userid 657) id 46E8142D82F; Mon, 23 Mar 2009 08:59:21 +0100 (CET) Date: Mon, 23 Mar 2009 08:59:21 +0100 From: Guido van Rooij To: Pawel Jakub Dawidek Message-ID: <20090323075921.GA38261@gvr.gvr.org> References: <20090311215518.GA25410@gvr.gvr.org> <20090312195904.GA1786@garage.freebsd.pl> <20090322120521.GA5584@kyanos.zer0.org> <20090322172749.GJ3102@garage.freebsd.pl> <20090322194335.GL2075@kyanos.zer0.org> <20090322203427.GL3102@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090322203427.GL3102@garage.freebsd.pl> Cc: freebsd-fs@freebsd.org Subject: Re: RFC: geli+gmirror problem + solution X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2009 07:59:23 -0000 On Sun, Mar 22, 2009 at 09:34:27PM +0100, Pawel Jakub Dawidek wrote: > > > > This fix looks like a straightforward MFC: > > http://svnweb.freebsd.org/viewvc/base/head/sys/geom/eli/g_eli.c?r1=172506&r2=189900 > > > > Do you agree, and if so, will you please MFC to 7-STABLE? That's > > where I'm encountering the problem. > > Let it settle for a week in HEAD, then I'll do MFC. FYI: I am running it on 7-RELEASE without any problem. -Guido From owner-freebsd-fs@FreeBSD.ORG Mon Mar 23 11:06:55 2009 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC790106564A for ; Mon, 23 Mar 2009 11:06:55 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 996618FC31 for ; Mon, 23 Mar 2009 11:06:55 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2NB6tvB003985 for ; Mon, 23 Mar 2009 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2NB6tQU003981 for freebsd-fs@FreeBSD.org; Mon, 23 Mar 2009 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 23 Mar 2009 11:06:55 GMT Message-Id: <200903231106.n2NB6tQU003981@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2009 11:06:56 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/132597 fs [tmpfs] [panic] tmpfs-related panic while interrupting o kern/132551 fs [zfs] ZFS locks up on extattr_list_link syscall o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132337 fs [zfs] [panic] kernel panic in zfs_fuid_create_cred o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132145 fs [panic] File System Hard Crashes f kern/132068 fs [zfs] page fault when using ZFS over NFS on 7.1-RELEAS o kern/131995 fs [nfs] Failure to mount NFSv4 server o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/131086 fs [ext2fs] mkfs.ext2 creates rotten partition o kern/131084 fs [xfs] xfs destroys itself after copying data o kern/131081 fs [zfs] User cannot delete a file when a ZFS dataset is o kern/130979 fs [smbfs] [panic] boot/kernel/smbfs.ko o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130229 fs [iconv] usermount fails on fs that need iconv o kern/130210 fs [nullfs] Error by check nullfs o bin/130105 fs [zfs] zfs send -R dumps core o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129174 fs [nfs] [zfs] [panic] NFS v3 Panic when under high load o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/129084 fs [udf] [panic] [lor] udf panic: getblk: size(67584) > M f kern/128829 fs smbd(8) causes periodic panic on 7-RELEASE o kern/128633 fs [zfs] [lor] lock order reversal in zfs o kern/128514 fs [zfs] [mpt] problems with ZFS and LSILogic SAS/SATA Ad f kern/128173 fs [ext2fs] ls gives "Input/output error" on mounted ext3 o kern/127420 fs [gjournal] [panic] Journal overflow on gmirrored gjour o kern/127213 fs [tmpfs] sendfile on tmpfs data corruption o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file f kern/125536 fs [ext2fs] ext 2 mounts cleanly but fails on commands li o kern/125149 fs [nfs] [panic] changing into .zfs dir from nfs client c f kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition o kern/122888 fs [zfs] zfs hang w/ prefetch on, zil off while running t o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o bin/118249 fs mv(1): moving a directory changes its mtime o kern/116170 fs [panic] Kernel panic when mounting /tmp o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po 49 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Mar 23 13:34:41 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 613141065672; Mon, 23 Mar 2009 13:34:41 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3669E8FC17; Mon, 23 Mar 2009 13:34:41 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (gavin@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2NDYfMP020201; Mon, 23 Mar 2009 13:34:41 GMT (envelope-from gavin@freefall.freebsd.org) Received: (from gavin@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2NDYf4k020197; Mon, 23 Mar 2009 13:34:41 GMT (envelope-from gavin) Date: Mon, 23 Mar 2009 13:34:41 GMT Message-Id: <200903231334.n2NDYf4k020197@freefall.freebsd.org> To: onemda@gmail.com, gavin@FreeBSD.org, freebsd-fs@FreeBSD.org From: gavin@FreeBSD.org Cc: Subject: Re: kern/129084: [udf] [panic] [lor] udf panic: getblk: size(67584) > MAXBSIZE(65536) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2009 13:34:41 -0000 Synopsis: [udf] [panic] [lor] udf panic: getblk: size(67584) > MAXBSIZE(65536) State-Changed-From-To: open->closed State-Changed-By: gavin State-Changed-When: Mon Mar 23 13:34:07 UTC 2009 State-Changed-Why: This has been fixed and merged to 7.x (confirmed by avg on IRC) http://www.freebsd.org/cgi/query-pr.cgi?pr=129084 From owner-freebsd-fs@FreeBSD.ORG Mon Mar 23 16:40:03 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DA1B4106566B for ; Mon, 23 Mar 2009 16:40:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AD8BD8FC0A for ; Mon, 23 Mar 2009 16:40:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2NGe3qf066432 for ; Mon, 23 Mar 2009 16:40:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2NGe3sj066425; Mon, 23 Mar 2009 16:40:03 GMT (envelope-from gnats) Date: Mon, 23 Mar 2009 16:40:03 GMT Message-Id: <200903231640.n2NGe3sj066425@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Jaakko Heinonen Cc: Subject: Re: bin/131341: makefs: error "Bad file descriptor" on the mount point of md-presentation makefs image X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Jaakko Heinonen List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2009 16:40:04 -0000 The following reply was made to PR bin/131341; it has been noted by GNATS. From: Jaakko Heinonen To: oleg.ginzburg@nevosoft.ru Cc: bug-followup@FreeBSD.org Subject: Re: bin/131341: makefs: error "Bad file descriptor" on the mount point of md-presentation makefs image Date: Mon, 23 Mar 2009 18:31:30 +0200 See this message: http://docs.freebsd.org/cgi/mid.cgi?20090323154536.GA2853 and a patch: http://www.saunalahti.fi/~jh3/patches/ffs-fs_fsbtodb.diff -- Jaakko From owner-freebsd-fs@FreeBSD.ORG Mon Mar 23 16:49:11 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 70C351065680 for ; Mon, 23 Mar 2009 16:49:11 +0000 (UTC) (envelope-from mel.flynn+fbsd.fs@mailing.thruhere.net) Received: from mailhub.rachie.is-a-geek.net (rachie.is-a-geek.net [66.230.99.27]) by mx1.freebsd.org (Postfix) with ESMTP id 423E58FC15 for ; Mon, 23 Mar 2009 16:49:11 +0000 (UTC) (envelope-from mel.flynn+fbsd.fs@mailing.thruhere.net) Received: from sarevok.dnr.servegame.org (gate.lan.rachie.is-a-geek.net [192.168.2.10]) by mailhub.rachie.is-a-geek.net (Postfix) with ESMTP id 5CC177E818 for ; Mon, 23 Mar 2009 08:33:53 -0800 (AKDT) From: Mel Flynn To: freebsd-fs@freebsd.org Date: Mon, 23 Mar 2009 17:33:50 +0100 User-Agent: KMail/1.11.0 (FreeBSD/8.0-CURRENT; KDE/4.2.0; i386; ; ) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200903231733.51671.mel.flynn+fbsd.fs@mailing.thruhere.net> Subject: Trying to understand how aio(4), mount(8) async (and gjournal) relate X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2009 16:49:11 -0000 Hello list, in an effort to try and understand the various async systems available, it's unclear to me how (if at all) the above relate. If one mounts a disk with async, does this internally use aio system calls, or is there a subsystem available which does largely the same? If so, why are there 2 subsystems? When using aio, for example with squid, does this mean the underlying provider needs to be mounted async or is this totally unrelated? Similarly if said disk is on a gjournalled partition, is the async mount redundant or is using aio redundant or neither? The systems themselves are properly documented - I just couldn't find any info on how they relate, or why they are unrelated. -- Mel From owner-freebsd-fs@FreeBSD.ORG Mon Mar 23 17:30:31 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B7171065670 for ; Mon, 23 Mar 2009 17:30:31 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outY.internet-mail-service.net (outy.internet-mail-service.net [216.240.47.248]) by mx1.freebsd.org (Postfix) with ESMTP id 127DD8FC1A for ; Mon, 23 Mar 2009 17:30:30 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 10FB0B980F; Mon, 23 Mar 2009 10:18:25 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id E6E442D6006; Mon, 23 Mar 2009 10:18:02 -0700 (PDT) Message-ID: <49C7C45B.7040708@elischer.org> Date: Mon, 23 Mar 2009 10:18:19 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Mel Flynn References: <200903231733.51671.mel.flynn+fbsd.fs@mailing.thruhere.net> In-Reply-To: <200903231733.51671.mel.flynn+fbsd.fs@mailing.thruhere.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Trying to understand how aio(4), mount(8) async (and gjournal) relate X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2009 17:30:31 -0000 Mel Flynn wrote: > Hello list, > > in an effort to try and understand the various async systems available, it's > unclear to me how (if at all) the above relate. > > If one mounts a disk with async, does this internally use aio system calls, or > is there a subsystem available which does largely the same? If so, why are > there 2 subsystems? the async mount operation tells the systewm that writes done by a process may be actually written to teh filesystem at any time later, and that it is ok to return "success" to the user immediately, regardless of whether the write has actually been done. This also applies to the metadata. It does not affect reads at all. > > When using aio, for example with squid, does this mean the underlying provider > needs to be mounted async or is this totally unrelated? Similarly if said disk > is on a gjournalled partition, is the async mount redundant or is using aio > redundant or neither? > no the aio system calls are implemented in a manner that allows the caller to request IO and then return at a later time to find out the result. it has no connection to the mount option. BTW for efficient async IO check out the relationship between the aio calls and the kqueue calls. > The systems themselves are properly documented - I just couldn't find any info > on how they relate, or why they are unrelated. From owner-freebsd-fs@FreeBSD.ORG Mon Mar 23 18:34:26 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2DFB106566C; Mon, 23 Mar 2009 18:34:26 +0000 (UTC) (envelope-from chris@young-alumni.com) Received: from mail.oldschoolpunx.net (cpe-72-177-10-243.austin.res.rr.com [72.177.10.243]) by mx1.freebsd.org (Postfix) with ESMTP id AAE668FC12; Mon, 23 Mar 2009 18:34:26 +0000 (UTC) (envelope-from chris@young-alumni.com) Received: by mail.oldschoolpunx.net (Postfix, from userid 58) id D96FA96E54; Mon, 23 Mar 2009 13:18:05 -0500 (CDT) Received: from [192.168.8.100] (unknown [192.168.8.100]) by mail.oldschoolpunx.net (Postfix) with ESMTPSA id 340AB96DBB; Mon, 23 Mar 2009 13:14:12 -0500 (CDT) Message-Id: <9DCF097D-421A-4F5F-8A48-D0286551C62C@young-alumni.com> From: Chris Ruiz To: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org In-Reply-To: <5f67a8c40903161109le12b8afuc25b8c1ec1b6f70c@mail.gmail.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Mon, 23 Mar 2009 13:14:10 -0500 References: <78cb3d3f0903151209r46837d70m914a23e30a19060e@mail.gmail.com> <4AE4493D5E9141E8812E4BC83FB5A2A5@PegaPegII> <5f67a8c40903161109le12b8afuc25b8c1ec1b6f70c@mail.gmail.com> X-Mailer: Apple Mail (2.930.3) Cc: Subject: Re: ETA for ZFS v. 13 Merge From HEAD ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2009 18:34:27 -0000 On Mar 16, 2009, at 1:09 PM, Zaphod Beeblebrox wrote: > On Sun, Mar 15, 2009 at 6:39 PM, Pegasus Mc Cleaft > wrote: > >> Hi Adrian, >> >> I am not sure, but I didnt think ZFS 13 was ever going to be >> merged into >> 7-stable. I thought the kernel memory requirements were to great >> (just going >> back in my memory on that one). Also, I think there are still a few >> bugs >> left with the zil being enabled (and/or prefetch) causing lockups >> on machine >> with a lot of IO. I know I have hit that bug a few times on my >> machine when >> using various torrent clients when they want to preallocate large >> amounts of >> diskspace. >> >> I personally cant wait until a later version of ZFS is imported that >> supports encryption. I can finally say good-bye to our GEOM ELI USB >> drives >> for backups!! Never the less, I am quite thankfull to thoes >> involved in >> porting V13 to FreeBSD. Its a wonderfull improvement and my FS of >> choice >> when installing on new machines (especially zfs boot) > > > I think that you're touching on two entirely separate points here... > What it > takes to upgrade ZFS in -STABLE and what it takes to bring ZFS > modules in to > FreeBSD. > > I sincerely hope that ZFSv13 is planned for -STABLE. Last we left > this > issue, testing and a few kernel improvements were in the way. None > of the > kernel improvements were going to change the API, so the project was > doable > in -STABLE. That said, time marches on, 8.0-RELEASE draws ever > nearer. > When we were still several years out on 8.0 and ZFS was causing me > more > problems, I was much more keen to push for the port. I would still > welcome > it with open arms, but I'm not convinced that anyone is going to > push it > forward. > > The issue of encryption (along with many other issues) is tied to the > ability of FreeBSD to compile and use ZFS modules. Just like netgraph > modules extend the function of netgraph.ko and geom modules extend > the base > geom function, ZFS is designed (in Solaris, at least) to take > modules. ZFS > encryption is a module. I'm not clear on compression --- it would > make > sense that it is a module, but it seemingly got copied into FreeBSD > as a > core feature (and it may also be so in solaris). > > Anyways... is there any plans to allow for ZFS modules in FreeBSD? AFAIK ZFS v13 requires changes to the kernel that would break the ABI, which is not allowed to change in a STABLE branch. With 8.0 coming within the next 6 months, I doubt that 7 will see a new version of ZFS. There are no problems running ZFS v13 with zil and prefetch enabled and I have not had any predictable out of kernel memory panics. For me, ZFS on CURRENT really is *that* much better. Also, OpenSolaris has yet to integrate ZFS on disk encryption into their source. The code is currently under review: http://opensolaris.org/os/project/zfs-crypto/ . OpenSolaris uses ZFS v14 now and on disk encryption will probably be synced to a newer version of ZFS, meaning that this would require another code sync with OpenSolaris. Chris From owner-freebsd-fs@FreeBSD.ORG Mon Mar 23 18:43:49 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79A33106564A; Mon, 23 Mar 2009 18:43:49 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 4F79A8FC16; Mon, 23 Mar 2009 18:43:49 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2NIhnUr041178; Mon, 23 Mar 2009 18:43:49 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2NIhnd5041174; Mon, 23 Mar 2009 18:43:49 GMT (envelope-from linimon) Date: Mon, 23 Mar 2009 18:43:49 GMT Message-Id: <200903231843.n2NIhnd5041174@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/132960: [zfs] [panic] panic:ffs_blkfree: freeing free frag X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2009 18:43:49 -0000 Old Synopsis: panic:ffs_blkfree: freeing free frag New Synopsis: [zfs] [panic] panic:ffs_blkfree: freeing free frag Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Mon Mar 23 18:43:01 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=132960 From owner-freebsd-fs@FreeBSD.ORG Mon Mar 23 21:09:19 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D5A710656BF; Mon, 23 Mar 2009 21:09:19 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from yx-out-2324.google.com (yx-out-2324.google.com [74.125.44.28]) by mx1.freebsd.org (Postfix) with ESMTP id 055E18FC17; Mon, 23 Mar 2009 21:09:18 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by yx-out-2324.google.com with SMTP id 8so1384247yxm.13 for ; Mon, 23 Mar 2009 14:09:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=RqvEaNtr0G+lRWmt5xB//Pjto3zMCblYprLM0t7AqvQ=; b=n7tvbVpWoVTPXcx3VXwYEXFzdmT/gCoynqMxLAhH/jm7DfKXEzgds3O8UN3Iwq9MTj 0q+tXq/K7xX+E/G2agZlyhLEyfZYUREKStJ9nRCpTdslAaK+HZJNw8ChcvN0MrLjX+MO AOcJOUMcAt8EVNUSmsYi0Mt6lid2ZWk8xw3m0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=dBMqF17hCvUpaBMPHChybEsKjRanSiu2xxAhs/ZdMVlicPUXD+4GAaEjuAW0EvRF/b zx29MAGiNjLtc0U0+NnKF9Jx6pzKk64bMYfERP7wf8cb6Kx+kcfy6HyesXQ15XST4uVO e2cAC2GDBQih8SsSk3z1drmr1w4HF0BuY9ksk= MIME-Version: 1.0 Received: by 10.151.45.2 with SMTP id x2mr11920744ybj.176.1237842558273; Mon, 23 Mar 2009 14:09:18 -0700 (PDT) In-Reply-To: <9DCF097D-421A-4F5F-8A48-D0286551C62C@young-alumni.com> References: <78cb3d3f0903151209r46837d70m914a23e30a19060e@mail.gmail.com> <4AE4493D5E9141E8812E4BC83FB5A2A5@PegaPegII> <5f67a8c40903161109le12b8afuc25b8c1ec1b6f70c@mail.gmail.com> <9DCF097D-421A-4F5F-8A48-D0286551C62C@young-alumni.com> Date: Mon, 23 Mar 2009 17:09:18 -0400 Message-ID: <5f67a8c40903231409q17fd0370wd2907525b8b6aff0@mail.gmail.com> From: Zaphod Beeblebrox To: Chris Ruiz Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: ETA for ZFS v. 13 Merge From HEAD ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2009 21:09:20 -0000 On Mon, Mar 23, 2009 at 2:14 PM, Chris Ruiz wrote: > > AFAIK ZFS v13 requires changes to the kernel that would break the ABI, > which is not allowed to change in a STABLE branch. With 8.0 coming within > the next 6 months, I doubt that 7 will see a new version of ZFS. Can we have someone who actually knows comment on this requirement? The last time this was discussed, I didn't hear this conclusion --- that ZFS 13 required ABI breaking kernel changes. > Also, OpenSolaris has yet to integrate ZFS on disk encryption into their > source. The code is currently under review: > http://opensolaris.org/os/project/zfs-crypto/ . OpenSolaris uses ZFS v14 > now and on disk encryption will probably be synced to a newer version of > ZFS, meaning that this would require another code sync with OpenSolaris. It should be said that importing updates from OpenSolaris needs to be easier. From owner-freebsd-fs@FreeBSD.ORG Mon Mar 23 21:31:30 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04955106568E for ; Mon, 23 Mar 2009 21:31:30 +0000 (UTC) (envelope-from mel.flynn+fbsd.fs@mailing.thruhere.net) Received: from mailhub.rachie.is-a-geek.net (rachie.is-a-geek.net [66.230.99.27]) by mx1.freebsd.org (Postfix) with ESMTP id A23A88FC0C for ; Mon, 23 Mar 2009 21:31:28 +0000 (UTC) (envelope-from mel.flynn+fbsd.fs@mailing.thruhere.net) Received: from sarevok.dnr.servegame.org (gate.lan.rachie.is-a-geek.net [192.168.2.10]) by mailhub.rachie.is-a-geek.net (Postfix) with ESMTP id 86C947E81A; Mon, 23 Mar 2009 13:31:27 -0800 (AKDT) From: Mel Flynn To: freebsd-fs@freebsd.org Date: Mon, 23 Mar 2009 22:31:26 +0100 User-Agent: KMail/1.11.0 (FreeBSD/8.0-CURRENT; KDE/4.2.0; i386; ; ) References: <200903231733.51671.mel.flynn+fbsd.fs@mailing.thruhere.net> <49C7C45B.7040708@elischer.org> In-Reply-To: <49C7C45B.7040708@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200903232231.26279.mel.flynn+fbsd.fs@mailing.thruhere.net> Cc: Julian Elischer Subject: Re: Trying to understand how aio(4), mount(8) async (and gjournal) relate X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2009 21:31:30 -0000 On Monday 23 March 2009 18:18:19 Julian Elischer wrote: > Mel Flynn wrote: > > Hello list, > > > > in an effort to try and understand the various async systems available, > > it's unclear to me how (if at all) the above relate. > > > > If one mounts a disk with async, does this internally use aio system > > calls, or is there a subsystem available which does largely the same? If > > so, why are there 2 subsystems? > > the async mount operation tells the systewm that writes done by a > process may be actually written to teh filesystem at any time later, > and that it is ok to return "success" to the user immediately, > regardless of whether the write has actually been done. > This also applies to the metadata. It does not affect reads at all. So it is up to the underlying geom provider to handle the write queue? > > > When using aio, for example with squid, does this mean the underlying > > provider needs to be mounted async or is this totally unrelated? > > Similarly if said disk is on a gjournalled partition, is the async mount > > redundant or is using aio redundant or neither? > > no the aio system calls are implemented in a manner that allows > the caller to request IO and then return at a later time to find out > the result. it has no connection to the mount option. So with aio_write(2) on an async gjournal fs, one faces a write queue on the fd used by aio(4) along with another made by gjournal? This leads me to believe that it's better to take one of those out of the equation for performance. > BTW for efficient async IO check out the relationship between the aio > calls and the kqueue calls. Thanks for the pointer, reading up ... -- Mel From owner-freebsd-fs@FreeBSD.ORG Mon Mar 23 23:45:48 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 754761065750 for ; Mon, 23 Mar 2009 23:45:48 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outQ.internet-mail-service.net (outq.internet-mail-service.net [216.240.47.240]) by mx1.freebsd.org (Postfix) with ESMTP id 559A38FC0A for ; Mon, 23 Mar 2009 23:45:48 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 54C1328BD3; Mon, 23 Mar 2009 16:45:54 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id C451D2D6017; Mon, 23 Mar 2009 16:45:45 -0700 (PDT) Message-ID: <49C81F3A.7010207@elischer.org> Date: Mon, 23 Mar 2009 16:46:02 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Mel Flynn References: <200903231733.51671.mel.flynn+fbsd.fs@mailing.thruhere.net> <49C7C45B.7040708@elischer.org> <200903232231.26279.mel.flynn+fbsd.fs@mailing.thruhere.net> In-Reply-To: <200903232231.26279.mel.flynn+fbsd.fs@mailing.thruhere.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Trying to understand how aio(4), mount(8) async (and gjournal) relate X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2009 23:45:51 -0000 Mel Flynn wrote: > On Monday 23 March 2009 18:18:19 Julian Elischer wrote: >> Mel Flynn wrote: >>> Hello list, >>> >>> in an effort to try and understand the various async systems available, >>> it's unclear to me how (if at all) the above relate. >>> >>> If one mounts a disk with async, does this internally use aio system >>> calls, or is there a subsystem available which does largely the same? If >>> so, why are there 2 subsystems? >> the async mount operation tells the systewm that writes done by a >> process may be actually written to teh filesystem at any time later, >> and that it is ok to return "success" to the user immediately, >> regardless of whether the write has actually been done. >> This also applies to the metadata. It does not affect reads at all. > > So it is up to the underlying geom provider to handle the write queue? The filesystem actually makes the I/O requests at a "later time". > >>> When using aio, for example with squid, does this mean the underlying >>> provider needs to be mounted async or is this totally unrelated? >>> Similarly if said disk is on a gjournalled partition, is the async mount >>> redundant or is using aio redundant or neither? >> no the aio system calls are implemented in a manner that allows >> the caller to request IO and then return at a later time to find out >> the result. it has no connection to the mount option. > > So with aio_write(2) on an async gjournal fs, one faces a write queue on the > fd used by aio(4) along with another made by gjournal? This leads me to > believe that it's better to take one of those out of the equation for > performance. oh, where to begin.... you do the aio operation. It is queued in the aio system, (unless it is acted upon immediately by the AIO thread) your thread then returns immediately. The aio thread associated with your process continues by (though the filesystem) requesting that the IO be done. The IO system (geom) queues the IO request and (probably immediately) starts operation on it. which results in the IO request being made to the hardware. On completion a notification is made by the driver to the geom layer which trickles the info back up to the AIO system which in turn notifies your process (when it asks) of the result of the operation. however, you have been working all (or much of) this time, so throughput and latency are not too bad. Notice that teh data is not necessarily queued everywhere if there is no work in front of it blovking the way. (I'm not a geom expert so that part is not too clear to me). async mounting plays no real part in this. > >> BTW for efficient async IO check out the relationship between the aio >> calls and the kqueue calls. > > Thanks for the pointer, reading up ... > From owner-freebsd-fs@FreeBSD.ORG Tue Mar 24 02:02:46 2009 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2D82106566B; Tue, 24 Mar 2009 02:02:46 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id F28608FC14; Tue, 24 Mar 2009 02:02:45 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2O1P7li059909; Mon, 23 Mar 2009 21:25:09 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C83673.3000604@aldan.algebra.com> Date: Mon, 23 Mar 2009 21:25:07 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: fs@FreeBSD.org, stable@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Mar 2009 02:02:47 -0000 Hello! I'm trying to migrate a filesystem from one disk to another using: dump a0hCf 0 32 - /old | restore -rf - (/old is already mounted read-only). The process runs for a while and then stops with: [...] DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 2009 DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 2009 DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 2009 unknown tape header type 1853384566 abort? [yn] Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD's dump's output? The system runs 7.0-STABLE from July 6th, i386. I just tried updating the dump and restore from source (latest 7.x) -- the error is the same... Please, advise. Thanks! -mi From owner-freebsd-fs@FreeBSD.ORG Tue Mar 24 05:41:36 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1FE0B1065670; Tue, 24 Mar 2009 05:41:36 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 3DC2D8FC08; Tue, 24 Mar 2009 05:41:34 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from inchoate.localnet (Inchoate.gsoft.com.au [203.31.81.30]) (authenticated bits=0) by cain.gsoft.com.au (8.13.8/8.13.8) with ESMTP id n2O57eNC023273 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Mar 2009 15:37:40 +1030 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: freebsd-stable@freebsd.org Date: Tue, 24 Mar 2009 15:37:10 +1030 User-Agent: KMail/1.10.4 (Linux/2.6.27-11-generic; KDE/4.1.4; i686; ; ) References: <49C83673.3000604@aldan.algebra.com> In-Reply-To: <49C83673.3000604@aldan.algebra.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1505730.BJrJCmrapF"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200903241537.36515.doconnor@gsoft.com.au> X-Spam-Score: -3.977 () ALL_TRUSTED,BAYES_00 X-Scanned-By: MIMEDefang 2.63 on 203.31.81.10 Cc: "Mikhail T." , stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Mar 2009 05:41:36 -0000 --nextPart1505730.BJrJCmrapF Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Tuesday 24 March 2009 11:55:07 Mikhail T. wrote: > I'm trying to migrate a filesystem from one disk to another using: > > dump a0hCf 0 32 - /old | restore -rf - > > (/old is already mounted read-only). The process runs for a while and > then stops with: > > [...] > DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 2009 > DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 2009 > DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 2009 > unknown tape header type 1853384566 > abort? [yn] > > Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD's > dump's output? What happens if you don't use the cache? Also, do you really want -h 0? That means you skip nodump marked files at a= =20 level 0 dump. =2D-=20 Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --nextPart1505730.BJrJCmrapF Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iD8DBQBJyGqB5ZPcIHs/zowRAlITAJ4s4JYFDaJAS6e5352mS0BvTdJ2jQCeJZXl lg+Tg9fdpzt2RajAmXyLN2I= =sX+i -----END PGP SIGNATURE----- --nextPart1505730.BJrJCmrapF-- From owner-freebsd-fs@FreeBSD.ORG Tue Mar 24 06:30:46 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6944B106566C for ; Tue, 24 Mar 2009 06:30:46 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id D9CAD8FC16 for ; Tue, 24 Mar 2009 06:30:43 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2O6Ubag060682; Tue, 24 Mar 2009 02:30:39 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C87E0D.5090501@aldan.algebra.com> Date: Tue, 24 Mar 2009 02:30:37 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: "Daniel O'Connor" References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> In-Reply-To: <200903241537.36515.doconnor@gsoft.com.au> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Mar 2009 06:30:46 -0000 Daniel O'Connor ΞΑΠΙΣΑΧ(ΜΑ): > On Tuesday 24 March 2009 11:55:07 Mikhail T. wrote: > >> I'm trying to migrate a filesystem from one disk to another using: >> >> dump a0hCf 0 32 - /old | restore -rf - >> >> (/old is already mounted read-only). The process runs for a while and >> then stops with: >> >> [...] >> DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 2009 >> DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 2009 >> DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 2009 >> unknown tape header type 1853384566 >> abort? [yn] >> >> Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD's >> dump's output? >> > > What happens if you don't use the cache? > No big difference: dump a0f - /old | restore -rf - [...] DUMP: 17.25% done, finished in 3:27 at Tue Mar 24 05:42:00 2009 DUMP: 20.36% done, finished in 3:09 at Tue Mar 24 05:28:13 2009 DUMP: 23.83% done, finished in 2:50 at Tue Mar 24 05:14:32 2009 unknown tape header type -621260722 abort? [yn] Looks like a junk value somewhere... Unitialized variable or some such. -mi From owner-freebsd-fs@FreeBSD.ORG Tue Mar 24 08:15:59 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A579106564A for ; Tue, 24 Mar 2009 08:15:59 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id C5BBC8FC1C for ; Tue, 24 Mar 2009 08:15:58 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1Lm1J9-000Ah6-Kf; Tue, 24 Mar 2009 09:44:23 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: "Mikhail T." In-reply-to: <49C87E0D.5090501@aldan.algebra.com> References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> Comments: In-reply-to "Mikhail T." message dated "Tue, 24 Mar 2009 02:30:37 -0400." Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Date: Tue, 24 Mar 2009 09:44:23 +0200 From: Danny Braniss Message-ID: Cc: Daniel O'Connor , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Mar 2009 08:15:59 -0000 > Daniel O'Connor =CE=C1=D0=C9=D3=C1=D7(=CC=C1): >=20 > On Tuesday 24 March 2009 11:55:07 Mikhail T. wrote: >=20 > =20 > >> I'm trying to migrate a filesystem from one disk to another using: > >> > >> dump a0hCf 0 32 - /old =7C restore -rf - > >> > >> (/old is already mounted read-only). The process runs for a while an= d> >> then stops with: > >> > >> =5B...=5D > >> DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 200= 9 > >> DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 200= 9 > >> DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 200= 9 > >> unknown tape header type 1853384566> >> abort? =5Byn=5D > >> > >> Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD'= s > >> dump's output? > >> =20 > >> > What happens if you don't use the cache? > > =20 > No big difference: > > dump a0f - /old =7C restore -rf - > =5B...=5D > DUMP: 17.25% done, finished in 3:27 at Tue Mar 24 05:42:00 2009 > DUMP: 20.36% done, finished in 3:09 at Tue Mar 24 05:28:13 2009 > DUMP: 23.83% done, finished in 2:50 at Tue Mar 24 05:14:32 2009 > unknown tape header type -621260722> abort? =5Byn=5D >=20 > Looks like a junk value somewhere... Unitialized variable or some such.= >=20 can you try splitting it in 2, ie no pipe? dump a0f some.file /old (or dump 0f - /old =7C gzip -c > file.dump.gz) restore rf some.file danny From owner-freebsd-fs@FreeBSD.ORG Tue Mar 24 12:41:14 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42E221065672 for ; Tue, 24 Mar 2009 12:41:14 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from fallbackmx07.syd.optusnet.com.au (fallbackmx07.syd.optusnet.com.au [211.29.132.9]) by mx1.freebsd.org (Postfix) with ESMTP id 3D0F38FC18 for ; Tue, 24 Mar 2009 12:41:13 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail04.syd.optusnet.com.au (mail04.syd.optusnet.com.au [211.29.132.185]) by fallbackmx07.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n2OCJ0nE021627 for ; Tue, 24 Mar 2009 23:19:00 +1100 Received: from besplex.bde.org (c122-107-120-227.carlnfd1.nsw.optusnet.com.au [122.107.120.227]) by mail04.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n2OCInZJ005236 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Mar 2009 23:18:52 +1100 Date: Tue, 24 Mar 2009 23:18:49 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Julian Elischer In-Reply-To: <49C7C45B.7040708@elischer.org> Message-ID: <20090324224001.D1670@besplex.bde.org> References: <200903231733.51671.mel.flynn+fbsd.fs@mailing.thruhere.net> <49C7C45B.7040708@elischer.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: Trying to understand how aio(4), mount(8) async (and gjournal) relate X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Mar 2009 12:41:14 -0000 On Mon, 23 Mar 2009, Julian Elischer wrote: > Mel Flynn wrote: >> If one mounts a disk with async, does this internally use aio system calls, >> or is there a subsystem available which does largely the same? If so, why >> are there 2 subsystems? > > the async mount operation tells the systewm that writes done by a process may > be actually written to teh filesystem at any time later, and that it is ok to > return "success" to the user immediately, regardless of whether the write has > actually been done. > This also applies to the metadata. It does not affect reads at all. Actually, this only (*) applies to the metadata. The async mount option does not affect writes, except for the metadata part of writes and for a small affect on the amount of asyncness of writes. The data part of writes is always (*) written asynchronously and "success" is returned to the user immediately, irrespective of whether the write has actually been done. (Success may be returned even if the write was attempted but failed. Writes are retried endlessly to a fault, so the data for a failed writes is not always lost. (The usual fault is panicing when the disk goes away.) It is necessary to use fsync() or the still-unimplemnted POSIX interface fdatasync() to ensure that the data has been written or get a report of non-success.) (*) Not quite only or always, since (1) The async mount option is incompatible with soft updates and is silently ignored if soft updates is configured. Soft updates essentially gives async everything, with stronger ordering of writes so that the file system is always consistent (but often out of date). (2) The sync mount option may be used to force synchronous writes of data. This option should give synchronous everything, but when it is mixed with the async mount option (and that option is not ignored) it gives the weird and undesirable behaviour of sync writes of data and async writes of metadata. >> When using aio, for example with squid, does this mean the underlying >> provider needs to be mounted async or is this totally unrelated? Similarly >> if said disk is on a gjournalled partition, is the async mount redundant or >> is using aio redundant or neither? > > no the aio system calls are implemented in a manner that allows > the caller to request IO and then return at a later time to find out > the result. it has no connection to the mount option. Yes, essentially Unix kernels always had and used async writes, and the relatively new and quite different aio interface lets applications do async writes too. Bruce From owner-freebsd-fs@FreeBSD.ORG Tue Mar 24 15:08:59 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E675910656BA for ; Tue, 24 Mar 2009 15:08:59 +0000 (UTC) (envelope-from hali@datapipe.com) Received: from EXFESMQ01.datapipe-corp.net (exfesmq01.datapipe-corp.net [64.106.130.69]) by mx1.freebsd.org (Postfix) with ESMTP id ACB178FC17 for ; Tue, 24 Mar 2009 15:08:59 +0000 (UTC) (envelope-from hali@datapipe.com) Received: from datapipe.net (192.168.128.20) by EXFESMQ01.datapipe-corp.net (64.106.130.71) with Microsoft SMTP Server (TLS) id 8.1.340.0; Tue, 24 Mar 2009 10:57:47 -0400 Date: Tue, 24 Mar 2009 10:58:41 -0400 From: Hussain Ali To: Message-ID: <20090324145841.GD17811@datapipe.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline User-Agent: mutt-ng/devel-r804 (Linux) Content-Transfer-Encoding: quoted-printable Subject: ZFS move from FreeBSD to Opensolaris ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Mar 2009 15:09:03 -0000 Hello, Was wondering if anyone has had success in migrating an external zpool from FreeBSD 7.x to OpenSolaris. Currently, I am testing this out with Vmware and while I can export/ import the pool on FreeBSD VM's , I haven't had much success with in Vmware to date : freebsd# zpool import pool: tank id: 7702735926967895150 state: ONLINE action: The pool can be imported using its name or numeric identifier. config: tank ONLINE da1 ONLINE da2 ONLINE da3 ONLINE da4 ONLINE opensolaris:~# zpool import pool: tank id: 7702735926967895150 state: UNAVAIL status: One or more devices contains corrupted data. action: The pool cannot be imported due to damaged devices or data. see: http://www.sun.com/msg/ZFS-8000-5E config: tank UNAVAIL insufficient replicas c4t1d0s8 UNAVAIL corrupted data c4t2d0s2 UNAVAIL corrupted data c4t3d0s8 UNAVAIL corrupted data c4t4d0s8 UNAVAIL corrupted data On a tangent, I can go from Mac -> OpenSolaris. I am guessing this maybe due to lack of EFI/GPT labels on the geom providers. This may also be an issue with how vmware does virtual disks, so I am going to test out some real hardware, but if anyone else has first hand experience I would appreciate the input. -- -hussain This message may contain confidential or privileged information. If you ar= e not the intended recipient, please advise us immediately and delete this = message. See http://www.datapipe.com/emaildisclaimer.aspx for further info= rmation on confidentiality and the risks of non-secure electronic communica= tion. If you cannot access these links, please notify us by reply message a= nd we will send the contents to you. From owner-freebsd-fs@FreeBSD.ORG Tue Mar 24 16:20:26 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 667401065D39; Tue, 24 Mar 2009 16:20:26 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id 912738FC12; Tue, 24 Mar 2009 16:20:25 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2OGJVeD065408; Tue, 24 Mar 2009 12:19:40 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C90813.8070705@aldan.algebra.com> Date: Tue, 24 Mar 2009 12:19:31 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: Danny Braniss References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> In-Reply-To: Content-Type: text/plain; charset=KOI8-U; format=flowed Content-Transfer-Encoding: 8bit Cc: Daniel O'Connor , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Mar 2009 16:20:31 -0000 Danny Braniss ΞΑΠΙΣΑΧ(ΜΑ): >> Daniel O'Connor ΞΑΠΙΣΑΧ(ΜΑ): >> >> On Tuesday 24 March 2009 11:55:07 Mikhail T. wrote: >> >> >> >>>> I'm trying to migrate a filesystem from one disk to another using: >>>> >>>> dump a0hCf 0 32 - /old | restore -rf - >>>> >>>> (/old is already mounted read-only). The process runs for a while and> >> then stops with: >>>> >>>> [...] >>>> DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 2009 >>>> DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 2009 >>>> DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 2009 >>>> unknown tape header type 1853384566> >> abort? [yn] >>>> >>>> Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD's >>>> dump's output? >>>> >>>> >>>> > > What happens if you don't use the cache? > >>> >>> >> No big difference: >> >>> dump a0f - /old | restore -rf - >>> >> [...] >> DUMP: 17.25% done, finished in 3:27 at Tue Mar 24 05:42:00 2009 >> DUMP: 20.36% done, finished in 3:09 at Tue Mar 24 05:28:13 2009 >> DUMP: 23.83% done, finished in 2:50 at Tue Mar 24 05:14:32 2009 >> unknown tape header type -621260722> abort? [yn] >> >> Looks like a junk value somewhere... Unitialized variable or some such. >> >> > can you try splitting it in 2, ie no pipe? > dump a0f some.file /old (or dump 0f - /old | gzip -c > file.dump.gz) > restore rf some.file > > danny > Well, the first part (the dump) runs almost to the completion, but hangs at the very end for some reason: dump 0aCf 64 /ibm/ibmo.0.2009-03-24.dump /old DUMP: WARNING: should use -L when dumping live read-write filesystems! DUMP: Date of this level 0 dump: Tue Mar 24 05:59:27 2009 DUMP: Date of last level 0 dump: the epoch DUMP: Dumping /dev/ad2s1e (/ibmo) to /ibm/ibmo.0.2009-03-24.dump DUMP: mapping (Pass I) [regular files] DUMP: Cache 64 MB, blocksize = 65536 DUMP: mapping (Pass II) [directories] DUMP: estimated 152357442 tape blocks. DUMP: dumping (Pass III) [directories] DUMP: dumping (Pass IV) [regular files] DUMP: 0.83% done, finished in 9:59 at Tue Mar 24 16:04:19 2009 DUMP: 2.74% done, finished in 5:55 at Tue Mar 24 12:05:07 2009 DUMP: 4.66% done, finished in 5:06 at Tue Mar 24 11:21:27 2009 DUMP: 6.58% done, finished in 4:43 at Tue Mar 24 11:03:37 2009 ... DUMP: 91.54% done, finished in 0:23 at Tue Mar 24 10:38:15 2009 DUMP: 93.41% done, finished in 0:18 at Tue Mar 24 10:38:02 2009 DUMP: 95.27% done, finished in 0:13 at Tue Mar 24 10:37:50 2009 DUMP: 97.15% done, finished in 0:07 at Tue Mar 24 10:37:36 2009 DUMP: 99.03% done, finished in 0:02 at Tue Mar 24 10:37:23 2009 DUMP: DUMP: 152769349 tape blocks on 1 volume DUMP: finished in 16706 seconds, throughput 9144 KBytes/sec [... Hang ...] load: 0.18 cmd: dump 10105 [sbwait] 72.53u 383.14s 0% 73048k load: 0.19 cmd: dump 10102 [sbwait] 164.93u 314.87s 0% 75008k load: 0.10 cmd: dump 10102 [running] 164.93u 314.87s 0% 75008k The timestamp on the output file is, indeed, 10:38 and the dumping process is hanging ever since then (over 90 minutes already). Yours, -mi From owner-freebsd-fs@FreeBSD.ORG Tue Mar 24 18:04:04 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 184381065672; Tue, 24 Mar 2009 18:04:04 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id 647BD8FC21; Tue, 24 Mar 2009 18:04:03 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2OI3Ctm067217; Tue, 24 Mar 2009 14:03:13 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C92060.4060303@aldan.algebra.com> Date: Tue, 24 Mar 2009 14:03:12 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: Danny Braniss References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> In-Reply-To: Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: Daniel O'Connor , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Mar 2009 18:04:05 -0000 Danny Braniss ΞΑΠΙΣΑΧ(ΜΑ): > can you try splitting it in 2, ie no pipe? > dump a0f some.file /old (or dump 0f - /old | gzip -c > file.dump.gz) > restore rf some.file > > Same problem: restore -rf ibmo.0.2009-03-24.dump load: 0.55 cmd: restore 11303 [nbufkv] 3.53u 3.91s 4% 27980k unknown tape header type 213474529 abort? [yn] Please, advise. Thanks! Yours, -mi From owner-freebsd-fs@FreeBSD.ORG Tue Mar 24 21:30:04 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E929D10656C2 for ; Tue, 24 Mar 2009 21:30:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 8E0B68FC24 for ; Tue, 24 Mar 2009 21:30:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2OLU4IN049456 for ; Tue, 24 Mar 2009 21:30:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2OLU4ec049453; Tue, 24 Mar 2009 21:30:04 GMT (envelope-from gnats) Date: Tue, 24 Mar 2009 21:30:04 GMT Message-Id: <200903242130.n2OLU4ec049453@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Oleg Ginzburg Cc: Subject: Re: bin/131341: makefs: error "Bad file descriptor" on the mount point of md-presentation makefs image X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Oleg Ginzburg List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Mar 2009 21:30:07 -0000 The following reply was made to PR bin/131341; it has been noted by GNATS. From: Oleg Ginzburg To: Jaakko Heinonen Cc: bug-followup@freebsd.org Subject: Re: bin/131341: makefs: error "Bad file descriptor" on the mount point of md-presentation makefs image Date: Tue, 24 Mar 2009 21:21:14 +0300 Hello Jaakko, I've apply yours http://www.saunalahti.fi/~jh3/patches/ffs-fs_fsbtodb.diff patch and this work fine for my, problem of PR-131341 is gone. Thanks a lot! On Monday 23 March 2009 19:31:30 Jaakko Heinonen wrote: > See this message: > > http://docs.freebsd.org/cgi/mid.cgi?20090323154536.GA2853 > > and a patch: > > http://www.saunalahti.fi/~jh3/patches/ffs-fs_fsbtodb.diff From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 00:42:27 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF7201065670 for ; Wed, 25 Mar 2009 00:42:27 +0000 (UTC) (envelope-from janm@transactionware.com) Received: from mail.transactionware.com (mail.transactionware.com [203.14.245.7]) by mx1.freebsd.org (Postfix) with SMTP id EDC078FC13 for ; Wed, 25 Mar 2009 00:42:25 +0000 (UTC) (envelope-from janm@transactionware.com) Received: (qmail 85689 invoked from network); 25 Mar 2009 00:27:40 -0000 Received: from midgard.transactionware.com (192.168.1.55) by dm.transactionware.com with SMTP; 25 Mar 2009 00:27:40 -0000 Received: (qmail 56590 invoked by uid 907); 25 Mar 2009 00:27:18 -0000 Received: from jmdesktop.transactionware.com (HELO [192.168.1.32]) (192.168.1.32) by midgard.transactionware.com (qpsmtpd/0.40) with ESMTP; Wed, 25 Mar 2009 11:27:18 +1100 Message-ID: <49C97A6F.70204@transactionware.com> Date: Wed, 25 Mar 2009 11:27:27 +1100 From: Jan Mikkelsen User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Bruce Evans References: <200903231733.51671.mel.flynn+fbsd.fs@mailing.thruhere.net> <49C7C45B.7040708@elischer.org> <20090324224001.D1670@besplex.bde.org> In-Reply-To: <20090324224001.D1670@besplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Julian Elischer Subject: Re: Trying to understand how aio(4), mount(8) async (and gjournal) relate X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 00:42:28 -0000 Hi, [Jumping into a conversation on aio, async mounts, etc.] I have had a few questions for a while that I haven't asked yet; these seems like an appropriate time to ask them! Is it reasonable to open a file with O_FSYNC and then use aio_write() to issue multiple writes, and then assume that the data is on disk when the aio completes? Can I get I/O parallelism using this approach? I recall reading (some time ago) that FreeBSD doesn't do I/O parallelism on a single file descriptor. Is that true? Do I need to open the file multiple times in order to get I/O parallelism? You can see where I'm going with this: What I'd really like to do is open a file with O_FSYNC | O_DIRECT | O_EXCL, and then do lots of aio operations on it using chunks that a multiple of the page size with buffers that are aligned on page boundaries. I'd like to know when aio writes are "really" complete to maintain various kinds of on-disk structures (eg. b-trees). I'd also like to avoid call fsync(2). Thanks, Jan Mikkelsen From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 01:05:38 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E265106566B for ; Wed, 25 Mar 2009 01:05:38 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (email.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 4DA488FC16 for ; Wed, 25 Mar 2009 01:05:38 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id 029C317E85; Wed, 25 Mar 2009 12:05:49 +1100 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.1.50.60] (ppp121-44-33-108.lns10.syd7.internode.on.net [121.44.33.108]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id CE892171BC; Wed, 25 Mar 2009 12:05:45 +1100 (EST) Message-ID: <49C982AF.5010305@modulus.org> Date: Wed, 25 Mar 2009 12:02:39 +1100 From: Andrew Snow User-Agent: Thunderbird 2.0.0.14 (X11/20080523) MIME-Version: 1.0 To: Hussain Ali , freebsd-fs@freebsd.org References: <20090324145841.GD17811@datapipe.com> In-Reply-To: <20090324145841.GD17811@datapipe.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: ZFS move from FreeBSD to Opensolaris ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 01:05:38 -0000 Hussain Ali wrote: > On a tangent, I can go from Mac -> OpenSolaris. I am guessing this maybe > due to lack of EFI/GPT labels on the geom providers. I believe Opensolaris writes an EFI label when it is given an entire disk to add to a pool. FreeBSD does not do this or handle this case, it just uses the entire GEOM provider with no labelling. - Andrew From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 01:14:56 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E1318106566B; Wed, 25 Mar 2009 01:14:56 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B5A758FC16; Wed, 25 Mar 2009 01:14:56 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2P1EuPr059162; Wed, 25 Mar 2009 01:14:56 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2P1Eu1b059158; Wed, 25 Mar 2009 01:14:56 GMT (envelope-from linimon) Date: Wed, 25 Mar 2009 01:14:56 GMT Message-Id: <200903250114.n2P1Eu1b059158@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/133020: [zfs] [panic] inappropriate panic caused by zfs. Panic: zfs_fuid_create X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 01:14:57 -0000 Old Synopsis: inappropriate panic caused by zfs. Panic: zfs_fuid_create New Synopsis: [zfs] [panic] inappropriate panic caused by zfs. Panic: zfs_fuid_create Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Wed Mar 25 01:14:39 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=133020 From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 01:19:05 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BF23106566C; Wed, 25 Mar 2009 01:19:05 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id 848C88FC1D; Wed, 25 Mar 2009 01:19:04 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2P1IuY9071528; Tue, 24 Mar 2009 21:19:01 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C98680.7020301@aldan.algebra.com> Date: Tue, 24 Mar 2009 21:18:56 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: Andrew Snow References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> <49C90813.8070705@aldan.algebra.com> <49C98202.9040403@modulus.org> In-Reply-To: <49C98202.9040403@modulus.org> Content-Type: text/plain; charset=KOI8-U; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 01:19:06 -0000 Andrew Snow ΞΑΠΙΣΑΧ(ΜΑ): > Mikhail T. wrote: >> dump 0aCf 64 /ibm/ibmo.0.2009-03-24.dump /old >> DUMP: WARNING: should use -L when dumping live read-write >> filesystems! > > I thought you said it was a read-only filesystem? It was yesterday. Today I remounted it rw to remove some junk-files, which I don't need to transfer. I don't believe, this is causing the problems. > In my experience, restore can sometimes throw warnings if you dump a > live filesystem. It might be causing your errors? If possible, can > you try completely unmounting the filesystem you are dumping and > trying again? I don't think, restore can even figure this out, much less throw a warning -- it is dump, that complains... But the dump started this morning is still hanging (in sbwait) -- I've never seen this before. I'm also very troubled, that such an important functionality (dump/restore!) is sooo problem-prone, and yet so few people seem to care... Is the official view, that dump is obsolete (and already bit-rotten), perhaps, and use of tar is encouraged instead? -mi From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 01:22:37 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96134106564A for ; Wed, 25 Mar 2009 01:22:37 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (email.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 542BA8FC1D for ; Wed, 25 Mar 2009 01:22:37 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id 5273617E9C; Wed, 25 Mar 2009 12:02:57 +1100 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.1.50.60] (ppp121-44-33-108.lns10.syd7.internode.on.net [121.44.33.108]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id 26DE217E85; Wed, 25 Mar 2009 12:02:53 +1100 (EST) Message-ID: <49C98202.9040403@modulus.org> Date: Wed, 25 Mar 2009 11:59:46 +1100 From: Andrew Snow User-Agent: Thunderbird 2.0.0.14 (X11/20080523) MIME-Version: 1.0 To: "Mikhail T." References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> <49C90813.8070705@aldan.algebra.com> In-Reply-To: <49C90813.8070705@aldan.algebra.com> Content-Type: text/plain; charset=KOI8-U; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 01:22:37 -0000 Mikhail T. wrote: > dump 0aCf 64 /ibm/ibmo.0.2009-03-24.dump /old > DUMP: WARNING: should use -L when dumping live read-write filesystems! I thought you said it was a read-only filesystem? In my experience, restore can sometimes throw warnings if you dump a live filesystem. It might be causing your errors? If possible, can you try completely unmounting the filesystem you are dumping and trying again? From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 01:34:21 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5E171065675 for ; Wed, 25 Mar 2009 01:34:21 +0000 (UTC) (envelope-from ota@j.email.ne.jp) Received: from mail2.asahi-net.or.jp (mail2.asahi-net.or.jp [202.224.39.198]) by mx1.freebsd.org (Postfix) with ESMTP id 72C428FC0A for ; Wed, 25 Mar 2009 01:34:21 +0000 (UTC) (envelope-from ota@j.email.ne.jp) Received: from localhost (pool-141-151-75-22.phlapa.east.verizon.net [141.151.75.22]) by mail2.asahi-net.or.jp (Postfix) with ESMTP id 652596E63E; Wed, 25 Mar 2009 10:15:54 +0900 (JST) Date: Tue, 24 Mar 2009 21:15:49 -0400 From: Yoshihiro Ota To: "Mikhail T." Message-Id: <20090324211549.27e65b90.ota@j.email.ne.jp> In-Reply-To: <49C87E0D.5090501@aldan.algebra.com> References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> X-Mailer: Sylpheed 2.6.0 (GTK+ 2.12.11; amd64-portbld-freebsd7.1) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-2022-JP Content-Transfer-Encoding: 7bit Cc: Daniel O'Connor , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 01:34:21 -0000 On Tue, 24 Mar 2009 02:30:37 -0400 "Mikhail T." wrote: > Daniel O'Connor $B'_'Q'a'Z'c'Q'S(B($B']'Q(B): > > On Tuesday 24 March 2009 11:55:07 Mikhail T. wrote: > > > >> I'm trying to migrate a filesystem from one disk to another using: > >> > >> dump a0hCf 0 32 - /old | restore -rf - > >> > >> (/old is already mounted read-only). The process runs for a while and > >> then stops with: > >> > >> [...] > >> DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 2009 > >> DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 2009 > >> DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 2009 > >> unknown tape header type 1853384566 > >> abort? [yn] > >> > >> Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD's > >> dump's output? > >> > > > > What happens if you don't use the cache? > > > No big difference: > > dump a0f - /old | restore -rf - > [...] > DUMP: 17.25% done, finished in 3:27 at Tue Mar 24 05:42:00 2009 > DUMP: 20.36% done, finished in 3:09 at Tue Mar 24 05:28:13 2009 > DUMP: 23.83% done, finished in 2:50 at Tue Mar 24 05:14:32 2009 > unknown tape header type -621260722 > abort? [yn] > > Looks like a junk value somewhere... Unitialized variable or some such. > > -mi -a option seems to be the problem. Try without it. Hiro From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 02:02:22 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 058A11065670; Wed, 25 Mar 2009 02:02:22 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 4E1C78FC16; Wed, 25 Mar 2009 02:02:21 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from inchoate.localnet (ppp121-45-33-47.lns10.adl2.internode.on.net [121.45.33.47]) (authenticated bits=0) by cain.gsoft.com.au (8.13.8/8.13.8) with ESMTP id n2P22DnM017234 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Mar 2009 12:32:13 +1030 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: freebsd-stable@freebsd.org Date: Wed, 25 Mar 2009 12:32:09 +1030 User-Agent: KMail/1.10.4 (Linux/2.6.27-11-generic; KDE/4.1.4; i686; ; ) References: <49C83673.3000604@aldan.algebra.com> <49C98202.9040403@modulus.org> <49C98680.7020301@aldan.algebra.com> In-Reply-To: <49C98680.7020301@aldan.algebra.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1370125.q0FYTSpdyJ"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200903251232.11418.doconnor@gsoft.com.au> X-Spam-Score: 0.238 () BAYES_00,MIME_CHARSET_FARAWAY,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.63 on 203.31.81.10 Cc: "Mikhail T." , fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 02:02:22 -0000 --nextPart1370125.q0FYTSpdyJ Content-Type: text/plain; charset="koi8-u" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Wednesday 25 March 2009 11:48:56 Mikhail T. wrote: > Andrew Snow =CE=C1=D0=C9=D3=C1=D7(=CC=C1): > > Mikhail T. wrote: > >> dump 0aCf 64 /ibm/ibmo.0.2009-03-24.dump /old > >> DUMP: WARNING: should use -L when dumping live read-write > >> filesystems! > > > > I thought you said it was a read-only filesystem? > > It was yesterday. Today I remounted it rw to remove some junk-files, > which I don't need to transfer. I don't believe, this is causing the > problems. > > > In my experience, restore can sometimes throw warnings if you dump a > > live filesystem. It might be causing your errors? If possible, can > > you try completely unmounting the filesystem you are dumping and > > trying again? > > I don't think, restore can even figure this out, much less throw a > warning -- it is dump, that complains... But the dump started this restore will emit a warning if dump writes a stream that is out of order=20 because of a live file system but that is not what you are seeing. > morning is still hanging (in sbwait) -- I've never seen this before. I'm > also very troubled, that such an important functionality (dump/restore!) > is sooo problem-prone, and yet so few people seem to care... Well, "works for me". > Is the official view, that dump is obsolete (and already bit-rotten), > perhaps, and use of tar is encouraged instead? I've never had dump fail but it IS rather crusty and slow.. That said tar=20 doesn't cover all the information I believe. =2D-=20 Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --nextPart1370125.q0FYTSpdyJ Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iD8DBQBJyZCj5ZPcIHs/zowRAuC5AKCsG52ocLa5Mw5KyHhhCsCQaLVgTwCfQgIX sNG6y1F8aM8Ge4/yK1uMyDc= =bb2i -----END PGP SIGNATURE----- --nextPart1370125.q0FYTSpdyJ-- From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 02:08:14 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 554551065670; Wed, 25 Mar 2009 02:08:14 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id 272878FC21; Wed, 25 Mar 2009 02:08:12 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2P2841H071637; Tue, 24 Mar 2009 22:08:08 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C99204.2050601@aldan.algebra.com> Date: Tue, 24 Mar 2009 22:08:04 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: "Daniel O'Connor" References: <49C83673.3000604@aldan.algebra.com> <49C98202.9040403@modulus.org> <49C98680.7020301@aldan.algebra.com> <200903251232.11418.doconnor@gsoft.com.au> In-Reply-To: <200903251232.11418.doconnor@gsoft.com.au> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 02:08:14 -0000 Daniel O'Connor ΞΑΠΙΣΑΧ(ΜΑ): >> morning is still hanging (in sbwait) -- I've never seen this before. I'm >> also very troubled, that such an important functionality (dump/restore!) >> is sooo problem-prone, and yet so few people seem to care... >> > > Well, "works for me". > Well, would like a login on this system to take a look for yourself? I can reproduce the problem easily. >> Is the official view, that dump is obsolete (and already bit-rotten), >> perhaps, and use of tar is encouraged instead? >> > > I've never had dump fail but it IS rather crusty and slow.. That said tar > doesn't cover all the information I believe. So, if dump/restore ain't it, does FreeBSD have a supported way of making filesystem-level backups, that's both modern and covers all aspects (like flags)? That said, I point out, that for me, dump is not failing (although it did hang this morning). It is the restore, which fails to read dump's output: unknown tape header type 213474529 abort? [yn] n resync restore, skipped 502 blocks expected next file 54, got 0 unknown tape header type -954356454 abort? [yn] n resync restore, skipped 29 blocks expected next file 54, got 0 unknown tape header type -1754938223 abort? [yn] n resync restore, skipped 482 blocks expected next file 54, got 0 unknown tape header type -915868704 abort? [yn] n resync restore, skipped 29 blocks expected next file 54, got 0 unknown tape header type 1790084751 abort? [yn] n resync restore, skipped 482 blocks expected next file 54, got 0 unknown tape header type 903667267 abort? [yn] n ... Thanks! -mi From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 02:40:25 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FD4E106566B; Wed, 25 Mar 2009 02:40:25 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id B12F08FC18; Wed, 25 Mar 2009 02:40:24 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2P2eNI2073473; Tue, 24 Mar 2009 22:40:23 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C99997.9050306@aldan.algebra.com> Date: Tue, 24 Mar 2009 22:40:23 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: Yoshihiro Ota References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> <20090324211549.27e65b90.ota@j.email.ne.jp> In-Reply-To: <20090324211549.27e65b90.ota@j.email.ne.jp> Content-Type: text/plain; charset=ISO-2022-JP Content-Transfer-Encoding: 7bit Cc: Daniel O'Connor , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 02:40:25 -0000 Yoshihiro Ota $B'_'Q'a'Z'c'Q'S(B($B']'Q(B): >> No big difference: >> dump a0f - /old | restore -rf - >> [...] >> DUMP: 17.25% done, finished in 3:27 at Tue Mar 24 05:42:00 2009 >> DUMP: 20.36% done, finished in 3:09 at Tue Mar 24 05:28:13 2009 >> DUMP: 23.83% done, finished in 2:50 at Tue Mar 24 05:14:32 2009 >> unknown tape header type -621260722 >> abort? [yn] >> >> Looks like a junk value somewhere... Unitialized variable or some such. >> >> -mi >> > > -a option seems to be the problem. > Try without it. > As could be expected, it failed exactly the same without the obviously unrelated -a option: root@symbion:/ibm (113) dump 0f - /ibmo | restore -rf - DUMP: Date of this level 0 dump: Tue Mar 24 21:31:19 2009 DUMP: Date of last level 0 dump: the epoch DUMP: Dumping /dev/ad2s1e (/ibmo) to standard output DUMP: mapping (Pass I) [regular files] DUMP: mapping (Pass II) [directories] DUMP: estimated 152357442 tape blocks. DUMP: dumping (Pass III) [directories] DUMP: dumping (Pass IV) [regular files] DUMP: 0.16% done, finished in 242:05 at Sat Apr 4 00:00:21 2009 DUMP: 4.29% done, finished in 10:24 at Wed Mar 25 08:24:02 2009 DUMP: 8.56% done, finished in 5:52 at Wed Mar 25 03:56:43 2009 DUMP: 11.76% done, finished in 4:35 at Wed Mar 25 02:43:29 2009 DUMP: 16.00% done, finished in 3:38 at Wed Mar 25 01:52:05 2009 DUMP: 19.28% done, finished in 3:15 at Wed Mar 25 01:33:43 2009 DUMP: 22.74% done, finished in 2:55 at Wed Mar 25 01:18:49 2009 unknown tape header type 1431655765 abort? [yn] n resync restore, skipped 9 blocks expected next file 1599492, got 0 DUMP: 24.50% done, finished in 3:27 at Wed Mar 25 02:05:41 2009 unknown tape header type 1508167078 abort? [yn] n resync restore, skipped 66 blocks expected next file 1599492, got 0 unknown tape header type -1493630979 abort? [yn] y dump core? [yn] n DUMP: Broken pipe DUMP: The ENTIRE dump is aborted. Now can one get /real/ support for the most basic functionality of the most advanced modern Unix in the world? Thanks, -mi From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 03:02:33 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AEBE1065670 for ; Wed, 25 Mar 2009 03:02:33 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (email.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 17BDA8FC16 for ; Wed, 25 Mar 2009 03:02:32 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id 0903017E57; Wed, 25 Mar 2009 14:02:44 +1100 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.1.50.60] (ppp121-44-33-108.lns10.syd7.internode.on.net [121.44.33.108]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id 6629817E6C; Wed, 25 Mar 2009 14:02:22 +1100 (EST) Message-ID: <49C99E03.9060604@modulus.org> Date: Wed, 25 Mar 2009 13:59:15 +1100 From: Andrew Snow User-Agent: Thunderbird 2.0.0.14 (X11/20080523) MIME-Version: 1.0 To: "Mikhail T." References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> <20090324211549.27e65b90.ota@j.email.ne.jp> <49C99997.9050306@aldan.algebra.com> In-Reply-To: <49C99997.9050306@aldan.algebra.com> Content-Type: text/plain; charset=ISO-2022-JP Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 03:02:33 -0000 Mikhail T. wrote: > Now can one get /real/ support for the most basic functionality of the > most advanced modern Unix in the world? Thanks, I think before this goes any further, you will need to try rebooting/unmouting it, running fsck on it, and then dump the unmounted partition and see how that goes. - Andrew From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 03:05:01 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA3811065692; Wed, 25 Mar 2009 03:05:01 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 2340E8FC1A; Wed, 25 Mar 2009 03:05:00 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from inchoate.localnet (ppp121-45-33-47.lns10.adl2.internode.on.net [121.45.33.47]) (authenticated bits=0) by cain.gsoft.com.au (8.13.8/8.13.8) with ESMTP id n2P34sKl020597 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Mar 2009 13:34:55 +1030 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: "Mikhail T." Date: Wed, 25 Mar 2009 13:34:31 +1030 User-Agent: KMail/1.10.4 (Linux/2.6.27-11-generic; KDE/4.1.4; i686; ; ) References: <49C83673.3000604@aldan.algebra.com> <200903251232.11418.doconnor@gsoft.com.au> <49C99204.2050601@aldan.algebra.com> In-Reply-To: <49C99204.2050601@aldan.algebra.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart12025682.40UkcpncZI"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200903251334.38350.doconnor@gsoft.com.au> X-Spam-Score: 0.238 () BAYES_00,MIME_CHARSET_FARAWAY,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.63 on 203.31.81.10 Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 03:05:03 -0000 --nextPart12025682.40UkcpncZI Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Wednesday 25 March 2009 12:38:04 Mikhail T. wrote: > Daniel O'Connor =CE=C1=D0=C9=D3=C1=D7(=CC=C1): > >> morning is still hanging (in sbwait) -- I've never seen this before. I= 'm > >> also very troubled, that such an important functionality (dump/restore= !) > >> is sooo problem-prone, and yet so few people seem to care... > > > > Well, "works for me". > > Well, would like a login on this system to take a look for yourself? I > can reproduce the problem easily. I don't know the internals of dump/restore :( > >> Is the official view, that dump is obsolete (and already bit-rotten), > >> perhaps, and use of tar is encouraged instead? > > > > I've never had dump fail but it IS rather crusty and slow.. That said t= ar > > doesn't cover all the information I believe. > > So, if dump/restore ain't it, does FreeBSD have a supported way of > making filesystem-level backups, that's both modern and covers all > aspects (like flags)? I would try a pax archive, eg.. tar --format pax --one-file-system -pcf - -C / . | tar -pxf - -C /mnt/newdi= sk According to the libarchive-formats page this handles ACLs & flags, my test= ing=20 shows it handles flags (at least). eg.. [midget 13:30] /tmp/test2 >touch foo [midget 13:30] /tmp/test2 >chflags uchg foo =20 [midget 13:30] /tmp/test2 >tar --format pax -zpcf /tmp/test.pax.gz foo [midget 13:30] /tmp/test2 >rm -f foo rm: foo: Operation not permitted [midget 13:30] /tmp/test2 >chflags nouchg foo [midget 13:30] /tmp/test2 >rm foo [midget 13:30] /tmp/test2 >tar -pxf /tmp/test.pax.gz =20 [midget 13:30] /tmp/test2 >ls -lao total 30 =20 drwxr-xr-x 2 darius wheel - 512 Mar 25 13:30 . drwxrwxrwt 53 root wheel - 28672 Mar 25 13:29 .. =2Drw-r--r-- 1 darius wheel uchg 0 Mar 25 13:30 foo [midget 13:30] /tmp/test2 >rm -f foo rm: foo: Operation not permitted > That said, I point out, that for me, dump is not failing (although it > did hang this morning). It is the restore, which fails to read dump's > output: You can't tell the difference between dump producing mangled output or rest= ore=20 bombing out on valid input.. =2D-=20 Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --nextPart12025682.40UkcpncZI Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iD8DBQBJyZ9A5ZPcIHs/zowRAjtjAJ9Re4V+zBWBrdjjltDwdYGwJSIaTwCdGlke F95N4/KuXGNR+A1p4WxbUCE= =KvTW -----END PGP SIGNATURE----- --nextPart12025682.40UkcpncZI-- From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 03:05:40 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F3CD010656C3; Wed, 25 Mar 2009 03:05:39 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id A94338FC16; Wed, 25 Mar 2009 03:05:35 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2P35Y7e073580; Tue, 24 Mar 2009 23:05:35 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C99F7E.2010108@aldan.algebra.com> Date: Tue, 24 Mar 2009 23:05:34 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: Andrew Snow References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> <20090324211549.27e65b90.ota@j.email.ne.jp> <49C99997.9050306@aldan.algebra.com> <49C99E03.9060604@modulus.org> In-Reply-To: <49C99E03.9060604@modulus.org> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 03:05:41 -0000 Andrew Snow ΞΑΠΙΣΑΧ(ΜΑ): > Mikhail T. wrote: > >> Now can one get /real/ support for the most basic functionality of the >> most advanced modern Unix in the world? Thanks, >> > > I think before this goes any further, you will need to try > rebooting/unmouting it, running fsck on it, and then dump the unmounted > partition and see how that goes. > The system's uptime is only 3 days -- I had to reboot it to put in the new disk... But I will try your suggestion next, after the current attempt (using restore's -v switch) ends. If it chokes on the same file every time, that would be a clue... Thanks, -mi From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 03:06:59 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 37D73106564A; Wed, 25 Mar 2009 03:06:59 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id E16EB8FC1F; Wed, 25 Mar 2009 03:06:58 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2P36wPG073585; Tue, 24 Mar 2009 23:06:58 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C99FD2.50609@aldan.algebra.com> Date: Tue, 24 Mar 2009 23:06:58 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: "Daniel O'Connor" References: <49C83673.3000604@aldan.algebra.com> <200903251232.11418.doconnor@gsoft.com.au> <49C99204.2050601@aldan.algebra.com> <200903251334.38350.doconnor@gsoft.com.au> In-Reply-To: <200903251334.38350.doconnor@gsoft.com.au> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 03:07:01 -0000 Daniel O'Connor ΞΑΠΙΣΑΧ(ΜΑ): >> That said, I point out, that for me, dump is not failing (although it >> did hang this morning). It is the restore, which fails to read dump's >> output: >> > > You can't tell the difference between dump producing mangled output or restore > bombing out on valid input.. > That's true. I just wanted to point out, that someone running dump only (to make backups) is not going to know, whether his dumps are usable (for whichever of the two reasons), until he needs them... -mi From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 03:59:51 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2282C106567A; Wed, 25 Mar 2009 03:59:51 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 874038FC0A; Wed, 25 Mar 2009 03:59:49 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from inchoate.localnet (ppp121-45-33-47.lns10.adl2.internode.on.net [121.45.33.47]) (authenticated bits=0) by cain.gsoft.com.au (8.13.8/8.13.8) with ESMTP id n2P3xhMN023346 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Mar 2009 14:29:43 +1030 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: "Mikhail T." Date: Wed, 25 Mar 2009 14:29:30 +1030 User-Agent: KMail/1.10.4 (Linux/2.6.27-11-generic; KDE/4.1.4; i686; ; ) References: <49C83673.3000604@aldan.algebra.com> <20090324211549.27e65b90.ota@j.email.ne.jp> <49C99997.9050306@aldan.algebra.com> In-Reply-To: <49C99997.9050306@aldan.algebra.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1914226.SHNIKI2sUl"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200903251429.40578.doconnor@gsoft.com.au> X-Spam-Score: -2.212 () BAYES_00,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.63 on 203.31.81.10 Cc: Yoshihiro Ota , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 03:59:51 -0000 --nextPart1914226.SHNIKI2sUl Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Wednesday 25 March 2009 13:10:23 Mikhail T. wrote: > Now can one get /real/ support for the most basic functionality of the > most advanced modern Unix in the world? Thanks, Maybe you should return it to the shop and ask for your money back. =2D-=20 Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --nextPart1914226.SHNIKI2sUl Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iD8DBQBJyawl5ZPcIHs/zowRAoQ8AJ44eqmk9Z8Fb2xnNt4a6+4kmqW3KACbBRb5 a3VXan3gOZv3FJHV1JjKy6Y= =4TsE -----END PGP SIGNATURE----- --nextPart1914226.SHNIKI2sUl-- From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 04:05:03 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBFAA1065673; Wed, 25 Mar 2009 04:05:03 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id 73F748FC14; Wed, 25 Mar 2009 04:05:03 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2P451T2073925; Wed, 25 Mar 2009 00:05:02 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C9AD6D.30305@aldan.algebra.com> Date: Wed, 25 Mar 2009 00:05:01 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: "Daniel O'Connor" References: <49C83673.3000604@aldan.algebra.com> <20090324211549.27e65b90.ota@j.email.ne.jp> <49C99997.9050306@aldan.algebra.com> <200903251429.40578.doconnor@gsoft.com.au> In-Reply-To: <200903251429.40578.doconnor@gsoft.com.au> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: Yoshihiro Ota , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 04:05:04 -0000 Daniel O'Connor ΞΑΠΙΣΑΧ(ΜΑ): > On Wednesday 25 March 2009 13:10:23 Mikhail T. wrote: > >> Now can one get /real/ support for the most basic functionality of the >> most advanced modern Unix in the world? Thanks, >> > > Maybe you should return it to the shop and ask for your money back. > Well, if this response is the best one can get, may be I should also revoke my own 15 years worth of contributions to the project... Except, why would I? I always supported people, who had problems with any of my work -- and the attitude of the rest of the contributors /used to be/ the same... -mi From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 04:22:53 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12E7F106564A; Wed, 25 Mar 2009 04:22:53 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 5DAD18FC1F; Wed, 25 Mar 2009 04:22:51 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from inchoate.localnet (ppp121-45-33-47.lns10.adl2.internode.on.net [121.45.33.47]) (authenticated bits=0) by cain.gsoft.com.au (8.13.8/8.13.8) with ESMTP id n2P4MiuX024275 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Mar 2009 14:52:45 +1030 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: "Mikhail T." Date: Wed, 25 Mar 2009 14:52:35 +1030 User-Agent: KMail/1.10.4 (Linux/2.6.27-11-generic; KDE/4.1.4; i686; ; ) References: <49C83673.3000604@aldan.algebra.com> <200903251429.40578.doconnor@gsoft.com.au> <49C9AD6D.30305@aldan.algebra.com> In-Reply-To: <49C9AD6D.30305@aldan.algebra.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2312064.hPhShRi3Ly"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200903251452.42301.doconnor@gsoft.com.au> X-Spam-Score: 0.238 () BAYES_00,MIME_CHARSET_FARAWAY,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.63 on 203.31.81.10 Cc: Yoshihiro Ota , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 04:22:53 -0000 --nextPart2312064.hPhShRi3Ly Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Wednesday 25 March 2009 14:35:01 Mikhail T. wrote: > Daniel O'Connor =CE=C1=D0=C9=D3=C1=D7(=CC=C1): > > On Wednesday 25 March 2009 13:10:23 Mikhail T. wrote: > >> Now can one get /real/ support for the most basic functionality of the > >> most advanced modern Unix in the world? Thanks, > > > > Maybe you should return it to the shop and ask for your money back. > > Well, if this response is the best one can get, may be I should also > revoke my own 15 years worth of contributions to the project... > > Except, why would I? I always supported people, who had problems with > any of my work -- and the attitude of the rest of the contributors /used > to be/ the same... People ARE helping you, just because they haven't come up with an answer is= no=20 reason to send snarky comments to the list. Outrage (fake or otherwise) that people don't seem to be taking your=20 particular problem seriously is unhelpful and probably counter-productive. =2D-=20 Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --nextPart2312064.hPhShRi3Ly Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iD8DBQBJybGM5ZPcIHs/zowRAqKZAJ4hB3UlBJc7gEqMVvJM2C4CwhK2+QCffHRT O02sDTpiYUpf8VdJCLugKPI= =0QEu -----END PGP SIGNATURE----- --nextPart2312064.hPhShRi3Ly-- From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 04:29:37 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A405106567B; Wed, 25 Mar 2009 04:29:37 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id 2AB598FC15; Wed, 25 Mar 2009 04:29:37 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2P4TaZR073998; Wed, 25 Mar 2009 00:29:36 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C9B330.10500@aldan.algebra.com> Date: Wed, 25 Mar 2009 00:29:36 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: "Daniel O'Connor" References: <49C83673.3000604@aldan.algebra.com> <200903251429.40578.doconnor@gsoft.com.au> <49C9AD6D.30305@aldan.algebra.com> <200903251452.42301.doconnor@gsoft.com.au> In-Reply-To: <200903251452.42301.doconnor@gsoft.com.au> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: Yoshihiro Ota , freebsd-stable@freebsd.org, fs@freebsd.org Subject: support quality (Re: dump | restore fails: unknown tape header type 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 04:29:38 -0000 Daniel O'Connor ΞΑΠΙΣΑΧ(ΜΑ): > People ARE helping you, just because they haven't come up with an answer is no > reason to send snarky comments to the list. > No, sorry, people aren't. They are trying, yes, but not even close. The suggestion to eliminate the -a switch (a no-op, in fact) was particularly unhelpful -- and deserving of mockery. Later on someone with a similar problem will find this thread with a search engine and will be trying to follow the posted advices -- to no avail, of course, plunging FreeBSD further into disrepute. > Outrage (fake or otherwise) that people don't seem to be taking your > particular problem seriously is unhelpful and probably counter-productive. It is fairly obvious by now, that no real help will be forthcoming, for whatever reason. Thus any talk of "productivity" is moot. Thanks for trying. -mi From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 04:30:49 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8211106566B; Wed, 25 Mar 2009 04:30:49 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id 771098FC14; Wed, 25 Mar 2009 04:30:49 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2P4UlKU074004; Wed, 25 Mar 2009 00:30:48 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49C9B377.5090903@aldan.algebra.com> Date: Wed, 25 Mar 2009 00:30:47 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: Andrew Snow References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> <20090324211549.27e65b90.ota@j.email.ne.jp> <49C99997.9050306@aldan.algebra.com> <49C99E03.9060604@modulus.org> In-Reply-To: <49C99E03.9060604@modulus.org> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 04:30:50 -0000 Andrew Snow ΞΑΠΙΣΑΧ(ΜΑ): > I think before this goes any further, you will need to try > rebooting/unmouting it, running fsck on it, and then dump the unmounted > partition and see how that goes. > > Rebooted, reran `fsck -y /old' (all clean). Same problem... -mi From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 04:54:39 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C04CA106566C; Wed, 25 Mar 2009 04:54:39 +0000 (UTC) (envelope-from markir@paradise.net.nz) Received: from smtp5.clear.net.nz (smtp5.clear.net.nz [203.97.33.68]) by mx1.freebsd.org (Postfix) with ESMTP id 88EB68FC13; Wed, 25 Mar 2009 04:54:39 +0000 (UTC) (envelope-from markir@paradise.net.nz) Received: from zmori.markir.net (121-73-175-180.dsl.telstraclear.net [121.73.175.180]) by smtp5.clear.net.nz (CLEAR Net Mail) with ESMTP id <0KH1000A0OXSO120@smtp5.clear.net.nz>; Wed, 25 Mar 2009 17:39:29 +1300 (NZDT) Date: Wed, 25 Mar 2009 17:39:27 +1300 From: Mark Kirkwood In-reply-to: <49C99204.2050601@aldan.algebra.com> To: "Mikhail T." Message-id: <49C9B57F.4010403@paradise.net.nz> MIME-version: 1.0 Content-type: text/plain; charset=KOI8-R; format=flowed Content-transfer-encoding: 7bit References: <49C83673.3000604@aldan.algebra.com> <49C98202.9040403@modulus.org> <49C98680.7020301@aldan.algebra.com> <200903251232.11418.doconnor@gsoft.com.au> <49C99204.2050601@aldan.algebra.com> User-Agent: Thunderbird 2.0.0.19 (X11/20090117) Cc: Daniel O'Connor , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 04:54:40 -0000 Mikhail T. wrote: > > That said, I point out, that for me, dump is not failing (although it > did hang this morning). It is the restore, which fails to read dump's > output: > > unknown tape header type 213474529 > abort? [yn] n > resync restore, skipped 502 blocks > expected next file 54, got 0 > unknown tape header type -954356454 > abort? [yn] n > resync restore, skipped 29 blocks > expected next file 54, got 0 > unknown tape header type -1754938223 > abort? [yn] n > resync restore, skipped 482 blocks > expected next file 54, got 0 > unknown tape header type -915868704 > abort? [yn] n > resync restore, skipped 29 blocks > expected next file 54, got 0 > unknown tape header type 1790084751 > abort? [yn] n > resync restore, skipped 482 blocks > expected next file 54, got 0 > unknown tape header type 903667267 > abort? [yn] n > ... > Is this link any help? http://www.mail-archive.com/freebsd-questions@freebsd.org/msg197899.html Other than that, I'd suggest checking the disk(s) with smartmontools to try to rule out hardware problems. regards Mark From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 05:22:45 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57574106566B for ; Wed, 25 Mar 2009 05:22:45 +0000 (UTC) (envelope-from freebsd-nospam@yaxom.com) Received: from gw.yaxom.com (gw.yaxom.com [59.167.217.197]) by mx1.freebsd.org (Postfix) with SMTP id 8423B8FC0A for ; Wed, 25 Mar 2009 05:22:44 +0000 (UTC) (envelope-from freebsd-nospam@yaxom.com) Received: (qmail 63059 invoked from network); 25 Mar 2009 14:56:01 +1000 Received: from joker.yaxom.com (172.16.1.10) by iliad.yaxom.com with SMTP; 25 Mar 2009 14:56:01 +1000 Received: (qmail 78754 invoked by uid 1001); 25 Mar 2009 14:56:01 +1000 Message-ID: Date: Wed, 25 Mar 2009 14:56:01 +1000 From: Greg Black To: "Mikhail T." References: <49C83673.3000604@aldan.algebra.com> <200903251232.11418.doconnor@gsoft.com.au> <49C99204.2050601@aldan.algebra.com> <200903251334.38350.doconnor@gsoft.com.au> <49C99FD2.50609@aldan.algebra.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <49C99FD2.50609@aldan.algebra.com> User-Agent: Mutt/1.4.2.2i; gjb-muttsend.sh 1.7 2004-10-05 X-Uptime: 124 days X-Operating-System: FreeBSD 6.2-RELEASE-p5 i386 X-Location: Brisbane, Australia; 27.49835S 152.98380E X-URL: http://www.gbch.net/gjb.html X-Blog: http://www.gbch.net/gjb/blog/ X-Image-URL: http://www.gbch.net/gjb/gjb-auug048.gif X-PGP-Key-Fingerprint: EBB2 2A92 A79D 1533 AC00 3C46 5D83 B6FB 4B04 B7D6 X-Request-PGP: http://www.gbch.net/keys/4B04B7D6.asc Cc: Daniel O'Connor , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 05:22:45 -0000 On 2009-03-24, Mikhail T. wrote: > That's true. I just wanted to point out, that someone running dump only > (to make backups) is not going to know, whether his dumps are usable > (for whichever of the two reasons), until he needs them... Such a person is not making backups and deserves what he gets. I haven't got anything to say about dump/restore because I haven't bothered with them for years. I do know that dumps from mounted file systems will often appear to work, but will fail when it matters. This is not a bug and is expected behaviour to which the solution is obvious. From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 07:58:14 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CEBB1065672; Wed, 25 Mar 2009 07:58:14 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id E47AC8FC12; Wed, 25 Mar 2009 07:58:13 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from inchoate.localnet (ppp121-45-35-198.lns10.adl2.internode.on.net [121.45.35.198]) (authenticated bits=0) by cain.gsoft.com.au (8.13.8/8.13.8) with ESMTP id n2P7w42E034543 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Mar 2009 18:28:05 +1030 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: "Mikhail T." Date: Wed, 25 Mar 2009 18:20:46 +1030 User-Agent: KMail/1.10.4 (Linux/2.6.27-11-generic; KDE/4.1.4; i686; ; ) References: <49C83673.3000604@aldan.algebra.com> <200903251452.42301.doconnor@gsoft.com.au> <49C9B330.10500@aldan.algebra.com> In-Reply-To: <49C9B330.10500@aldan.algebra.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart65162929.sPH3db2Mya"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200903251820.54749.doconnor@gsoft.com.au> X-Spam-Score: 0.238 () BAYES_00,MIME_CHARSET_FARAWAY,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.63 on 203.31.81.10 Cc: Yoshihiro Ota , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape header type 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 07:58:15 -0000 --nextPart65162929.sPH3db2Mya Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Wednesday 25 March 2009 14:59:36 Mikhail T. wrote: > Daniel O'Connor =CE=C1=D0=C9=D3=C1=D7(=CC=C1): > > People ARE helping you, just because they haven't come up with an answer > > is no reason to send snarky comments to the list. > > No, sorry, people aren't. They are trying, yes, but not even close. The > suggestion to eliminate the -a switch (a no-op, in fact) was > particularly unhelpful -- and deserving of mockery. Later on someone > with a similar problem will find this thread with a search engine and > will be trying to follow the posted advices -- to no avail, of course, > plunging FreeBSD further into disrepute. I don't see how whining about it's going to change it. Insulting people for= =20 having a helpful attitude (even if it didn't solve your problem) is not goi= ng=20 to reward them for their time and effort. > > Outrage (fake or otherwise) that people don't seem to be taking your > > particular problem seriously is unhelpful and probably > > counter-productive. > > It is fairly obvious by now, that no real help will be forthcoming, for > whatever reason. Thus any talk of "productivity" is moot. Thanks for > trying. Except that I offered you a way of transferring your files that would prese= rve=20 file flags and so on. Yes, dump is broken for you, deal with it. It is quite possible your FS is= =20 corrupt, and/or your disk is damaged. =2D-=20 Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --nextPart65162929.sPH3db2Mya Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iD8DBQBJyeJX5ZPcIHs/zowRAj12AJ9wj89PXuH9qJWZtjliK35VGGRdwwCfRoTl ldjsy0SEyRhUUQPq8Dtjuvk= =qFBg -----END PGP SIGNATURE----- --nextPart65162929.sPH3db2Mya-- From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 08:31:54 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8CCFE1065688; Wed, 25 Mar 2009 08:31:54 +0000 (UTC) (envelope-from admin@kkip.pl) Received: from mainframe.kkip.pl (kkip.pl [87.105.164.78]) by mx1.freebsd.org (Postfix) with ESMTP id 3B3958FC24; Wed, 25 Mar 2009 08:31:54 +0000 (UTC) (envelope-from admin@kkip.pl) Received: from mb01.admin.lan.kkip.pl ([10.66.3.254]) by mainframe.kkip.pl with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1LmO8r-000NgI-TE; Wed, 25 Mar 2009 09:07:19 +0100 Message-ID: <49C9E635.5010106@kkip.pl> Date: Wed, 25 Mar 2009 09:07:17 +0100 From: Bartosz Stec User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Daniel O'Connor References: <49C83673.3000604@aldan.algebra.com> <200903251452.42301.doconnor@gsoft.com.au> <49C9B330.10500@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> In-Reply-To: <200903251820.54749.doconnor@gsoft.com.au> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-Authenticated-User: admin@kkip.pl X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Spam-Score: -8.9 X-Spam-Score-Int: -88 X-Exim-Version: 4.69 (build at 26-Jan-2009 09:48:47) X-Date: 2009-03-25 09:07:19 X-Connected-IP: 10.66.3.254:2704 X-Message-Linecount: 32 X-Body-Linecount: 16 X-Message-Size: 1352 X-Body-Size: 586 X-Received-Count: 1 X-Recipient-Count: 5 X-Local-Recipient-Count: 5 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 Cc: Yoshihiro Ota , "Mikhail T." , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape header type 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 08:31:56 -0000 > Yes, dump is broken for you, deal with it. It is quite possible your FS is > corrupt, and/or your disk is damaged. > > ..and/or it is some other hardware problem, maybe you also should test your memory with memtest or something similiar? I'm using dump/restore very frequently and I had never seen such problem. Neither on -RELAESE, -STABLE, nor -CURRENT. So I think you should make sure that your problem is not hardware/filesystem dependent before you point dump/restore as a couse of the problem. Peter Jeremy already gives you good hints to do that. -- Bartosz Stec. From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 08:55:48 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F165106566B; Wed, 25 Mar 2009 08:55:48 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 777528FC12; Wed, 25 Mar 2009 08:55:47 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from inchoate.localnet (ppp121-45-35-198.lns10.adl2.internode.on.net [121.45.35.198]) (authenticated bits=0) by cain.gsoft.com.au (8.13.8/8.13.8) with ESMTP id n2P8teUS037433 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Mar 2009 19:25:40 +1030 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: Bartosz Stec Date: Wed, 25 Mar 2009 19:25:28 +1030 User-Agent: KMail/1.10.4 (Linux/2.6.27-11-generic; KDE/4.1.4; i686; ; ) References: <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <49C9E635.5010106@kkip.pl> In-Reply-To: <49C9E635.5010106@kkip.pl> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1981183.VGV1rVWxO9"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200903251925.36108.doconnor@gsoft.com.au> X-Spam-Score: -2.212 () BAYES_00,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.63 on 203.31.81.10 Cc: Yoshihiro Ota , "Mikhail T." , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape header type 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 08:55:48 -0000 --nextPart1981183.VGV1rVWxO9 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Wednesday 25 March 2009 18:37:17 Bartosz Stec wrote: > > Yes, dump is broken for you, deal with it. It is quite possible your FS > > is corrupt, and/or your disk is damaged. > > ..and/or it is some other hardware problem, maybe you also should test > your memory with memtest or something similiar? I'm using dump/restore > very frequently and I had never seen such problem. Neither on -RELAESE, > -STABLE, nor -CURRENT. > So I think you should make sure that your problem is not > hardware/filesystem dependent before you point dump/restore as a couse > of the problem. Peter Jeremy already gives you good hints to do that. One other thing would be to make absolutely sure that your version of dump = &=20 restore are in sync, the are very machine/version dependent. =2D-=20 Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --nextPart1981183.VGV1rVWxO9 Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iD8DBQBJyfGB5ZPcIHs/zowRArtlAJ9i3tGIjjQSSdom2UuIJmfWoGi8SACcChHH Iguso4K+yDKpck9e4CcKcmo= =JMMX -----END PGP SIGNATURE----- --nextPart1981183.VGV1rVWxO9-- From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 12:32:44 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1828B1065677; Wed, 25 Mar 2009 12:32:44 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from mail.sagedata.net (mail.sagedata.net [63.214.156.21]) by mx1.freebsd.org (Postfix) with ESMTP id CC6EB8FC14; Wed, 25 Mar 2009 12:32:43 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from sagemaster (sageweb.net [65.68.247.73]) by mail.sagedata.net (8.14.3/8.14.3) with SMTP id n2PCLd9p007104; Wed, 25 Mar 2009 07:21:39 -0500 (CDT) (envelope-from jacks@sage-american.com) X-Authentication-Warning: mail.sagedata.net: Host sageweb.net [65.68.247.73] claimed to be sagemaster Message-Id: <3.0.1.32.20090325072137.00ee6b48@sage-american.com> X-Sender: jacks@sage-american.com X-Mailer: Windows Eudora Pro Version 3.0.1 (32) Date: Wed, 25 Mar 2009 07:21:37 -0500 To: "Daniel O'Connor" , Bartosz Stec From: "Jack L. Stone" In-Reply-To: <200903251925.36108.doconnor@gsoft.com.au> References: <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <49C9E635.5010106@kkip.pl> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Scanned-By: milter-spamc/1.13.385 (mail.sagedata.net [63.214.156.21]); Wed, 25 Mar 2009 07:21:40 -0500 X-Scanned-By: milter-sender/1.16.915 (mail.sagedata.net [63.214.156.21]); Wed, 25 Mar 2009 07:21:40 -0500 X-Virus-Scanned: ClamAV 0.94.2/9164/Tue Mar 24 23:02:31 2009 on mail.sagedata.net X-Virus-Status: Clean X-Spam-Status: NO, hits=-10.00 required=4.50 X-Spam-Report: Content analysis details: (-10.0 points, 4.5 required) | | pts rule name description | ---- ---------------------- -------------------------------------------------- | -10 ALL_TRUSTED Passed through trusted hosts only via SMTP | Cc: Yoshihiro Ota , "Mikhail T." , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 12:32:44 -0000 At 07:25 PM 3.25.2009 +1030, Daniel O'Connor wrote: >On Wednesday 25 March 2009 18:37:17 Bartosz Stec wrote: >> > Yes, dump is broken for you, deal with it. It is quite possible your FS >> > is corrupt, and/or your disk is damaged. >> >> ..and/or it is some other hardware problem, maybe you also should test >> your memory with memtest or something similiar? I'm using dump/restore >> very frequently and I had never seen such problem. Neither on -RELAESE, >> -STABLE, nor -CURRENT. >> So I think you should make sure that your problem is not >> hardware/filesystem dependent before you point dump/restore as a couse >> of the problem. Peter Jeremy already gives you good hints to do that. > >One other thing would be to make absolutely sure that your version of dump & >restore are in sync, the are very machine/version dependent. > >-- I've been watching this thread with some interest since we've had some similar problems with dump/restore which we use every morning via cron scripts on a number of servers to produce bootable clones as part of our backup program. Have been doing this for years and also never saw a problem as most of you say. We prefer dump/restore for backups. However, last month upon upon upgrading those servers from FBSD-6.3px (RELEASE) to 7.0px (RELEASE) we found that about one-half of the servers had a similar problem as the original poster while the other half did not. All of the servers (rackmounts) use the same (type) hardware. We spent many hours trying to solve the problem with those that failed to dump/restore. Also, searched for any others with the problem and only found a very few, but without solutions to this issue. (Indeed, the only one was a reference to any efforts to restore an older OS version which didn't apply here). And, indeed we tried everything suggested here to fix the proble without success. Sometimes the problem was dump which would reach 99% and never finish -- it would stick there and would overlap with another cron start the next day, and the next day, and the next day. (The servers that did work fooled us and we found out about this issue on the others when the overlaps appeared and drew our attention). That's when our work to try and solve the issues started and went on for days. Our script that has always worked contained this (after scraping and making fresh FS): /sbin/dump -D /root/dumpdates -0auL -f - / | /sbin/restore -rf - Indeed, the first thing we did was to remove the pipe and tried to restore from a file. However, because the dumps would not go past the 99%, no file to restore from! There were some exceptions when the dump would complete, but was not reliable. When these reached the restore level, restore would go crazy with errors. SOLUTION The "clones" are a very important pasrt of our backup program. Since the dump side of the problems simply stuck and provided no error message at all and the errors from any restores were not useful, our only solution was to revert back to FBSD-6.3 on those servers with this issue and dump/restore went back to working again. We left those that were working on FBSD-7.0-R and they continue to work okay. We could only conclude that the problem was perhaps something with hardeware, perhaps the way memory was handled in 7.0, but that is only a guess. Once again, every suggestion on this thread was tried during our long efforts to fix the issue. Perhaps there is yet another suggestion? In the meantime, we've decided to wait for 7.2R (7.1 did not fix the problems either). /Jack (^_^) Happy trails, Jack L. Stone System Admin Sage-american From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 12:50:39 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF4641065925 for ; Wed, 25 Mar 2009 12:50:39 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail08.syd.optusnet.com.au (mail08.syd.optusnet.com.au [211.29.132.189]) by mx1.freebsd.org (Postfix) with ESMTP id 442A48FC1B for ; Wed, 25 Mar 2009 12:50:38 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c122-107-120-227.carlnfd1.nsw.optusnet.com.au (c122-107-120-227.carlnfd1.nsw.optusnet.com.au [122.107.120.227]) by mail08.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n2PCoVTo001993 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Mar 2009 23:50:33 +1100 Date: Wed, 25 Mar 2009 23:50:31 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Jan Mikkelsen In-Reply-To: <49C97A6F.70204@transactionware.com> Message-ID: <20090325223213.P35996@delplex.bde.org> References: <200903231733.51671.mel.flynn+fbsd.fs@mailing.thruhere.net> <49C7C45B.7040708@elischer.org> <20090324224001.D1670@besplex.bde.org> <49C97A6F.70204@transactionware.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org, Julian Elischer Subject: Re: Trying to understand how aio(4), mount(8) async (and gjournal) relate X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 12:50:40 -0000 On Wed, 25 Mar 2009, Jan Mikkelsen wrote: > [Jumping into a conversation on aio, async mounts, etc.] > > I have had a few questions for a while that I haven't asked yet; these seems > like an appropriate time to ask them! > > Is it reasonable to open a file with O_FSYNC and then use aio_write() to > issue multiple writes, and then assume that the data is on disk when the aio > completes? I know very little about aio, but looking at the sources seems to show that O_FSYNC (or mounting with the sync option) just defeats the asyncness of aio. aio seems to use only fo_write() for writing, so at lower (file system) levels, O_FSYNC has the same behaviour as for write(2) -- it syncs the i/o at the end of the call in the usual case where fo_write = vn_write. > Can I get I/O parallelism using this approach? Apparently not. > I recall reading (some time > ago) that FreeBSD doesn't do I/O parallelism on a single file descriptor. Is > that true? Do I need to open the file multiple times in order to get I/O > parallelism? The fs part of vn_write() is serialized, now using the exclusive vnode lock. The code is essentially: vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); error = VOP_WRITE(...); /* this soon reaches foofs_write() */ VOP_UNLOCK(vp); In the usual case without O_FSYNC, foofs should try to only schedule the i/o (by writing it to the buffer cache and not waiting), so that the actual i/o is done in parallel later. However, foofs might need to do some physical input in order to tell where to write (e.g., reading indirect block(s)) or some physical output of metadata needed for consistency (e.g., writing indirect blocks), and any such i/o is serialized. (I think most file systems avoid writing to the inode on every foofs_write(), though not doing requires tricks to maintain consistency. No tricks seem to be available for indirect blocks, so ffs without soft updates always writes them synchronously (except in my version where the async mount option actually works for indurect blocks).) O_FSYNC should cause almost all writes related to the file to be synced at the end of foofs_write(). Thus it forces all i/o to be serialized. Some excepions to "all": - at least in ffs, bitmap blocks are not synced. This is safe since fsck can always recover bitmap blocks. - at least in ffs, directories above the file are not synced by fsync() for the file. This is normally harmless because critical directory operations are normally synchronous (or ordered relative to everything including related file operations in the case of soft updates), and fsync() is not specified to do this (?), but perhaps careful applications should fsync() all the directories too, and with the async mount option, even the most critical directory operation (creation of the file's directory entry) is asynchronous (except bugs make it partly synchronus). - at least in ffs, with the async mount option, fsync() is more broken than it should be broken -- it syncs everything except for the most critical metadata (the inode) and directories above the file. > You can see where I'm going with this: What I'd really like to do is open a > file with O_FSYNC | O_DIRECT | O_EXCL, and then do lots of aio operations on > it using chunks that a multiple of the page size with buffers that are > aligned on page boundaries. I'd like to know when aio writes are "really" > complete to maintain various kinds of on-disk structures (eg. b-trees). I'd > also like to avoid call fsync(2). Calling fsync() or aio_waitcomplete() seems to be necessary. More global options like the sync mount flag and O_FSYNC don't provide enough control. I can't find any aio interfaces to select or poll for completion. It seems to have only aio_return() to test for completion, with the possibly unwanted side effect of doing the completion if possible. I don't trust aio_return() to test that _all_ the things that would be done by the file system for fsync(2) have been done. aio_waitcomplete ensures doing these things by calling the file system (VOP_FSYNC()), but aio_return() doesn't seem to go near the file system. BTW, I just remembered that there is no mount option or file flag to give fully sync metadata. At least in ffs, all inode-change operations (chmod(), chown(), fchmod(), fchown(), etc.) are async, irrespective of mount options and O_FSYNC. It takes a syscall calling VOP_FSYNC() or an unrelated inode update to sync the metadata for these operations. Bruce From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 14:56:38 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7F551065674; Wed, 25 Mar 2009 14:56:38 +0000 (UTC) (envelope-from mozart@mozart.lib.uchicago.edu) Received: from mozart.lib.uchicago.edu (mozart.lib.uchicago.edu [128.135.53.59]) by mx1.freebsd.org (Postfix) with ESMTP id 945628FC20; Wed, 25 Mar 2009 14:56:38 +0000 (UTC) (envelope-from mozart@mozart.lib.uchicago.edu) Received: from localhost (localhost [127.0.0.1]) by mozart.lib.uchicago.edu (8.14.3/8.14.3) with ESMTP id n2PEjnP3075631; Wed, 25 Mar 2009 09:45:49 -0500 (CDT) (envelope-from mozart@mozart.lib.uchicago.edu) Date: Wed, 25 Mar 2009 09:45:48 -0500 Message-ID: From: Peggy Wilkins To: "Jack L. Stone" In-Reply-To: <3.0.1.32.20090325072137.00ee6b48@sage-american.com> References: <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> User-Agent: Wanderlust/2.14.0 (Africa) SEMI/1.14.6 (Maruoka) FLIM/1.14.8 (=?ISO-8859-4?Q?Shij=F2?=) APEL/10.6 MULE XEmacs/21.4 (patch 21) (Educational Television) (i386--freebsd) Organization: University of Chicago Library X-Face: )C,7auub1,qHwu.Spk;Y6'8pld<:g8_\/=Zo/=E3y5*[m?C;;5Wc'B9=V+I:v 7>8Y=yx{[yy0$Zu@q+d&+$`jV{"Gk"Q0A#d]btd#$qjAR),qXF+GuE#?9Q4_xai{V&". MJluE"^Oj41R'ypsHcp2_`6kxl;T7?5eW57j#Muert'z`Y Y03pPAp3uY@x9DhN?yWwH.!jV MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Sender: Peggy Wilkins X-Dispatcher: imput version 20070423(IM149) Lines: 42 Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 14:56:39 -0000 >>>>> Jack L Stone writes: >> I've been watching this thread with some interest since we've had some >> similar problems with dump/restore which we use every morning via cron >> scripts on a number of servers to produce bootable clones as part of our >> backup program. Have been doing this for years and also never saw a problem >> as most of you say. We prefer dump/restore for backups. >> However, last month upon upon upgrading those servers from FBSD-6.3px >> (RELEASE) to 7.0px (RELEASE) we found that about one-half of the servers >> had a similar problem as the original poster while the other half did not. >> All of the servers (rackmounts) use the same (type) hardware. We spent many >> hours trying to solve the problem with those that failed to dump/restore. >> Also, searched for any others with the problem and only found a very few, >> but without solutions to this issue. (Indeed, the only one was a reference >> to any efforts to restore an older OS version which didn't apply here). [snip] >> SOLUTION >> The "clones" are a very important pasrt of our backup program. Since the >> dump side of the problems simply stuck and provided no error message at all >> and the errors from any restores were not useful, our only solution was to >> revert back to FBSD-6.3 on those servers with this issue and dump/restore >> went back to working again. We left those that were working on FBSD-7.0-R >> and they continue to work okay. I was seeing this same problem on all my 64-bit systems: FreeBSD-7 dump would hang at a random point. Dump continues to work flawlessly for me on FreeBSD-7/i386. I ran across this which includes a patch: http://www.freebsd.org/cgi/query-pr.cgi?pr=121684 The kernel patch linked to there solved the problem for me, but I am running many production systems and am unwilling to apply this patch to -RELEASE every time there is a kernel update (I just use the standard GENERIC kernel which I get via freebsd-update). I now live without dump on amd64. Apparently this fix is waiting on some related issue; and I will be very happy when it makes it to the officially released kernel. plw From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 15:59:38 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DBC181065672; Wed, 25 Mar 2009 15:59:38 +0000 (UTC) (envelope-from mr@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B1D0D8FC1E; Wed, 25 Mar 2009 15:59:38 +0000 (UTC) (envelope-from mr@FreeBSD.org) Received: from freefall.freebsd.org (mr@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2PFxcqk094723; Wed, 25 Mar 2009 15:59:38 GMT (envelope-from mr@freefall.freebsd.org) Received: (from mr@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2PFxcFZ094719; Wed, 25 Mar 2009 15:59:38 GMT (envelope-from mr) Date: Wed, 25 Mar 2009 15:59:38 GMT Message-Id: <200903251559.n2PFxcFZ094719@freefall.freebsd.org> To: mr@FreeBSD.org, freebsd-fs@FreeBSD.org, pjd@FreeBSD.org From: mr@FreeBSD.org Cc: Subject: Re: kern/133020: [zfs] [panic] inappropriate panic caused by zfs. Panic: zfs_fuid_create X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 15:59:39 -0000 Synopsis: [zfs] [panic] inappropriate panic caused by zfs. Panic: zfs_fuid_create Responsible-Changed-From-To: freebsd-fs->pjd Responsible-Changed-By: mr Responsible-Changed-When: Wed Mar 25 15:57:50 UTC 2009 Responsible-Changed-Why: Over to maintainer. http://www.freebsd.org/cgi/query-pr.cgi?pr=133020 From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 16:05:01 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 098C21065673; Wed, 25 Mar 2009 16:05:01 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id AAA4D8FC0A; Wed, 25 Mar 2009 16:05:00 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2PG4IfL083523; Wed, 25 Mar 2009 12:04:27 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49CA5602.9050001@aldan.algebra.com> Date: Wed, 25 Mar 2009 12:04:18 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: Greg Black References: <49C83673.3000604@aldan.algebra.com> <200903251232.11418.doconnor@gsoft.com.au> <49C99204.2050601@aldan.algebra.com> <200903251334.38350.doconnor@gsoft.com.au> <49C99FD2.50609@aldan.algebra.com> In-Reply-To: Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: Daniel O'Connor , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 16:05:02 -0000 Greg Black ΞΑΠΙΣΑΧ(ΜΑ): > On 2009-03-24, Mikhail T. wrote: > >> That's true. I just wanted to point out, that someone running dump only >> (to make backups) is not going to know, whether his dumps are usable >> (for whichever of the two reasons), until he needs them... >> > > Such a person is not making backups and deserves what he gets. > But he *is* making backups -- kindly re-read my paragraph above... He just is not routinely using them and thus does not know, that they aren't usable... It is not unreasonable to expect the two utilities to "just work", so I wouldn't be blaming such a person for not testing restore. That such a scenario is possible, despite the user making diligent regular backups, is a very bad sign... > I haven't got anything to say about dump/restore because I haven't > bothered with them for years. I do know that dumps from mounted file > systems will often appear to work, but will fail when it matters. This > is not a bug and is expected behaviour to which the solution is obvious. > FS being mounted read-only should not be a problem. And even read-write mounted filesystems will be Ok, although possibly inconsistent (a file modified during backup may get to into dump "as of" any point). But an idle FS is Ok to dump. Generally, when dumping read-write mounted filesystems, one is supposed to use snapshots, but that is very buggy on its own (leading to kernel crashes), so it is safer to rely on the FS being "idle", if it can not be forced into read-only for some other reason. -mi From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 16:11:39 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1B871065672; Wed, 25 Mar 2009 16:11:39 +0000 (UTC) (envelope-from dimitry@andric.com) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 61DC48FC21; Wed, 25 Mar 2009 16:11:39 +0000 (UTC) (envelope-from dimitry@andric.com) Received: from [IPv6:2001:7b8:3a7:0:d93e:519d:eeb6:9049] (unknown [IPv6:2001:7b8:3a7:0:d93e:519d:eeb6:9049]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 73F155C43; Wed, 25 Mar 2009 17:11:38 +0100 (CET) Message-ID: <49CA57BB.2070409@andric.com> Date: Wed, 25 Mar 2009 17:11:39 +0100 From: Dimitry Andric User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.1b4pre) Gecko/20090324 Shredder/3.0b3pre MIME-Version: 1.0 To: "Mikhail T." References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> In-Reply-To: <49C87E0D.5090501@aldan.algebra.com> Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: Daniel O'Connor , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 16:11:40 -0000 On 2009-03-24 07:30, Mikhail T. wrote: > dump a0f - /old | restore -rf - > [...] > DUMP: 17.25% done, finished in 3:27 at Tue Mar 24 05:42:00 2009 > DUMP: 20.36% done, finished in 3:09 at Tue Mar 24 05:28:13 2009 > DUMP: 23.83% done, finished in 2:50 at Tue Mar 24 05:14:32 2009 > unknown tape header type -621260722 > abort? [yn] > > Looks like a junk value somewhere... Unitialized variable or some such. Hmm, I can't reproduce this at all; I use dump and restore quite regularly in this way, and I have never encountered this issue (except for the occasional 'expected file NNN, got file MMM', which is usually harmless). Maybe the dump output gets corrupted in some way? (E.g. faulty RAM, or disk?) If you are dumping a live filesystem, could it possibly help to add the -L option? From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 17:22:09 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82D9D10658D0 for ; Wed, 25 Mar 2009 17:22:09 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outW.internet-mail-service.net (outw.internet-mail-service.net [216.240.47.246]) by mx1.freebsd.org (Postfix) with ESMTP id 60D898FC1A for ; Wed, 25 Mar 2009 17:22:09 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id B6E8783D41; Wed, 25 Mar 2009 10:22:28 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 9D3F22D6017; Wed, 25 Mar 2009 10:22:06 -0700 (PDT) Message-ID: <49CA684F.70604@elischer.org> Date: Wed, 25 Mar 2009 10:22:23 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Bruce Evans References: <200903231733.51671.mel.flynn+fbsd.fs@mailing.thruhere.net> <49C7C45B.7040708@elischer.org> <20090324224001.D1670@besplex.bde.org> <49C97A6F.70204@transactionware.com> <20090325223213.P35996@delplex.bde.org> In-Reply-To: <20090325223213.P35996@delplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Trying to understand how aio(4), mount(8) async (and gjournal) relate X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 17:22:13 -0000 Bruce Evans wrote: > On Wed, 25 Mar 2009, Jan Mikkelsen wrote: > >> [Jumping into a conversation on aio, async mounts, etc.] >> >> I have had a few questions for a while that I haven't asked yet; these >> seems like an appropriate time to ask them! >> >> Is it reasonable to open a file with O_FSYNC and then use aio_write() >> to issue multiple writes, and then assume that the data is on disk >> when the aio completes? > > I know very little about aio, but looking at the sources seems to show that > O_FSYNC (or mounting with the sync option) just defeats the asyncness of > aio. aio seems to use only fo_write() for writing, so at lower (file > system) levels, O_FSYNC has the same behaviour as for write(2) -- it syncs > the i/o at the end of the call in the usual case where fo_write = vn_write. > >> Can I get I/O parallelism using this approach? > > Apparently not. > >> I recall reading (some time ago) that FreeBSD doesn't do I/O >> parallelism on a single file descriptor. Is that true? Do I need to >> open the file multiple times in order to get I/O parallelism? > > The fs part of vn_write() is serialized, now using the exclusive vnode > lock. > The code is essentially: > > vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); > error = VOP_WRITE(...); /* this soon reaches foofs_write() */ > VOP_UNLOCK(vp); > > In the usual case without O_FSYNC, foofs should try to only schedule > the i/o (by writing it to the buffer cache and not waiting), so that > the actual i/o is done in parallel later. However, foofs might need > to do some physical input in order to tell where to write (e.g., reading > indirect block(s)) or some physical output of metadata needed for > consistency (e.g., writing indirect blocks), and any such i/o is > serialized. (I think most file systems avoid writing to the inode on > every foofs_write(), though not doing requires tricks to maintain > consistency. No tricks seem to be available for indirect blocks, so > ffs without soft updates always writes them synchronously (except in > my version where the async mount option actually works for indurect > blocks).) > > O_FSYNC should cause almost all writes related to the file to be > synced at the end of foofs_write(). Thus it forces all i/o to be > serialized. > Some excepions to "all": > - at least in ffs, bitmap blocks are not synced. This is safe since > fsck can always recover bitmap blocks. > - at least in ffs, directories above the file are not synced by fsync() > for the file. This is normally harmless because critical directory > operations are normally synchronous (or ordered relative to everything > including related file operations in the case of soft updates), and > fsync() is not specified to do this (?), but perhaps careful > applications should fsync() all the directories too, and with the > async mount option, even the most critical directory operation > (creation of the file's directory entry) is asynchronous (except > bugs make it partly synchronus). > - at least in ffs, with the async mount option, fsync() is more broken than > it should be broken -- it syncs everything except for the most critical > metadata (the inode) and directories above the file. > >> You can see where I'm going with this: What I'd really like to do is >> open a file with O_FSYNC | O_DIRECT | O_EXCL, and then do lots of aio >> operations on it using chunks that a multiple of the page size with >> buffers that are aligned on page boundaries. I'd like to know when >> aio writes are "really" complete to maintain various kinds of on-disk >> structures (eg. b-trees). I'd also like to avoid call fsync(2). > > Calling fsync() or aio_waitcomplete() seems to be necessary. More > global options like the sync mount flag and O_FSYNC don't provide > enough control. I can't find any aio interfaces to select or poll for > completion. it does have a comprehensive interface with kqueue. > It seems to have only aio_return() to test for completion, > with the possibly unwanted side effect of doing the completion if > possible. I don't trust aio_return() to test that _all_ the things > that would be done by the file system for fsync(2) have been done. > aio_waitcomplete ensures doing these things by calling the file system > (VOP_FSYNC()), but aio_return() doesn't seem to go near the file system. > > BTW, I just remembered that there is no mount option or file flag to > give fully sync metadata. At least in ffs, all inode-change operations > (chmod(), chown(), fchmod(), fchown(), etc.) are async, irrespective of > mount options and O_FSYNC. It takes a syscall calling VOP_FSYNC() or > an unrelated inode update to sync the metadata for these operations. > > Bruce From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 17:30:17 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0910F1065B68; Wed, 25 Mar 2009 17:30:17 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 06B588FC13; Wed, 25 Mar 2009 17:30:16 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (brucec@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2PHUFJM019393; Wed, 25 Mar 2009 17:30:15 GMT (envelope-from brucec@freefall.freebsd.org) Received: (from brucec@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2PHUFCQ019385; Wed, 25 Mar 2009 17:30:15 GMT (envelope-from brucec) Date: Wed, 25 Mar 2009 17:30:15 GMT Message-Id: <200903251730.n2PHUFCQ019385@freefall.freebsd.org> To: brucec@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: brucec@FreeBSD.org Cc: Subject: Re: kern/89991: [ufs] softupdates with mount -ur causes fs UNREFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 17:30:21 -0000 Synopsis: [ufs] softupdates with mount -ur causes fs UNREFS Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: brucec Responsible-Changed-When: Wed Mar 25 17:29:30 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=89991 From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 17:31:01 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C1EA10656E1; Wed, 25 Mar 2009 17:31:01 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 2FC648FC16; Wed, 25 Mar 2009 17:31:01 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (brucec@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2PHV1R1022419; Wed, 25 Mar 2009 17:31:01 GMT (envelope-from brucec@freefall.freebsd.org) Received: (from brucec@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2PHV1I7022415; Wed, 25 Mar 2009 17:31:01 GMT (envelope-from brucec) Date: Wed, 25 Mar 2009 17:31:01 GMT Message-Id: <200903251731.n2PHV1I7022415@freefall.freebsd.org> To: brucec@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: brucec@FreeBSD.org Cc: Subject: Re: kern/94480: [libufs] [patch] bread(3) & bwrite(3) can crash under low memory conditions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 17:31:04 -0000 Synopsis: [libufs] [patch] bread(3) & bwrite(3) can crash under low memory conditions Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: brucec Responsible-Changed-When: Wed Mar 25 17:30:41 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=94480 From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 18:13:39 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7FDDD1065673 for ; Wed, 25 Mar 2009 18:13:39 +0000 (UTC) (envelope-from hywel@hmallett.co.uk) Received: from lisbon.directrouter.com (lisbon.directrouter.com [72.249.30.130]) by mx1.freebsd.org (Postfix) with ESMTP id 4EB868FC1D for ; Wed, 25 Mar 2009 18:13:39 +0000 (UTC) (envelope-from hywel@hmallett.co.uk) Received: from localhost ([127.0.0.1]) by lisbon.directrouter.com with esmtpa (Exim 4.69) (envelope-from ) id 1LmWkp-0008He-7n for freebsd-fs@freebsd.org; Wed, 25 Mar 2009 12:19:03 -0500 Received: from 82.111.203.2 ([82.111.203.2]) by www.hmallett.co.uk (Horde MIME library) with HTTP; Wed, 25 Mar 2009 17:19:03 +0000 Message-ID: <20090325171903.x4eqs9ceo8w00c04@www.hmallett.co.uk> Date: Wed, 25 Mar 2009 17:19:03 +0000 From: Hywel Mallett To: freebsd-fs@freebsd.org References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> <49CA57BB.2070409@andric.com> In-Reply-To: <49CA57BB.2070409@andric.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.6) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - lisbon.directrouter.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - hmallett.co.uk X-Source: X-Source-Args: X-Source-Dir: Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 18:13:40 -0000 Quoting Dimitry Andric : > On 2009-03-24 07:30, Mikhail T. wrote: >> dump a0f - /old | restore -rf - >> [...] >> DUMP: 17.25% done, finished in 3:27 at Tue Mar 24 05:42:00 2009 >> DUMP: 20.36% done, finished in 3:09 at Tue Mar 24 05:28:13 2009 >> DUMP: 23.83% done, finished in 2:50 at Tue Mar 24 05:14:32 2009 >> unknown tape header type -621260722 >> abort? [yn] >> >> Looks like a junk value somewhere... Unitialized variable or some such. Looking at the restore code (tape.c, in findinode), restore is =20 expecting a header type in the range 1-6, so the header type =20 -621260722 is way out. Assuming that findinode is being passed the =20 correct variable, it would indicate that dump is writing the header =20 (or at least the header type incorrectly). I can't work out where this =20 header is getting written though. It looks like plenty of data gets =20 written into a header (such as inode, magic number, checksum). I =20 wonder if one of these values is overflowing and overwriting the =20 header type? > > Maybe the dump output gets corrupted in some way? (E.g. faulty RAM, or > disk?) If you are dumping a live filesystem, could it possibly help to > add the -L option? It might be worth fscking the original volume (though I suspect the OP =20 has done this already), and also passing the -D option to restore, as =20 restore will then try and continue, rather than abort on getting the =20 invalid header type. Fixing the root cause would be better, but that =20 might be a workaround for now. From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 18:31:43 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26B5910657BD; Wed, 25 Mar 2009 18:31:43 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 758958FC1D; Wed, 25 Mar 2009 18:31:38 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (brucec@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2PIVcKL007710; Wed, 25 Mar 2009 18:31:38 GMT (envelope-from brucec@freefall.freebsd.org) Received: (from brucec@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2PIVcFr007703; Wed, 25 Mar 2009 18:31:38 GMT (envelope-from brucec) Date: Wed, 25 Mar 2009 18:31:38 GMT Message-Id: <200903251831.n2PIVcFr007703@freefall.freebsd.org> To: brucec@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: brucec@FreeBSD.org Cc: Subject: Re: kern/51685: [hang] Unbounded inode allocation causes kernel to lock up X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 18:31:54 -0000 Synopsis: [hang] Unbounded inode allocation causes kernel to lock up Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: brucec Responsible-Changed-When: Wed Mar 25 18:31:16 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=51685 From owner-freebsd-fs@FreeBSD.ORG Wed Mar 25 22:13:01 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 519741065675 for ; Wed, 25 Mar 2009 22:13:01 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from mx02.qsc.de (mx02.qsc.de [213.148.130.14]) by mx1.freebsd.org (Postfix) with ESMTP id DAE698FC23 for ; Wed, 25 Mar 2009 22:13:00 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from r55.edvax.de (port-92-196-37-253.dynamic.qsc.de [92.196.37.253]) by mx02.qsc.de (Postfix) with ESMTP id C1D4E16C00DF; Wed, 25 Mar 2009 22:48:12 +0100 (CET) Received: from r55.edvax.de (localhost [127.0.0.1]) by r55.edvax.de (8.14.2/8.14.2) with SMTP id n2PLm7Fw001620; Wed, 25 Mar 2009 22:48:07 +0100 (CET) (envelope-from freebsd@edvax.de) Date: Wed, 25 Mar 2009 22:48:07 +0100 From: Polytropon To: FreeBSD FS Message-Id: <20090325224807.81016dd9.freebsd@edvax.de> Organization: EDVAX X-Mailer: Sylpheed 2.4.7 (GTK+ 2.12.1; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: FreeBSD Questions Subject: Repairing a defective UFS 2 partition with another BSD's fsck X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Polytropon List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2009 22:13:01 -0000 Dear -fs list, last night I had an idea how I could have a chance to repair my defective UFS partition. To remember, this is the whole story: http://kerneltrap.org/mailarchive/freebsd-fs/2008/11/2/3894714 "Repairing a defective UFS 2 partition with fsck_ffs (or other means)" I thought about the following: As far as I know, UFS isn't only used by FreeBSD, but also by OpenBSD and NetBSD. Variations should be less than, let's say, with Solaris UFS. So it may be that I can use the fsck utility of OpenBSD or NetBSD to check and repair the UFS dd duplicate where FreeBSD's fsck fails with fsck_ffs: bad inode number 306176 to nextinode To use with NetBSD, I've got a NetBSD Live! 2007 live system CD here. For OpenBSD... well, I don't know if they offer any live file system? The only commands I'd need are mount and fsck (both for UFS only). The system should be bootable via CD. Then I would need a shell to do something like this: # mkdir /temp # mount -t ufs -o rw /dev/wd1s1c /temp # cd /temp/rescue # fsck -t ufs -yf ad1s1f.dd In case NetBSD's or OpenBSD's fsck can't operate on the bare file, I'd think about something like: # mkdir /temp # mount -t ufs -o rw /dev/wd1s1c /temp # cd /temp/rescue # mdconfig -a -t vnode -u 10 -f ad1s1f.dd # fsck -t ufs -yf /dev/md10 What do you think, is this even possible? Should I try it? Please keep me CC because I'm on the questions@ list only. Thanks! I'll cross-post it to questions@ in case someone there as an idea for this really strange problem - remember, I'm the second (!) being on this planet having encountered this particular problem. I hope that's an acceptable behaviour. :-) -- Polytropon >From Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 09:37:02 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A6F51065670 for ; Thu, 26 Mar 2009 09:37:02 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from fallbackmx08.syd.optusnet.com.au (fallbackmx08.syd.optusnet.com.au [211.29.132.10]) by mx1.freebsd.org (Postfix) with ESMTP id 2A14D8FC1E for ; Thu, 26 Mar 2009 09:37:00 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail34.syd.optusnet.com.au (mail34.syd.optusnet.com.au [211.29.133.218]) by fallbackmx08.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n2Q9PEjK025659 for ; Thu, 26 Mar 2009 20:25:14 +1100 Received: from server.vk2pj.dyndns.org (c122-106-216-167.belrs3.nsw.optusnet.com.au [122.106.216.167]) by mail34.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n2Q9P44b013340 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 26 Mar 2009 20:25:05 +1100 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id n2Q9P4sO057618; Thu, 26 Mar 2009 20:25:04 +1100 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id n2Q9P4N6057617; Thu, 26 Mar 2009 20:25:04 +1100 (EST) (envelope-from peter) Date: Thu, 26 Mar 2009 20:25:04 +1100 From: Peter Jeremy To: "Daniel O'Connor" Message-ID: <20090326092504.GF56137@server.vk2pj.dyndns.org> References: <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <49C9E635.5010106@kkip.pl> <200903251925.36108.doconnor@gsoft.com.au> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Mjqg7Yu+0hL22rav" Content-Disposition: inline In-Reply-To: <200903251925.36108.doconnor@gsoft.com.au> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.19 (2009-01-05) Cc: "Mikhail T." , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape header type 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 09:37:02 -0000 --Mjqg7Yu+0hL22rav Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2009-Mar-25 19:25:28 +1030, Daniel O'Connor wrot= e: >One other thing would be to make absolutely sure that your version of dump= &=20 >restore are in sync, the are very machine/version dependent. Actually, they aren't - the archive format is very stable. (This is a fairly important requirement - you don't want to suddenly be unable to restore your backups after an upgrade). I can restore dumps made on FreeBSD4.8/i386 and FreeBSD4.9/alpha on a FreeBSD-current/amd64 system without problems. It's possible that you might have problems with a backup made using a very recent dump on a very old restore if you've used filesystem features that didn't exist when that restore was built. --=20 Peter Jeremy --Mjqg7Yu+0hL22rav Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (FreeBSD) iEYEARECAAYFAknLSfAACgkQ/opHv/APuIcudACgxNIAZcEZ2+95dCWS8+S1eIG/ 4msAn3S4G8J1LQPI/oCK+QF+teL5OmKF =eC27 -----END PGP SIGNATURE----- --Mjqg7Yu+0hL22rav-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 09:48:25 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44433106566B; Thu, 26 Mar 2009 09:48:25 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail17.syd.optusnet.com.au (mail17.syd.optusnet.com.au [211.29.132.198]) by mx1.freebsd.org (Postfix) with ESMTP id C2CC68FC20; Thu, 26 Mar 2009 09:48:24 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (c122-106-216-167.belrs3.nsw.optusnet.com.au [122.106.216.167]) by mail17.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n2Q9mMBX015716 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 26 Mar 2009 20:48:23 +1100 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id n2Q9mMtS057790; Thu, 26 Mar 2009 20:48:22 +1100 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id n2Q9mL5l057789; Thu, 26 Mar 2009 20:48:21 +1100 (EST) (envelope-from peter) Date: Thu, 26 Mar 2009 20:48:21 +1100 From: Peter Jeremy To: "Jack L. Stone" Message-ID: <20090326094821.GG56137@server.vk2pj.dyndns.org> References: <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <49C9E635.5010106@kkip.pl> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sLx0z+5FKKtIVDwd" Content-Disposition: inline In-Reply-To: <3.0.1.32.20090325072137.00ee6b48@sage-american.com> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.19 (2009-01-05) Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 09:48:25 -0000 --sLx0z+5FKKtIVDwd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2009-Mar-25 07:21:37 -0500, "Jack L. Stone" wr= ote: >However, last month upon upon upgrading those servers from FBSD-6.3px >(RELEASE) to 7.0px (RELEASE) we found that about one-half of the servers >had a similar problem as the original poster while the other half did not. =2E.. >the next day, and the next day, and the next day. (The servers that did >work fooled us and we found out about this issue on the others when the >overlaps appeared and drew our attention). That's when our work to try and >solve the issues started and went on for days. It's not clear to me whether all your servers have the problem and you only initially noticed it on some of them or some of your servers work and others dont. In the latter case, you are probably in a very good position to identify the problem since it is related to some difference between your servers. >We could only conclude that the problem was perhaps something with >hardeware, perhaps the way memory was handled in 7.0, but that is only a >guess. If you are talking about server-grade hardware (ECC RAM etc) then it's unlikely to be RAM corruption. About the only thing I can think of would be that if you have RAM above 4GB, you might be running foul of an address being truncated somewhere (particularly in a device driver). (The amd64 user memory map changed between 7.x and 8.x but I don't think there was any change between 6.x & 7.x). The pre-emption changes in 7.x and/or moving to an SMP host would seem to increase the probability of hitting the problem fixed in the patch mentioned later (kern/121684). --=20 Peter Jeremy --sLx0z+5FKKtIVDwd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (FreeBSD) iEYEARECAAYFAknLT2UACgkQ/opHv/APuIfFAwCbBy84eVmPrGqb0ZvfXzQc7990 fVcAn1RbsWihH+PZ17zrvEebStBsrcCt =2jIt -----END PGP SIGNATURE----- --sLx0z+5FKKtIVDwd-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 09:56:03 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 671ED106567C; Thu, 26 Mar 2009 09:56:03 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id BEECD8FC2B; Thu, 26 Mar 2009 09:55:55 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from inchoate.localnet (ppp121-45-126-50.lns11.adl6.internode.on.net [121.45.126.50]) (authenticated bits=0) by cain.gsoft.com.au (8.13.8/8.13.8) with ESMTP id n2Q9tkhv036809 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 26 Mar 2009 20:25:47 +1030 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: Peter Jeremy Date: Thu, 26 Mar 2009 20:25:42 +1030 User-Agent: KMail/1.10.4 (Linux/2.6.27-11-generic; KDE/4.1.4; i686; ; ) References: <49C83673.3000604@aldan.algebra.com> <200903251925.36108.doconnor@gsoft.com.au> <20090326092504.GF56137@server.vk2pj.dyndns.org> In-Reply-To: <20090326092504.GF56137@server.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart11060867.Y2l71WW5bB"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200903262025.43890.doconnor@gsoft.com.au> X-Spam-Score: -2.212 () BAYES_00,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.63 on 203.31.81.10 Cc: "Mikhail T." , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape header type 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 09:56:09 -0000 --nextPart11060867.Y2l71WW5bB Content-Type: text/plain; charset="iso-8859-15" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Thursday 26 March 2009 19:55:04 Peter Jeremy wrote: > On 2009-Mar-25 19:25:28 +1030, Daniel O'Connor =20 wrote: > >One other thing would be to make absolutely sure that your version of du= mp > > & restore are in sync, the are very machine/version dependent. > > Actually, they aren't - the archive format is very stable. (This is a > fairly important requirement - you don't want to suddenly be unable to > restore your backups after an upgrade). I can restore dumps made on > FreeBSD4.8/i386 and FreeBSD4.9/alpha on a FreeBSD-current/amd64 system > without problems. Hmm interesting.. I must confess I haven't had problems recently but I tend= to=20 use tar these days anyway since it's more portable. > It's possible that you might have problems with a backup made using a > very recent dump on a very old restore if you've used filesystem > features that didn't exist when that restore was built. Could be, too long ago to remember :) =2D-=20 Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --nextPart11060867.Y2l71WW5bB Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iD8DBQBJy1Ef5ZPcIHs/zowRAhr7AKClfJgj0nrgniAAEvpCB0F6+NgFiQCfRlhM WWs7qbDMmH4Gv12MURc+p/8= =GAR6 -----END PGP SIGNATURE----- --nextPart11060867.Y2l71WW5bB-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 11:53:40 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6F69106566B; Thu, 26 Mar 2009 11:53:40 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from mail.sagedata.net (mail.sagedata.net [63.214.156.21]) by mx1.freebsd.org (Postfix) with ESMTP id C0B8E8FC13; Thu, 26 Mar 2009 11:53:40 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from sagemaster (sageweb.net [65.68.247.73]) by mail.sagedata.net (8.14.3/8.14.3) with SMTP id n2QBrd1b067467; Thu, 26 Mar 2009 06:53:40 -0500 (CDT) (envelope-from jacks@sage-american.com) X-Authentication-Warning: mail.sagedata.net: Host sageweb.net [65.68.247.73] claimed to be sagemaster Message-Id: <3.0.1.32.20090326065337.00f081e0@sage-american.com> X-Sender: jacks@sage-american.com X-Mailer: Windows Eudora Pro Version 3.0.1 (32) Date: Thu, 26 Mar 2009 06:53:37 -0500 To: Peggy Wilkins From: "Jack L. Stone" In-Reply-To: References: <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Scanned-By: milter-spamc/1.13.385 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 06:53:40 -0500 X-Scanned-By: milter-sender/1.16.915 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 06:53:40 -0500 X-Virus-Scanned: ClamAV 0.94.2/9169/Wed Mar 25 23:13:48 2009 on mail.sagedata.net X-Virus-Status: Clean X-Spam-Status: NO, hits=-10.00 required=4.50 X-Spam-Report: Content analysis details: (-10.0 points, 4.5 required) | | pts rule name description | ---- ---------------------- -------------------------------------------------- | -10 ALL_TRUSTED Passed through trusted hosts only via SMTP | Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 11:53:41 -0000 At 09:45 AM 3.25.2009 -0500, Peggy Wilkins wrote: >>>>>> Jack L Stone writes: > > >> I've been watching this thread with some interest since we've had some > >> similar problems with dump/restore which we use every morning via cron > >> scripts on a number of servers to produce bootable clones as part of our > >> backup program. Have been doing this for years and also never saw a problem > >> as most of you say. We prefer dump/restore for backups. > > >> However, last month upon upon upgrading those servers from FBSD-6.3px > >> (RELEASE) to 7.0px (RELEASE) we found that about one-half of the servers > >> had a similar problem as the original poster while the other half did not. > >> All of the servers (rackmounts) use the same (type) hardware. We spent many > >> hours trying to solve the problem with those that failed to dump/restore. > >> Also, searched for any others with the problem and only found a very few, > >> but without solutions to this issue. (Indeed, the only one was a reference > >> to any efforts to restore an older OS version which didn't apply here). > [snip] > >> SOLUTION > >> The "clones" are a very important pasrt of our backup program. Since the > >> dump side of the problems simply stuck and provided no error message at all > >> and the errors from any restores were not useful, our only solution was to > >> revert back to FBSD-6.3 on those servers with this issue and dump/restore > >> went back to working again. We left those that were working on FBSD-7.0-R > >> and they continue to work okay. > >I was seeing this same problem on all my 64-bit systems: FreeBSD-7 >dump would hang at a random point. Dump continues to work flawlessly >for me on FreeBSD-7/i386. > >I ran across this which includes a patch: > >http://www.freebsd.org/cgi/query-pr.cgi?pr=121684 > >The kernel patch linked to there solved the problem for me, but I am >running many production systems and am unwilling to apply this patch >to -RELEASE every time there is a kernel update (I just use the >standard GENERIC kernel which I get via freebsd-update). I now live >without dump on amd64. Apparently this fix is waiting on some related >issue; and I will be very happy when it makes it to the officially >released kernel. > >plw > Thanks for the reply. Forgot to mention, our machines are all i386 with the problem -- so are the ones without the problem. Yes, I found that patch too and tried it on one of the servers -- no joy. Guess we'll continue to wait also for now. Maybe 7.2/i386....or, until someone finds the solution since we're out of ideas and stuck with 6.3 in order to use dump that we have trusted. Jack (^_^) Happy trails, Jack L. Stone System Admin Sage-american From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 12:08:11 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3117B1065951; Thu, 26 Mar 2009 12:08:11 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from mail.sagedata.net (mail.sagedata.net [63.214.156.21]) by mx1.freebsd.org (Postfix) with ESMTP id 019B58FC15; Thu, 26 Mar 2009 12:08:10 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from sagemaster (sageweb.net [65.68.247.73]) by mail.sagedata.net (8.14.3/8.14.3) with SMTP id n2QC8AnC068164; Thu, 26 Mar 2009 07:08:10 -0500 (CDT) (envelope-from jacks@sage-american.com) X-Authentication-Warning: mail.sagedata.net: Host sageweb.net [65.68.247.73] claimed to be sagemaster Message-Id: <3.0.1.32.20090326070807.00f081e0@sage-american.com> X-Sender: jacks@sage-american.com X-Mailer: Windows Eudora Pro Version 3.0.1 (32) Date: Thu, 26 Mar 2009 07:08:07 -0500 To: Mike Tancsa From: "Jack L. Stone" In-Reply-To: <200903261157.n2QBv2Z1038026@lava.sentex.ca> References: <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Scanned-By: milter-spamc/1.13.385 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 07:08:10 -0500 X-Scanned-By: milter-sender/1.16.915 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 07:08:10 -0500 X-Virus-Scanned: ClamAV 0.94.2/9169/Wed Mar 25 23:13:48 2009 on mail.sagedata.net X-Virus-Status: Clean X-Spam-Status: NO, hits=-10.00 required=4.50 X-Spam-Report: Content analysis details: (-10.0 points, 4.5 required) | | pts rule name description | ---- ---------------------- -------------------------------------------------- | -10 ALL_TRUSTED Passed through trusted hosts only via SMTP | Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 12:08:38 -0000 At 07:57 AM 3.26.2009 -0400, Mike Tancsa wrote: >> >>Thanks for the reply. Forgot to mention, our machines are all i386 with the >>problem -- so are the ones without the problem. >> >>Yes, I found that patch too and tried it on one of the servers -- no joy. >> >>Guess we'll continue to wait also for now. Maybe 7.2/i386....or, until >>someone finds the solution since we're out of ideas and stuck with 6.3 in >>order to use dump that we have trusted. > > >Hi, > I didnt see it in the thread clearly, but did you try >creating a dump without the -L... i.e. without snapshots enabled ? > > ---Mike > Yes, but it's for running a dump on a (L)ive FS and just spits out warnings to that effect and has no effect on solving the problem(s). Jack (^_^) Happy trails, Jack L. Stone System Admin Sage-american From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 12:11:50 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4BA251065791 for ; Thu, 26 Mar 2009 12:11:50 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by mx1.freebsd.org (Postfix) with ESMTP id 0ED128FC28 for ; Thu, 26 Mar 2009 12:11:49 +0000 (UTC) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.14.3/8.14.3) with ESMTP id n2QBv2Z1038026; Thu, 26 Mar 2009 07:57:02 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <200903261157.n2QBv2Z1038026@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 26 Mar 2009 07:57:39 -0400 To: "Jack L. Stone" From: Mike Tancsa In-Reply-To: <3.0.1.32.20090326065337.00f081e0@sage-american.com> References: <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 12:11:54 -0000 At 07:53 AM 3/26/2009, Jack L. Stone wrote: > > >> SOLUTION > > >> The "clones" are a very important pasrt of our backup program. >Since the > > >> dump side of the problems simply stuck and provided no error >message at all > > >> and the errors from any restores were not useful, our only solution >was to > > >> revert back to FBSD-6.3 on those servers with this issue and >dump/restore > > >> went back to working again. We left those that were working on >FBSD-7.0-R > > >> and they continue to work okay. > > > >Thanks for the reply. Forgot to mention, our machines are all i386 with the >problem -- so are the ones without the problem. > >Yes, I found that patch too and tried it on one of the servers -- no joy. > >Guess we'll continue to wait also for now. Maybe 7.2/i386....or, until >someone finds the solution since we're out of ideas and stuck with 6.3 in >order to use dump that we have trusted. Hi, I didnt see it in the thread clearly, but did you try creating a dump without the -L... i.e. without snapshots enabled ? ---Mike From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 12:32:31 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FA97106564A for ; Thu, 26 Mar 2009 12:32:31 +0000 (UTC) (envelope-from cptsalek@gmail.com) Received: from mail-ew0-f171.google.com (mail-ew0-f171.google.com [209.85.219.171]) by mx1.freebsd.org (Postfix) with ESMTP id C1A2A8FC0A for ; Thu, 26 Mar 2009 12:32:30 +0000 (UTC) (envelope-from cptsalek@gmail.com) Received: by ewy19 with SMTP id 19so504478ewy.43 for ; Thu, 26 Mar 2009 05:32:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=CJRSZBApzAvaH6K6Hyerdvp6/L2XWzNft/rI2hyOfy4=; b=FdMVkr2FuWIfUnFyNKZX6zCqaKV8jJrY/s49tF6PdvQRTaY+ggl4IJEna46Z/5P/OS yWQxTJSRlZR1b9LcFHcvs13q5IUiAUPKH+R8a3tgr0lk/yxb/VeD+469sXYieuyKEFfJ ZbCTBWBvawykgRHFtq7htlMM6UCN9nf+XHqBg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=OIUbmRAwJF/KYOx+Wa5EL7QVgbRRgH57tcrVyMQ4olnUML5zOqNUJBo09Q/jH5EE8s xFXmH3JxAa/OV4YUT4O7jkLO1NY8Wjdx3yZWcjkT+8kcs0zE6e/RDqfeSrf5R1m6u5Vs eH0oDPRGyKLbXWMhk+NR1oVWARbSNJMWFFOr4= MIME-Version: 1.0 Received: by 10.210.127.10 with SMTP id z10mr4363470ebc.66.1238069111372; Thu, 26 Mar 2009 05:05:11 -0700 (PDT) In-Reply-To: <3.0.1.32.20090326065337.00f081e0@sage-american.com> References: <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> Date: Thu, 26 Mar 2009 13:05:11 +0100 Message-ID: <14989d6e0903260505n41122fb4j768ccecf7c5833a0@mail.gmail.com> From: Christian Walther To: "Jack L. Stone" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 12:32:31 -0000 Hi, I followed this thread out of interest since I do not suffer from this error. But I wonder if truss could shed some light into this issue. If for example a dump hangs at 99%, it might be an idea to set up truss to trace the dump process. Yes, this will produce lots of output, but maybe it gives a hint as soon as dump hangs. Christian Walther From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 13:32:12 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD993106564A; Thu, 26 Mar 2009 13:32:12 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by mx1.freebsd.org (Postfix) with ESMTP id A1BAD8FC1D; Thu, 26 Mar 2009 13:32:12 +0000 (UTC) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.14.3/8.14.3) with ESMTP id n2QDVd4b038485; Thu, 26 Mar 2009 09:31:39 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <200903261331.n2QDVd4b038485@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 26 Mar 2009 09:32:17 -0400 To: "Jack L. Stone" From: Mike Tancsa In-Reply-To: <3.0.1.32.20090326070807.00f081e0@sage-american.com> References: <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 13:32:13 -0000 At 08:08 AM 3/26/2009, Jack L. Stone wrote: >Yes, but it's for running a dump on a (L)ive FS and just spits out warnings >to that effect and has no effect on solving the problem(s). Unless the filesystem is very busy, you will get your data backed up. If you have things like databases, I still would not trust snapshots. Better to use pg_dump or mysqldump or the app that comes with whatever DB you are using... When backing up things like / and /usr, I would hazard a guess that most things are not changing while the backup is running, at least they dont in my environments. I have never had a problem with things like /home and even /var or /mail which are changing quite a bit. We dont restore much in the course of our daily routine, but we have always been able to restore people's Maildir when they accidentally have deleted stuff and it all worked without issue over the years. ---Mike From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 14:01:33 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9690A106566C; Thu, 26 Mar 2009 14:01:33 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id 4F8968FC17; Thu, 26 Mar 2009 14:01:33 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 492FB23C4E9; Thu, 26 Mar 2009 15:01:32 +0100 (CET) Date: Thu, 26 Mar 2009 15:01:32 +0100 From: Peter Schuller To: Mike Tancsa Message-ID: <20090326140131.GA45201@hyperion.scode.org> References: <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="J/dobhs11T7y2rNN" Content-Disposition: inline In-Reply-To: <200903261331.n2QDVd4b038485@lava.sentex.ca> User-Agent: Mutt/1.5.19 (2009-01-05) Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 14:01:34 -0000 --J/dobhs11T7y2rNN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > >Yes, but it's for running a dump on a (L)ive FS and just spits out warni= ngs > >to that effect and has no effect on solving the problem(s). >=20 > Unless the filesystem is very busy, you will get your data backed up.=20 > If you have things like databases, I still would not trust=20 > snapshots. =20 Uh. If backuping up a live database from a snapshot is not trustworthy, either the snapshot facility is broken or the database is broken (i.e., not crash-safe to begin with). That said there are plenty of other reasos to use proper dump tools (data portability, confirming the ability to actually read all rows =66rom a table, using a more often exercised code path and perhaps less likely to have edge case bugs, etc). --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --J/dobhs11T7y2rNN Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAknLirsACgkQDNor2+l1i31ZEgCdEqNBg7vRQorydzFVUmRMFI3H UVcAnA6sie1EgWonOybD7Ce/RRoZ/DgD =3wsx -----END PGP SIGNATURE----- --J/dobhs11T7y2rNN-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 14:15:41 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57561106568B; Thu, 26 Mar 2009 14:15:41 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by mx1.freebsd.org (Postfix) with ESMTP id 17A308FC24; Thu, 26 Mar 2009 14:15:40 +0000 (UTC) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.14.3/8.14.3) with ESMTP id n2QEF7hF038711; Thu, 26 Mar 2009 10:15:08 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <200903261415.n2QEF7hF038711@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 26 Mar 2009 10:15:45 -0400 To: Peter Schuller From: Mike Tancsa In-Reply-To: <20090326140131.GA45201@hyperion.scode.org> References: <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 14:15:43 -0000 At 10:01 AM 3/26/2009, Peter Schuller wrote: > > > > Unless the filesystem is very busy, you will get your data backed up. > > If you have things like databases, I still would not trust > > snapshots. > >Uh. If backuping up a live database from a snapshot is not >trustworthy, either the snapshot facility is broken or the database is >broken (i.e., not crash-safe to begin with). ... or the database is configured in a risky way... But judging by this thread, it seems dump with -L is indeed broken for some people. Hence, I suggested dumping the database using the database's backup tools and trying dump without -L ---Mike From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 14:35:06 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F280106568B for ; Thu, 26 Mar 2009 14:35:06 +0000 (UTC) (envelope-from karl@denninger.net) Received: from FS.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id 534568FC2D for ; Thu, 26 Mar 2009 14:35:06 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (localhost [127.0.0.1]) by FS.denninger.net (8.14.3/8.13.1) with SMTP id n2QEHQHW097205 for ; Thu, 26 Mar 2009 08:17:28 -0600 (CST) (envelope-from karl@denninger.net) Received: from [127.0.0.1] [192.168.1.40] by Spamblock-sys (LOCAL); Thu Mar 26 08:17:26 2009 Message-ID: <49CB8E74.9090508@denninger.net> Date: Thu, 26 Mar 2009 09:17:24 -0500 From: Karl Denninger User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Peter Schuller References: <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> In-Reply-To: <20090326140131.GA45201@hyperion.scode.org> Content-Type: multipart/mixed; boundary="------------010506020106090705000204" X-Antivirus: avast! (VPS 090325-0, 03/25/2009), Outbound message X-Antivirus-Status: Clean X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org, Mike Tancsa Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 14:35:07 -0000 This is a multi-part message in MIME format. --------------010506020106090705000204 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Peter Schuller wrote: >>> Yes, but it's for running a dump on a (L)ive FS and just spits out warnings >>> to that effect and has no effect on solving the problem(s). >>> >> Unless the filesystem is very busy, you will get your data backed up. >> If you have things like databases, I still would not trust >> snapshots. >> > > Uh. If backuping up a live database from a snapshot is not > trustworthy, either the snapshot facility is broken or the database is > broken (i.e., not crash-safe to begin with). > > That said there are plenty of other reasos to use proper dump tools > (data portability, confirming the ability to actually read all rows > from a table, using a more often exercised code path and perhaps less > likely to have edge case bugs, etc). > The issue with backing up a database live comes in when the filesystem where the database and transaction log files are DIFFERS. You can get into a pathological case in that instance. If the transaction log and database itself are both on the same snapshotted entity (that is, the snapshot is pulled at the same instant in time for both) what you get BETTER be restorable or your database's transaction log facility doesn't really do what it promises to do! -- -- Karl Denninger karl@denninger.net --------------010506020106090705000204-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 14:49:22 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A8783106566C; Thu, 26 Mar 2009 14:49:22 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id 638058FC14; Thu, 26 Mar 2009 14:49:22 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 8D79923C4E8; Thu, 26 Mar 2009 15:49:21 +0100 (CET) Date: Thu, 26 Mar 2009 15:49:21 +0100 From: Peter Schuller To: Mike Tancsa Message-ID: <20090326144916.GB46345@hyperion.scode.org> References: <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> <200903261415.n2QEF7hF038711@lava.sentex.ca> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EuxKj2iCbKjpUGkD" Content-Disposition: inline In-Reply-To: <200903261415.n2QEF7hF038711@lava.sentex.ca> User-Agent: Mutt/1.5.19 (2009-01-05) Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 14:49:23 -0000 --EuxKj2iCbKjpUGkD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > ... or the database is configured in a risky way... But judging by=20 > this thread, it seems dump with -L is indeed broken for some=20 > people. Hence, I suggested dumping the database using the database's=20 > backup tools and trying dump without -L Well, if dump -L is really broken I'd recommend just not using dump unless you absolutely have to. That is assuming snapshots are not broken so that you can still use mksnap_ffs with your favorite backup tool. --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --EuxKj2iCbKjpUGkD Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAknLlesACgkQDNor2+l1i31pWQCg7JKitrc1ZcyG3DprKKFt2LDS cdsAnRmjvH/i6AWKtfHHHRQhw6g8Ub3A =G1pu -----END PGP SIGNATURE----- --EuxKj2iCbKjpUGkD-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 14:52:34 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDD281065713; Thu, 26 Mar 2009 14:52:34 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id 95CAA8FC1A; Thu, 26 Mar 2009 14:52:34 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id E6C2423C4FF; Thu, 26 Mar 2009 15:52:33 +0100 (CET) Date: Thu, 26 Mar 2009 15:52:33 +0100 From: Peter Schuller To: Karl Denninger Message-ID: <20090326145230.GC46345@hyperion.scode.org> References: <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> <49CB8E74.9090508@denninger.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="8NvZYKFJsRX2Djef" Content-Disposition: inline In-Reply-To: <49CB8E74.9090508@denninger.net> User-Agent: Mutt/1.5.19 (2009-01-05) Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org, Mike Tancsa Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 14:52:38 -0000 --8NvZYKFJsRX2Djef Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > The issue with backing up a database live comes in when the filesystem=20 > where the database and transaction log files are DIFFERS. You can get=20 > into a pathological case in that instance. >=20 > If the transaction log and database itself are both on the same=20 > snapshotted entity (that is, the snapshot is pulled at the same instant= =20 > in time for both) what you get BETTER be restorable or your database's=20 > transaction log facility doesn't really do what it promises to do! Absolutely. Doing things like snapshot based backups of databases assumes you know what you're doing since it is not something which is documented as an official procedure in your typical database administrator guide. Personally, while I would use such schemes, I would always use a plain fully supported regular dump as a fallback position. I would only rely on snapshot based processes to do fancy stuff (such as near-realtime hot standby with zfs snaps + serialized incrementals). --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --8NvZYKFJsRX2Djef Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAknLlq0ACgkQDNor2+l1i31tTgCdFx23SoVJBUKLPlnVAvAGV8rv zrAAoM7Wu7Qb0S8BlRJXsWmtNtOX+Dm6 =5ZS7 -----END PGP SIGNATURE----- --8NvZYKFJsRX2Djef-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 14:57:20 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 253C1106566C for ; Thu, 26 Mar 2009 14:57:20 +0000 (UTC) (envelope-from karl@denninger.net) Received: from FS.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id CF8238FC1D for ; Thu, 26 Mar 2009 14:57:19 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (localhost [127.0.0.1]) by FS.denninger.net (8.14.3/8.13.1) with SMTP id n2QEvJ9o098818 for ; Thu, 26 Mar 2009 08:57:20 -0600 (CST) (envelope-from karl@denninger.net) Received: from [127.0.0.1] [192.168.1.40] by Spamblock-sys (LOCAL); Thu Mar 26 08:57:19 2009 Message-ID: <49CB97CD.2010303@denninger.net> Date: Thu, 26 Mar 2009 09:57:17 -0500 From: Karl Denninger User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Peter Schuller References: <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> <49CB8E74.9090508@denninger.net> <20090326145230.GC46345@hyperion.scode.org> In-Reply-To: <20090326145230.GC46345@hyperion.scode.org> Content-Type: multipart/mixed; boundary="------------060006040003040509080806" X-Antivirus: avast! (VPS 090325-0, 03/25/2009), Outbound message X-Antivirus-Status: Clean X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 14:57:20 -0000 This is a multi-part message in MIME format. --------------060006040003040509080806 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Peter Schuller wrote: >> The issue with backing up a database live comes in when the filesystem >> where the database and transaction log files are DIFFERS. You can get >> into a pathological case in that instance. >> >> If the transaction log and database itself are both on the same >> snapshotted entity (that is, the snapshot is pulled at the same instant >> in time for both) what you get BETTER be restorable or your database's >> transaction log facility doesn't really do what it promises to do! >> > > Absolutely. Doing things like snapshot based backups of databases > assumes you know what you're doing since it is not something which is > documented as an official procedure in your typical database > administrator guide. > > Personally, while I would use such schemes, I would always use a plain > fully supported regular dump as a fallback position. I would only rely > on snapshot based processes to do fancy stuff (such as near-realtime > hot standby with zfs snaps + serialized incrementals). > To add to this what SHOULD (ha!) work is to dump the database partition FIRST and THEN dump the Transaction Log partition. If you do it in the other order you WILL get screwed, as you will have transactions committed in the database that are not in the XLOG. That is essentially guaranteed to blow up in your face. As always any backup scheme has to be TESTED so you can prove to your own satisfaction that it is RESTORABLE. I can't tell you how many business clients I have run into (and not only on Unix machines) that have wind up with lots of backups and NONE of them can be restored - because they never TESTED their backup strategy. -- -- Karl Denninger karl@denninger.net --------------060006040003040509080806-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 15:11:01 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ACBFA1065670; Thu, 26 Mar 2009 15:11:01 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id 65A7D8FC0C; Thu, 26 Mar 2009 15:11:01 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 95D4323C4E8; Thu, 26 Mar 2009 16:11:00 +0100 (CET) Date: Thu, 26 Mar 2009 16:11:00 +0100 From: Peter Schuller To: Karl Denninger Message-ID: <20090326151100.GA46864@hyperion.scode.org> References: <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> <49CB8E74.9090508@denninger.net> <20090326145230.GC46345@hyperion.scode.org> <49CB97CD.2010303@denninger.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="TB36FDmn/VVEgNH/" Content-Disposition: inline In-Reply-To: <49CB97CD.2010303@denninger.net> User-Agent: Mutt/1.5.19 (2009-01-05) Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 15:11:04 -0000 --TB36FDmn/VVEgNH/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > To add to this what SHOULD (ha!) work is to dump the database partition= =20 > FIRST and THEN dump the Transaction Log partition. Depending on how the database works; specifically when old data in the log may or may not be expunged. You could do this with PostgreSQL by using it's PITR support for example. But to the extent that you have a log which is only supposed to be used for internal reasons (such as with pg by default), you'd likely be in trouble anyway unless you had a specific reason to know that it is safe. > As always any backup scheme has to be TESTED so you can prove to your=20 > own satisfaction that it is RESTORABLE. I can't tell you how many=20 > business clients I have run into (and not only on Unix machines) that=20 > have wind up with lots of backups and NONE of them can be restored -=20 > because they never TESTED their backup strategy. Very true, but it is equally dangerous to rely on testing *only*; a backup system can be very very broken yet appear to work during testing, either because backups only break sometimes or because they break in ways that do not obviously and immediately blow up in your face. --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --TB36FDmn/VVEgNH/ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAknLmwMACgkQDNor2+l1i31jFQCfQ1YTYyriN8KwFSkAMrWvNTQo m6MAn1Mq0krTUayQbZt6oHjFbqHb5JMA =NlCs -----END PGP SIGNATURE----- --TB36FDmn/VVEgNH/-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 15:16:01 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32F4510656D7 for ; Thu, 26 Mar 2009 15:16:01 +0000 (UTC) (envelope-from karl@denninger.net) Received: from FS.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id E90DE8FC18 for ; Thu, 26 Mar 2009 15:16:00 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (localhost [127.0.0.1]) by FS.denninger.net (8.14.3/8.13.1) with SMTP id n2QFG0TB099643 for ; Thu, 26 Mar 2009 09:16:01 -0600 (CST) (envelope-from karl@denninger.net) Received: from [127.0.0.1] [192.168.1.40] by Spamblock-sys (LOCAL); Thu Mar 26 09:16:01 2009 Message-ID: <49CB9C2E.8060401@denninger.net> Date: Thu, 26 Mar 2009 10:15:58 -0500 From: Karl Denninger User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Peter Schuller References: <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> <49CB8E74.9090508@denninger.net> <20090326145230.GC46345@hyperion.scode.org> <49CB97CD.2010303@denninger.net> <20090326151100.GA46864@hyperion.scode.org> In-Reply-To: <20090326151100.GA46864@hyperion.scode.org> Content-Type: multipart/mixed; boundary="------------090708050607050807000208" X-Antivirus: avast! (VPS 090325-0, 03/25/2009), Outbound message X-Antivirus-Status: Clean X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 15:16:05 -0000 This is a multi-part message in MIME format. --------------090708050607050807000208 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Peter Schuller wrote: >> To add to this what SHOULD (ha!) work is to dump the database partition >> FIRST and THEN dump the Transaction Log partition. >> > > Depending on how the database works; specifically when old data in the > log may or may not be expunged. You could do this with PostgreSQL by > using it's PITR support for example. But to the extent that you have a > log which is only supposed to be used for internal reasons (such as > with pg by default), you'd likely be in trouble anyway unless you had > a specific reason to know that it is safe. > True. If the log rolls on you while you're taking the dumps you're DEAD. Both design and verification are important :) -- -- Karl Denninger karl@denninger.net --------------090708050607050807000208-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 15:24:16 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 746071065670; Thu, 26 Mar 2009 15:24:16 +0000 (UTC) (envelope-from jake@poptart.org) Received: from plum.flirble.org (plum.flirble.org [194.70.3.20]) by mx1.freebsd.org (Postfix) with ESMTP id 322EE8FC12; Thu, 26 Mar 2009 15:24:15 +0000 (UTC) (envelope-from jake@poptart.org) Received: from jacob (helo=localhost) by plum.flirble.org with local-esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Lmqq2-0005KC-I4; Thu, 26 Mar 2009 14:45:46 +0000 Date: Thu, 26 Mar 2009 14:45:45 +0000 (GMT) From: Jake Scott To: Peter Schuller In-Reply-To: <20090326140131.GA45201@hyperion.scode.org> Message-ID: References: <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org, Mike Tancsa Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 15:24:17 -0000 Hi.. On Thu, 26 Mar 2009, Peter Schuller wrote: >>> Yes, but it's for running a dump on a (L)ive FS and just spits out warnings >>> to that effect and has no effect on solving the problem(s). >> >> Unless the filesystem is very busy, you will get your data backed up. >> If you have things like databases, I still would not trust >> snapshots. > > Uh. If backuping up a live database from a snapshot is not > trustworthy, either the snapshot facility is broken or the database is > broken (i.e., not crash-safe to begin with). Exactly right - if you backup a database by relying on storage snapshots then the best you will end up with is a database that needs to be recovered when you restore those volumes (crash consistent). That's not a good place to be in when your production DB has just blown up. In addition, there are all sorts of complications which mean that you might need to freeze IO on multiple volumes simultaneously in order for this to have a chance of being successful (maintaining write-order-fidelity). I would strongly discourage anyone from using this method of backup for anything that is considered production, thought it might do you for making QA clones of a running database. > That said there are plenty of other reasos to use proper dump tools > (data portability, confirming the ability to actually read all rows > from a table, using a more often exercised code path and perhaps less > likely to have edge case bugs, etc). Absolutely. You really must use a tool that interacts with the database to perform the backup. Most commercial DBs have hooks that allow the backup routines to call out to custom snapshot facilities. One would usually request a backup through the database, which would then freeze IO to its data files and maybe log files, deal with flushing caches etc and then call your snapshot routine. I'm not aware that MySQL and Postgres do though so the best you can do is a dump. Jake From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 15:40:50 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E0AEE10656C5; Thu, 26 Mar 2009 15:40:49 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id 93BB18FC18; Thu, 26 Mar 2009 15:40:49 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id C596923C4E9; Thu, 26 Mar 2009 16:40:48 +0100 (CET) Date: Thu, 26 Mar 2009 16:40:48 +0100 From: Peter Schuller To: Jake Scott Message-ID: <20090326154048.GA47539@hyperion.scode.org> References: <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="nFreZHaLTZJo0R7j" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.19 (2009-01-05) Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org, Mike Tancsa Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 15:40:57 -0000 --nFreZHaLTZJo0R7j Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > Exactly right - if you backup a database by relying on storage snapshots= =20 > then the best you will end up with is a database that needs to be=20 > recovered when you restore those volumes (crash consistent). That's not = a=20 > good place to be in when your production DB has just blown up. If you have a production db that you care about, your database better not have trouble recoverying from a crash consistent state. But again I'm not suggesting that snapshot based backups be the primary method of backing up. In terms of time-to-recovery, having a crash consistent DB can be a lot quicker to recover than grabbing a dump, whose restoration will tend to be a lot slower than copying files. > Absolutely. You really must use a tool that interacts with the database= =20 > to perform the backup. Most commercial DBs have hooks that allow the=20 > backup routines to call out to custom snapshot facilities. One would=20 > usually request a backup through the database, which would then freeze IO= =20 > to its data files and maybe log files, deal with flushing caches etc and= =20 > then call your snapshot routine. I'm not aware that MySQL and Postgres d= o=20 > though so the best you can do is a dump. I do not think "really must" is appropriate since clearly you can recover without DB specific integration. There may be reasons why it's better to have DB specific integration though (for example, limiting the amount of log reply that will be needed at recovery). The implication above that you cannot use snapshot based mechanisms with PostgreSQL and MySQL is not true; it's just that if you do you have to know what you're doing. --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --nFreZHaLTZJo0R7j Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAknLogAACgkQDNor2+l1i30jXwCfRpNF3CpzFl7K0QdjaQN8kuLr RtgAn2Vw6k/WuNwtIdtOXodlxhKsKaf2 =s3Az -----END PGP SIGNATURE----- --nFreZHaLTZJo0R7j-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 15:46:22 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CDB1B1065673 for ; Thu, 26 Mar 2009 15:46:22 +0000 (UTC) (envelope-from karl@denninger.net) Received: from FS.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id 939528FC17 for ; Thu, 26 Mar 2009 15:46:22 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (localhost [127.0.0.1]) by FS.denninger.net (8.14.3/8.13.1) with SMTP id n2QFkLbe001091 for ; Thu, 26 Mar 2009 09:46:22 -0600 (CST) (envelope-from karl@denninger.net) Received: from [127.0.0.1] [192.168.1.40] by Spamblock-sys (LOCAL); Thu Mar 26 09:46:22 2009 Message-ID: <49CBA34B.9070708@denninger.net> Date: Thu, 26 Mar 2009 10:46:19 -0500 From: Karl Denninger User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Jake Scott References: <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> In-Reply-To: Content-Type: multipart/mixed; boundary="------------030704080901050100070208" X-Antivirus: avast! (VPS 090325-0, 03/25/2009), Outbound message X-Antivirus-Status: Clean X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "Jack L. Stone" , fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 15:46:23 -0000 This is a multi-part message in MIME format. --------------030704080901050100070208 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Jake Scott wrote: > >> That said there are plenty of other reasos to use proper dump tools >> (data portability, confirming the ability to actually read all rows >> from a table, using a more often exercised code path and perhaps less >> likely to have edge case bugs, etc). > > Absolutely. You really must use a tool that interacts with the > database to perform the backup. Most commercial DBs have hooks that > allow the backup routines to call out to custom snapshot facilities. > One would usually request a backup through the database, which would > then freeze IO to its data files and maybe log files, deal with > flushing caches etc and then call your snapshot routine. I'm not > aware that MySQL and Postgres do though so the best you can do is a dump. > > Jake VERY careful thought has to go into backup strategy with production databases. Hooks that call out and snapshot are not necessarily good enough although they're "necessary" to get a dump that restores without the database going into log-replay mode. It is not difficult to do this with Postgresql; you can quiesce the database, snapshot and then release it, then dump the snapshots. This gives you transaction-complete dumps (as opposed to "crashed and rolled forward" dumps). The latter ("crashed and rolled forward"), if its sufficient, is trivially able to be done by having Postgresql (and most other databases) keep a sufficient number of log segments that a rollover cannot happen during the dump process itself, and either snapshotting both filesystems at once, keeping both on the same filesystem (undesirable for performance reasons) or dumping the database first and XLOG second. However, whether either of these approaches is sufficient is another matter. One of the real problems with live transaction processing systems is a means to know when there is a failure exactly what you lost. This is not a trivial problem to solve and requires plenty of thought before implementation, especially if you cannot afford the outage time necessary to take the snapshots - in some cases even that (relatively) short outage time is unacceptable. -- -- Karl Denninger karl@denninger.net --------------030704080901050100070208-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 15:53:51 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F401E106564A; Thu, 26 Mar 2009 15:53:50 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id AEB8B8FC1F; Thu, 26 Mar 2009 15:53:50 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id D5CBF23C4E9; Thu, 26 Mar 2009 16:53:49 +0100 (CET) Date: Thu, 26 Mar 2009 16:53:49 +0100 From: Peter Schuller To: Karl Denninger Message-ID: <20090326155349.GA47932@hyperion.scode.org> References: <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> <49CBA34B.9070708@denninger.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="azLHFNyN32YCQGCU" Content-Disposition: inline In-Reply-To: <49CBA34B.9070708@denninger.net> User-Agent: Mutt/1.5.19 (2009-01-05) Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 15:53:51 -0000 --azLHFNyN32YCQGCU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > However, whether either of these approaches is sufficient is another=20 > matter. One of the real problems with live transaction processing=20 > systems is a means to know when there is a failure exactly what you=20 > lost. This is not a trivial problem to solve and requires plenty of=20 > thought before implementation, especially if you cannot afford the=20 > outage time necessary to take the snapshots - in some cases even that=20 > (relatively) short outage time is unacceptable. I would like to point out that if the backup strategy is correct, a COMMIT is guaranteed to be correctly honored, and the problem of determining what was lost has more to do with the birds-eye view of to what point in time the database was reverted as part of emergency recovery, than any difficulty in understanding what actually happens during snapshot recovery. I completely understand that you have various requirements in production that makes it a non-solution to just get a consistent snapshot at some arbitrary point in time without synchronizing with other software components somehow, but such issuse are into the realm of application design and integration with the backup procedure, and we are no longer talking about the viability of obtaining a consistent backup of a single database through snapshotting. --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --azLHFNyN32YCQGCU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAknLpQ0ACgkQDNor2+l1i31CzQCg5UoDaK/ztD4cfo0QV+NkpFfX uaUAnjhT2maKRMhWwSQ8XZMUOMPrMhdh =jjGC -----END PGP SIGNATURE----- --azLHFNyN32YCQGCU-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 16:04:59 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE4F0106564A for ; Thu, 26 Mar 2009 16:04:59 +0000 (UTC) (envelope-from peter@simons-rock.edu) Received: from hedwig.simons-rock.edu (hedwig.simons-rock.edu [208.81.88.14]) by mx1.freebsd.org (Postfix) with ESMTP id AE9618FC0C for ; Thu, 26 Mar 2009 16:04:59 +0000 (UTC) (envelope-from peter@simons-rock.edu) Received: from cesium.hyperfine.info (c2.8d.5646.static.theplanet.com [70.86.141.194]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by hedwig.simons-rock.edu (Postfix) with ESMTP id B74002BB36B; Thu, 26 Mar 2009 11:35:30 -0400 (EDT) Date: Thu, 26 Mar 2009 11:35:29 -0400 From: "Peter C. Lai" To: Jake Scott Message-ID: <20090326153528.GH13398@cesium.hyperfine.info> References: <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.17 (2007-11-01) Cc: "Jack L. Stone" , fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 16:05:00 -0000 On 2009-03-26 02:45:45PM +0000, Jake Scott wrote: > Absolutely. You really must use a tool that interacts with the database to > perform the backup. Most commercial DBs have hooks that allow the backup > routines to call out to custom snapshot facilities. One would usually > request a backup through the database, which would then freeze IO to its > data files and maybe log files, deal with flushing caches etc and then call > your snapshot routine. I'm not aware that MySQL and Postgres do though so > the best you can do is a dump. With MySQL at least, you can (ab)use the replication facilities so that you can set up a "slave" and do the fs-level dump while the slave is in a "frozen" state - the last time I played with MySQL, you could basically desync your slave for a period of time (basically until transaction logs are purged on the master), during which the slave will be consistent; do the fs-level backup then kick the master to sync with the slave again. -- =========================================================== Peter C. Lai | Bard College at Simon's Rock Systems Administrator | 84 Alford Rd. Information Technology Svcs. | Gt. Barrington, MA 01230 USA peter AT simons-rock.edu | (413) 528-7428 =========================================================== From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 17:01:10 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A3A8106566C; Thu, 26 Mar 2009 17:01:10 +0000 (UTC) (envelope-from dwmalone@maths.tcd.ie) Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [IPv6:2001:770:10:300::86e2:510b]) by mx1.freebsd.org (Postfix) with SMTP id 4530D8FC15; Thu, 26 Mar 2009 17:01:09 +0000 (UTC) (envelope-from dwmalone@maths.tcd.ie) Received: from walton.maths.tcd.ie ([134.226.81.10] helo=walton.maths.tcd.ie) by salmon.maths.tcd.ie with SMTP id ; 26 Mar 2009 17:01:07 +0000 (GMT) Date: Thu, 26 Mar 2009 17:01:06 +0000 From: David Malone To: "Mikhail T." Message-ID: <20090326170106.GA28100@walton.maths.tcd.ie> References: <49C83673.3000604@aldan.algebra.com> <200903241537.36515.doconnor@gsoft.com.au> <49C87E0D.5090501@aldan.algebra.com> <49C92060.4060303@aldan.algebra.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <49C92060.4060303@aldan.algebra.com> User-Agent: Mutt/1.5.6i Sender: dwmalone@maths.tcd.ie Cc: Danny Braniss , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 17:01:10 -0000 On Tue, Mar 24, 2009 at 02:03:12PM -0400, Mikhail T. wrote: > Same problem: > > restore -rf ibmo.0.2009-03-24.dump > load: 0.55 cmd: restore 11303 [nbufkv] 3.53u 3.91s 4% 27980k > unknown tape header type 213474529 > abort? [yn] > > Please, advise. Thanks! Yours, Hi Mikhail, If you actually need to get a dump back that restore can't read, you can try the "-D" option that I added a few years ago. Dump and restore expect things to be in a block format, but if (say) dump outputs a few bytes into the stream due to a bug, then the entire end of the dump can become unreadable. The -D option to restore tells it to try hard to get back in sync again. I'd guess you've tripped over either a bug in dump or restore. If you can file a PR, particularly with access to a sample dump, then I can have a look and see if I can figure out what's going on. Daid. From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 17:07:34 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ABC3010656C1; Thu, 26 Mar 2009 17:07:34 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from mail.sagedata.net (mail.sagedata.net [63.214.156.21]) by mx1.freebsd.org (Postfix) with ESMTP id 650C98FC08; Thu, 26 Mar 2009 17:07:34 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from sagemaster (sageweb.net [65.68.247.73]) by mail.sagedata.net (8.14.3/8.14.3) with SMTP id n2QH7XGP084261; Thu, 26 Mar 2009 12:07:33 -0500 (CDT) (envelope-from jacks@sage-american.com) X-Authentication-Warning: mail.sagedata.net: Host sageweb.net [65.68.247.73] claimed to be sagemaster Message-Id: <3.0.1.32.20090326120732.00ee69f0@sage-american.com> X-Sender: jacks@sage-american.com X-Mailer: Windows Eudora Pro Version 3.0.1 (32) Date: Thu, 26 Mar 2009 12:07:32 -0500 To: Mike Tancsa From: "Jack L. Stone" In-Reply-To: <200903261331.n2QDVd4b038485@lava.sentex.ca> References: <3.0.1.32.20090326070807.00f081e0@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Scanned-By: milter-spamc/1.13.385 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 12:07:34 -0500 X-Scanned-By: milter-sender/1.16.915 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 12:07:34 -0500 X-Virus-Scanned: ClamAV 0.94.2/9170/Thu Mar 26 10:37:38 2009 on mail.sagedata.net X-Virus-Status: Clean X-Spam-Status: NO, hits=-10.00 required=4.50 X-Spam-Report: Content analysis details: (-10.0 points, 4.5 required) | | pts rule name description | ---- ---------------------- -------------------------------------------------- | -10 ALL_TRUSTED Passed through trusted hosts only via SMTP | Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 17:07:35 -0000 At 09:32 AM 3.26.2009 -0400, Mike Tancsa wrote: >At 08:08 AM 3/26/2009, Jack L. Stone wrote: > >>Yes, but it's for running a dump on a (L)ive FS and just spits out warnings >>to that effect and has no effect on solving the problem(s). > >Unless the filesystem is very busy, you will get your data backed up. >If you have things like databases, I still would not trust >snapshots. Better to use pg_dump or mysqldump or the app that comes >with whatever DB you are using... When backing up things like / and >/usr, I would hazard a guess that most things are not changing while >the backup is running, at least they dont in my environments. I have >never had a problem with things like /home and even /var or /mail >which are changing quite a bit. We dont restore much in the course >of our daily routine, but we have always been able to restore >people's Maildir when they accidentally have deleted stuff and it all >worked without issue over the years. > > ---Mike Yes, we have been using the "L" switch for as long as it has existed because we have and always before that, backed up live FSes. And, as said before, we do a dump/restore dump every morning to produce a bootable clone. The clones have always worked for the many years we have done this. Indeed, we have always used the MySQL's own dump separately to backup DBs. But, that's OT. Jack (^_^) Happy trails, Jack L. Stone System Admin Sage-american From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 17:09:25 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B74E61065687; Thu, 26 Mar 2009 17:09:25 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from mail.sagedata.net (mail.sagedata.net [63.214.156.21]) by mx1.freebsd.org (Postfix) with ESMTP id 71A628FC28; Thu, 26 Mar 2009 17:09:25 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from sagemaster (sageweb.net [65.68.247.73]) by mail.sagedata.net (8.14.3/8.14.3) with SMTP id n2QH9Oqh084376; Thu, 26 Mar 2009 12:09:24 -0500 (CDT) (envelope-from jacks@sage-american.com) X-Authentication-Warning: mail.sagedata.net: Host sageweb.net [65.68.247.73] claimed to be sagemaster Message-Id: <3.0.1.32.20090326120922.00ee69f0@sage-american.com> X-Sender: jacks@sage-american.com X-Mailer: Windows Eudora Pro Version 3.0.1 (32) Date: Thu, 26 Mar 2009 12:09:22 -0500 To: Mike Tancsa , Peter Schuller From: "Jack L. Stone" In-Reply-To: <200903261415.n2QEF7hF038711@lava.sentex.ca> References: <20090326140131.GA45201@hyperion.scode.org> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Scanned-By: milter-spamc/1.13.385 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 12:09:25 -0500 X-Scanned-By: milter-sender/1.16.915 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 12:09:24 -0500 X-Virus-Scanned: ClamAV 0.94.2/9170/Thu Mar 26 10:37:38 2009 on mail.sagedata.net X-Virus-Status: Clean X-Spam-Status: NO, hits=-10.00 required=4.50 X-Spam-Report: Content analysis details: (-10.0 points, 4.5 required) | | pts rule name description | ---- ---------------------- -------------------------------------------------- | -10 ALL_TRUSTED Passed through trusted hosts only via SMTP | Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 17:09:27 -0000 At 10:15 AM 3.26.2009 -0400, Mike Tancsa wrote: >At 10:01 AM 3/26/2009, Peter Schuller wrote: >> > >> > Unless the filesystem is very busy, you will get your data backed up. >> > If you have things like databases, I still would not trust >> > snapshots. >> >>Uh. If backuping up a live database from a snapshot is not >>trustworthy, either the snapshot facility is broken or the database is >>broken (i.e., not crash-safe to begin with). > >... or the database is configured in a risky way... But judging by >this thread, it seems dump with -L is indeed broken for some >people. Hence, I suggested dumping the database using the database's >backup tools and trying dump without -L > > ---Mike Mike: The "L" for dumping is definitely NOT the problem. Jack (^_^) Happy trails, Jack L. Stone System Admin Sage-american From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 17:11:15 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 061581065733; Thu, 26 Mar 2009 17:11:15 +0000 (UTC) (envelope-from petefrench@ticketswitch.com) Received: from constantine.ticketswitch.com (constantine.ticketswitch.com [IPv6:2002:57e0:1d4e:1::3]) by mx1.freebsd.org (Postfix) with ESMTP id BB79E8FC25; Thu, 26 Mar 2009 17:11:14 +0000 (UTC) (envelope-from petefrench@ticketswitch.com) Received: from dilbert.rattatosk ([10.64.50.6] helo=dilbert.ticketswitch.com) by constantine.ticketswitch.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Lmt6i-0004rZ-V7; Thu, 26 Mar 2009 17:11:08 +0000 Received: from petefrench by dilbert.ticketswitch.com with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Lmt6i-0000PI-T4; Thu, 26 Mar 2009 17:11:08 +0000 To: jake@poptart.org, peter.schuller@infidyne.com In-Reply-To: Message-Id: From: Pete French Date: Thu, 26 Mar 2009 17:11:08 +0000 Cc: jacks@sage-american.com, freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 17:11:16 -0000 > Absolutely. You really must use a tool that interacts with the database > to perform the backup. Most commercial DBs have hooks that allow the > backup routines to call out to custom snapshot facilities. One would > usually request a backup through the database, which would then freeze IO > to its data files and maybe log files, deal with flushing caches etc and > then call your snapshot routine. I'm not aware that MySQL and Postgres do > though so the best you can do is a dump. mysql can do this - you can flush the tables and acuire a lock simultaneously so that you can then snapshot the uderlying filesystem and then release the lock to let everything continue. I use this for taking database snapshots and it works fine. I stop my slaves before snapshotting to avoid log files changing underneath me too .... like this... #!/bin/sh /usr/local/bin/mysql -usnapuser -psnapuser < Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E36B21065786; Thu, 26 Mar 2009 17:11:59 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from mail.sagedata.net (mail.sagedata.net [63.214.156.21]) by mx1.freebsd.org (Postfix) with ESMTP id B57478FC21; Thu, 26 Mar 2009 17:11:59 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from sagemaster (sageweb.net [65.68.247.73]) by mail.sagedata.net (8.14.3/8.14.3) with SMTP id n2QHBwbV084589; Thu, 26 Mar 2009 12:11:59 -0500 (CDT) (envelope-from jacks@sage-american.com) X-Authentication-Warning: mail.sagedata.net: Host sageweb.net [65.68.247.73] claimed to be sagemaster Message-Id: <3.0.1.32.20090326121157.00ee69f0@sage-american.com> X-Sender: jacks@sage-american.com X-Mailer: Windows Eudora Pro Version 3.0.1 (32) Date: Thu, 26 Mar 2009 12:11:57 -0500 To: Peter Schuller , Mike Tancsa From: "Jack L. Stone" In-Reply-To: <20090326144916.GB46345@hyperion.scode.org> References: <200903261415.n2QEF7hF038711@lava.sentex.ca> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> <200903261415.n2QEF7hF038711@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Scanned-By: milter-spamc/1.13.385 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 12:11:59 -0500 X-Scanned-By: milter-sender/1.16.915 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 12:11:59 -0500 X-Virus-Scanned: ClamAV 0.94.2/9170/Thu Mar 26 10:37:38 2009 on mail.sagedata.net X-Virus-Status: Clean X-Spam-Status: NO, hits=-10.00 required=4.50 X-Spam-Report: Content analysis details: (-10.0 points, 4.5 required) | | pts rule name description | ---- ---------------------- -------------------------------------------------- | -10 ALL_TRUSTED Passed through trusted hosts only via SMTP | Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tapeheadertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 17:12:02 -0000 At 03:49 PM 3.26.2009 +0100, Peter Schuller wrote: >> ... or the database is configured in a risky way... But judging by >> this thread, it seems dump with -L is indeed broken for some >> people. Hence, I suggested dumping the database using the database's >> backup tools and trying dump without -L > >Well, if dump -L is really broken I'd recommend just not using dump >unless you absolutely have to. That is assuming snapshots are not >broken so that you can still use mksnap_ffs with your favorite backup >tool. > >-- >/ Peter Schuller > No one has said the dump "L" is broken -- and is NOT but now may mislead others just tuning into this thread. Jack (^_^) Happy trails, Jack L. Stone System Admin Sage-american From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 17:23:49 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EEAC1065724; Thu, 26 Mar 2009 17:23:49 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from mail.sagedata.net (mail.sagedata.net [63.214.156.21]) by mx1.freebsd.org (Postfix) with ESMTP id C6EF08FC15; Thu, 26 Mar 2009 17:23:48 +0000 (UTC) (envelope-from jacks@sage-american.com) Received: from sagemaster (sageweb.net [65.68.247.73]) by mail.sagedata.net (8.14.3/8.14.3) with SMTP id n2QHNjH2085212; Thu, 26 Mar 2009 12:23:45 -0500 (CDT) (envelope-from jacks@sage-american.com) X-Authentication-Warning: mail.sagedata.net: Host sageweb.net [65.68.247.73] claimed to be sagemaster Message-Id: <3.0.1.32.20090326122344.00ee69f0@sage-american.com> X-Sender: jacks@sage-american.com X-Mailer: Windows Eudora Pro Version 3.0.1 (32) Date: Thu, 26 Mar 2009 12:23:44 -0500 To: Karl Denninger , Jake Scott From: "Jack L. Stone" In-Reply-To: <49CBA34B.9070708@denninger.net> References: <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Scanned-By: milter-spamc/1.13.385 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 12:23:46 -0500 X-Scanned-By: milter-sender/1.16.915 (mail.sagedata.net [63.214.156.21]); Thu, 26 Mar 2009 12:23:46 -0500 X-Virus-Scanned: ClamAV 0.94.2/9170/Thu Mar 26 10:37:38 2009 on mail.sagedata.net X-Virus-Status: Clean X-Spam-Status: NO, hits=-10.00 required=4.50 X-Spam-Report: Content analysis details: (-10.0 points, 4.5 required) | | pts rule name description | ---- ---------------------- -------------------------------------------------- | -10 ALL_TRUSTED Passed through trusted hosts only via SMTP | Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 17:23:52 -0000 At 10:46 AM 3.26.2009 -0500, Karl Denninger wrote: >Jake Scott wrote: >> >>> That said there are plenty of other reasos to use proper dump tools >>> (data portability, confirming the ability to actually read all rows >>> from a table, using a more often exercised code path and perhaps less >>> likely to have edge case bugs, etc). >> This thread has drifted off the main issue of using the FS dump/restore problem. Although, the DBs may be included as a part of this dump, it should NOT be relied upon as a backup/restore of DBs. We use MySQL's dump/backup tools strictly for the DBs. The dumps of the DBs (using MySQL's own dump tool) here are run by cron jobs every hour or few hours at most. That keeps our DB backups very recent and near current as possible is a restore is needed. Jack (^_^) Happy trails, Jack L. Stone System Admin Sage-american From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 17:36:44 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AD4810656CA for ; Thu, 26 Mar 2009 17:36:44 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-fx0-f167.google.com (mail-fx0-f167.google.com [209.85.220.167]) by mx1.freebsd.org (Postfix) with ESMTP id A541F8FC19 for ; Thu, 26 Mar 2009 17:36:43 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: by fxm11 with SMTP id 11so649000fxm.43 for ; Thu, 26 Mar 2009 10:36:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:subject:from:to:cc :in-reply-to:references:content-type:date:message-id:mime-version :x-mailer:content-transfer-encoding; bh=4X+bXXEfsdrDlXkMPxAHfDK5WuKa5x7vOhQU3ZA9bPM=; b=KqVrgu14mhALpedyKTBRpIl89xAKphxpdfOfLvP04qggL114W3hwq/J2eVReh+YlbZ IbKK3Ij9GdJL9Cs5qWTA+0P5fr9A1/sqR3SoX5vMaLObPwQWZyBlXTzULu2vVSHSFQ9Q UcNKa+jeWhc3lmV5HirCCYiQ7dVv+M+qraT3A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=subject:from:to:cc:in-reply-to:references:content-type:date :message-id:mime-version:x-mailer:content-transfer-encoding; b=si7Y3NHPt9X/QBfGSByiCVmmrOCz5el53TGBEke8/4wwQ4zaug3fZmW2ZFnGRixPy0 uuui0/VkXINPehKl6tLYq0z7IVw+ajqUTx2EZit7/PDkDTP5WuDksGAt4Pc63VbfdFbf SjK3PbIKNds2pTykFHr4vbzJ3BUSeAnvGDJRs= Received: by 10.86.94.11 with SMTP id r11mr1399444fgb.53.1238087477636; Thu, 26 Mar 2009 10:11:17 -0700 (PDT) Received: from ?127.0.0.1? (87-194-39-182.bethere.co.uk [87.194.39.182]) by mx.google.com with ESMTPS id e11sm643209fga.20.2009.03.26.10.11.16 (version=SSLv3 cipher=RC4-MD5); Thu, 26 Mar 2009 10:11:17 -0700 (PDT) From: Tom Evans To: Jake Scott In-Reply-To: References: <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> Content-Type: text/plain Date: Thu, 26 Mar 2009 17:11:15 +0000 Message-Id: <1238087475.7491.44.camel@strangepork.mintel.co.uk> Mime-Version: 1.0 X-Mailer: Evolution 2.22.3.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: "Jack L. Stone" , fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tape headertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 17:36:44 -0000 On Thu, 2009-03-26 at 14:45 +0000, Jake Scott wrote: .. > Absolutely. You really must use a tool that interacts with the database > to perform the backup. Most commercial DBs have hooks that allow the > backup routines to call out to custom snapshot facilities. One would > usually request a backup through the database, which would then freeze IO > to its data files and maybe log files, deal with flushing caches etc and > then call your snapshot routine. I'm not aware that MySQL and Postgres do > though so the best you can do is a dump. > > > Jake > Just to add, mysql has a utility (mysqlhotcopy) to allow you to directly copy MyISAM databases with a guarentee of consistency (thus avoiding the conversion from MyISAM data -> SQL, and no need to reimport when recovering). It isn't exactly online though, any writes will be blocked until the hotcopy finishes. Still, it is only MyISAM, and not much call for that these days.. Cheers Tom From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 18:18:58 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5EA0106566C; Thu, 26 Mar 2009 18:18:58 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by mx1.freebsd.org (Postfix) with ESMTP id A97688FC1B; Thu, 26 Mar 2009 18:18:58 +0000 (UTC) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.14.3/8.14.3) with ESMTP id n2QIIPih039864; Thu, 26 Mar 2009 14:18:25 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <200903261818.n2QIIPih039864@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 26 Mar 2009 14:19:03 -0400 To: "Jack L. Stone" From: Mike Tancsa In-Reply-To: <3.0.1.32.20090326121157.00ee69f0@sage-american.com> References: <200903261415.n2QEF7hF038711@lava.sentex.ca> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> <200903261415.n2QEF7hF038711@lava.sentex.ca> <3.0.1.32.20090326121157.00ee69f0@sage-american.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tapeheadertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 18:18:59 -0000 At 01:11 PM 3/26/2009, Jack L. Stone wrote: >No one has said the dump "L" is broken -- and is NOT but now may mislead >others just tuning into this thread. OK, sorry I misunderstood that restore worked from dump files without -L.... I wonder if the original poster just needs to do an fsck on the disk. The only time I had dump crap out on me (other than with -L) was on a dirty filesystem. I regularly do stuff like cd /usr;dump 0f - /usr | (cd /tmp/usr; restore -rf - ) to duplicate partitions ---Mike From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 18:48:03 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71F8A1065670 for ; Thu, 26 Mar 2009 18:48:03 +0000 (UTC) (envelope-from karl@denninger.net) Received: from FS.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id 0F31B8FC17 for ; Thu, 26 Mar 2009 18:48:02 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (localhost [127.0.0.1]) by FS.denninger.net (8.14.3/8.13.1) with SMTP id n2QIm2tc009144 for ; Thu, 26 Mar 2009 12:48:02 -0600 (CST) (envelope-from karl@denninger.net) Received: from [127.0.0.1] [192.168.1.40] by Spamblock-sys (LOCAL); Thu Mar 26 12:48:02 2009 Message-ID: <49CBCDE0.60401@denninger.net> Date: Thu, 26 Mar 2009 13:48:00 -0500 From: Karl Denninger User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Mike Tancsa References: <200903261415.n2QEF7hF038711@lava.sentex.ca> <49C9E635.5010106@kkip.pl> <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> <200903261415.n2QEF7hF038711@lava.sentex.ca> <3.0.1.32.20090326121157.00ee69f0@sage-american.com> <200903261818.n2QIIPih039864@lava.sentex.ca> In-Reply-To: <200903261818.n2QIIPih039864@lava.sentex.ca> Content-Type: multipart/mixed; boundary="------------080600020607090902090707" X-Antivirus: avast! (VPS 090325-0, 03/25/2009), Outbound message X-Antivirus-Status: Clean X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "Jack L. Stone" , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: support quality (Re: dump | restore fails: unknown tapeheadertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 18:48:03 -0000 This is a multi-part message in MIME format. --------------080600020607090902090707 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Mike Tancsa wrote: > At 01:11 PM 3/26/2009, Jack L. Stone wrote: > >> No one has said the dump "L" is broken -- and is NOT but now may mislead >> others just tuning into this thread. > > > OK, sorry I misunderstood that restore worked from dump files without > -L.... > I wonder if the original poster just needs to do an fsck on the disk. > The only time I had dump crap out on me (other than with -L) was on a > dirty filesystem. > > I regularly do stuff like > cd /usr;dump 0f - /usr | (cd /tmp/usr; restore -rf - ) > to duplicate partitions > > ---Mike I do as well but if the original user is trying to restore into a filesystem that is either not clean or not EMPTY for a full restore he could run into some trouble. I've never had dump/restore fail by the way, other than due to bad (tape) media (yes, it was that long ago :) ) -- -- Karl Denninger karl@denninger.net --------------080600020607090902090707-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 18:48:38 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0381E106566C; Thu, 26 Mar 2009 18:48:38 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (cl-1361.ams-04.nl.sixxs.net [IPv6:2001:960:2:550::2]) by mx1.freebsd.org (Postfix) with ESMTP id B1CA48FC19; Thu, 26 Mar 2009 18:48:37 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 8A7F023C4E9; Thu, 26 Mar 2009 19:48:36 +0100 (CET) Date: Thu, 26 Mar 2009 19:48:36 +0100 From: Peter Schuller To: "Jack L. Stone" Message-ID: <20090326184835.GA53678@hyperion.scode.org> References: <49C83673.3000604@aldan.algebra.com> <200903251820.54749.doconnor@gsoft.com.au> <200903251925.36108.doconnor@gsoft.com.au> <3.0.1.32.20090325072137.00ee6b48@sage-american.com> <3.0.1.32.20090326065337.00f081e0@sage-american.com> <3.0.1.32.20090326070807.00f081e0@sage-american.com> <200903261331.n2QDVd4b038485@lava.sentex.ca> <20090326140131.GA45201@hyperion.scode.org> <200903261415.n2QEF7hF038711@lava.sentex.ca> <3.0.1.32.20090326121157.00ee69f0@sage-american.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0F1p//8PRICkK4MW" Content-Disposition: inline In-Reply-To: <3.0.1.32.20090326121157.00ee69f0@sage-american.com> User-Agent: Mutt/1.5.19 (2009-01-05) Cc: freebsd-stable@freebsd.org, fs@freebsd.org, Mike Tancsa Subject: Re: support quality (Re: dump | restore fails: unknown tapeheadertype 1853384566) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 18:48:39 -0000 --0F1p//8PRICkK4MW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > >> ... or the database is configured in a risky way... But judging by=20 > >> this thread, it seems dump with -L is indeed broken for some=20 > >> people. Hence, I suggested dumping the database using the database's= =20 > >> backup tools and trying dump without -L > > > >Well, if dump -L is really broken I'd recommend just not using dump > >unless you absolutely have to. That is assuming snapshots are not > >broken so that you can still use mksnap_ffs with your favorite backup > >tool. > > > >--=20 > >/ Peter Schuller > > >=20 > No one has said the dump "L" is broken -- and is NOT but now may mislead > others just tuning into this thread. I was commenting on the specific statement that I quoted above. I am not claiming anything about dump being broken. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --0F1p//8PRICkK4MW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAknLzgMACgkQDNor2+l1i30mCACdHi1kdOQ9T9IelzGGvhV/MPZB 9UMAn2Hvl/WX+uyoDMS1++6btL1gfuEU =4aWc -----END PGP SIGNATURE----- --0F1p//8PRICkK4MW-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 19:10:08 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4BD91065673 for ; Thu, 26 Mar 2009 19:10:08 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B559E8FC21 for ; Thu, 26 Mar 2009 19:10:08 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2QJA8QO015950 for ; Thu, 26 Mar 2009 19:10:08 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2QJA8HF015949; Thu, 26 Mar 2009 19:10:08 GMT (envelope-from gnats) Date: Thu, 26 Mar 2009 19:10:08 GMT Message-Id: <200903261910.n2QJA8HF015949@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Jaakko Heinonen Cc: Subject: Re: kern/132068: [zfs] page fault when using ZFS over NFS on 7.1-RELEASE/amd64 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Jaakko Heinonen List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 19:10:09 -0000 The following reply was made to PR kern/132068; it has been noted by GNATS. From: Jaakko Heinonen To: Edward Fisk <7ogcg7g02@sneakemail.com> Cc: bug-followup@FreeBSD.org Subject: Re: kern/132068: [zfs] page fault when using ZFS over NFS on 7.1-RELEASE/amd64 Date: Thu, 26 Mar 2009 21:01:10 +0200 I was able to trigger a panic on current using this patch: %%% Index: sys/nfsserver/nfs_srvsubs.c =================================================================== --- sys/nfsserver/nfs_srvsubs.c (revision 190316) +++ sys/nfsserver/nfs_srvsubs.c (working copy) @@ -1169,6 +1169,8 @@ nfsrv_fhtovp(fhandle_t *fhp, int lockfla vfs_unbusy(mp); if (error) goto out; + if ((*vpp)->v_type == VBAD) + panic("VBAD *vpp in nfsrv_fhtovp()"); #ifdef MNT_EXNORESPORT if (!(exflags & (MNT_EXNORESPORT|MNT_EXPUBLIC))) { saddr = (struct sockaddr_in *)nam; %%% #2 0xc08537d2 in panic (fmt=Variable "fmt" is not available.) at /home/jaakko/src/head/sys/kern/kern_shutdown.c:576 #3 0xc0a1c17e in nfsrv_fhtovp (fhp=0xf48ecae4, lockflag=1, vpp=0xf48ecad8, vfslockedp=0xf48ecac4, nfsd=0xf48ecbb0, slp=0x0, nam=0xc63aeac4, rdonlyp=0xf48ecad0, pubflag=1) at /home/jaakko/src/head/sys/nfsserver/nfs_srvsubs.c:1173 #4 0xc0a0ff20 in nfsrv_commit (nfsd=0xf48ecbb0, slp=0x0, mrq=0xf48ecba8) at /home/jaakko/src/head/sys/nfsserver/nfs_serv.c:3836 #5 0xc0a1b1a7 in nfssvc_program (rqst=0xca95d000, xprt=0xc63aea00) at /home/jaakko/src/head/sys/nfsserver/nfs_srvkrpc.c:420 #6 0xc0a35fe2 in svc_run_internal (pool=0xc57a7a80, ismaster=0) at /home/jaakko/src/head/sys/rpc/svc.c:883 #7 0xc0a362c0 in svc_thread_start (arg=0xc57a7a80) at /home/jaakko/src/head/sys/rpc/svc.c:1188 #8 0xc082fd38 in fork_exit (callout=0xc0a362b0 , arg=0xc57a7a80, frame=0xf48ecd38) at /home/jaakko/src/head/sys/kern/kern_fork.c:821 #9 0xc0b41630 in fork_trampoline () at /home/jaakko/src/head/sys/i386/i386/exception.s:270 I now know what is going on. The vnode may be reclaimed during zfs_zget() because it doesn't hold the vnode lock (except when a new znode is created). I tried to modify zfs_zget() to grab the vnode lock and check if the vnode is doomed. I was able to trigger the condition. However eventually vnode locking deadlocked with lookup(). I used exclusive locking so maybe one could avoid deadlocking with shared locking but it isn't a real solution especially because shared lookups can be disabled. Also other deadlock scenarios may exist. On 2009-03-05, Edward Fisk wrote: > The machine did a panic earlier, but was unfortunately unable to do a dump. > > panic: Bad link elm 0xffffff014f3e2400 prev->next != elm > xprt_unregister_locked() at xprt_unregister_locked+0xad This is a different issue. You may have already seen this discussion: http://lists.freebsd.org/pipermail/freebsd-current/2009-March/005097.html -- Jaakko From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 22:11:49 2009 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5EBAC1065672; Thu, 26 Mar 2009 22:11:49 +0000 (UTC) (envelope-from eugene@donpac.ru) Received: from cops.gladchenko.ru (cops.gladchenko.ru [195.161.106.100]) by mx1.freebsd.org (Postfix) with ESMTP id 8A19B8FC29; Thu, 26 Mar 2009 22:11:47 +0000 (UTC) (envelope-from eugene@donpac.ru) Received: from cerberus.rnd.cbr.ru (cerberus.rnd.cbr.ru [194.84.224.97]) by cops.gladchenko.ru (8.14.3/8.14.2) with ESMTP id n2QLb2Ma018234; Fri, 27 Mar 2009 00:37:03 +0300 (MSK) (envelope-from eugene@donpac.ru) Received: from [10.10.10.2] by cerberus.rnd.cbr.ru with ESMTP id n2QLaukR059546; Fri, 27 Mar 2009 00:36:57 +0300 (MSK) (envelope-from eugene@donpac.ru) X-Antivirus: Dr.Web (R) for Mail Servers on cops.gladchenko.ru host X-AntiVirus: Checked by Dr.Web [version: 4.44, engine: 4.44.0.09170, virus records: 553984, updated: 26.03.2009] Date: Fri, 27 Mar 2009 00:36:48 +0300 From: Eugene Gladchenko X-Priority: 3 (Normal) Message-ID: <725997207.20090327003648@donpac.ru> To: "Mikhail T." In-Reply-To: <49C83673.3000604@aldan.algebra.com> References: <49C83673.3000604@aldan.algebra.com> MIME-Version: 1.0 Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="----------E4199112C0777C1" X-Antivirus-Code: 100000 X-Spam-Status: No, score=-101.3 required=5.0 tests=AWL,BAYES_00,RDNS_NONE, USER_IN_WHITELIST autolearn=no version=3.2.5 X-Spam-Report: * -100 USER_IN_WHITELIST From: address is in the user's white-list * -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1% * [score: 0.0023] * 0.1 RDNS_NONE Delivered to trusted network by a host with no rDNS * 1.2 AWL AWL: From: address is in the auto white-list X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on cops.gladchenko.ru X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: stable@FreeBSD.org, fs@FreeBSD.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Eugene Gladchenko List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 22:11:50 -0000 This is a cryptographically signed message in MIME format. ------------E4199112C0777C1 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Mikhail, MT> I'm trying to migrate a filesystem from one disk to another using: MT> dump a0hCf 0 32 - /old | restore -rf - MT> (/old is already mounted read-only). The process runs for a while and MT> then stops with: MT> [...] MT> DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 2009 MT> DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 2009 MT> DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 2009 MT> unknown tape header type 1853384566 MT> abort? [yn] MT> Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD's MT> dump's output? Is your /tmp large enough to restore? --=20 Eugene Gladchenko EVG15-RIPE ------------E4199112C0777C1-- From owner-freebsd-fs@FreeBSD.ORG Thu Mar 26 22:48:35 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C3C11065670; Thu, 26 Mar 2009 22:48:35 +0000 (UTC) (envelope-from listreader@lazlarlyricon.com) Received: from proxy2.bredband.net (proxy2.bredband.net [195.54.101.72]) by mx1.freebsd.org (Postfix) with ESMTP id 35EAF8FC17; Thu, 26 Mar 2009 22:48:35 +0000 (UTC) (envelope-from listreader@lazlarlyricon.com) Received: from ironport2.bredband.com (195.54.101.122) by proxy2.bredband.net (7.3.139) id 49CB855700032DD1; Thu, 26 Mar 2009 23:27:54 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgMtAE+ey0lV44ctPGdsb2JhbACBUJQnAQEBATW+HoN3Bg X-IronPort-AV: E=Sophos;i="4.38,428,1233529200"; d="scan'208";a="468348048" Received: from c-2d87e355.09-42-6e6b7010.cust.bredbandsbolaget.se (HELO trapper.homedns.org) ([85.227.135.45]) by ironport2.bredband.com with ESMTP; 26 Mar 2009 23:27:53 +0100 Message-ID: <49CC0168.70502@lazlarlyricon.com> Date: Thu, 26 Mar 2009 23:27:52 +0100 From: Rolf G Nielsen User-Agent: Thunderbird 2.0.0.19 (X11/20090221) MIME-Version: 1.0 To: "Mikhail T." References: <49C83673.3000604@aldan.algebra.com> In-Reply-To: <49C83673.3000604@aldan.algebra.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2009 22:48:36 -0000 Mikhail T. wrote: > Hello! > > I'm trying to migrate a filesystem from one disk to another using: > > dump a0hCf 0 32 - /old | restore -rf - Try dump -a0 -h 0 -C 32 -f - /old instead. Your command line should work, but I've had trouble with similar ones and have been able to solve it by separating the options. > > (/old is already mounted read-only). The process runs for a while and > then stops with: > > [...] > DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 2009 > DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 2009 > DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 2009 > unknown tape header type 1853384566 > abort? [yn] > > Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD's > dump's output? > > The system runs 7.0-STABLE from July 6th, i386. I just tried updating > the dump and restore from source (latest 7.x) -- the error is the same... > > Please, advise. Thanks! > > -mi > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > > > -- Sincerly, Rolf Nielsen From owner-freebsd-fs@FreeBSD.ORG Fri Mar 27 01:11:53 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCD6E1065672 for ; Fri, 27 Mar 2009 01:11:53 +0000 (UTC) (envelope-from freebsd-nospam@yaxom.com) Received: from gw.yaxom.com (gw.yaxom.com [59.167.217.197]) by mx1.freebsd.org (Postfix) with SMTP id 24AC18FC18 for ; Fri, 27 Mar 2009 01:11:52 +0000 (UTC) (envelope-from freebsd-nospam@yaxom.com) Received: (qmail 75227 invoked from network); 27 Mar 2009 11:11:51 +1000 Received: from joker.yaxom.com (172.16.1.10) by iliad.yaxom.com with SMTP; 27 Mar 2009 11:11:51 +1000 Received: (qmail 2891 invoked by uid 1001); 27 Mar 2009 11:11:51 +1000 Message-ID: Date: Fri, 27 Mar 2009 11:11:50 +1000 From: Greg Black To: "Mikhail T." References: <49C83673.3000604@aldan.algebra.com> <200903251232.11418.doconnor@gsoft.com.au> <49C99204.2050601@aldan.algebra.com> <200903251334.38350.doconnor@gsoft.com.au> <49C99FD2.50609@aldan.algebra.com> <49CA5602.9050001@aldan.algebra.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <49CA5602.9050001@aldan.algebra.com> User-Agent: Mutt/1.4.2.2i; gjb-muttsend.sh 1.7 2004-10-05 X-Uptime: 126 days X-Operating-System: FreeBSD 6.2-RELEASE-p5 i386 X-Location: Brisbane, Australia; 27.49835S 152.98380E X-URL: http://www.gbch.net/gjb.html X-Blog: http://www.gbch.net/gjb/blog/ X-Image-URL: http://www.gbch.net/gjb/gjb-auug048.gif X-PGP-Key-Fingerprint: EBB2 2A92 A79D 1533 AC00 3C46 5D83 B6FB 4B04 B7D6 X-Request-PGP: http://www.gbch.net/keys/4B04B7D6.asc Cc: Daniel O'Connor , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Mar 2009 01:11:54 -0000 On 2009-03-25, Mikhail T. wrote: > Greg Black ???????(??): >> On 2009-03-24, Mikhail T. wrote: >> >>> That's true. I just wanted to point out, that someone running dump only >>> (to make backups) is not going to know, whether his dumps are usable >>> (for whichever of the two reasons), until he needs them... >> >> Such a person is not making backups and deserves what he gets. >> > But he *is* making backups -- kindly re-read my paragraph above... He > just is not routinely using them and thus does not know, that they > aren't usable... It is not unreasonable to expect the two utilities to > "just work", so I wouldn't be blaming such a person for not testing > restore. Sorry, this person is *not* making backups in any meaningful fashion. Unless you verify regularly (preferably every time you make a backup) that you can restore both parts of the backup and the entire thing, you are not making backups. You may be lucky, but you probably won't be lucky when it really matters. This is a long-standing issue with backups. But people who don't understand the tools they are using tend to make these mistakes. The result is bad. The solution is to employ competent people who both understand the issues and have the authority to carry out their task. From owner-freebsd-fs@FreeBSD.ORG Fri Mar 27 01:42:26 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE9A8106566B for ; Fri, 27 Mar 2009 01:42:26 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx22.fluidhosting.com [204.14.89.5]) by mx1.freebsd.org (Postfix) with ESMTP id 68BB68FC13 for ; Fri, 27 Mar 2009 01:42:26 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: (qmail 3650 invoked by uid 399); 27 Mar 2009 01:15:43 -0000 Received: from localhost (HELO lap.dougb.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 27 Mar 2009 01:15:43 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <49CC28BD.1070501@FreeBSD.org> Date: Thu, 26 Mar 2009 18:15:41 -0700 From: Doug Barton Organization: http://www.FreeBSD.org/ User-Agent: Thunderbird 2.0.0.21 (X11/20090321) MIME-Version: 1.0 To: Greg Black References: <49C83673.3000604@aldan.algebra.com> <200903251232.11418.doconnor@gsoft.com.au> <49C99204.2050601@aldan.algebra.com> <200903251334.38350.doconnor@gsoft.com.au> <49C99FD2.50609@aldan.algebra.com> <49CA5602.9050001@aldan.algebra.com> In-Reply-To: X-Enigmail-Version: 0.95.7 OpenPGP: id=D5B2F0FB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "Mikhail T." , freebsd-stable@freebsd.org, fs@freebsd.org Subject: Re: dump | restore fails: unknown tape header type 1853384566 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Mar 2009 01:42:27 -0000 Greg Black wrote: > Sorry, this person is *not* making backups in any meaningful fashion. > Unless you verify regularly (preferably every time you make a backup) > that you can restore both parts of the backup and the entire thing, you > are not making backups. You may be lucky, but you probably won't be > lucky when it really matters. s/making backups/employing an effective data recovery strategy/ Please don't feed the "proper use of language" trolls. :) Meanwhile, we may want to consider that this whole thread is veering off topic ... Doug -- This .signature sanitized for your protection From owner-freebsd-fs@FreeBSD.ORG Fri Mar 27 01:52:48 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1886E106564A; Fri, 27 Mar 2009 01:52:48 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id EED798FC0C; Fri, 27 Mar 2009 01:52:46 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2R1qYiG039571; Thu, 26 Mar 2009 21:52:37 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49CC3162.5090201@aldan.algebra.com> Date: Thu, 26 Mar 2009 21:52:34 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: Greg Black References: <49C83673.3000604@aldan.algebra.com> <200903251232.11418.doconnor@gsoft.com.au> <49C99204.2050601@aldan.algebra.com> <200903251334.38350.doconnor@gsoft.com.au> <49C99FD2.50609@aldan.algebra.com> <49CA5602.9050001@aldan.algebra.com> In-Reply-To: Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: Daniel O'Connor , freebsd-stable@freebsd.org, fs@freebsd.org Subject: backup strategy (Re: dump | restore fails) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Mar 2009 01:52:48 -0000 Greg Black ΞΑΠΙΣΑΧ(ΜΑ): > Sorry, this person is *not* making backups in any meaningful fashion. > Unless you verify regularly (preferably every time you make a backup) > that you can restore both parts of the backup and the entire thing, you > are not making backups. To qualify for your (and your kind's) recognition then, a person needs to have at least as much extra storage capacity as the largest filesystem they are backing up. They also need non-trivial scripting abilities, because the OS doesn't include anything like what you are describing (and I already do consider scheduling dumps via cron "trivial", which may be a stretch). Yours may thus be an acceptable requirement for a multi-computer shop with dedicated system administration personnel, but for a private home user with only one computer this simply is not reasonable. Stating this as a requirement is ridiculous -- unless you are prepared to say, that such people should not own a computer (with worthy data) at all. And that's even more ridiculous... Make your pick. I would agree with you, if the chosen backup method involved some complex or third-party tools. But if the simple, OS-supplied orthogonal dump/restore don't work together, then the OS is broken -- plain and simple, and pointing a finger at the user: "Well, it is all your fault, because you relied on us providing you with working utilities, ha-ha-ha!" -- is the lamest excuse imaginable. -mi P.S. Some people have actually volunteered to help debug this problem and I'm working on providing them with data (the troublesome partition is, sadly, over 170Gb, so it takes a while). Any results/conclusions will be posted under the original subject. P.P.S. The data transferred fine using tar, but that is not the point -- the bug (confirmed by at least one more person) -- needs to be fixed before a higher-profile embarrassment... From owner-freebsd-fs@FreeBSD.ORG Fri Mar 27 02:25:27 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40CAE106566B for ; Fri, 27 Mar 2009 02:25:27 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (email.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id F034E8FC0A for ; Fri, 27 Mar 2009 02:25:26 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id 646BE17E9B; Fri, 27 Mar 2009 13:25:37 +1100 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.1.50.60] (ppp121-44-35-16.lns10.syd7.internode.on.net [121.44.35.16]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id 611BB17E8F; Fri, 27 Mar 2009 13:25:33 +1100 (EST) Message-ID: <49CC3889.9000201@modulus.org> Date: Fri, 27 Mar 2009 13:23:05 +1100 From: Andrew Snow User-Agent: Thunderbird 2.0.0.14 (X11/20080523) MIME-Version: 1.0 To: "Mikhail T." , freebsd-stable@freebsd.org, freebsd-fs@freebsd.org References: <49C83673.3000604@aldan.algebra.com> <200903251232.11418.doconnor@gsoft.com.au> <49C99204.2050601@aldan.algebra.com> <200903251334.38350.doconnor@gsoft.com.au> <49C99FD2.50609@aldan.algebra.com> <49CA5602.9050001@aldan.algebra.com> <49CC3162.5090201@aldan.algebra.com> In-Reply-To: <49CC3162.5090201@aldan.algebra.com> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: backup strategy (Re: dump | restore fails) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Mar 2009 02:25:27 -0000 Mikhail T. wrote: > To qualify for your (and your kind's) recognition then, a person > needs to have at least as much extra storage capacity as the > largest filesystem they are backing up. They also need > non-trivial scripting abilities, because the OS doesn't > include anything like what you are describing Mikhail, users would be well advised to check their backups using this option, without having to have the space to restore: -N Do the extraction normally, but do not actually write any changes to disk. This can be used to check the integrity of dump media or other test purposes. RESTORE(8) FreeBSD System Manager's Manual From owner-freebsd-fs@FreeBSD.ORG Fri Mar 27 03:02:49 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AFE9F106564A for ; Fri, 27 Mar 2009 03:02:49 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by mx1.freebsd.org (Postfix) with ESMTP id 8587C8FC08 for ; Fri, 27 Mar 2009 03:02:46 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.14.3/8.14.3) with ESMTP id n2R2SkvO039674; Thu, 26 Mar 2009 22:28:49 -0400 (EDT) (envelope-from mi+thun@aldan.algebra.com) Message-ID: <49CC39DE.3060604@aldan.algebra.com> Date: Thu, 26 Mar 2009 22:28:46 -0400 From: "Mikhail T." User-Agent: Thunderbird 2.0.0.18 (X11/20081126) MIME-Version: 1.0 To: Andrew Snow References: <49C83673.3000604@aldan.algebra.com> <200903251232.11418.doconnor@gsoft.com.au> <49C99204.2050601@aldan.algebra.com> <200903251334.38350.doconnor@gsoft.com.au> <49C99FD2.50609@aldan.algebra.com> <49CA5602.9050001@aldan.algebra.com> <49CC3162.5090201@aldan.algebra.com> <49CC3889.9000201@modulus.org> In-Reply-To: <49CC3889.9000201@modulus.org> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: backup strategy (Re: dump | restore fails) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Mar 2009 03:02:49 -0000 Andrew Snow ΞΑΠΙΣΑΧ(ΜΑ): > Mikhail T. wrote: > > To qualify for your (and your kind's) recognition then, a person > > needs to have at least as much extra storage capacity as the > > largest filesystem they are backing up. They also need > > non-trivial scripting abilities, because the OS doesn't > > include anything like what you are describing > > Mikhail, users would be well advised to check their backups using this > option, without having to have the space to restore: > > -N Do the extraction normally, but do not actually write any > changes to disk. This can be used to check the integrity of dump media > or other test purposes. And then another "purist" will reject this as a fake and not sufficiently conclusive test... Just ask around... Because, you see, until you extract all data back and run a bit-by-bit comparison with the original, you don't really know, do you? -mi From owner-freebsd-fs@FreeBSD.ORG Fri Mar 27 20:12:34 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA6EA1065672; Fri, 27 Mar 2009 20:12:34 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 800CA8FC15; Fri, 27 Mar 2009 20:12:34 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2RKCXim090526; Fri, 27 Mar 2009 20:12:34 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2RKCXTW090513; Fri, 27 Mar 2009 20:12:33 GMT (envelope-from linimon) Date: Fri, 27 Mar 2009 20:12:33 GMT Message-Id: <200903272012.n2RKCXTW090513@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/133134: [zfs] Missing ZFS zpool labels X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Mar 2009 20:12:35 -0000 Old Synopsis: Missing ZFS zpool labels New Synopsis: [zfs] Missing ZFS zpool labels Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Fri Mar 27 20:12:07 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=133134 From owner-freebsd-fs@FreeBSD.ORG Sat Mar 28 17:53:41 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B1BF1065670 for ; Sat, 28 Mar 2009 17:53:41 +0000 (UTC) (envelope-from juri_mian@yahoo.com) Received: from n71.bullet.mail.sp1.yahoo.com (n71.bullet.mail.sp1.yahoo.com [98.136.44.36]) by mx1.freebsd.org (Postfix) with SMTP id 491348FC18 for ; Sat, 28 Mar 2009 17:53:41 +0000 (UTC) (envelope-from juri_mian@yahoo.com) Received: from [216.252.122.219] by n71.bullet.mail.sp1.yahoo.com with NNFMP; 28 Mar 2009 17:40:34 -0000 Received: from [69.147.65.165] by t4.bullet.sp1.yahoo.com with NNFMP; 28 Mar 2009 17:40:34 -0000 Received: from [127.0.0.1] by omp500.mail.sp1.yahoo.com with NNFMP; 28 Mar 2009 17:40:34 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 433965.68838.bm@omp500.mail.sp1.yahoo.com Received: (qmail 178 invoked by uid 60001); 28 Mar 2009 17:40:34 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1238262034; bh=EHueQM6FqVBXsn0fjDFkdBLSu/sWCrD/G5shdle2ja8=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=MOj9b0+kl4r9rD6wRuK+m6gA32Dlac4d//N0SjhJO40HXhRr1XDmvsMLJPWkjLaJs6/L/1r7HB77d3UP8cY/54f/zhgFGQPLa3qirABTCTT2K51Gn2qPST44FwqflFm8gW9/ZQzdotNyFJU36K54bCSUOYJ2rkkeFRYVutx1onk= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=sjDXZfUSstaruQ41DGMvrsvbAG9eFVRg4UTSsEmc+C4l708DmW4AWeBZC7MzzRXl4ETMdm67e1GDu/1zdAdZfZT6gelIN+cRjoJGDRuHpmzmJMEz6sNfqydiJkKUdTkmjlmOhhavQp6Cj3qhsoQLbSQnW8cJ5QEpPFx3SX0Xmes=; Message-ID: <286854.110.qm@web45606.mail.sp1.yahoo.com> X-YMail-OSG: KR7HVykVM1kjTSvm2sjPJwfargQBi8eAJQHJHkEa01neRoHKRuABEJpIi5Hbq_BySgfe5VEUAShgdCF0XZom8sXpS.88t4hMJk9dpiotTLlbqAcdMe940syRD9G_mqRq0VTnLd9M61hVQd5PTQKz.QhJhhRW8zdWf7iPpbTvgAA55kJ5bl7RmuzLbNfXQlDgXXAQyr9s781YFGd3mNtO_vc9SD7tuda6W6R8N_KtnjMS8Cu.kjY- Received: from [71.63.232.32] by web45606.mail.sp1.yahoo.com via HTTP; Sat, 28 Mar 2009 10:40:33 PDT X-Mailer: YahooMailClassic/5.1.20 YahooMailWebService/0.7.289.1 Date: Sat, 28 Mar 2009 10:40:33 -0700 (PDT) From: Juri Mianovich To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailman-Approved-At: Sat, 28 Mar 2009 18:25:05 +0000 Subject: quick advice needed RE: Intel Integrated Server RAID and FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Mar 2009 17:53:41 -0000 Hello, I am considering this motherboard: http://www.intel.com/products/server/motherboards/S5000PSL/S5000PSL-overview.htm The Intel S5000 PSL Server board, with "Intel Integrated Server RAID". I plan on using the onboard raid for a simple boot mirror. I have two quick questions: 1. What chipset/controller is this "Intel Integrated Server RAID" and is it well supported in FreeBSD ? Will sysinstall recognize it ? Is there a CLI tool ofr viewing status once FreeBSD is running ? 2. Is this _real_ hardware raid ? As in, FreeBSD will know _nothing_ about the raid and will simply see one disk (which is a mirror) ? I am seeing discussion that I don't understand about how this is somehow software raid, but If it's all built on the board ... ? Performance isn't a real issue since it is just a boot mirror. I just want to hide the raid layer from the OS and do my raid work (rebuilds, etc.) through the BIOS. Thanks. From owner-freebsd-fs@FreeBSD.ORG Sat Mar 28 23:26:15 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D63C010656C2; Sat, 28 Mar 2009 23:26:15 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id ABBA28FC1D; Sat, 28 Mar 2009 23:26:15 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (brucec@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2SNQFpE083931; Sat, 28 Mar 2009 23:26:15 GMT (envelope-from brucec@freefall.freebsd.org) Received: (from brucec@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2SNQF2g083927; Sat, 28 Mar 2009 23:26:15 GMT (envelope-from brucec) Date: Sat, 28 Mar 2009 23:26:15 GMT Message-Id: <200903282326.n2SNQF2g083927@freefall.freebsd.org> To: brucec@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: brucec@FreeBSD.org Cc: Subject: Re: kern/72424: panic: ffs_blkfree: freeing free block in ffs/ffs_alloc.c:1750 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Mar 2009 23:26:16 -0000 Synopsis: panic: ffs_blkfree: freeing free block in ffs/ffs_alloc.c:1750 Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: brucec Responsible-Changed-When: Sat Mar 28 23:25:51 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=72424 From owner-freebsd-fs@FreeBSD.ORG Sat Mar 28 23:28:22 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C549106566B; Sat, 28 Mar 2009 23:28:22 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 220A08FC0A; Sat, 28 Mar 2009 23:28:22 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (brucec@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2SNSMMD083999; Sat, 28 Mar 2009 23:28:22 GMT (envelope-from brucec@freefall.freebsd.org) Received: (from brucec@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2SNSMDQ083995; Sat, 28 Mar 2009 23:28:22 GMT (envelope-from brucec) Date: Sat, 28 Mar 2009 23:28:22 GMT Message-Id: <200903282328.n2SNSMDQ083995@freefall.freebsd.org> To: brucec@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: brucec@FreeBSD.org Cc: Subject: Re: kern/62824: [panic] softdep_setup_inomapdep: found inode [5.2-CURRENT] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Mar 2009 23:28:22 -0000 Synopsis: [panic] softdep_setup_inomapdep: found inode [5.2-CURRENT] Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: brucec Responsible-Changed-When: Sat Mar 28 23:28:00 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=62824 From owner-freebsd-fs@FreeBSD.ORG Sat Mar 28 23:30:53 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66DDD106566C; Sat, 28 Mar 2009 23:30:53 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3C9398FC16; Sat, 28 Mar 2009 23:30:53 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (brucec@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2SNUrdL088761; Sat, 28 Mar 2009 23:30:53 GMT (envelope-from brucec@freefall.freebsd.org) Received: (from brucec@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2SNUr00088757; Sat, 28 Mar 2009 23:30:53 GMT (envelope-from brucec) Date: Sat, 28 Mar 2009 23:30:53 GMT Message-Id: <200903282330.n2SNUr00088757@freefall.freebsd.org> To: brucec@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: brucec@FreeBSD.org Cc: Subject: Re: kern/92272: [ffs] [hang] Filling a filesystem while creating a snapshot on it locks the system X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Mar 2009 23:30:53 -0000 Synopsis: [ffs] [hang] Filling a filesystem while creating a snapshot on it locks the system Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: brucec Responsible-Changed-When: Sat Mar 28 23:30:32 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=92272 From owner-freebsd-fs@FreeBSD.ORG Sat Mar 28 23:32:35 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3260110656C4; Sat, 28 Mar 2009 23:32:35 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 084528FC19; Sat, 28 Mar 2009 23:32:35 +0000 (UTC) (envelope-from brucec@FreeBSD.org) Received: from freefall.freebsd.org (brucec@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2SNWYGc098216; Sat, 28 Mar 2009 23:32:34 GMT (envelope-from brucec@freefall.freebsd.org) Received: (from brucec@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2SNWYOc098212; Sat, 28 Mar 2009 23:32:34 GMT (envelope-from brucec) Date: Sat, 28 Mar 2009 23:32:34 GMT Message-Id: <200903282332.n2SNWYOc098212@freefall.freebsd.org> To: brucec@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: brucec@FreeBSD.org Cc: Subject: Re: kern/94769: [ufs] Multiple file deletions on multi-snapshotted filesystems causes hang X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Mar 2009 23:32:35 -0000 Synopsis: [ufs] Multiple file deletions on multi-snapshotted filesystems causes hang Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: brucec Responsible-Changed-When: Sat Mar 28 23:32:14 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=94769