From owner-freebsd-fs@FreeBSD.ORG Sun Feb 10 00:44:14 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE43B16A4DA; Sun, 10 Feb 2008 00:44:14 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (gate.funkthat.com [69.17.45.168]) by mx1.freebsd.org (Postfix) with ESMTP id 69F2913C442; Sun, 10 Feb 2008 00:44:13 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (oosbiwcaljco04a8@localhost.funkthat.com [127.0.0.1]) by hydrogen.funkthat.com (8.13.6/8.13.3) with ESMTP id m1A0TDeY067357; Sat, 9 Feb 2008 16:29:13 -0800 (PST) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.13.6/8.13.3/Submit) id m1A0TCZs067356; Sat, 9 Feb 2008 16:29:12 -0800 (PST) (envelope-from jmg) Date: Sat, 9 Feb 2008 16:29:12 -0800 From: John-Mark Gurney To: Nikolay Pavlov Message-ID: <20080210002912.GA7399@funkthat.com> Mail-Followup-To: Nikolay Pavlov , Joao Barros , Attilio Rao , Yar Tikhiy , Doug Barton , Jeff Roberson , freebsd-fs@freebsd.org, Scot Hetzel , freebsd-arch@freebsd.org References: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> <200802071941.23199.qpadla@gmail.com> <70e8236f0802071018n389afa3bu161eaa5c6563cbc0@mail.gmail.com> <200802072052.56918.qpadla@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200802072052.56918.qpadla@gmail.com> User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 5.4-RELEASE-p6 i386 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (hydrogen.funkthat.com [127.0.0.1]); Sat, 09 Feb 2008 16:29:13 -0800 (PST) Cc: freebsd-fs@freebsd.org, Doug Barton , Jeff Roberson , Yar Tikhiy , Attilio Rao , Scot Hetzel , freebsd-arch@freebsd.org Subject: Re: [RFC] Remove NTFS kernel support X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: John-Mark Gurney List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2008 00:44:14 -0000 Nikolay Pavlov wrote this message on Thu, Feb 07, 2008 at 20:52 +0200: > On Thursday 07 February 2008 20:18:42 Joao Barros wrote: > > On Feb 7, 2008 5:41 PM, Nikolay Pavlov wrote: > > > On Thursday 07 February 2008 14:47:41 Eric Anderson wrote: > > > > FUSE is slow, requires a port (unless PUFFS is ported, which I've > > > > probed about before). > > > > > > I think this is not an argument: > > > http://www.ntfs-3g.org/performance.html > > > > Eric has valid points. > > How relevant is a benchmark on Linux to your argument? > > But it's a userland application. This page is demonstration of it's > potential performance that could be achieved, but were is the FreeBSD NTFS > implementation stats? Let me ask you: compered to what FUSE is slow? Kernel NTFS support is about 10x faster than ntfs-3g on FreeBSD (I think this also depends upon the size of the file). This is because ntfs-3g depends upon the block device that linux provides to userland. There are patches that make ntfs-3g have it's own block cache that makes it perform decently on FreeBSD, but until those patches are integrated, using ntfs-3g is a non-starter if you use NTFS for >4GB file support. It's faster to use samba to a Windows box than it is to use ntfs-3g to write large files. (And that's even w/ how much slower samba is that nfs.) I don't have any hard core benchmarks handy. Even on MacOSX ntfs-3g is sooo slow. It's so slow, that I don't even both hooking up NTFS disks to my MacOSX box anymore either. Though I will say that once ntfs-3g has decided that they want to target other platforms than Linux and address these performance issues, I will be one of the first asking for our current NTFS code to be removed and replaced by ntfs-3g, but until that time, we need to keep the current code. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-fs@FreeBSD.ORG Mon Feb 11 11:07:04 2008 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87C0A16A41B for ; Mon, 11 Feb 2008 11:07:04 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 6CE9B13C457 for ; Mon, 11 Feb 2008 11:07:04 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m1BB74r2007373 for ; Mon, 11 Feb 2008 11:07:04 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m1BB73kX007369 for freebsd-fs@FreeBSD.org; Mon, 11 Feb 2008 11:07:03 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 11 Feb 2008 11:07:03 GMT Message-Id: <200802111107.m1BB73kX007369@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2008 11:07:04 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o kern/116170 fs [panic] Kernel panic when mounting /tmp 3 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o bin/118249 fs mv(1): moving a directory changes its mtime 2 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Feb 11 17:11:07 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5B6216A418; Mon, 11 Feb 2008 17:11:07 +0000 (UTC) (envelope-from gavin.atkinson@ury.york.ac.uk) Received: from mail-gw1.york.ac.uk (mail-gw1.york.ac.uk [144.32.128.246]) by mx1.freebsd.org (Postfix) with ESMTP id 61D6113C4D9; Mon, 11 Feb 2008 17:11:06 +0000 (UTC) (envelope-from gavin.atkinson@ury.york.ac.uk) Received: from mail-gw7.york.ac.uk (mail-gw7.york.ac.uk [144.32.129.30]) by mail-gw1.york.ac.uk (8.13.6/8.13.6) with ESMTP id m1BGdEX8006034; Mon, 11 Feb 2008 16:39:14 GMT Received: from buffy-128.york.ac.uk ([144.32.128.160] helo=buffy.york.ac.uk) by mail-gw7.york.ac.uk with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1JObgY-0003B9-Jr; Mon, 11 Feb 2008 16:39:14 +0000 Received: from buffy.york.ac.uk (localhost [127.0.0.1]) by buffy.york.ac.uk (8.14.2/8.14.2) with ESMTP id m1BGdEfU028425; Mon, 11 Feb 2008 16:39:14 GMT (envelope-from gavin.atkinson@ury.york.ac.uk) Received: (from ga9@localhost) by buffy.york.ac.uk (8.14.2/8.14.2/Submit) id m1BGdDes028424; Mon, 11 Feb 2008 16:39:13 GMT (envelope-from gavin.atkinson@ury.york.ac.uk) X-Authentication-Warning: buffy.york.ac.uk: ga9 set sender to gavin.atkinson@ury.york.ac.uk using -f From: Gavin Atkinson To: Joe Peterson In-Reply-To: <47ACF0AE.3040802@skyrush.com> References: <47ACD7D4.5050905@skyrush.com> <47ACDE82.1050100@skyrush.com> <20080208173517.rdtobnxqg4g004c4@www.wolves.k12.mo.us> <47ACF0AE.3040802@skyrush.com> Content-Type: text/plain Content-Transfer-Encoding: 7bit Date: Mon, 11 Feb 2008 16:39:13 +0000 Message-Id: <1202747953.27277.7.camel@buffy.york.ac.uk> Mime-Version: 1.0 X-Mailer: Evolution 2.10.1 FreeBSD GNOME Team Port X-York-MailScanner: Found to be clean X-York-MailScanner-From: gavin.atkinson@ury.york.ac.uk Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Analysis of disk file block with ZFS checksum error X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2008 17:11:08 -0000 On Fri, 2008-02-08 at 17:15 -0700, Joe Peterson wrote: > Chris Dillon wrote: > > That is a chunk of a Mozilla Mork-format database. Perhaps the > > Firefox URL history or address book from Thunderbird. > > Interesting (thanks to all who recognized Mork). I do use Firefox and > Thunderbird, so it's feasible, but how the heck would a piece of one of > those files find its way into 1/2 of a ZFS block in one of my mp3 files? > I wonder if it could have been done on write when the file was copied > to the ZFS pool (maybe some write-caching issue?), but I thought ZFS > would have verified the block after write. It seems unlikely that it > would get changed later - I never rewrote that file after the original > copy... Are the datestamps (Thu Jan 24 23:20:58 2008) found within the corrupt block before or after the datestamp of the file it was found within? i.e. was the corrupt block on the disk before or after the mp3 was written there? You could possibly confirm this by grepping for that datestamp in the files in your home directory, and with the aid of http://developer.mozilla.org/en/docs/Mork_Structure#Rows, try to establish exactly what the datestamp means (ie was it the time you visited a URL, etc). Gavin From owner-freebsd-fs@FreeBSD.ORG Mon Feb 11 19:39:13 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62F8D16A419; Mon, 11 Feb 2008 19:39:13 +0000 (UTC) (envelope-from joe@skyrush.com) Received: from shadow.wildlava.net (shadow.wildlava.net [67.40.138.81]) by mx1.freebsd.org (Postfix) with ESMTP id 1DBED13C447; Mon, 11 Feb 2008 19:39:12 +0000 (UTC) (envelope-from joe@skyrush.com) Received: from crater.wildlava.net (crater.wildlava.net [67.40.138.82]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by shadow.wildlava.net (Postfix) with ESMTP id 2B5F98F3B2; Mon, 11 Feb 2008 12:39:12 -0700 (MST) Message-ID: <47B0A45C.4090909@skyrush.com> Date: Mon, 11 Feb 2008 12:39:08 -0700 From: Joe Peterson User-Agent: Thunderbird 2.0.0.9 (X11/20071208) MIME-Version: 1.0 To: Gavin Atkinson References: <47ACD7D4.5050905@skyrush.com> <47ACDE82.1050100@skyrush.com> <20080208173517.rdtobnxqg4g004c4@www.wolves.k12.mo.us> <47ACF0AE.3040802@skyrush.com> <1202747953.27277.7.camel@buffy.york.ac.uk> In-Reply-To: <1202747953.27277.7.camel@buffy.york.ac.uk> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Analysis of disk file block with ZFS checksum error X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2008 19:39:13 -0000 Gavin Atkinson wrote: > Are the datestamps (Thu Jan 24 23:20:58 2008) found within the corrupt > block before or after the datestamp of the file it was found within? > i.e. was the corrupt block on the disk before or after the mp3 was > written there? Hi Gavin, those dated are later than the original copy (I do not have the file timestamps to prove this, but according to my email record, I am pretty sure of this). So the corrupt block is later than the original write. If this is the case, I assume that the block got written, by mistake, into the middle of the mp3 file. Someone else suggested that it could be caused by a bad transfer block number or bad drive command (corrupted on the way to the drive, since these are not checksummed in the hardware). If the block went to the wrong place, AND if it was a HW glitch, I suppose the best ZFS could then do is retry the write (if its failure was even detected - still not sure if ZFS does a re-check of the disk data checksum after the disk write), not knowing until the later scrub that the block had corrupted a file. I think that anything is possible, but I know I was getting periodic DMA timeouts, etc. around that time. I hesitate, although it is tempting, to use this evidence to focus blame purely on bad HW, given that others seem to be seeing DMA problems too, and there is reasonable doubt whether their problems are HW related or not. In my case, I have been free of DMA errors (cross your fingers) after re-installed FreeBSD completely (giving it a larger boot partition and redoing the ZFS slice too), and before this, I changed the IDE cable just to eliminate one more variable. Therefore, there are too many variables to reach a firm conclusion, since even if the cable was "bad", I never saw one DMA error or other indication of anything wrong with HW from the Linux side (and I've been using that HW with both Linux and FreeBSD 6.2 for months now - no apparent flakiness of any kind on either system). So either it *was* bad and FreeBSD 7.0 was being more "honest", FreeBSD's drivers and/or ZFS was stressing the HW and revealing weaknesses in the cable, or it was a SW issue that got cleared somehow when I re-installed. Is it possible that the problem lies in the ATA drivers in FreeBSD or even in ZFS and just looks like HW issues? I do not have enough info/expertise to know. If not, then it may very well be true that HW problems are pretty widespread (and that disk HW cannot, in fact, be trusted), and there really *is* a strong need for ZFS *now* to protect our data. If there is a possibility that SW could be involved, any hints on how to further debug this would be of great help to those still experiencing recent DMA errors. I just want to be more sure one way or the other, but I know this issue is not an easy one (however, it's the kind of problem that should receive the highest priority, IMHO). -Joe From owner-freebsd-fs@FreeBSD.ORG Mon Feb 11 22:24:46 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55F3B16A417; Mon, 11 Feb 2008 22:24:46 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from falcon.cybervisiontech.com (falcon.cybervisiontech.com [217.20.163.9]) by mx1.freebsd.org (Postfix) with ESMTP id C747A13C467; Mon, 11 Feb 2008 22:24:45 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from localhost (localhost [127.0.0.1]) by falcon.cybervisiontech.com (Postfix) with ESMTP id 8B6B0744003; Tue, 12 Feb 2008 00:24:43 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at falcon.cybervisiontech.com Received: from falcon.cybervisiontech.com ([127.0.0.1]) by localhost (falcon.cybervisiontech.com [127.0.0.1]) (amavisd-new, port 10027) with ESMTP id iVkIyE8l+1M6; Tue, 12 Feb 2008 00:24:43 +0200 (EET) Received: from [10.74.70.239] (unknown [193.138.145.53]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by falcon.cybervisiontech.com (Postfix) with ESMTP id 491FB744001; Tue, 12 Feb 2008 00:24:08 +0200 (EET) Message-ID: <47B0CAC8.2050603@icyb.net.ua> Date: Tue, 12 Feb 2008 00:23:04 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.9 (X11/20071208) MIME-Version: 1.0 To: Poul-Henning Kamp References: <200612221824.kBMIOhfM049471@freefall.freebsd.org> <47A2EDB0.8000801@icyb.net.ua> <47A2F404.7010208@icyb.net.ua> <47A7339E.4070708@icyb.net.ua> <47A73562.4010309@samsco.org> <47A9E062.3040409@icyb.net.ua> In-Reply-To: <47A9E062.3040409@icyb.net.ua> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, freebsd-hackers@FreeBSD.org, Pav Lucistnik , Bruce Evans Subject: Re: fs/udf: vm pages "overlap" while reading large dir X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2008 22:24:46 -0000 on 06/02/2008 18:29 Andriy Gapon said the following: > Small summary of the above long description. > For directory reading fs/udf performs bread() on a (underlying) device > vnode. It passes block number as if block size was 512 bytes (i.e. > byte_offset_within_dev/512). On the other hand vm page index calculation > code uses the following formula: (blkno*bo_bsize)/PAGE_SIZE. > For CD/DVD devices bo_bsize is set to 2048. This results in page index > overlap when reading sufficiently many blocks from adjacent starting blocks. > That is, it seems that in "normal" cases page index gets calculated as > byte_offset/PAGE_SIZE, but in this case page index gets to be > 4*byte_offset/PAGE_SIZE. More details are in the quoted above text. > > With a lot of help from Bruce Evance I found this commit which seems to > have made a big difference: > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/vfs_bio.c.diff?r1=1.458;r2=1.459;f=h > > Before this change page index for device vnodes was always > (blkno*512)/PAGE_SIZE. This is because of the vn_isdisk() case. > > udf_mountfs device vnode is passed to g_vfs_open() (in geom_vfs.c) and > the latter has the following line of code: > bo->bo_bsize = pp->sectorsize; > Where pp is geom provider for the device in question. > > I am not sure if the mentioned above vfs_bio.c change broke something > else in reading from device vnodes or if it fixed something for that. I > am also not sure what would be the consequences of setting bo_bsize to > 512 for vnodes of "disk" devices. It would most probably fix current > code in UDF, but I am not sure if it might break anything else. > > Just wanted to let the more knowledgeable people know and make a decision. > > P.S. bo_bsize seems to be referenced only in a handful of files and in > most of them it seems that it is related to "file" vnodes (as opposed to > "device" vnodes). > Paul, do you have any comment or opinion on the above? [sorry if clamped too much of the previous context, but it's all in archives] And a little continuation: Just for the sake of experiment I tried to emulated the previous behavior by changing geom_vfs.c, function g_vfs_open(), so that the relevant code reads as follows: if (vn_isdisk(vp, NULL)) bo->bo_bsize = DEV_BSIZE; else bo->bo_bsize = pp->sectorsize; It didn't break anything for me and it re-enabled current (CVS) fs/udf code to work as before. (patch from kern/78987 still has to be applied for large directories to be handled). So udf (on DVD) is fixed, ufs (on HDD), msdosfs (on HDD) and cd9660 (on CD) are not broken. Of course, the change should have affected only filesystems on CD/DVD (where sector/block size is not 512 bytes). Of course, unlike Bruce I don't use msdosfs on CD/DVD media (e.g. DVD-RAM), so my test is somewhat incomplete. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Mon Feb 11 22:39:10 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A72DC16A419; Mon, 11 Feb 2008 22:39:10 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from falcon.cybervisiontech.com (falcon.cybervisiontech.com [217.20.163.9]) by mx1.freebsd.org (Postfix) with ESMTP id 23C9913C478; Mon, 11 Feb 2008 22:39:10 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from localhost (localhost [127.0.0.1]) by falcon.cybervisiontech.com (Postfix) with ESMTP id E156C43E41B; Tue, 12 Feb 2008 00:39:08 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at falcon.cybervisiontech.com Received: from falcon.cybervisiontech.com ([127.0.0.1]) by localhost (falcon.cybervisiontech.com [127.0.0.1]) (amavisd-new, port 10027) with ESMTP id dzAZRB52lhr7; Tue, 12 Feb 2008 00:39:08 +0200 (EET) Received: from [10.74.70.239] (unknown [193.138.145.53]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by falcon.cybervisiontech.com (Postfix) with ESMTP id 2AA6A43E387; Tue, 12 Feb 2008 00:38:42 +0200 (EET) Message-ID: <47B0CE54.6090204@icyb.net.ua> Date: Tue, 12 Feb 2008 00:38:12 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.9 (X11/20071208) MIME-Version: 1.0 To: Poul-Henning Kamp References: <200612221824.kBMIOhfM049471@freefall.freebsd.org> <47A2EDB0.8000801@icyb.net.ua> <47A2F404.7010208@icyb.net.ua> <47A7339E.4070708@icyb.net.ua> <47A73562.4010309@samsco.org> <47A9E062.3040409@icyb.net.ua> In-Reply-To: <47A9E062.3040409@icyb.net.ua> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, freebsd-hackers@FreeBSD.org, Pav Lucistnik , Bruce Evans Subject: Re: fs/udf: vm pages "overlap" while reading large dir X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2008 22:39:10 -0000 on 06/02/2008 18:29 Andriy Gapon said the following: > Small summary of the above long description. > For directory reading fs/udf performs bread() on a (underlying) device > vnode. It passes block number as if block size was 512 bytes (i.e. > byte_offset_within_dev/512). On the other hand vm page index calculation > code uses the following formula: (blkno*bo_bsize)/PAGE_SIZE. > For CD/DVD devices bo_bsize is set to 2048. This results in page index > overlap when reading sufficiently many blocks from adjacent starting blocks. > That is, it seems that in "normal" cases page index gets calculated as > byte_offset/PAGE_SIZE, but in this case page index gets to be > 4*byte_offset/PAGE_SIZE. More details are in the quoted above text. > > With a lot of help from Bruce Evance I found this commit which seems to > have made a big difference: > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/vfs_bio.c.diff?r1=1.458;r2=1.459;f=h > > Before this change page index for device vnodes was always > (blkno*512)/PAGE_SIZE. This is because of the vn_isdisk() case. > > udf_mountfs device vnode is passed to g_vfs_open() (in geom_vfs.c) and > the latter has the following line of code: > bo->bo_bsize = pp->sectorsize; > Where pp is geom provider for the device in question. > > I am not sure if the mentioned above vfs_bio.c change broke something > else in reading from device vnodes or if it fixed something for that. I > am also not sure what would be the consequences of setting bo_bsize to > 512 for vnodes of "disk" devices. It would most probably fix current > code in UDF, but I am not sure if it might break anything else. > > Just wanted to let the more knowledgeable people know and make a decision. > > P.S. bo_bsize seems to be referenced only in a handful of files and in > most of them it seems that it is related to "file" vnodes (as opposed to > "device" vnodes). > Poul-Henning, I am sorry for this duplicate post and very sorry for misspelling your name in the first one. do you have any comment or opinion on the above? [sorry if clamped too much of the previous context, but it's all in archives] And a little continuation: Just for the sake of experiment I tried to emulated the previous behavior by changing geom_vfs.c, function g_vfs_open(), so that the relevant code reads as follows: if (vn_isdisk(vp, NULL)) bo->bo_bsize = DEV_BSIZE; else bo->bo_bsize = pp->sectorsize; It didn't break anything for me and it re-enabled current (CVS) fs/udf code to work as before. (patch from kern/78987 still has to be applied for large directories to be handled). So udf (on DVD) is fixed, ufs (on HDD), msdosfs (on HDD) and cd9660 (on CD) are not broken. Of course, the change should have affected only filesystems on CD/DVD (where sector/block size is not 512 bytes). Of course, unlike Bruce I don't use msdosfs on CD/DVD media (e.g. DVD-RAM), so my test is somewhat incomplete. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 06:42:56 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD97016A419; Tue, 12 Feb 2008 06:42:56 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (lefty.soaustin.net [66.135.55.46]) by mx1.freebsd.org (Postfix) with ESMTP id 8A7F113C459; Tue, 12 Feb 2008 06:42:56 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: by mail.soaustin.net (Postfix, from userid 502) id 2BC048C134; Thu, 7 Feb 2008 00:33:11 -0600 (CST) Date: Thu, 7 Feb 2008 00:33:11 -0600 To: Attilio Rao Message-ID: <20080207063311.GA6118@soaustin.net> References: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> User-Agent: Mutt/1.5.13 (2006-08-11) From: linimon@lonesome.com (Mark Linimon) Cc: Yar Tikhiy , Doug Barton , Jeff Roberson , freebsd-fs@freebsd.org, Scot Hetzel , freebsd-arch@freebsd.org Subject: Re: [RFC] Remove NTFS kernel support X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 06:42:56 -0000 On Thu, Feb 07, 2008 at 02:00:41AM +0100, Attilio Rao wrote: > - Do you know a good reason to not use FUSE ntfs implementation? We just had a PR come in that claims the ports FUSE NTFS implementation isn't working: 120286 alepulve ports open serious medium current-us sysutils/fusefs-ntfs makes fatal trap/page fault in FreeBSD-7.0 http://www.freebsd.org/cgi/query-pr.cgi?pr=120286&cat= mcl From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 09:25:00 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B585716A418; Tue, 12 Feb 2008 09:25:00 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 738FF13C461; Tue, 12 Feb 2008 09:25:00 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 999E117104; Tue, 12 Feb 2008 08:53:40 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.2/8.14.2) with ESMTP id m1C8rbF0003028; Tue, 12 Feb 2008 08:53:38 GMT (envelope-from phk@critter.freebsd.dk) To: Andriy Gapon From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 12 Feb 2008 00:38:12 +0200." <47B0CE54.6090204@icyb.net.ua> Date: Tue, 12 Feb 2008 08:53:37 +0000 Message-ID: <3027.1202806417@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: freebsd-fs@FreeBSD.org, freebsd-hackers@FreeBSD.org, Pav Lucistnik , Bruce Evans Subject: Re: fs/udf: vm pages "overlap" while reading large dir X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 09:25:00 -0000 In message <47B0CE54.6090204@icyb.net.ua>, Andriy Gapon writes: >on 06/02/2008 18:29 Andriy Gapon said the following: >> Small summary of the above long description. >> For directory reading fs/udf performs bread() on a (underlying) device >> vnode. It passes block number as if block size was 512 bytes (i.e. >> byte_offset_within_dev/512). We have three sizes of relevance here, the sectorsize of the provider, the blocksize of the filesystem and the page size of the system. In general it is adventurous to have any of them be anything but powers of two, and it is at best ill-adviced and more likely asking for trouble to do requests that are not multiple of and aligned to the sectorsize of the provider. So up front, I would say that it is an UDF bug to do 512 byte reads off a 2k sector provider. Making the buffer cache handle this is possible, but it is not the direction we have planned for the buffercache (ie: that it should become a wrapper for using the VM system, rather than being a separately allocated cache). So if the objective is to make UDF work in the short term, your change might work, if the objective is to move FreeBSD's kernel architecture forward, the right thing to do is to fix UDF to not access subsectors. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 11:04:39 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C92B916A469 for ; Tue, 12 Feb 2008 11:04:39 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.190]) by mx1.freebsd.org (Postfix) with ESMTP id 9683013C4CC for ; Tue, 12 Feb 2008 11:04:39 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: by rv-out-0910.google.com with SMTP id g13so3915756rvb.43 for ; Tue, 12 Feb 2008 03:04:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition:x-google-sender-auth; bh=54We2nklyjn5pgrep8D0xncxzDyQB+4GK/+p0lw9ZPM=; b=fOz3n3IIruuIZVmZ+0jlVfPPtFIsJSkFXm9WwfmJUouHm7TTb+lSEvqWJL/X/0V4ug6R0gW0Phy3+orcp5XD8dSxcPKtPFTGD9SXmTnONrGp4mtZCUKWRWKF+vRIu2mUD8wqYwNWDYeeihO90aMVceGK+UfQBudEwsFBS7TlOd8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition:x-google-sender-auth; b=WZRHylyBOuh+EIhBX1AvxW/EP7iB52NIMRVgehvDSS9PS6B/QAD9GPzPd9WwdPou2rmxP0vYrwJBJcwm1bP8iGR4B2AB+2kMKGZVbkcXqJ/LTMN0sGFBVSJcpWnPT7qi+YekHOyeVBf7M5I+CUm6eKz6w1ed39D055oGfeTAu+M= Received: by 10.141.177.2 with SMTP id e2mr795357rvp.268.1202814278716; Tue, 12 Feb 2008 03:04:38 -0800 (PST) Received: by 10.141.170.18 with HTTP; Tue, 12 Feb 2008 03:04:38 -0800 (PST) Message-ID: <2e77fc10802120304n32fd1c42m52e6bc617ba07c35@mail.gmail.com> Date: Tue, 12 Feb 2008 13:04:38 +0200 From: "Niki Denev" Sender: ndenev@gmail.com To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Google-Sender-Auth: d4283ee0e431e47f Subject: ZFS panic when changing zfs dataset mountpoint X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 11:04:40 -0000 Hi, I got the following panic when trying to set/change the mountpoint property of a dataset. I did : # zfs set mountpoint=/usr/ports zfs2/ports and the machine crashed. The datased had one snapshot taken. Here is what i was able to extract from the dump : # kgdb -q /boot/kernel/kernel.symbols /var/crash/vmcore.0 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xc0 fault code = supervisor write data, page not present instruction pointer = 0x8:0xffffffff804800d5 stack pointer = 0x10:0xffffffffd7a71980 frame pointer = 0x10:0xffffffffd7a71a20 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 71366 (zfs) trap number = 12 panic: page fault cpuid = 0 Uptime: 7d21h41m6s Physical memory: 2034 MB Dumping 1879 MB: 1864 1848 1832 1816 1800 1784 1768 1752 1736 1720 1704 1688 1672 1656 1640 1624 1608 1592 1576 1560 1544 1528 1512 1496 1480 1464 1448 1432 1416 1400 1384 1368 1352 1336 1320 1304 1288 1272 1256 1240 1224 1208 1192 1176 1160 1144 1128 1112 1096 1080 1064 1048 1032 1016 1000 984 968 952 936 920 904 888 872 856 840 824 808 792 776 760 744 728 712 696 680 664 648 632 616 600 584 568 552 536 520 504 488 472 456 440 424 408 392 376 360 344 328 312 296 280 264 248 232 216 200 184 168 152 136 120 104 88 72 56 40 24 8 #0 doadump () at pcpu.h:194 194 pcpu.h: No such file or directory. in pcpu.h (kgdb) add-symbol-file /boot/kernel/zfs.ko.symbols 0xffffffff80bfc000 add symbol table from file "/boot/kernel/zfs.ko.symbols" at .text_addr = 0xffffffff80bfc000 (y or n) y Reading symbols from /boot/kernel/zfs.ko.symbols...done. (kgdb) list *0xffffffff804800d5 0xffffffff804800d5 is in _sx_xlock (atomic.h:142). 137 atomic.h: No such file or directory. in atomic.h (kgdb) bt #0 doadump () at pcpu.h:194 #1 0x0000000000000004 in avl_balance2child () #2 0xffffffff80478619 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #3 0xffffffff80478a1d in panic (fmt=0x104
) at /usr/src/sys/kern/kern_shutdown.c:563 #4 0xffffffff8074f154 in trap_fatal (frame=0xffffff001483b6a0, eva=18446742974905006288) at /usr/src/sys/amd64/amd64/trap.c:724 #5 0xffffffff8074f525 in trap_pfault (frame=0xffffffffd7a718d0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:641 #6 0xffffffff8074fe68 in trap (frame=0xffffffffd7a718d0) at /usr/src/sys/amd64/amd64/trap.c:410 #7 0xffffffff80735ace in calltrap () at /usr/src/sys/amd64/amd64/exception.S:169 #8 0xffffffff804800d5 in _sx_xlock (sx=0xa0, opts=0, file=0xffffffff80c697f0 "/usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c", line=1069) at atomic.h:142 #9 0xffffffff80c50b2a in zfsctl_umount_snapshots (vfsp=Variable "vfsp" is not available. ) at /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:1069 #10 0xffffffff80c57978 in zfs_umount (vfsp=0xffffff00014f5650, fflag=0, td=0xffffff001483b6a0) at /usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:692 #11 0xffffffff804ebfce in dounmount (mp=0xffffff00014f5650, flags=0, td=0xffffff001483b6a0) at /usr/src/sys/kern/vfs_mount.c:1286 #12 0xffffffff804ec7b5 in unmount (td=0xffffff001483b6a0, uap=0xffffffffd7a71be0) at /usr/src/sys/kern/vfs_mount.c:1182 #13 0xffffffff8074f7a7 in syscall (frame=0xffffffffd7a71c70) at /usr/src/sys/amd64/amd64/trap.c:852 #14 0xffffffff80735cdb in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:290 #15 0x0000000800f1396c in ?? () Previous frame inner to this frame (corrupt stack?) I'll be glad to provide more info if needed. The machine is running : FreeBSD fyut.bg.freebsd.org 7.0-PRERELEASE FreeBSD 7.0-PRERELEASE #0: Sat Feb 2 17:27:18 EET 2008 root@fyut.bg.freebsd.org:/usr/obj/usr/src/sys/FYUT amd64 From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 11:24:40 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12F9E16A469; Tue, 12 Feb 2008 11:24:40 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from falcon.cybervisiontech.com (falcon.cybervisiontech.com [217.20.163.9]) by mx1.freebsd.org (Postfix) with ESMTP id 7746113C45E; Tue, 12 Feb 2008 11:24:38 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from localhost (localhost [127.0.0.1]) by falcon.cybervisiontech.com (Postfix) with ESMTP id C710143D123; Tue, 12 Feb 2008 13:24:36 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at falcon.cybervisiontech.com Received: from falcon.cybervisiontech.com ([127.0.0.1]) by localhost (falcon.cybervisiontech.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Pnl6RIIlY1lj; Tue, 12 Feb 2008 13:24:36 +0200 (EET) Received: from [10.2.1.87] (gateway.cybervisiontech.com.ua [88.81.251.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by falcon.cybervisiontech.com (Postfix) with ESMTP id D19EA43D097; Tue, 12 Feb 2008 13:24:35 +0200 (EET) Message-ID: <47B181F2.2070808@icyb.net.ua> Date: Tue, 12 Feb 2008 13:24:34 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.9 (X11/20080123) MIME-Version: 1.0 To: Poul-Henning Kamp References: <3027.1202806417@critter.freebsd.dk> In-Reply-To: <3027.1202806417@critter.freebsd.dk> Content-Type: text/plain; charset=KOI8-U Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, freebsd-hackers@FreeBSD.org, Pav Lucistnik , Bruce Evans Subject: Re: fs/udf: vm pages "overlap" while reading large dir X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 11:24:40 -0000 on 12/02/2008 10:53 Poul-Henning Kamp said the following: > In message <47B0CE54.6090204@icyb.net.ua>, Andriy Gapon writes: >> on 06/02/2008 18:29 Andriy Gapon said the following: >>> Small summary of the above long description. >>> For directory reading fs/udf performs bread() on a (underlying) device >>> vnode. It passes block number as if block size was 512 bytes (i.e. >>> byte_offset_within_dev/512). > > We have three sizes of relevance here, the sectorsize of the provider, > the blocksize of the filesystem and the page size of the system. > > In general it is adventurous to have any of them be anything but > powers of two, and it is at best ill-adviced and more likely asking > for trouble to do requests that are not multiple of and aligned to > the sectorsize of the provider. > > So up front, I would say that it is an UDF bug to do 512 byte reads off > a 2k sector provider. > > Making the buffer cache handle this is possible, but it is not the > direction we have planned for the buffercache (ie: that it should > become a wrapper for using the VM system, rather than being a > separately allocated cache). > > So if the objective is to make UDF work in the short term, your > change might work, if the objective is to move FreeBSD's kernel > architecture forward, the right thing to do is to fix UDF to not > access subsectors. > Poul, I agree with what you say, but I think that I didn't properly explain what is UDF code doing and why it might be important in general. Let me try to do it step-by-step (sorry if I'll say something too obvious). And please correct me if I misunderstand something in the fundamental steps. 1.1. UDF is typically used with CD/DVD media, so provider's sector size is 2048. 1.2. udf vfs mount method calls g_vfs_open. 1.3. g_vfs_open creates vobject for a device vnode. 1.4. g_vfs_open sets bo_bsize=pp->sectorsize in the device vnode's bufobj. 1.5. g_vfs_open also overrides bo_ops for the bufobj. 2.1. UDF directory reading code performs bread() via the device vnode. [*] 2.2. this code passes to bread a size that is multiple of 2048. 2.3. this code passes to bread blkno that is calculated as 4*sector, where sector is a number of a physical 2048-byte sector. [**] [*] - this seems to be an uncommon thing among filesystems. [**] - I think that this is a requirement of buffcache system, because internally it performs many calculations that seem to assume that block size is always 512. E.g. breadn() code has the following: bp->b_iooffset = dbtob(bp->b_blkno); <-- effectively multiplies by 512 bstrategy(bp); And g_vfs_strategy has the following: bip->bio_offset = bp->b_iooffset; So, if udf code would pass blkno==sector, then bio_offset would be incorrect. Or maybe g_vfs_strategy should do some translation here from b_iooffset to bio_offset taking into account bo_bsize ?? So that the actual, non-adjusted, sector number could be passed to the bread() ? 3.1. for a fresh buf getlbk would assign the following: bsize = bo->bo_bsize; offset = blkno * bsize; ... bp->b_blkno = bp->b_lblkno = blkno; bp->b_offset = offset; <--- this is where this discussion started so b_offset of a buffer is 4*sector*2048. This is a source of the trouble. 3.2. getblk would set bp->b_flags |= B_VMIO; if a vnode has a vobject and our device vnode has it (step 1.3). 3.3. consequently allocbuf will execute code for B_VMIO case. 3.4. allocbuf will lookup/allocate pages by index which is calculated from "base index" of OFF_TO_IDX(bp->b_offset), which is, in essence, bp->b_offset divided by page size, 4096. So our "base index" is 2*sector. 4.1. Let's assume we bread() (via the device vnode, as described above) from physical sector N and we read 6 physical sectors (6*2048 bytes). 4.2 Sectors will get "mapped"/tied/bound to VM pages as follows (according to the above calculations): sectors[N, N+1] -> page[2*N], sectors[N+2, N+3] -> page[2*N + 1], /* the next page */ sectors[N+4, N+5] -> page[2*N + 2] /* the next page */ 4.5 Now lets consider "the next read" of X sectors but now starting from sector N+1; repeating the calculations we get the following mapping: sectors[(N+1), (N+1) + 1] -> page[2*(N+1)] = page[2N +2] But this page already has cached data from sectors[N+4, N+5]. Problem!! Theoretical calculations show it and practical testing confirms that. So this is a proof that bread()-ing via a device vnode is broken if: C1) the vnode was "set up" by g_vfs_open(); C2) sector size of underlying geom provider is not 512, but any non-trivial multiple of it; C3) bread size is sufficiently big; Current UDF code for directory reading is the only known to me place that meets all the 3 above conditions (for sufficiently large directories to meet condition C3). So I stated this, now let the wise speak. I already have a patch that makes UDF read directories via the directory vnodes. But the problem in general would remain. Maybe g_vfs_open is a correct place to change, maybe g_vfs_strategy is the place, maybe something, maybe "don't bother". I don't know. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 11:47:38 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3C9916A417; Tue, 12 Feb 2008 11:47:38 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 7004213C45B; Tue, 12 Feb 2008 11:47:38 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id CA95717104; Tue, 12 Feb 2008 11:47:36 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.2/8.14.2) with ESMTP id m1CBlYLx004300; Tue, 12 Feb 2008 11:47:34 GMT (envelope-from phk@critter.freebsd.dk) To: Andriy Gapon From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 12 Feb 2008 13:24:34 +0200." <47B181F2.2070808@icyb.net.ua> Date: Tue, 12 Feb 2008 11:47:34 +0000 Message-ID: <4299.1202816854@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: freebsd-fs@FreeBSD.org, freebsd-hackers@FreeBSD.org, Pav Lucistnik , Bruce Evans Subject: Re: fs/udf: vm pages "overlap" while reading large dir X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 11:47:38 -0000 In message <47B181F2.2070808@icyb.net.ua>, Andriy Gapon writes: >2.3. this code passes to bread blkno that is calculated as 4*sector, >where sector is a number of a physical 2048-byte sector. [**] >[**] - I think that this is a requirement of buffcache system, because >internally it performs many calculations that seem to assume that block >size is always 512. Yes, the buf-cache works in 512 bytes units throughout. >3.1. for a fresh buf getlbk would assign the following: >bsize = bo->bo_bsize; >offset = blkno * bsize; That's clearly wrong. We need to assert that the blkno is aligned to the start of a sector and use the 512 byte units, so I guess it would be: offset = dbtob(blkno); KASSERT(!(offset & (bsize - 1)), ("suitable diagnostic")); -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 12:33:24 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A41AF16A41A; Tue, 12 Feb 2008 12:33:24 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from falcon.cybervisiontech.com (falcon.cybervisiontech.com [217.20.163.9]) by mx1.freebsd.org (Postfix) with ESMTP id 4BE3413C4EF; Tue, 12 Feb 2008 12:33:24 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from localhost (localhost [127.0.0.1]) by falcon.cybervisiontech.com (Postfix) with ESMTP id 3885C43E5F2; Tue, 12 Feb 2008 14:33:22 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at falcon.cybervisiontech.com Received: from falcon.cybervisiontech.com ([127.0.0.1]) by localhost (falcon.cybervisiontech.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FxO0vAEilxdU; Tue, 12 Feb 2008 14:33:22 +0200 (EET) Received: from [10.2.1.87] (gateway.cybervisiontech.com.ua [88.81.251.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by falcon.cybervisiontech.com (Postfix) with ESMTP id 9276C43DB18; Tue, 12 Feb 2008 14:33:21 +0200 (EET) Message-ID: <47B19210.3010508@icyb.net.ua> Date: Tue, 12 Feb 2008 14:33:20 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.9 (X11/20080123) MIME-Version: 1.0 To: Poul-Henning Kamp References: <4299.1202816854@critter.freebsd.dk> In-Reply-To: <4299.1202816854@critter.freebsd.dk> Content-Type: text/plain; charset=KOI8-U Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, freebsd-hackers@FreeBSD.org, Pav Lucistnik , Bruce Evans Subject: Re: fs/udf: vm pages "overlap" while reading large dir X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 12:33:24 -0000 on 12/02/2008 13:47 Poul-Henning Kamp said the following: > In message <47B181F2.2070808@icyb.net.ua>, Andriy Gapon writes: > >> 2.3. this code passes to bread blkno that is calculated as 4*sector, >> where sector is a number of a physical 2048-byte sector. [**] >> [**] - I think that this is a requirement of buffcache system, because >> internally it performs many calculations that seem to assume that block >> size is always 512. > > Yes, the buf-cache works in 512 bytes units throughout. > >> 3.1. for a fresh buf getlbk would assign the following: >> bsize = bo->bo_bsize; >> offset = blkno * bsize; > > That's clearly wrong. > > We need to assert that the blkno is aligned to the start of a sector > and use the 512 byte units, so I guess it would be: > > offset = dbtob(blkno); > KASSERT(!(offset & (bsize - 1)), ("suitable diagnostic")); > > Thank you for this very insightful and neat suggestion! I think that it must work but I will try test it tonight on whatever media and FS-s I have available. Thank you again! P.S. hope to not get '"suitable diagnostic"' from something like msdosfs :-) -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 13:11:35 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2517116A469; Tue, 12 Feb 2008 13:11:35 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail18.syd.optusnet.com.au (mail18.syd.optusnet.com.au [211.29.132.199]) by mx1.freebsd.org (Postfix) with ESMTP id 9F37813C448; Tue, 12 Feb 2008 13:11:34 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c211-30-219-213.carlnfd3.nsw.optusnet.com.au (c211-30-219-213.carlnfd3.nsw.optusnet.com.au [211.30.219.213]) by mail18.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m1CDBSUA008800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 13 Feb 2008 00:11:32 +1100 Date: Wed, 13 Feb 2008 00:11:28 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Poul-Henning Kamp In-Reply-To: <4299.1202816854@critter.freebsd.dk> Message-ID: <20080212234018.O92415@delplex.bde.org> References: <4299.1202816854@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, Pav Lucistnik , Andriy Gapon , Bruce Evans Subject: Re: fs/udf: vm pages "overlap" while reading large dir X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 13:11:35 -0000 On Tue, 12 Feb 2008, Poul-Henning Kamp wrote: > In message <47B181F2.2070808@icyb.net.ua>, Andriy Gapon writes: > >> 2.3. this code passes to bread blkno that is calculated as 4*sector, >> where sector is a number of a physical 2048-byte sector. [**] >> [**] - I think that this is a requirement of buffcache system, because >> internally it performs many calculations that seem to assume that block >> size is always 512. > > Yes, the buf-cache works in 512 bytes units throughout. I missed the dbtob() conversions in vfs_bio.c when I replied previously So blkno cannot be a cookie; it must be for a 512-block. So how did the cases where bsize != DEV_BSIZE in getblk() ever work? >> 3.1. for a fresh buf getlbk would assign the following: >> bsize = bo->bo_bsize; >> offset = blkno * bsize; > > That's clearly wrong. If units were always 512-blocks, then anything except bsize = DEV_BSIZE would be clearly wrong. Things aren't that simple (but probably should be). Even RELENG_3 has bsize = f_iosize (possibly != 512) for non-disks. That seems to include nfs(client). In fact, nfs_getcacheblk() does weird scaling which seems to be mainly to compensate for for weird scaling here. It calls getblk() with a bn arg that seems to be f_iosize units. Then at then end, for the VREG case, it sets bp->b_blkno to this bn scaled to normal DEV_BSIZE units. bp->b_blkno seems to have DEV_BSIZE units for all uses of it in nfs. > We need to assert that the blkno is aligned to the start of a sector > and use the 512 byte units, so I guess it would be: > > offset = dbtob(blkno); > KASSERT(!(offset & (bsize - 1)), ("suitable diagnostic")); Barely worth checking. The current bug has nothing to do with this. The offset is just wrong at this point, after using a scale factor that is inconsistent with the units of blkno, for all (?) disk (and other?) file systems whose sector size isn't 512. Bruce From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 13:59:18 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10F2016A41A; Tue, 12 Feb 2008 13:59:18 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from falcon.cybervisiontech.com (falcon.cybervisiontech.com [217.20.163.9]) by mx1.freebsd.org (Postfix) with ESMTP id 776DB13C4CE; Tue, 12 Feb 2008 13:59:17 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from localhost (localhost [127.0.0.1]) by falcon.cybervisiontech.com (Postfix) with ESMTP id 935FF43D123; Tue, 12 Feb 2008 15:59:15 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at falcon.cybervisiontech.com Received: from falcon.cybervisiontech.com ([127.0.0.1]) by localhost (falcon.cybervisiontech.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UdP0cuSbTNyG; Tue, 12 Feb 2008 15:59:15 +0200 (EET) Received: from [10.2.1.87] (gateway.cybervisiontech.com.ua [88.81.251.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by falcon.cybervisiontech.com (Postfix) with ESMTP id C7A9A43CBDF; Tue, 12 Feb 2008 15:59:14 +0200 (EET) Message-ID: <47B1A631.1000504@icyb.net.ua> Date: Tue, 12 Feb 2008 15:59:13 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.9 (X11/20080123) MIME-Version: 1.0 To: Bruce Evans References: <4299.1202816854@critter.freebsd.dk> <20080212234018.O92415@delplex.bde.org> In-Reply-To: <20080212234018.O92415@delplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, Poul-Henning Kamp , Pav Lucistnik , Bruce Evans Subject: Re: fs/udf: vm pages "overlap" while reading large dir X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 13:59:18 -0000 on 12/02/2008 15:11 Bruce Evans said the following: > On Tue, 12 Feb 2008, Poul-Henning Kamp wrote: > >> In message <47B181F2.2070808@icyb.net.ua>, Andriy Gapon writes: >> >>> 2.3. this code passes to bread blkno that is calculated as 4*sector, >>> where sector is a number of a physical 2048-byte sector. [**] >>> [**] - I think that this is a requirement of buffcache system, because >>> internally it performs many calculations that seem to assume that block >>> size is always 512. >> Yes, the buf-cache works in 512 bytes units throughout. > > I missed the dbtob() conversions in vfs_bio.c when I replied previously > So blkno cannot be a cookie; it must be for a 512-block. So how did > the cases where bsize != DEV_BSIZE in getblk() ever work? It seems that this is because specific VOP_STRATEGY and/or BO_STRATEGY kicked in and did the right thing, see below. >>> 3.1. for a fresh buf getlbk would assign the following: >>> bsize = bo->bo_bsize; >>> offset = blkno * bsize; >> That's clearly wrong. > > If units were always 512-blocks, then anything except bsize = DEV_BSIZE > would be clearly wrong. Things aren't that simple (but probably should > be). Even RELENG_3 has bsize = f_iosize (possibly != 512) for non-disks. O, I missed this obvious thing! Bruce, thank you for putting me back on the ground :-) Even in UDF case, when we bread() via a file or directory vnode we pass a logical 2048-byte block number (within the file) to bread(). In this case the code in getblk() does the right thing when it calculates offset as blkno * 2048. Calculating it assuming 512-byte units would be a disaster. And the actual reading works correctly because udf_strategy is called for such vnodes and it translates block numbers from physical to logical and also correctly re-calculates b_iooffset for actual reading. So b_iooffset value set in breadn() (using bdtob) is completely ignored. I remember that this is why g_vfs_* was my primary target. It seems that I revert my opinion back: either g_vfs_open should be smart about setting bo_bsize, or g_vfs_strategy should be smart about calculating bio_offset, or individual filesystems should not depend on g_vfs_* always doing the best thing for them. In any case, it seems that it is the g_vfs_* that is currently inconsistent: it sets bo_bsize to a somewhat arbitrary value, but expects that it would always be provided with correct b_iooffset (the latter being rigidly calculated via bdtob() in buffcache code). So this leaves filesystems dead in the water while reading via device vnode: provide blkno in 512-byte units - screw up VM cache, provide blkno in bo_bsize units - screw up reading from disk. Not sure if the FS-s should have wits to set bo_bsize properly and/or provide proper bo_ops - we are talking about a device vnode here. Should filesystems be involved in the business of setting its properties? Not sure. But it seems that there are many possibilities for "fixups" and various filesystems [have to] do stuff in their unique ways (*_STRATEGY, etc). -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 15:58:21 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A56C116A41B; Tue, 12 Feb 2008 15:58:21 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail16.syd.optusnet.com.au (mail16.syd.optusnet.com.au [211.29.132.197]) by mx1.freebsd.org (Postfix) with ESMTP id 34D7613C4DB; Tue, 12 Feb 2008 15:58:21 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from besplex.bde.org (c211-30-219-213.carlnfd3.nsw.optusnet.com.au [211.30.219.213]) by mail16.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m1CFwC0G014582 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 13 Feb 2008 02:58:18 +1100 Date: Wed, 13 Feb 2008 02:58:12 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Andriy Gapon In-Reply-To: <47B1A631.1000504@icyb.net.ua> Message-ID: <20080213015738.L24074@besplex.bde.org> References: <4299.1202816854@critter.freebsd.dk> <20080212234018.O92415@delplex.bde.org> <47B1A631.1000504@icyb.net.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Bruce Evans , freebsd-hackers@freebsd.org, freebsd-fs@freebsd.org, Poul-Henning Kamp , Pav Lucistnik Subject: Re: fs/udf: vm pages "overlap" while reading large dir X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 15:58:21 -0000 On Tue, 12 Feb 2008, Andriy Gapon wrote: > on 12/02/2008 15:11 Bruce Evans said the following: >> On Tue, 12 Feb 2008, Poul-Henning Kamp wrote: >> >>> In message <47B181F2.2070808@icyb.net.ua>, Andriy Gapon writes: >>>> 3.1. for a fresh buf getlbk would assign the following: >>>> bsize = bo->bo_bsize; >>>> offset = blkno * bsize; >>> That's clearly wrong. >> >> If units were always 512-blocks, then anything except bsize = DEV_BSIZE >> would be clearly wrong. Things aren't that simple (but probably should >> be). Even RELENG_3 has bsize = f_iosize (possibly != 512) for non-disks. > > O, I missed this obvious thing! Me too. > Bruce, thank you for putting me back on the ground :-) > Even in UDF case, when we bread() via a file or directory vnode we pass > a logical 2048-byte block number (within the file) to bread(). In this > case the code in getblk() does the right thing when it calculates offset > as blkno * 2048. Calculating it assuming 512-byte units would be a disaster. I think the size is mnt_stat.f_iosize in general (but not for device vnodes). > And the actual reading works correctly because udf_strategy is called > for such vnodes and it translates block numbers from physical to logical > and also correctly re-calculates b_iooffset for actual reading. So > b_iooffset value set in breadn() (using bdtob) is completely ignored. Is this setting ever used (and the corresponding setting for writing) ever used? > I remember that this is why g_vfs_* was my primary target. > It seems that I revert my opinion back: either g_vfs_open should be > smart about setting bo_bsize, or g_vfs_strategy should be smart about > calculating bio_offset, or individual filesystems should not depend on > g_vfs_* always doing the best thing for them. In fact, ffs already overrides the setting of bo_bsize for the device vnode to a different wrong setting -- g_vfs_open() sets the sector size, and ffs_mount() changes the setting to fs_bsize, but ffs actually needs the setting to be DEV_BSIZE (I think). Other bugs from this: - ffs_rawread wants the sector size, and it assumes that this is in bo_bsize for the device vnode, but ffs_mount() has changed this to fs_bsize. - multiple r/o mounts are supposed to work, but don't, since there is only one device vnode with a shared bufobj, but the bufobj needs to be per-file-system since all mounts write to it. Various bad things including panics result. There is a well-know panic from bo_private becoming garbage on unmount. I just noticed more possibilities for panics. bo_ops points to static storage, so it never becomes complete garbage. However, at least ffs sets it blindly early on in ffs_mountfs(), before looking at the file system to see if ffs can mount it. Thus if the file system is already mounted by another ffs, then ffs clobbers the other fs's bo_ops. The ffs mount will presumably fail, leaving bo_ops clobbered. Also, a successful g_vfs_open() has clobbered bo_ops, bo_private and bo_bsize a little earlier. g_vfs_open() is fundamental to looking at the file system, since I/O is not set up until it completes. Clobbering the pointers is most dangerous, but just clobbering bo_bsize breaks blkno calculations for any code that uses bo_bsize. Apart from these bugs, the fix for the blkno calculations for device vp's may be as simple as setting bo_bsize to DEV_BSIZE for the device vp of all disk file systems (since all disk file systems use expect this size). Currently, ffs seems to be the only file system that overrides g_vfs_open()'s default of the sector size. Nothing in any of fs/, gnu/fs/ and contrib/ references bo_bsize. > In any case, it seems that it is the g_vfs_* that is currently > inconsistent: it sets bo_bsize to a somewhat arbitrary value, but > expects that it would always be provided with correct b_iooffset (the > latter being rigidly calculated via bdtob() in buffcache code). So this > leaves filesystems dead in the water while reading via device vnode: > provide blkno in 512-byte units - screw up VM cache, provide blkno in > bo_bsize units - screw up reading from disk. I think I/O on the disk doesn't use bo_bsize, but only the sector size (to scale the byte count to a sector count). Otherwise, ffs's override would break I/O. geom is another place that has few references to bo_bsize -- it just clobbers it in g_vfs_open(). > Not sure if the FS-s should have wits to set bo_bsize properly and/or > provide proper bo_ops - we are talking about a device vnode here. > Should filesystems be involved in the business of setting its > properties? Not sure. Yes. They can set it more easily as they can tell g_vfs_open() what to set it to. Except, until the bugs are fixed properly, g_vfs_open() can more easily do tests to prevent clobbering. For bo_bsize and bo_ops, sharing a common value is safe and safe changes can be detected by setting to a special value on last unmount. For bo_private, a safety check would probably disallow multiple mounts (since cp is dynamically allocated (?)). Bruce From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 16:08:16 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68B5916A41B; Tue, 12 Feb 2008 16:08:16 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from falcon.cybervisiontech.com (falcon.cybervisiontech.com [217.20.163.9]) by mx1.freebsd.org (Postfix) with ESMTP id 1461613C44B; Tue, 12 Feb 2008 16:08:16 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from localhost (localhost [127.0.0.1]) by falcon.cybervisiontech.com (Postfix) with ESMTP id 6327843E22E; Tue, 12 Feb 2008 18:08:14 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at falcon.cybervisiontech.com Received: from falcon.cybervisiontech.com ([127.0.0.1]) by localhost (falcon.cybervisiontech.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fSmxEWbeB4dd; Tue, 12 Feb 2008 18:08:14 +0200 (EET) Received: from [10.2.1.87] (gateway.cybervisiontech.com.ua [88.81.251.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by falcon.cybervisiontech.com (Postfix) with ESMTP id 6D83343E905; Tue, 12 Feb 2008 18:08:13 +0200 (EET) Message-ID: <47B1C46C.2010906@icyb.net.ua> Date: Tue, 12 Feb 2008 18:08:12 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.9 (X11/20080123) MIME-Version: 1.0 To: Bruce Evans References: <4299.1202816854@critter.freebsd.dk> <20080212234018.O92415@delplex.bde.org> <47B1A631.1000504@icyb.net.ua> <20080213015738.L24074@besplex.bde.org> In-Reply-To: <20080213015738.L24074@besplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, Poul-Henning Kamp , Pav Lucistnik , Bruce Evans Subject: Re: fs/udf: vm pages "overlap" while reading large dir X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 16:08:16 -0000 on 12/02/2008 17:58 Bruce Evans said the following: > On Tue, 12 Feb 2008, Andriy Gapon wrote: >> And the actual reading works correctly because udf_strategy is called >> for such vnodes and it translates block numbers from physical to logical >> and also correctly re-calculates b_iooffset for actual reading. So >> b_iooffset value set in breadn() (using bdtob) is completely ignored. > > Is this setting ever used (and the corresponding setting for writing) > ever used? Bruce, replying only to this part (digesting the others): yes, it is used by g_vfs_strategy which is a bufobj startegy after g_vfs_open. It passes b_iooffset as is to bio_offset. P.S. of course, I am speaking about the current sources - I know you have almost an OS of your own, kidding :-) -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 19:06:56 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1066916A417 for ; Tue, 12 Feb 2008 19:06:56 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from fallbackmx09.syd.optusnet.com.au (fallbackmx09.syd.optusnet.com.au [211.29.132.242]) by mx1.freebsd.org (Postfix) with ESMTP id 6DD9013C467 for ; Tue, 12 Feb 2008 19:06:54 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail06.syd.optusnet.com.au (mail06.syd.optusnet.com.au [211.29.132.187]) by fallbackmx09.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m1CCXgAM025626 for ; Tue, 12 Feb 2008 23:33:42 +1100 Received: from c211-30-219-213.carlnfd3.nsw.optusnet.com.au (c211-30-219-213.carlnfd3.nsw.optusnet.com.au [211.30.219.213]) by mail06.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m1CCXa7n029892 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 12 Feb 2008 23:33:38 +1100 Date: Tue, 12 Feb 2008 23:33:36 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Poul-Henning Kamp In-Reply-To: <3027.1202806417@critter.freebsd.dk> Message-ID: <20080212220951.I92195@delplex.bde.org> References: <3027.1202806417@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, Pav Lucistnik , Andriy Gapon , Bruce Evans Subject: Re: fs/udf: vm pages "overlap" while reading large dir X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 19:06:56 -0000 On Tue, 12 Feb 2008, Poul-Henning Kamp wrote: > In message <47B0CE54.6090204@icyb.net.ua>, Andriy Gapon writes: >> on 06/02/2008 18:29 Andriy Gapon said the following: >>> Small summary of the above long description. >>> For directory reading fs/udf performs bread() on a (underlying) device >>> vnode. It passes block number as if block size was 512 bytes (i.e. >>> byte_offset_within_dev/512). > > We have three sizes of relevance here, the sectorsize of the provider, > the blocksize of the filesystem and the page size of the system. 4. The block size required for bread() and friends (almost always DEV_BSIZE). > In general it is adventurous to have any of them be anything but > powers of two, and it is at best ill-adviced and more likely asking > for trouble to do requests that are not multiple of and aligned to > the sectorsize of the provider. > > So up front, I would say that it is an UDF bug to do 512 byte reads off > a 2k sector provider. > > Making the buffer cache handle this is possible, but it is not the > direction we have planned for the buffercache (ie: that it should > become a wrapper for using the VM system, rather than being a > separately allocated cache). > > So if the objective is to make UDF work in the short term, your > change might work, if the objective is to move FreeBSD's kernel > architecture forward, the right thing to do is to fix UDF to not > access subsectors. This bug has nothing to do with subsectors, and very little to do with udf. udf is just depending on bread()'s API being unbroken. That API requires starting with blocks consisting of whole sectors (else the subsequent I/O would fail) and converting to blocks of size DEV_BSIZE, phyexcept for unusual (non-disk?) file systems like nfs (and no others?). All (?) disk file systems do this conversion. The bug is just more noticeable for udf since it is used more often on disks with a block size != DEV_BSIZE, and it apparently does something that makes the bug mess up VM more than other file systems. Of course, it might be better for the API to take blocks in units of the sector size, but that is not what it takes, and this can't be changed easily or safely. The standard macro btodb() is often used to convert from bytes to blocks of this size, and doesn't support bo_bsize or the bufobj pointer that would be needed to reach bo_bsize. ffs mostly uses its fsbtodb() macro, which converts blocks in ffs block (frag?)-sized units to blocks in DEV_BSIZE units so as to pass them to unbroken bread() and friends. ffs initializes bo_bsize to its block (not frag) size, and then never uses it directly except in ffs_rawread(), where it is used to check that the I/O is in units of whole sectors (otherwise, ffs_rawread() has DEV_BSIZE more hard-coded than the rest of ffs). The details of fsbtodb() are interesting. They show signs of previous attempts to use units of sectors. From ffs/fs.h: % /* % * Turn filesystem block numbers into disk block addresses. % * This maps filesystem blocks to device size blocks. % */ % #define fsbtodb(fs, b) ((daddr_t)(b) << (fs)->fs_fsbtodb) % #define dbtofsb(fs, b) ((b) >> (fs)->fs_fsbtodb) Creation of fs_fsbtodb is left to newfs. The units of DEV_BSIZE are thus built in to the on-disk data struct (in a fairly easy to change and/or override way, but there are a lot of existing disks). From newfs.c: % realsectorsize = sectorsize; % if (sectorsize != DEV_BSIZE) { /* XXX */ % int secperblk = sectorsize / DEV_BSIZE; % % sectorsize = DEV_BSIZE; % fssize *= secperblk; % if (pp != NULL) % pp->p_size *= secperblk; % } % mkfs(pp, special); Though mkfs() appears to support disk blocks of size = sectorsize, its sectorsize parameter is hacked on here, so it always generates fs_fsbtodb and other derived parameters for disk blocks of size DEV_BSIZE, as is required for the unbroken bread() API. msdosfs is similar to ffs here, except it constructs the shift count at mount time, to arrange for always converting to DEV_BSIZE blocks for passing the bread() and friends. udf is effectively similar, but pessimal and disorganized. For the conversion for bread(), it uses btodb(). In udf_bmap(), it constructs a shift count on every call by subtracting DEV_BSHIFT from its block shift count. In udf_strategy(), on every call it constructs a multiplier instead of a shift count, by dividing its block size by DEV_BSIZE. It's weird that the strategy routing, which will soon end up sending sectors to the disk, is converting to DEV_BSIZE units, a size that cannot be the size for udf's normal media. cd9660 uses btodb() for one conversion for bread() and ( - DEV_BSHIFT) in 7 other conversions to DEV_BSIZE units. So it's surprising that any file systems work on CDs and DVDs. Maybe the bug affects udf more because udf crosses page boundaries more. It's also surprising that the bug has such small effects. This seems to be because the blkno arg to bread() and friends is little more than a cookie. Each fs's strategy routine converts it to a byte offset later, and could do this using any unique cookie, but actually uses a simple shift forth and back. All fs's are consistent in their conversion to and from DEV_BSIZE or other units. Not much more than getblk() needs more than a cookie or uses an inconsistent conversion to a byte offset. Bruce From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 19:26:00 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A28716A417; Tue, 12 Feb 2008 19:26:00 +0000 (UTC) (envelope-from csaba@beastie.creo.hu) Received: from beastie.creo.hu (www.creo.hu [217.113.62.14]) by mx1.freebsd.org (Postfix) with ESMTP id B474913C447; Tue, 12 Feb 2008 19:25:59 +0000 (UTC) (envelope-from csaba@beastie.creo.hu) Received: from beastie.creo.hu (localhost [127.0.0.1]) by beastie.creo.hu (8.14.1/8.14.1) with ESMTP id m1CJ28dF013175 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 12 Feb 2008 20:02:08 +0100 (CET) (envelope-from csaba@beastie.creo.hu) Received: (from csaba@localhost) by beastie.creo.hu (8.14.1/8.14.1/Submit) id m1CJ28qu013171; Tue, 12 Feb 2008 20:02:08 +0100 (CET) (envelope-from csaba) Date: Tue, 12 Feb 2008 20:02:07 +0100 From: Csaba Henk To: Dag-Erling Smorgrav Message-ID: <20080212190207.GB49155@beastie.creo.hu> References: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> <70e8236f0802070321n9097d3fy1b39f637b3c2a06@mail.gmail.com> <867ihdc34c.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <867ihdc34c.fsf@ds4.des.no> User-Agent: Mutt/1.5.16 (2007-06-09) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (beastie.creo.hu [127.0.0.1]); Tue, 12 Feb 2008 20:02:08 +0100 (CET) Cc: freebsd-fs@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] Remove NTFS kernel support X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 19:26:00 -0000 On Sat, Feb 09, 2008 at 08:59:47PM +0100, Dag-Erling Sm??rgrav wrote: > Csaba Henk writes: > > Userspace components are LGPL / GPL licensed, kernel components are > > BSD licensed. > > Are you planning to have both the kernel part and the userland part > committed to base? > > How much work would you guess it would take to reimplement the userland > part under a BSD license? Well, I just started to work on a from scratch FUSE daemon library. The story is as follows: I wanted to put together a FUSE interface to sysctls, and I started it from scratch (so that it shall be clean licensing-wise, and for the fun of it). Then as things evolved, the generic code was distilled out to a library I named "folly", while the actual work on the sysctl fs has stalled. As a proof-of-concept I also wrote an userspace nullfs using libfolly. So I think: fuse4bsd (ie, the kld + the mount util) + libfolly + sysctl fs could go to base under BSD license. It also might make sense to rebase ntfs-3g atop of folly -- although it won't help ntfs-3g being GPL'd. So for ntfs-3g aka fusefs-ntfs there is no choice just GPL unless the authors decide to relincense it. (Rewriting ntfs-3g from scratch is not an option, there is too much expertise and man hour in that code base to be worth to waste time on it.) AFAIK libntfs is also GPL'd -- I don't know if its developers have ever thought of relicensing it to LGPL, I can imagine that they would be willing to speak about it. The fuse library is LGPL-d (although license sensitive developers should be aware of the fact that the skeletal/basic fs examples which ship with it and are a good base for writing new fs-es are under GPL!). Anyway, these are self contained bits, so I don't see it too much problematic to put all the userspace stuff into base (YMMV, of course...). Regarding libfolly: if anyone is interested, I can publish a snapshot, although the API is a moving target, and commenting/documenting it / polishing the command line interface it offers is still on the TODO list. (It's a small lib so its not that much work but now that I hit the POC level, the priority is of fuse4bsd...) Regards, Csaba From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 20:11:35 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D01C316A418; Tue, 12 Feb 2008 20:11:35 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 8F86313C4DD; Tue, 12 Feb 2008 20:11:35 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id 38B6C2084; Tue, 12 Feb 2008 21:11:29 +0100 (CET) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: -0.3/3.0 X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on tim.des.no Received: from ds4.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id A034A2049; Tue, 12 Feb 2008 21:11:28 +0100 (CET) Received: by ds4.des.no (Postfix, from userid 1001) id 7BC3B844A4; Tue, 12 Feb 2008 21:11:28 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Csaba Henk References: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> <70e8236f0802070321n9097d3fy1b39f637b3c2a06@mail.gmail.com> <867ihdc34c.fsf@ds4.des.no> <20080212190207.GB49155@beastie.creo.hu> Date: Tue, 12 Feb 2008 21:11:28 +0100 In-Reply-To: <20080212190207.GB49155@beastie.creo.hu> (Csaba Henk's message of "Tue\, 12 Feb 2008 20\:02\:07 +0100") Message-ID: <86d4r2540f.fsf@ds4.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] Remove NTFS kernel support X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 20:11:35 -0000 Csaba Henk writes: > Dag-Erling Sm=C3=B8rgrav writes: > > How much work would you guess it would take to reimplement the > > userland part under a BSD license? > Well, I just started to work on a from scratch FUSE daemon library. > [...] > So I think: fuse4bsd (ie, the kld + the mount util) + libfolly + sysctl > fs could go to base under BSD license. It also might make sense to rebase > ntfs-3g atop of folly -- although it won't help ntfs-3g being GPL'd. That doesn't matter; ntfs-3g can still be a port. What does matter is that if libfolly exports the same API as libfuse, we can have a complete BSD-licensed FUSE implementation in the base system, with minimal effort required to port FUSE-based file systems. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Tue Feb 12 23:54:01 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4968B16A418 for ; Tue, 12 Feb 2008 23:54:01 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 013EB13C43E for ; Tue, 12 Feb 2008 23:54:00 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JP4wo-0001GX-LY for freebsd-fs@freebsd.org; Tue, 12 Feb 2008 23:53:58 +0000 Received: from 78-1-70-143.adsl.net.t-com.hr ([78.1.70.143]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 12 Feb 2008 23:53:58 +0000 Received: from ivoras by 78-1-70-143.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 12 Feb 2008 23:53:58 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Wed, 13 Feb 2008 00:53:39 +0100 Lines: 39 Message-ID: References: <2e77fc10802120304n32fd1c42m52e6bc617ba07c35@mail.gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig644D36C1BCA4562F47C6E6DA" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 78-1-70-143.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) In-Reply-To: <2e77fc10802120304n32fd1c42m52e6bc617ba07c35@mail.gmail.com> X-Enigmail-Version: 0.95.6 Sender: news Subject: Re: ZFS panic when changing zfs dataset mountpoint X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2008 23:54:01 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig644D36C1BCA4562F47C6E6DA Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Niki Denev wrote: > Hi, >=20 > I got the following panic when trying to set/change the mountpoint > property of a dataset. >=20 > I did : > # zfs set mountpoint=3D/usr/ports zfs2/ports > and the machine crashed. >=20 > The datased had one snapshot taken. >=20 > Here is what i was able to extract from the dump : You call the zpool "zfs2" - do you perhaps have another one? (Just=20 guessing here - I remember people sometime complain of unusual errors=20 when there's more than one zpool). --------------enig644D36C1BCA4562F47C6E6DA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHsjGNldnAQVacBcgRAua4AJ9ZVOxkf1TR7TOV7VxTZ8pBkNEGOgCcCTTN 3y3R5GM5s3WEESCoo2ljm7o= =dPPI -----END PGP SIGNATURE----- --------------enig644D36C1BCA4562F47C6E6DA-- From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 05:44:18 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA34516A417 for ; Wed, 13 Feb 2008 05:44:18 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.185]) by mx1.freebsd.org (Postfix) with ESMTP id 8365613C457 for ; Wed, 13 Feb 2008 05:44:18 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: by rv-out-0910.google.com with SMTP id g13so4114559rvb.43 for ; Tue, 12 Feb 2008 21:44:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=aZO/5pipJIJCI1+XFhrotAj+N9GZcYNLKdEp4hK0S+w=; b=H+nh5J41fcG0Sij630FHHIcJ4lWTbTCvF1sFW43PCL36LWKBUDZ5QGA68vEbQugglS2QtfeBXjJsJ6ISfCnhmKUEfnKh52/hE86DZrv+EXY2B628gF2grAGQl2F283jdL9jV6AYIaMZo3GUmk8bGKg0vWIfhjt/LoarEi25jPEE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=YfmbUnrGmsR0CXQjRH+vTmirFQRGOU9gmnMOowzkw4p7Qwm4PKv18jdPCMyjMauxOSkRgPP2RhvITfo2L2b562Lo+Fox7pjLI40WDYjd6BEFGCazEjMVYfCyUHJLehiaLUQIGcvTwg1w+utC4fKKzG2nEWJ+Cg3Z2sR36yLPzMU= Received: by 10.141.33.21 with SMTP id l21mr1571266rvj.105.1202881458127; Tue, 12 Feb 2008 21:44:18 -0800 (PST) Received: by 10.141.170.18 with HTTP; Tue, 12 Feb 2008 21:44:18 -0800 (PST) Message-ID: <2e77fc10802122144r409a1c6by41075cb8b06719e3@mail.gmail.com> Date: Wed, 13 Feb 2008 07:44:18 +0200 From: "Niki Denev" Sender: ndenev@gmail.com To: "Ivan Voras" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <2e77fc10802120304n32fd1c42m52e6bc617ba07c35@mail.gmail.com> X-Google-Sender-Auth: d1d745bd2d7f82c0 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS panic when changing zfs dataset mountpoint X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 05:44:19 -0000 On Feb 13, 2008 1:53 AM, Ivan Voras wrote: > Niki Denev wrote: > > Hi, > > > > I got the following panic when trying to set/change the mountpoint > > property of a dataset. > > > > I did : > > # zfs set mountpoint=/usr/ports zfs2/ports > > and the machine crashed. > > > > The datased had one snapshot taken. > > > > Here is what i was able to extract from the dump : > > You call the zpool "zfs2" - do you perhaps have another one? (Just > guessing here - I remember people sometime complain of unusual errors > when there's more than one zpool). > Yes, there are two pools in the system, and this is the second one. % gpart show => 34 2929686461 da0 GPT (1.5TB) 34 128 1 freebsd-boot (65.5KB) 162 104857600 2 freebsd-ufs (53.7GB) 104857762 16777216 3 freebsd-swap (8.6GB) 121634978 2808051517 4 freebsd-zfs (1.4TB) => 34 976562109 da1 GPT (500.0GB) 34 976562109 1 freebsd-zfs (500.0GB) % zpool status pool: zfs state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM zfs ONLINE 0 0 0 da0p4 ONLINE 0 0 0 errors: No known data errors pool: zfs2 state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM zfs2 ONLINE 0 0 0 da1p1 ONLINE 0 0 0 errors: No known data errors --Niki From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 09:04:00 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A768616A417; Wed, 13 Feb 2008 09:04:00 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 765A113C468; Wed, 13 Feb 2008 09:04:00 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 7928846C53; Wed, 13 Feb 2008 04:03:59 -0500 (EST) Date: Wed, 13 Feb 2008 09:03:59 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= In-Reply-To: <86d4r2540f.fsf@ds4.des.no> Message-ID: <20080213085903.U13849@fledge.watson.org> References: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> <70e8236f0802070321n9097d3fy1b39f637b3c2a06@mail.gmail.com> <867ihdc34c.fsf@ds4.des.no> <20080212190207.GB49155@beastie.creo.hu> <86d4r2540f.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="621616949-1602308289-1202893439=:13849" Cc: Csaba Henk , freebsd-fs@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] Remove NTFS kernel support X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 09:04:00 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --621616949-1602308289-1202893439=:13849 Content-Type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Tue, 12 Feb 2008, Dag-Erling Sm=F8rgrav wrote: > Csaba Henk writes: >> Dag-Erling Sm=F8rgrav writes: >>> How much work would you guess it would take to reimplement the >>> userland part under a BSD license? >> Well, I just started to work on a from scratch FUSE daemon library. [...= ]=20 >> So I think: fuse4bsd (ie, the kld + the mount util) + libfolly + sysctl = fs=20 >> could go to base under BSD license. It also might make sense to rebase= =20 >> ntfs-3g atop of folly -- although it won't help ntfs-3g being GPL'd. > > That doesn't matter; ntfs-3g can still be a port. > > What does matter is that if libfolly exports the same API as libfuse, we = can=20 > have a complete BSD-licensed FUSE implementation in the base system, with= =20 > minimal effort required to port FUSE-based file systems. Has there been any work to add more mature interfaces to fuse over the last= =20 couple of years? When I looked at it previously, and that was a year or tw= o=20 ago, fuse didn't work well with our notion of "referenced" vs. "open" vnode= s,=20 and required explicit data copies from cache files into the kernel to be=20 exposed via fuse. These are both areas where nnpfs, the userspace file sys= tem=20 framework for Arla, does much better, as they offer improved handling of=20 memory mapping (which persists after file descriptor close(), as in execve(= )=20 and with shared libraries) and performance (no need to feed data for files = to=20 the kernel, you can just point the kernel at a persistent cache file, possi= bly=20 cached from a previous session, allowing normal faulting of cache data into= =20 memory rather than requiring that pages pass through user space). My=20 understanding is that the NetBSD user space fs work offers a more mature=20 back-end interface than fuse, but allows the less complex fuse API to be us= ed,=20 but I've not done any detailed reading. These are areas where I assumed th= at=20 over time we'd see functional improvements in fuse, so I guess I'm wonderin= g=20 if that has happened? Robert N M Watson Computer Laboratory University of Cambridge --621616949-1602308289-1202893439=:13849-- From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 10:42:00 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A746F16A419 for ; Wed, 13 Feb 2008 10:42:00 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 611DB13C4DD for ; Wed, 13 Feb 2008 10:42:00 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JPF3u-0004Ab-Bn for freebsd-fs@freebsd.org; Wed, 13 Feb 2008 10:41:58 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 13 Feb 2008 10:41:58 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 13 Feb 2008 10:41:58 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Wed, 13 Feb 2008 11:43:42 +0100 Lines: 29 Message-ID: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig01E7B574F72BB82562D9E71C" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.6 (X11/20071022) X-Enigmail-Version: 0.95.0 Sender: news Cc: freebsd-current@freebsd.org Subject: ZFS lockup in "zfs" state X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 10:42:00 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig01E7B574F72BB82562D9E71C Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, A machine just wedged its ZFS file systems (UFS file systems were still running), all unresponsive processes were in "zfs" state. This is 7.0-RC1 on i386 with 2 GB RAM. It has happened at least once before. I have a kernel core dump (dumped from kdb/ddb on hotkey), if anyone's interested. --------------enig01E7B574F72BB82562D9E71C Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHssneldnAQVacBcgRAnd2AJsFjthK2XfGyGOpLTVr/j2YBux4pwCgm5ww ws9R75FIlsy1P5uexcsMREw= =FxLQ -----END PGP SIGNATURE----- --------------enig01E7B574F72BB82562D9E71C-- From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 11:12:13 2008 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6390016A41B for ; Wed, 13 Feb 2008 11:12:13 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 40C6513C4E5 for ; Wed, 13 Feb 2008 11:12:13 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id C03A546BBC; Wed, 13 Feb 2008 06:12:12 -0500 (EST) Date: Wed, 13 Feb 2008 11:12:12 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Jan Harkes In-Reply-To: <20080122040252.GI30266@cs.cmu.edu> Message-ID: <20080213105632.A13849@fledge.watson.org> References: <20080119165056.E3375@fledge.watson.org> <20080122010003.B29737@fledge.watson.org> <20080122040252.GI30266@cs.cmu.edu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: fs@FreeBSD.org Subject: Re: Various FreeBSD Coda fixes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 11:12:13 -0000 On Mon, 21 Jan 2008, Jan Harkes wrote: >> - ".." and "." sometimes appear to have problems in the root directory of the >> root volume of a realm (i.e., /coda/testserver.coda.cs.cmu.edu). I'm not >> sure if this is a Coda client bug or a kernel bug, quite possibly the >> latter. This most likely won't be fixed for 7.0 unless it jumps out at me >> tomorrow. > > Known Coda client problem. Across the volume mount we have 2 different > objects, the root of the volume and the mountlink object on which is it > mounted. If you do a low-level readdir on the parent you see the identifier > of the mountlink and not the volume root. So stat('.') in the volume root > cannot be found in the readdir('..') information, the only way to match it > up right now is to stat() every entry you got back from readdir. > >> - getpwd() appears to have problems, possibly related to the previous bug >> if it's unable to recurse to the root. Because Coda doesn't use the global > > Correct. I really see this as a Coda client issue, although is has been > fixed in the Linux kernel module by peeking in the in-kernel directory > cache. Effectively similar to calling stat(2) on all children as long as > they are cached, and the components of the path we're looking up are > guaranteed to be cached because they are held pinned down by the cwd > reference of the process that calls getcwd. I explored this issue a bit more, and have some local patches to make Coda on FreeBSD use the global VFS namecache, which seems to help with some of these problems. However, there is one case it's not helping with: stat("..", &sb) on the root of /coda/testserver.coda.cs.cmu.edu returns ENOENT once the namecache is flushed as a result of a CODA_PURGEUSER (i.e., cunlog). It looks like Venus is returning a lookup error: cinnamon-coda# clog guest username: guest@testserver.coda.cs.cmu.edu Password: cinnamon-coda# cd /coda/testserver.coda.cs.cmu.edu/ cinnamon-coda# cul cinnamon-coda# cunlog cinnamon-coda# stat .. stat: ..: stat: No such file or directory Here's the kernel module debug output: Doing a call for 9.663 vcwrite got a call for 9.663 lookup: .. in [2.7f000000.1.1] Doing a call for 10.664 vcwrite got a call for 10.664 lookup error on [2.7f000000.1.1] (..)2 So it looks like venus_lookup() isn't returning /coda's fid on a lookup of ".." in /coda/testserver.coda.cs.cmu.edu. The BSD Coda kernel module (and presumably the original Mach code it was based on) has a credential-tagged namecache, so actually flushes the namecache for the uid passed into CODA_PURGEUSER. Linux's Coda module has an access cache that it flushes for all users on CODA_PURGEUSER, but doesn't touch the namecache. I can adopt the same approach in FreeBSD, but I think venus probably should know how to resolve ".." for the kernel module, rather than relying solely on the namecache. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 11:33:21 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5919F16A477 for ; Wed, 13 Feb 2008 11:33:21 +0000 (UTC) (envelope-from jdc@parodius.com) Received: from mx01.sc1.parodius.com (mx01.sc1.parodius.com [72.20.106.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4C55513C47E for ; Wed, 13 Feb 2008 11:33:21 +0000 (UTC) (envelope-from jdc@parodius.com) Received: by mx01.sc1.parodius.com (Postfix, from userid 1000) id 38CFE1CC064; Wed, 13 Feb 2008 03:33:21 -0800 (PST) Date: Wed, 13 Feb 2008 03:33:21 -0800 From: Jeremy Chadwick To: Ivan Voras Message-ID: <20080213113321.GA52329@eos.sc1.parodius.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.16 (2007-06-09) Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: ZFS lockup in "zfs" state X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 11:33:21 -0000 On Wed, Feb 13, 2008 at 11:43:42AM +0100, Ivan Voras wrote: > Hi, > > A machine just wedged its ZFS file systems (UFS file systems were still > running), all unresponsive processes were in "zfs" state. This is > 7.0-RC1 on i386 with 2 GB RAM. It has happened at least once before. > > I have a kernel core dump (dumped from kdb/ddb on hotkey), if anyone's > interested. Were you copying any data from a ZFS pool to a UFS filesystem, or ZFS-to-ZFS, at that time? Does your coredump's backtrace look at all similar to the below report? http://lists.freebsd.org/pipermail/freebsd-stable/2008-January/040047.html -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 11:51:57 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 232FB16A468 for ; Wed, 13 Feb 2008 11:51:57 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id DA03A13C47E for ; Wed, 13 Feb 2008 11:51:55 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JPG9V-0007tL-4B for freebsd-fs@freebsd.org; Wed, 13 Feb 2008 11:51:49 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 13 Feb 2008 11:51:49 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 13 Feb 2008 11:51:49 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Wed, 13 Feb 2008 12:53:31 +0100 Lines: 51 Message-ID: References: <20080213113321.GA52329@eos.sc1.parodius.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig843EF36E5FEFD2230858901E" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.6 (X11/20071022) In-Reply-To: <20080213113321.GA52329@eos.sc1.parodius.com> X-Enigmail-Version: 0.95.0 Sender: news Cc: freebsd-current@freebsd.org Subject: Re: ZFS lockup in "zfs" state X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 11:51:57 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig843EF36E5FEFD2230858901E Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Jeremy Chadwick wrote: > On Wed, Feb 13, 2008 at 11:43:42AM +0100, Ivan Voras wrote: >> Hi, >> >> A machine just wedged its ZFS file systems (UFS file systems were stil= l >> running), all unresponsive processes were in "zfs" state. This is >> 7.0-RC1 on i386 with 2 GB RAM. It has happened at least once before. >> >> I have a kernel core dump (dumped from kdb/ddb on hotkey), if anyone's= >> interested. >=20 > Were you copying any data from a ZFS pool to a UFS filesystem, or > ZFS-to-ZFS, at that time?=20 I don't know - the machine is a server that's been doing it's job (web server, mysql, java). Depending on when the problem started, there could have been a rsync from ZFS to UFS for backup purposes. > Does your coredump's backtrace look at all > similar to the below report? >=20 > http://lists.freebsd.org/pipermail/freebsd-stable/2008-January/040047.h= tml I vaguely remember there being something about sockbuf when I had the online debugger running and java (tomcat) is definitely running on the server. Can you advise me how to extract this kind of information from a kernel core dump (kgdb doesn't have show alllocks, etc :) )? --------------enig843EF36E5FEFD2230858901E Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHsto7ldnAQVacBcgRAgXoAKD4grqr9a7Rnf12RQeVAEN/YmXkUwCdE7/j 8odXT2wiXG4nWgcdCGucewQ= =+MUx -----END PGP SIGNATURE----- --------------enig843EF36E5FEFD2230858901E-- From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 13:04:40 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6DA9F16A418; Wed, 13 Feb 2008 13:04:40 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from falcon.cybervisiontech.com (falcon.cybervisiontech.com [217.20.163.9]) by mx1.freebsd.org (Postfix) with ESMTP id DEC7213C4E1; Wed, 13 Feb 2008 13:04:39 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from localhost (localhost [127.0.0.1]) by falcon.cybervisiontech.com (Postfix) with ESMTP id BEA1243F36C; Wed, 13 Feb 2008 15:04:37 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at falcon.cybervisiontech.com Received: from falcon.cybervisiontech.com ([127.0.0.1]) by localhost (falcon.cybervisiontech.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fnqWw7irepm9; Wed, 13 Feb 2008 15:04:37 +0200 (EET) Received: from [10.2.1.87] (gateway.cybervisiontech.com.ua [88.81.251.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by falcon.cybervisiontech.com (Postfix) with ESMTP id 3498343F366; Wed, 13 Feb 2008 15:04:36 +0200 (EET) Message-ID: <47B2EAE3.3010600@icyb.net.ua> Date: Wed, 13 Feb 2008 15:04:35 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.9 (X11/20080123) MIME-Version: 1.0 To: Bruce Evans References: <4299.1202816854@critter.freebsd.dk> <20080212234018.O92415@delplex.bde.org> <47B1A631.1000504@icyb.net.ua> <20080213015738.L24074@besplex.bde.org> In-Reply-To: <20080213015738.L24074@besplex.bde.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, Poul-Henning Kamp , freebsd-arch@freebsd.org Subject: device/disk vnode use by filesystems: vfs_bio, geom_vfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 13:04:40 -0000 I changed Subject to reflect more general discussion than what we originally had (UDF specifics). I also CC arch, because, I think, these issues are core architectural/design issues. ufs/ffs is quite a complex beast, I don't pretend I understand its internal workings well. on 12/02/2008 17:58 Bruce Evans said the following: > In fact, ffs already overrides the setting of bo_bsize for the device > vnode to a different wrong setting -- g_vfs_open() sets the sector size, > and ffs_mount() changes the setting to fs_bsize, but ffs actually needs > the setting to be DEV_BSIZE (I think). Latest ffs code doesn't seem to do that. It calls g_vfs_open which sets bo_bsize on disk vnode and that's it. The code does override indeed bo_ops from those that were set by g_vfs_open. What you said is done in ffs_vget, it assigns bo_bsize=fs_bsize for new vnodes for ffs files. > Other bugs from this: > - ffs_rawread wants the sector size, and it assumes that this is in bo_bsize > for the device vnode, but ffs_mount() has changed this to fs_bsize. In the latest code this is not true. bo_bsize is underlying GEOM provider's sector size ("device sector size"). > - multiple r/o mounts are supposed to work, but don't, since there is only > one device vnode with a shared bufobj, but the bufobj needs to be > per-file-system since all mounts write to it. Various bad things > including panics result. There is a well-know panic from bo_private > becoming garbage on unmount. I agree. Here's another example (more familiar to me) - many DVD disks have both CD9660 and UDF fs structures referencing the same data. mkisofs can also produce such images with -udf option. In theory, there should be nothing that would prevent simultaneous RO mount of such a disk via both methods. In practice at least bo_private would be overridden by the second mount. Well, even two simultaneous RO CD9660 mounts of the same device can/will result in a problem (at least one umount). I agree that this is indeed an architectural problem. Concurrent filesystems using the same device should not fight over its vnode/bufobj. They should get their private structures, but I have no idea how. > I just noticed more possibilities for > panics. bo_ops points to static storage, so it never becomes complete > garbage. However, at least ffs sets it blindly early on in > ffs_mountfs(), before looking at the file system to see if ffs can > mount it. Thus if the file system is already mounted by another > ffs, then ffs clobbers the other fs's bo_ops. The ffs mount will > presumably fail, leaving bo_ops clobbered. I agree. > Also, a successful > g_vfs_open() has clobbered bo_ops, bo_private and bo_bsize a little > earlier. I agree. > g_vfs_open() is fundamental to looking at the file system, > since I/O is not set up until it completes. Clobbering the pointers > is most dangerous, but just clobbering bo_bsize breaks blkno > calculations for any code that uses bo_bsize. I don't think there is any current code that uses bo_bsize for this purpose. > Apart from these bugs, the fix for the blkno calculations for device > vp's may be as simple as setting bo_bsize to DEV_BSIZE for the device > vp of all disk file systems (since all disk file systems use expect > this size). Currently, ffs seems to be the only file system that > overrides g_vfs_open()'s default of the sector size. Nothing in any > of fs/, gnu/fs/ and contrib/ references bo_bsize. It is the first possibility. And I actually currently run a system with this patch in place and do not see any regressions. Another possibility, which requires more changes but is more "elegant" (in my opinion), is to keep bo_bsize as it is. Instead, always use non-adjusted block numbers in bread*/bwrite and modify g_vfs_strategy to set bio_offset to blkno*bo_bsize. That is, filesystems would not have to hack block numbers via shifting them by (bshift - DEV_BSHIFT). In this case the code should be very similar whether you work via a device vnode or via a file vnode. This is because filesystems already pass logical block number when they work via a file vnode. > > Yes. They can set it more easily as they can tell g_vfs_open() what to > set it to. Except, until the bugs are fixed properly, g_vfs_open() can > more easily do tests to prevent clobbering. For bo_bsize and bo_ops, > sharing a common value is safe and safe changes can be detected by > setting to a special value on last unmount. For bo_private, a safety > check would probably disallow multiple mounts (since cp is dynamically > allocated (?)). I agree. We either should prohibit a concurrent use of a device vnode by multiple filesystems; or should have some reference counting - maybe this would work for multiple mounts for the same fs type; or design some way to have that data private to filesystems. One tough thing here is a lack of encapsulation: g_vfs_open can make its own changes in v_bufobj, then filesystem code can modify it further, so there is no single point of control. The simplest solution is to disallow this practice altogether. Another approach that comes to my mind is some kind of vnode "cloning", so that each filesystem has its own device vnode. This should work for multiple RO mounts, maybe with some resource usage penalties. RW mounts must be exclusive of course. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 13:31:56 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ACEE116A468; Wed, 13 Feb 2008 13:31:56 +0000 (UTC) (envelope-from jdc@parodius.com) Received: from mx01.sc1.parodius.com (mx01.sc1.parodius.com [72.20.106.3]) by mx1.freebsd.org (Postfix) with ESMTP id A18BC13C46A; Wed, 13 Feb 2008 13:31:56 +0000 (UTC) (envelope-from jdc@parodius.com) Received: by mx01.sc1.parodius.com (Postfix, from userid 1000) id 623D81CC038; Wed, 13 Feb 2008 05:31:56 -0800 (PST) Date: Wed, 13 Feb 2008 05:31:56 -0800 From: Jeremy Chadwick To: Ivan Voras Message-ID: <20080213133156.GA5645@eos.sc1.parodius.com> References: <20080213113321.GA52329@eos.sc1.parodius.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.16 (2007-06-09) Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: ZFS lockup in "zfs" state X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 13:31:56 -0000 On Wed, Feb 13, 2008 at 12:53:31PM +0100, Ivan Voras wrote: > Jeremy Chadwick wrote: > > On Wed, Feb 13, 2008 at 11:43:42AM +0100, Ivan Voras wrote: > >> Hi, > >> > >> A machine just wedged its ZFS file systems (UFS file systems were still > >> running), all unresponsive processes were in "zfs" state. This is > >> 7.0-RC1 on i386 with 2 GB RAM. It has happened at least once before. > >> > >> I have a kernel core dump (dumped from kdb/ddb on hotkey), if anyone's > >> interested. > > > > Were you copying any data from a ZFS pool to a UFS filesystem, or > > ZFS-to-ZFS, at that time? > > I don't know - the machine is a server that's been doing it's job (web > server, mysql, java). Depending on when the problem started, there could > have been a rsync from ZFS to UFS for backup purposes. > > > Does your coredump's backtrace look at all > > similar to the below report? > > > > http://lists.freebsd.org/pipermail/freebsd-stable/2008-January/040047.html > > I vaguely remember there being something about sockbuf when I had the > online debugger running and java (tomcat) is definitely running on the > server. Can you advise me how to extract this kind of information from a > kernel core dump (kgdb doesn't have show alllocks, etc :) )? I personally don't know. kris@ should be able to help with this. Volunteering people for stuff again, heh. :-) -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 17:01:20 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED7DF16A469; Wed, 13 Feb 2008 17:01:20 +0000 (UTC) (envelope-from csaba@beastie.creo.hu) Received: from beastie.creo.hu (www.creo.hu [217.113.62.14]) by mx1.freebsd.org (Postfix) with ESMTP id BFFED13C4F5; Wed, 13 Feb 2008 17:01:19 +0000 (UTC) (envelope-from csaba@beastie.creo.hu) Received: from beastie.creo.hu (localhost [127.0.0.1]) by beastie.creo.hu (8.14.1/8.14.1) with ESMTP id m1DGxNZf070350 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 13 Feb 2008 17:59:24 +0100 (CET) (envelope-from csaba@beastie.creo.hu) Received: (from csaba@localhost) by beastie.creo.hu (8.14.1/8.14.1/Submit) id m1DGxNER070349; Wed, 13 Feb 2008 17:59:23 +0100 (CET) (envelope-from csaba) Date: Wed, 13 Feb 2008 17:59:23 +0100 From: Csaba Henk To: Dag-Erling Smorgrav Message-ID: <20080213165923.GD49155@beastie.creo.hu> References: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> <70e8236f0802070321n9097d3fy1b39f637b3c2a06@mail.gmail.com> <867ihdc34c.fsf@ds4.des.no> <20080212190207.GB49155@beastie.creo.hu> <86d4r2540f.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86d4r2540f.fsf@ds4.des.no> User-Agent: Mutt/1.5.16 (2007-06-09) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (beastie.creo.hu [127.0.0.1]); Wed, 13 Feb 2008 17:59:24 +0100 (CET) Cc: freebsd-fs@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] Remove NTFS kernel support X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 17:01:21 -0000 On Tue, Feb 12, 2008 at 09:11:28PM +0100, Dag-Erling Sm??rgrav wrote: > Csaba Henk writes: > > So I think: fuse4bsd (ie, the kld + the mount util) + libfolly + sysctl > > fs could go to base under BSD license. It also might make sense to rebase > > ntfs-3g atop of folly -- although it won't help ntfs-3g being GPL'd. > > That doesn't matter; ntfs-3g can still be a port. > > What does matter is that if libfolly exports the same API as libfuse, we > can have a complete BSD-licensed FUSE implementation in the base system, > with minimal effort required to port FUSE-based file systems. No, it's a different API. I don't see it would have much importance to clone the libfuse API, because: - If FreeBSD wants a filesystem for its own internal purposes (like a filesytem interface to sysctls) it can use folly. - The huge majority of FUSE based filesystems in common use are GPL licensed anyway, and given this, I don't see the gain of having them backed with a BSD licensed library instead of a LGPL'd one. And they are as fine in ports as ntfs-3g is. Csaba From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 17:23:17 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D39BA16A418; Wed, 13 Feb 2008 17:23:17 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 978EE13C442; Wed, 13 Feb 2008 17:23:17 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id 16BF8207F; Wed, 13 Feb 2008 18:23:11 +0100 (CET) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: -0.3/3.0 X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on tim.des.no Received: from ds4.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id EF371207E; Wed, 13 Feb 2008 18:23:10 +0100 (CET) Received: by ds4.des.no (Postfix, from userid 1001) id C6860844A3; Wed, 13 Feb 2008 18:23:10 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Csaba Henk References: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> <70e8236f0802070321n9097d3fy1b39f637b3c2a06@mail.gmail.com> <867ihdc34c.fsf@ds4.des.no> <20080212190207.GB49155@beastie.creo.hu> <86d4r2540f.fsf@ds4.des.no> <20080213165923.GD49155@beastie.creo.hu> Date: Wed, 13 Feb 2008 18:23:10 +0100 In-Reply-To: <20080213165923.GD49155@beastie.creo.hu> (Csaba Henk's message of "Wed\, 13 Feb 2008 17\:59\:23 +0100") Message-ID: <86zlu493ep.fsf@ds4.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] Remove NTFS kernel support X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 17:23:17 -0000 Csaba Henk writes: > - The huge majority of FUSE based filesystems in common use are GPL > licensed anyway, and given this, I don't see the gain of having > them backed with a BSD licensed library instead of a LGPL'd one. > And they are as fine in ports as ntfs-3g is. My implicit assumption was that it would be much harder to maintain a FUSE library in ports, since it is so tightly bound to the FUSE kernel module. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 17:26:25 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE03216A41A for ; Wed, 13 Feb 2008 17:26:25 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id AC35213C45A for ; Wed, 13 Feb 2008 17:26:25 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JPLNF-0000XD-Of for freebsd-fs@freebsd.org; Wed, 13 Feb 2008 17:26:21 +0000 Received: from 213.202.123.79 ([213.202.123.79]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 13 Feb 2008 17:26:21 +0000 Received: from ivoras by 213.202.123.79 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 13 Feb 2008 17:26:21 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Wed, 13 Feb 2008 18:26:16 +0100 Lines: 21 Message-ID: References: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> <70e8236f0802070321n9097d3fy1b39f637b3c2a06@mail.gmail.com> <867ihdc34c.fsf@ds4.des.no> <20080212190207.GB49155@beastie.creo.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 213.202.123.79 User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) In-Reply-To: <20080212190207.GB49155@beastie.creo.hu> X-Enigmail-Version: 0.94.1.0 Sender: news Cc: freebsd-arch@freebsd.org Subject: FUSE/libfolly (was: Re: [RFC] Remove NTFS kernel support) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 17:26:26 -0000 Csaba Henk wrote: > Well, I just started to work on a from scratch FUSE daemon library. > > The story is as follows: I wanted to put together a FUSE interface to > sysctls, and I started it from scratch (so that it shall be clean > licensing-wise, and for the fun of it). Then as things evolved, the > generic code was distilled out to a library I named "folly", while the > actual work on the sysctl fs has stalled. As a proof-of-concept I also > wrote an userspace nullfs using libfolly. > > So I think: fuse4bsd (ie, the kld + the mount util) + libfolly + sysctl > fs could go to base under BSD license. It also might make sense to rebase > ntfs-3g atop of folly -- although it won't help ntfs-3g being GPL'd. Just for clarification: libfolly is the userland part to the kernel fuse4bsd interface; it implements the original FUSE API? So, in general, it should allow the use of 3rd party FUSE components by changing "-lfuse" to "-lfolly"? (what about include files?) Offtopic: Have you seen NetBSD's PUFFS? From owner-freebsd-fs@FreeBSD.ORG Wed Feb 13 23:36:40 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D43516A417; Wed, 13 Feb 2008 23:36:40 +0000 (UTC) (envelope-from jakob@dev.citybikes.cz) Received: from dev.citybikes.cz (r5o136.net.upc.cz [86.49.14.136]) by mx1.freebsd.org (Postfix) with ESMTP id 9953613C455; Wed, 13 Feb 2008 23:36:39 +0000 (UTC) (envelope-from jakob@dev.citybikes.cz) Received: from dev.citybikes.cz (localhost [127.0.0.1]) by dev.citybikes.cz (8.14.2/8.14.2) with ESMTP id m1DN4YbG001619; Thu, 14 Feb 2008 00:04:34 +0100 (CET) (envelope-from jakob@dev.citybikes.cz) Received: (from jakob@localhost) by dev.citybikes.cz (8.14.2/8.14.2/Submit) id m1DN4XUA001618; Thu, 14 Feb 2008 00:04:34 +0100 (CET) (envelope-from jakob) Date: Thu, 14 Feb 2008 00:04:33 +0100 From: Jakub Siroky To: Kris Kennaway Message-ID: <20080214000433.6d8e1f34@dev.citybikes.cz> In-Reply-To: <47921AE2.1060004@FreeBSD.org> References: <20080118120140.2a8170a0@dev> <47921931.9050606@FreeBSD.org> <47921AE2.1060004@FreeBSD.org> X-Mailer: Claws Mail 3.0.2 (GTK+ 2.12.1; amd64-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: infinite loop when copying to ext2fs / ext2_new_block() X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2008 23:36:40 -0000 This is the dump when I place a panic call inside the loop and then use kgdb: Panic String: ext2_new_block: bit already set for block 5346 panic() at panic+0x17a - added otherwise loop occurs, especially when the volume is almost full ext2_new_block() at ext2_new_block+0x892 ext2_alloc() at ext2_alloc+0x189 ext2_balloc() at ext2_balloc+0x1f3 ext2_write() at ext2_write+0x178 VOP_WRITE_APV() at VOP_WRITE_APV+0x131 vn_write() at vn_write+0x1ce dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x4c write() at write+0x54 syscall() at syscall+0x1f6 Xfast_syscall() at Xfast_syscall+0xab --- syscall (4, FreeBSD ELF64, write), rip = 0x800713d3c, rsp = 0x7fffffffe948, rbp = 0x19df2 --- Regards, Jakub On Sat, 19 Jan 2008 16:44:34 +0100 Kris Kennaway wrote: > Kris Kennaway wrote: > > Jakub Siroky wrote: > >> I have two large ext2fs partitions (368 and 313GB) to hold data > >> shared between several OSes. While there were no problems on > >> 6-STABLE branch I was quite disappointed after upgrade to > >> 7-STABLE. Whenever I copy/write to ext2fs partition the system > >> freezes totally without crashdump. So I set debugging settings to > >> kernel config (DEBUG,WITNESS,..) and in console I reproduced error > >> situation ending with full screen of unstoppable running text with > >> lot of memory addresses and a few recognisable words: 'new block > >> bit set for ext already' - again with no crashdump. Then I have > >> formatted 1GB partition with ext2fs and the problem on this small > >> partition appears only sometimes. > > > > OK, I am able to reproduce this. > > > > Kris > > > > Is anyone able to look at this? I could not spot a candidate change > that has not been merged to 6.x. > > Kris From owner-freebsd-fs@FreeBSD.ORG Thu Feb 14 10:17:08 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DA83516A419; Thu, 14 Feb 2008 10:17:08 +0000 (UTC) (envelope-from csaba@beastie.creo.hu) Received: from beastie.creo.hu (www.creo.hu [217.113.62.14]) by mx1.freebsd.org (Postfix) with ESMTP id 3361813C469; Thu, 14 Feb 2008 10:17:07 +0000 (UTC) (envelope-from csaba@beastie.creo.hu) Received: from beastie.creo.hu (localhost [127.0.0.1]) by beastie.creo.hu (8.14.1/8.14.1) with ESMTP id m1EAFBrj015449 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 14 Feb 2008 11:15:11 +0100 (CET) (envelope-from csaba@beastie.creo.hu) Received: (from csaba@localhost) by beastie.creo.hu (8.14.1/8.14.1/Submit) id m1EAFBnV015448; Thu, 14 Feb 2008 11:15:11 +0100 (CET) (envelope-from csaba) Date: Thu, 14 Feb 2008 11:15:11 +0100 From: Csaba Henk To: Dag-Erling Smorgrav Message-ID: <20080214101511.GE49155@beastie.creo.hu> References: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> <70e8236f0802070321n9097d3fy1b39f637b3c2a06@mail.gmail.com> <867ihdc34c.fsf@ds4.des.no> <20080212190207.GB49155@beastie.creo.hu> <86d4r2540f.fsf@ds4.des.no> <20080213165923.GD49155@beastie.creo.hu> <86zlu493ep.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86zlu493ep.fsf@ds4.des.no> User-Agent: Mutt/1.5.16 (2007-06-09) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (beastie.creo.hu [127.0.0.1]); Thu, 14 Feb 2008 11:15:12 +0100 (CET) Cc: freebsd-fs@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] Remove NTFS kernel support X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2008 10:17:09 -0000 On Wed, Feb 13, 2008 at 06:23:10PM +0100, Dag-Erling Smorgrav wrote: > My implicit assumption was that it would be much harder to maintain a > FUSE library in ports, since it is so tightly bound to the FUSE kernel > module. So then, just to clean up more implicit principles: also you mean that LGPL'd code can't go to the base system unless absolutely necessary? If yes, why so? FreeBSD has embraced recently a big chunk of CDDL'd code without making much fuss about licensing, and for practical purposes, I don't see much difference between CDDL and LGPL (altough the latter is worded undisputedly sickly :)). Regarding your assumption: I think it's not true. The only thing that the lib and the module have in common is the header fuse_kernel.h which defines the data structures and constants used in the kernel/userspace protocol. This protocol is versioned and the kernel and the daemon negotiate the highest proto version supported by both parties and shall communicate according to that. Csaba From owner-freebsd-fs@FreeBSD.ORG Thu Feb 14 12:32:11 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C26F16A420; Thu, 14 Feb 2008 12:32:11 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 41BE213C4DB; Thu, 14 Feb 2008 12:32:11 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id A5491207E; Thu, 14 Feb 2008 13:32:00 +0100 (CET) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: -0.3/3.0 X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on tim.des.no Received: from ds4.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id 956122049; Thu, 14 Feb 2008 13:32:00 +0100 (CET) Received: by ds4.des.no (Postfix, from userid 1001) id 7A29784492; Thu, 14 Feb 2008 13:32:00 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Csaba Henk References: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> <70e8236f0802070321n9097d3fy1b39f637b3c2a06@mail.gmail.com> <867ihdc34c.fsf@ds4.des.no> <20080212190207.GB49155@beastie.creo.hu> <86d4r2540f.fsf@ds4.des.no> <20080213165923.GD49155@beastie.creo.hu> <86zlu493ep.fsf@ds4.des.no> <20080214101511.GE49155@beastie.creo.hu> Date: Thu, 14 Feb 2008 13:32:00 +0100 In-Reply-To: <20080214101511.GE49155@beastie.creo.hu> (Csaba Henk's message of "Thu\, 14 Feb 2008 11\:15\:11 +0100") Message-ID: <86d4qzlnwf.fsf@ds4.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] Remove NTFS kernel support X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2008 12:32:11 -0000 Csaba Henk writes: > So then, just to clean up more implicit principles: also you mean that > LGPL'd code can't go to the base system unless absolutely necessary? I mean that we shouldn't if a suitable BSD-licensed alternative exists. > Regarding your assumption: I think it's not true. The only thing that > the lib and the module have in common is the header fuse_kernel.h which > defines the data structures and constants used in the kernel/userspace > protocol. If those data structures and constants change, the port would have to deal with supporting multiple FreeBSD versions simultaneously. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Thu Feb 14 13:59:32 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AC5C16A418; Thu, 14 Feb 2008 13:59:32 +0000 (UTC) (envelope-from csaba@beastie.creo.hu) Received: from beastie.creo.hu (www.creo.hu [217.113.62.14]) by mx1.freebsd.org (Postfix) with ESMTP id EFB0813C46E; Thu, 14 Feb 2008 13:59:31 +0000 (UTC) (envelope-from csaba@beastie.creo.hu) Received: from beastie.creo.hu (localhost [127.0.0.1]) by beastie.creo.hu (8.14.1/8.14.1) with ESMTP id m1EDvZPt023709 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 14 Feb 2008 14:57:35 +0100 (CET) (envelope-from csaba@beastie.creo.hu) Received: (from csaba@localhost) by beastie.creo.hu (8.14.1/8.14.1/Submit) id m1EDvW46023708; Thu, 14 Feb 2008 14:57:32 +0100 (CET) (envelope-from csaba) Date: Thu, 14 Feb 2008 14:57:32 +0100 From: Csaba Henk To: Dag-Erling Smorgrav Message-ID: <20080214135732.GF49155@beastie.creo.hu> References: <3bbf2fe10802061700p253e68b8s704deb3e5e4ad086@mail.gmail.com> <70e8236f0802070321n9097d3fy1b39f637b3c2a06@mail.gmail.com> <867ihdc34c.fsf@ds4.des.no> <20080212190207.GB49155@beastie.creo.hu> <86d4r2540f.fsf@ds4.des.no> <20080213165923.GD49155@beastie.creo.hu> <86zlu493ep.fsf@ds4.des.no> <20080214101511.GE49155@beastie.creo.hu> <86d4qzlnwf.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86d4qzlnwf.fsf@ds4.des.no> User-Agent: Mutt/1.5.16 (2007-06-09) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (beastie.creo.hu [127.0.0.1]); Thu, 14 Feb 2008 14:57:35 +0100 (CET) Cc: freebsd-fs@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] Remove NTFS kernel support X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2008 13:59:32 -0000 On Thu, Feb 14, 2008 at 01:32:00PM +0100, Dag-Erling Smorgrav wrote: > Csaba Henk writes: > > Regarding your assumption: I think it's not true. The only thing that > > the lib and the module have in common is the header fuse_kernel.h which > > defines the data structures and constants used in the kernel/userspace > > protocol. > > If those data structures and constants change, the port would have to > deal with supporting multiple FreeBSD versions simultaneously. No -- that's what the negotiation thingy is for. It's not a problem if the kld and the lib were compiled with different revisions of that header. Csaba From owner-freebsd-fs@FreeBSD.ORG Thu Feb 14 19:58:15 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 70F3316A417 for ; Thu, 14 Feb 2008 19:58:15 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (unknown [IPv6:2a01:170:102f::2]) by mx1.freebsd.org (Postfix) with ESMTP id F21DF13C465 for ; Thu, 14 Feb 2008 19:58:14 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (localhost [127.0.0.1]) by lurza.secnetix.de (8.14.1/8.14.1) with ESMTP id m1EJwD9m078517; Thu, 14 Feb 2008 20:58:13 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.14.1/8.14.1/Submit) id m1EJwCoZ078516; Thu, 14 Feb 2008 20:58:12 +0100 (CET) (envelope-from olli) Date: Thu, 14 Feb 2008 20:58:12 +0100 (CET) Message-Id: <200802141958.m1EJwCoZ078516@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (FreeBSD/6.2-STABLE-20070808 (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Thu, 14 Feb 2008 20:58:13 +0100 (CET) Cc: Subject: UFS2 corruption (RELENG_7, amd64) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2008 19:58:15 -0000 Hi, We have a problem with a large file system (1.5 TB). This is a UFS2 that was newfs'ed on FreeBSD/amd64 RELENG_7 from December 17. It was formatted with bsize 64K and fsize 8K because of performance reasons. Especially fsck is significantly faster with these settings. There's no disklabel; the whole raw disk is used. Also, the inode density was reduced to one inode per 256 KB, so there are 5.7 million inodes available, of which 33,000 are actually in use currently. The file system contains a busy news spool area. It is mounted with soft-updates and noatime. After while of usage, some corruption seems to occur, and df(1) output becomes very strange, see below. This happened several times already. ================== snip ================== Filesystem 1K-blocks Used Avail Capacity /dev/da45 1463107704 -1202122007974910440 1202122009320969528 -89306778483% # umount /dev/da45 # fsck -y /dev/da45 ** /dev/da45 ** Last Mounted on [...] ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames UNALLOCATED I=2107428 OWNER=2283830233 MODE=0 SIZE=0 MTIME=Feb 5 08:43 2008 NAME=/D.0131ae2e/B.0786 UNEXPECTED SOFT UPDATE INCONSISTENCY REMOVE? yes ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes BLK(S) MISSING IN BIT MAPS SALVAGE? yes 24667 files, 116050264 used, 66838199 free (5135 frags, 8354133 blocks, 0.0% fragmentation) ***** FILE SYSTEM WAS MODIFIED ***** ================== snip ================== After a while, the same thing happens again. In dmesg I see the following messages, but I have no idea if they are related to the above problem: bad block 724405012343388976, ino 759812 pid 48 (softdepflush), uid 0 inumber 759812 on /news2/spool/news/22: bad block bad block -4916881576150019921, ino 759812 pid 48 (softdepflush), uid 0 inumber 759812 on /news2/spool/news/22: bad block bad block -131736542334903744, ino 759812 pid 48 (softdepflush), uid 0 inumber 759812 on /news2/spool/news/22: bad block bad block -6421737673234919641, ino 759812 pid 48 (softdepflush), uid 0 inumber 759812 on /news2/spool/news/22: bad block handle_workitem_freeblocks: block count I do not see any SCSI error messages, so I don't think it is a hardware problem. These are the related boot messages: mpt0: da49 at mpt0 bus 0 target 10 lun 4 da49: Fixed Direct Access SCSI-4 device da49: 300.000MB/s transfers da49: Command Queueing Enabled da49: 1430288MB (2929229824 512 byte sectors: 255H 63S/T 182336C) Did anyone encounter similar problems? Could it be caused by the nonstandard bsize/fsize settings? Are there any known problems, especially on amd64? Any hint or advice is very much appreciated! Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Perl will consistently give you what you want, unless what you want is consistency." -- Larry Wall From owner-freebsd-fs@FreeBSD.ORG Thu Feb 14 20:18:01 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8714D16A469 for ; Thu, 14 Feb 2008 20:18:01 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id DF4B313C4E8; Thu, 14 Feb 2008 20:18:00 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47B4A1F7.2040208@FreeBSD.org> Date: Thu, 14 Feb 2008 21:17:59 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Oliver Fromme References: <200802141958.m1EJwCoZ078516@lurza.secnetix.de> In-Reply-To: <200802141958.m1EJwCoZ078516@lurza.secnetix.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.ORG Subject: Re: UFS2 corruption (RELENG_7, amd64) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2008 20:18:01 -0000 Oliver Fromme wrote: > Did anyone encounter similar problems? Could it be > caused by the nonstandard bsize/fsize settings? It could be. Depending on how urgent this is you might either try reverting to standard settings, or try to reproduce on another system. When I have the time I will also try to reproduce this. Kris From owner-freebsd-fs@FreeBSD.ORG Thu Feb 14 20:36:44 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C974816A417 for ; Thu, 14 Feb 2008 20:36:44 +0000 (UTC) (envelope-from ahurt@goflexitllc.com) Received: from mail.goflexitllc.com (net1.goflexitllc.com [66.180.163.38]) by mx1.freebsd.org (Postfix) with ESMTP id 9A73613C44B for ; Thu, 14 Feb 2008 20:36:44 +0000 (UTC) (envelope-from ahurt@goflexitllc.com) Received: (qmail 7747 invoked by uid 89); 14 Feb 2008 20:12:33 -0000 Received: from unknown (HELO ?172.20.10.187?) (ahurt@anbcs.com@72.158.187.62) by mail.goflexitllc.com with ESMTPA; 14 Feb 2008 20:12:33 -0000 Message-ID: <47B4A00E.1050105@goflexitllc.com> Date: Thu, 14 Feb 2008 14:09:50 -0600 From: Aaron Hurt Organization: Flex I.T., LLC User-Agent: Thunderbird 2.0.0.9 (X11/20071212) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <200802141958.m1EJwCoZ078516@lurza.secnetix.de> In-Reply-To: <200802141958.m1EJwCoZ078516@lurza.secnetix.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: UFS2 corruption (RELENG_7, amd64) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2008 20:36:44 -0000 Have you checked to make sure that there are no physical problems with any of the disks in the array...possibly checked for SMART errors on the drives in the array? Aaron Hurt Managing Partner Flex I.T., LLC 611 Commerce Street Suite 3117 Nashville, TN 37203 Phone: 615.438.7101 E-mail: aaron@goflexitllc.com Oliver Fromme wrote: > Hi, > > We have a problem with a large file system (1.5 TB). > This is a UFS2 that was newfs'ed on FreeBSD/amd64 > RELENG_7 from December 17. > > It was formatted with bsize 64K and fsize 8K because of > performance reasons. Especially fsck is significantly > faster with these settings. There's no disklabel; > the whole raw disk is used. > > Also, the inode density was reduced to one inode per > 256 KB, so there are 5.7 million inodes available, > of which 33,000 are actually in use currently. > The file system contains a busy news spool area. > It is mounted with soft-updates and noatime. > > After while of usage, some corruption seems to occur, > and df(1) output becomes very strange, see below. > This happened several times already. > > ================== snip ================== > > Filesystem 1K-blocks Used Avail Capacity > /dev/da45 1463107704 -1202122007974910440 1202122009320969528 -89306778483% > > # umount /dev/da45 > # fsck -y /dev/da45 > ** /dev/da45 > ** Last Mounted on [...] > ** Phase 1 - Check Blocks and Sizes > ** Phase 2 - Check Pathnames > UNALLOCATED I=2107428 OWNER=2283830233 MODE=0 > SIZE=0 MTIME=Feb 5 08:43 2008 > NAME=/D.0131ae2e/B.0786 > > UNEXPECTED SOFT UPDATE INCONSISTENCY > > REMOVE? yes > > ** Phase 3 - Check Connectivity > ** Phase 4 - Check Reference Counts > ** Phase 5 - Check Cyl groups > FREE BLK COUNT(S) WRONG IN SUPERBLK > SALVAGE? yes > > SUMMARY INFORMATION BAD > SALVAGE? yes > > BLK(S) MISSING IN BIT MAPS > SALVAGE? yes > > 24667 files, 116050264 used, 66838199 free (5135 frags, 8354133 blocks, 0.0% fragmentation) > > ***** FILE SYSTEM WAS MODIFIED ***** > > ================== snip ================== > > After a while, the same thing happens again. > > In dmesg I see the following messages, but I have no > idea if they are related to the above problem: > > bad block 724405012343388976, ino 759812 > pid 48 (softdepflush), uid 0 inumber 759812 on /news2/spool/news/22: bad block > bad block -4916881576150019921, ino 759812 > pid 48 (softdepflush), uid 0 inumber 759812 on /news2/spool/news/22: bad block > bad block -131736542334903744, ino 759812 > pid 48 (softdepflush), uid 0 inumber 759812 on /news2/spool/news/22: bad block > bad block -6421737673234919641, ino 759812 > pid 48 (softdepflush), uid 0 inumber 759812 on /news2/spool/news/22: bad block > handle_workitem_freeblocks: block count > > I do not see any SCSI error messages, so I don't > think it is a hardware problem. These are the > related boot messages: > > mpt0: > da49 at mpt0 bus 0 target 10 lun 4 > da49: Fixed Direct Access SCSI-4 device > da49: 300.000MB/s transfers > da49: Command Queueing Enabled > da49: 1430288MB (2929229824 512 byte sectors: 255H 63S/T 182336C) > > Did anyone encounter similar problems? Could it be > caused by the nonstandard bsize/fsize settings? > Are there any known problems, especially on amd64? > > Any hint or advice is very much appreciated! > > Best regards > Oliver > > From owner-freebsd-fs@FreeBSD.ORG Thu Feb 14 21:03:32 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ADF2316A419 for ; Thu, 14 Feb 2008 21:03:32 +0000 (UTC) (envelope-from cdillon@wolves.k12.mo.us) Received: from mail.wolves.k12.mo.us (mail.wolves.k12.mo.us [207.160.214.1]) by mx1.freebsd.org (Postfix) with ESMTP id 9A32013C45B for ; Thu, 14 Feb 2008 21:03:32 +0000 (UTC) (envelope-from cdillon@wolves.k12.mo.us) Received: from localhost (localhost [127.0.0.1]) by mail.wolves.k12.mo.us (Postfix) with ESMTP id EBA3CB89B; Thu, 14 Feb 2008 15:03:31 -0600 (CST) X-Virus-Scanned: amavisd-new at wolves.k12.mo.us Received: from mail.wolves.k12.mo.us ([127.0.0.1]) by localhost (mail.wolves.k12.mo.us [127.0.0.1]) (amavisd-new, port 10024) with LMTP id j77V58BNQhg4; Thu, 14 Feb 2008 15:03:30 -0600 (CST) Received: from wolves.k12.mo.us (localhost [127.0.0.1]) by mail.wolves.k12.mo.us (Postfix) with ESMTP id 75AD1B89A; Thu, 14 Feb 2008 15:03:30 -0600 (CST) Received: from rstech21.int.wolves.k12.mo.us (rstech21.int.wolves.k12.mo.us [10.1.3.201]) by www.wolves.k12.mo.us (Horde MIME library) with HTTP; Thu, 14 Feb 2008 15:03:30 -0600 Message-ID: <20080214150330.bj28qojhko0w4kgw@www.wolves.k12.mo.us> Date: Thu, 14 Feb 2008 15:03:30 -0600 From: Chris Dillon To: Oliver Fromme References: <200802141958.m1EJwCoZ078516@lurza.secnetix.de> In-Reply-To: <200802141958.m1EJwCoZ078516@lurza.secnetix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.5) / FreeBSD-6.2 Cc: freebsd-fs@FreeBSD.ORG Subject: Re: UFS2 corruption (RELENG_7, amd64) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2008 21:03:32 -0000 Quoting Oliver Fromme : ...snip... > # fsck -y /dev/da45 > ** /dev/da45 > ** Last Mounted on [...] > ** Phase 1 - Check Blocks and Sizes > ** Phase 2 - Check Pathnames > UNALLOCATED I=3D2107428 OWNER=3D2283830233 MODE=3D0 > SIZE=3D0 MTIME=3DFeb 5 08:43 2008 > NAME=3D/D.0131ae2e/B.0786 > > UNEXPECTED SOFT UPDATE INCONSISTENCY > > REMOVE? yes I've been seeing "UNEXPECTED SOFT UPDATE INCONSISTENCY" fsck errors =20 show up in some of my filesytem snapshots on a RELENG_6 AMD64 box for =20 years (actually since RELENG_5), which will eventually lead to "panic: =20 snapblkfree: inconsistent block type" if those snapshots are mounted =20 and used. There are no SCSI or "bad block" errors or any error of any =20 other kind that shows up on my system before the problems occur. I =20 think I used default newfs settings when creating the (comparatively =20 small) 300GB filesystem. The problem happens rarely and on a =20 production system so I've never been able to pin it down to anything =20 in particular. I've found that others have seen the same panic, likely from the same cause: http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/115645 (see first =20 reply in Audit-Trail) http://lists.freebsd.org/pipermail/freebsd-stable/2006-August/027916.html http://lists.freebsd.org/pipermail/freebsd-stable/2006-August/027917.html I did very recently determine that the "snapblkfree: inconsistent =20 block type" panic I see is due directly to the "soft update =20 inconsistency" in the snapshots, so there is some filesystem =20 corruption happening somewhere which leads to the panic later. I have =20 been regularly fscking the snapshots and deleting any of them that =20 have errors so they won't panic the system later. It has kept the =20 system panic-free for quite some time now. A commonality you will notice amongst all our systems is that they =20 are AMD64. I've been unable to find anyone having this problem on i386. --=20 Chris Dillon - NetEng/SysAdm Reeds Spring R-IV School District Technology Department 175 Elementary Rd. Reeds Spring, MO 65737 Voice: 417-272-8266 Fax: 417-272-0015 From owner-freebsd-fs@FreeBSD.ORG Fri Feb 15 09:17:24 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F41816A419 for ; Fri, 15 Feb 2008 09:17:24 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (unknown [IPv6:2a01:170:102f::2]) by mx1.freebsd.org (Postfix) with ESMTP id 8DE8913C448 for ; Fri, 15 Feb 2008 09:17:23 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (localhost [127.0.0.1]) by lurza.secnetix.de (8.14.1/8.14.1) with ESMTP id m1F9HMkg010428; Fri, 15 Feb 2008 10:17:22 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.14.1/8.14.1/Submit) id m1F9HJWw010427; Fri, 15 Feb 2008 10:17:19 +0100 (CET) (envelope-from olli) Date: Fri, 15 Feb 2008 10:17:19 +0100 (CET) Message-Id: <200802150917.m1F9HJWw010427@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG, ahurt@goflexitllc.com In-Reply-To: <47B4A00E.1050105@goflexitllc.com> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (FreeBSD/6.2-STABLE-20070808 (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Fri, 15 Feb 2008 10:17:22 +0100 (CET) Cc: Subject: Re: UFS2 corruption (RELENG_7, amd64) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Feb 2008 09:17:24 -0000 Aaron Hurt wrote: > Have you checked to make sure that there are no physical problems with > any of the disks in the array...possibly checked for SMART errors on the > drives in the array? As I wrote: > > I do not see any SCSI error messages, so I don't > > think it is a hardware problem Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-fs@FreeBSD.ORG Fri Feb 15 09:23:20 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B164B16A41A for ; Fri, 15 Feb 2008 09:23:20 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (unknown [IPv6:2a01:170:102f::2]) by mx1.freebsd.org (Postfix) with ESMTP id 0205C13C461 for ; Fri, 15 Feb 2008 09:23:19 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (localhost [127.0.0.1]) by lurza.secnetix.de (8.14.1/8.14.1) with ESMTP id m1F9NI36010777; Fri, 15 Feb 2008 10:23:18 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.14.1/8.14.1/Submit) id m1F9NHNI010776; Fri, 15 Feb 2008 10:23:17 +0100 (CET) (envelope-from olli) Date: Fri, 15 Feb 2008 10:23:17 +0100 (CET) Message-Id: <200802150923.m1F9NHNI010776@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG, cdillon@wolves.k12.mo.us In-Reply-To: <20080214150330.bj28qojhko0w4kgw@www.wolves.k12.mo.us> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (FreeBSD/6.2-STABLE-20070808 (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Fri, 15 Feb 2008 10:23:18 +0100 (CET) Cc: Subject: Re: UFS2 corruption (RELENG_7, amd64) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Feb 2008 09:23:20 -0000 Chris Dillon wrote: > Oliver Fromme wrote: > ...snip... > > > # fsck -y /dev/da45 > > ** /dev/da45 > > ** Last Mounted on [...] > > ** Phase 1 - Check Blocks and Sizes > > ** Phase 2 - Check Pathnames > > UNALLOCATED I=2107428 OWNER=2283830233 MODE=0 > > SIZE=0 MTIME=Feb 5 08:43 2008 > > NAME=/D.0131ae2e/B.0786 > > > > UNEXPECTED SOFT UPDATE INCONSISTENCY > > > > REMOVE? yes > > I've been seeing "UNEXPECTED SOFT UPDATE INCONSISTENCY" fsck errors > show up in some of my filesytem snapshots on a RELENG_6 AMD64 box for > years (actually since RELENG_5), which will eventually lead to "panic: > snapblkfree: inconsistent block type" if those snapshots are mounted > and used. We do not use snapshots, and we don't see such panics. So this is probably unrelated. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "I made up the term 'object-oriented', and I can tell you I didn't have C++ in mind." -- Alan Kay, OOPSLA '97 From owner-freebsd-fs@FreeBSD.ORG Fri Feb 15 10:58:36 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED55416A417 for ; Fri, 15 Feb 2008 10:58:36 +0000 (UTC) (envelope-from jhs@berklix.org) Received: from flat.berklix.org (flat.berklix.org [83.236.223.115]) by mx1.freebsd.org (Postfix) with ESMTP id 48DD513C44B for ; Fri, 15 Feb 2008 10:58:35 +0000 (UTC) (envelope-from jhs@berklix.org) Received: from js.berklix.net (p549A491F.dip.t-dialin.net [84.154.73.31]) (authenticated bits=0) by flat.berklix.org (8.13.8/8.13.8) with ESMTP id m1FAwYD0084318; Fri, 15 Feb 2008 11:58:34 +0100 (CET) (envelope-from jhs@berklix.org) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by js.berklix.net (8.13.8/8.13.8) with ESMTP id m1FB0i2a086538; Fri, 15 Feb 2008 12:00:44 +0100 (CET) (envelope-from jhs@berklix.org) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.13.8/8.13.8) with ESMTP id m1FB0TvO054046; Fri, 15 Feb 2008 12:00:34 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <200802151100.m1FB0TvO054046@fire.js.berklix.net> To: Oliver Fromme In-reply-to: <200802150923.m1F9NHNI010776@lurza.secnetix.de> References: <200802150923.m1F9NHNI010776@lurza.secnetix.de> Comments: In-reply-to Oliver Fromme message dated "Fri, 15 Feb 2008 10:23:17 +0100." Date: Fri, 15 Feb 2008 12:00:29 +0100 From: "Julian H. Stacey" Cc: freebsd-fs@freebsd.org Subject: Re: UFS2 corruption (RELENG_7, amd64) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Feb 2008 10:58:37 -0000 Oliver Fromme wrote: > Chris Dillon wrote: > > Oliver Fromme wrote: > > ...snip... > > > > > # fsck -y /dev/da45 > > > ** /dev/da45 > > > ** Last Mounted on [...] > > > ** Phase 1 - Check Blocks and Sizes > > > ** Phase 2 - Check Pathnames > > > UNALLOCATED I=2107428 OWNER=2283830233 MODE=0 > > > SIZE=0 MTIME=Feb 5 08:43 2008 > > > NAME=/D.0131ae2e/B.0786 > > > > > > UNEXPECTED SOFT UPDATE INCONSISTENCY > > > > > > REMOVE? yes > > > > I've been seeing "UNEXPECTED SOFT UPDATE INCONSISTENCY" fsck errors > > show up in some of my filesytem snapshots on a RELENG_6 AMD64 box for > > years (actually since RELENG_5), which will eventually lead to "panic: > > snapblkfree: inconsistent block type" if those snapshots are mounted > > and used. > > We do not use snapshots, and we don't see such panics. > So this is probably unrelated. > > Best regards > Oliver I've never enabled snapshots either, but was suprised to find one recently! Though I recall it had been around a long while, it was also certainly bigger than the physical disk it was on, (presumably a sparse file). I just deleted it, & guessed some flag or config file presence/ absence/ corruption etc had somehow triggered it. (No one else would have enabled snapshots on it). As it had hung, crashed or been troublesome lately, I did a foreground fsck rather background, & I think a 2nd fsck -y straight after 1st, which I think fixed some more, (sometimes does). Maybe schedule an outage, take off line & single user { foreground fsck, & repeat till you get a clean run where it fixes nothing || backup & newfs }. Good luck! Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com From owner-freebsd-fs@FreeBSD.ORG Fri Feb 15 11:13:21 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 788BA16A417 for ; Fri, 15 Feb 2008 11:13:21 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (unknown [IPv6:2a01:170:102f::2]) by mx1.freebsd.org (Postfix) with ESMTP id CF0C913C46E for ; Fri, 15 Feb 2008 11:13:20 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (localhost [127.0.0.1]) by lurza.secnetix.de (8.14.1/8.14.1) with ESMTP id m1FBDHxd015196; Fri, 15 Feb 2008 12:13:18 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.14.1/8.14.1/Submit) id m1FBDHYG015194; Fri, 15 Feb 2008 12:13:17 +0100 (CET) (envelope-from olli) From: Oliver Fromme Message-Id: <200802151113.m1FBDHYG015194@lurza.secnetix.de> To: jhs@berklix.org (Julian H. Stacey) Date: Fri, 15 Feb 2008 12:13:17 +0100 (CET) In-Reply-To: <200802151100.m1FB0TvO054046@fire.js.berklix.net> X-Mailer: ELM [version 2.5 PL8] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Fri, 15 Feb 2008 12:13:18 +0100 (CET) Cc: freebsd-fs@freebsd.org Subject: Re: UFS2 corruption (RELENG_7, amd64) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Feb 2008 11:13:21 -0000 Julian H. Stacey wrote: > Oliver Fromme wrote: > > Chris Dillon wrote: > > > I've been seeing "UNEXPECTED SOFT UPDATE INCONSISTENCY" fsck errors > > > show up in some of my filesytem snapshots on a RELENG_6 AMD64 box for > > > years (actually since RELENG_5), which will eventually lead to "panic: > > > snapblkfree: inconsistent block type" if those snapshots are mounted > > > and used. > > > > We do not use snapshots, and we don't see such panics. > > So this is probably unrelated. > > I've never enabled snapshots either, but was suprised to find one > recently! Though I recall it had been around a long while, it was > also certainly bigger than the physical disk it was on, (presumably > a sparse file). Yes, snapshots are sparse files, basically. > I just deleted it, & guessed some flag or > config file presence/ absence/ corruption etc had somehow > triggered it. (No one else would have enabled snapshots on it). If you haven't disabled background fsck in /etc/rc.conf, then snapshots will be created upon reboot after an unclean shutdown. Maybe that's where your snapshot came from. Another way to create snapshots implicitely is with dump(8), but you have to pass the -L option, so that shouldn't happen without you knowing. > Maybe schedule an outage, take off line & single user { foreground > fsck, & repeat till you get a clean run where it fixes nothing || > backup & newfs }. Hm. Yes, I think I will do an fsck -y -f on it until it is clean. But I fear the problem will occur again after a while. > Good luck! Thank you! Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Being really good at C++ is like being really good at using rocks to sharpen sticks." -- Thant Tessman From owner-freebsd-fs@FreeBSD.ORG Sat Feb 16 10:56:55 2008 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B40EE16A41B for ; Sat, 16 Feb 2008 10:56:55 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 062B513C4D5 for ; Sat, 16 Feb 2008 10:56:55 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 311FA46B46 for ; Sat, 16 Feb 2008 05:56:54 -0500 (EST) Date: Sat, 16 Feb 2008 10:56:54 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: freebsd-fs@FreeBSD.org Message-ID: <20080216105116.Q93919@fledge.watson.org> MIME-Version: 1.0 Content-Type: MULTIPART/Mixed; BOUNDARY="621616949-433016993-1203135147=:93919" Content-ID: <20080216105116.G93919@fledge.watson.org> Cc: Subject: Patches to get Arla running on FreeBSD 8-CURRENT (fwd) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Feb 2008 10:56:55 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --621616949-433016993-1203135147=:93919 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII; FORMAT=flowed Content-ID: <20080216105116.A93919@fledge.watson.org> FYI--looks like Arla is now at least minimally running on FreeBSD 8.x. The continuing plan is: (1) Make sure Arla is actually stable on FreeBSD 8-CURRENT and add FreeBSD 7-STABLE to the mix, get more people testing it on recent FreeBSD versions. With any luck the attached patch will go back into the base Arla tree in some form or another. (2) Import the nnpfs kernel module into the base CVS tree in order to avoid future VFS-related lag, but retain Arla in the ports collection. Likely de-ifdef the module once imported. The goal would be to get to the point where an AFS client can be used out of the box in FreeBSD 7.1 by simply installing the Arla port. Right now it appears the Heimdal in the base system is too old to work with the latest Arla, requiring installing the Heimdal port to be installed. I'm not sure who's maintianing Heimdal in the base tree currently, but we should get that fixed on general principle. Robert N M Watson Computer Laboratory University of Cambridge ---------- Forwarded message ---------- Date: Sat, 16 Feb 2008 04:12:27 +0000 (GMT) From: Robert Watson To: arla-drinkers@stacken.kth.se Cc: afs@FreeBSD.org Subject: Patches to get Arla running on FreeBSD 8-CURRENT Dear Arla folk, I've spent the evening getting Arla (checked out of anoncvs) up and running on FreeBSD 8-CURRENT, and have it at least minimally running (/afs mounted, some directory access, read a few files, etc), although arlad core dumped fairly quickly during use. I've not yet attempted to debug that problem though. I've attached a patch that does a few things: First, it does a few minor Arla cleanups that appear to be necessary to build on FreeBSD 8: A few general Arla ifdef fixes, etc, such as testing defined(SunOS) before using the value, likewise on __NetBSD_Version__, OpenBSD, etc. Fix two build dependency issues I ran into regarding building arlalib before dependent tools and an include that broke (at least on FreeBSD). Second, it adds new autoconf and ifdef parts to get Arla building on FreeBSD 8, such as handling VFS changes that appeared in FreeBSD 7.x and 8.x, the priv(9) kernel privilege framework, some include problems I ran into with using /usr/src/sys before /usr/include/sys (which doesn't work for generated files such as vnode_if.h), and things along those lines. Unfortunately, I'm not set up to easily build test on other platforms, and I've also not had a chance to try this on FreeBSD 7 -- my guess is some minor tweaks may be required with respect to both of those, but hopefully these are steps in the right direction and someone with a bit more Arla experience can sort out what I've done into things with keeping and things with fixing :-). I'll investigate the arlad crash tomorrow. Patch also up at: http://www.watson.org/~robert/freebsd/20080216-arla.diff Robert N M Watson Computer Laboratory University of Cambridge --621616949-433016993-1203135147=:93919 Content-Type: TEXT/X-DIFF; CHARSET=US-ASCII; NAME=20080216-arla.diff Content-Transfer-Encoding: BASE64 Content-ID: <20080216041227.P93919@fledge.watson.org> Content-Description: Content-Disposition: ATTACHMENT; FILENAME=20080216-arla.diff SW5kZXg6IE1ha2VmaWxlLmFtDQo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09DQpS Q1MgZmlsZTogL3N0YWNrZW4tY3ZzL2FybGEvTWFrZWZpbGUuYW0sdg0KcmV0 cmlldmluZyByZXZpc2lvbiAxLjUNCmRpZmYgLXUgLXIxLjUgTWFrZWZpbGUu YW0NCi0tLSBNYWtlZmlsZS5hbQkyNSBKYW4gMjAwNyAxMToyMzowNSAtMDAw MAkxLjUNCisrKyBNYWtlZmlsZS5hbQkxNiBGZWIgMjAwOCAwMzo0OTo0OSAt MDAwMA0KQEAgLTE5LDYgKzE5LDcgQEANCiAJCWxpYi9idWZkaXIgXA0KIAkJ JChyeGthZCkgXA0KIAkJbm5wZnMgXA0KKwkJYXBwbC9saWIgXA0KIAkJYXJs YWQgXA0KIAkJY29uZiBcDQogCQlhcHBsIFwNCkluZGV4OiBjb25maWd1cmUu aW4NCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvc3RhY2tl bi1jdnMvYXJsYS9jb25maWd1cmUuaW4sdg0KcmV0cmlldmluZyByZXZpc2lv biAxLjc1OA0KZGlmZiAtdSAtcjEuNzU4IGNvbmZpZ3VyZS5pbg0KLS0tIGNv bmZpZ3VyZS5pbgkyNSBKYW4gMjAwNyAwOToyODo0NiAtMDAwMAkxLjc1OA0K KysrIGNvbmZpZ3VyZS5pbgkxNiBGZWIgMjAwOCAwMzo0OTo1MiAtMDAwMA0K QEAgLTM5Niw3ICszOTYsNyBAQA0KICAgS0VSTkVMPWBzeXNjdGwgLW4ga2Vy bi5ib290ZmlsZWANCiAgIEFDX1dFUlJPUihCU0RfV0VSUk9SKQ0KICAgTk5Q RlNfU1VCRElSPWZyZWVic2QNCi0gIEtFUk5FTF9JTkNMVURFPS1JYChjZCAk c3JjZGlyOyBwd2QpYC9ubnBmcy9ic2QNCisgIEtFUk5FTF9JTkNMVURFPS1J L3Vzci9pbmNsdWRlIC1JYChjZCAkc3JjZGlyOyBwd2QpYC9ubnBmcy9ic2QN CiAgIEtFUk5FTF9TUkNTPSdic2Qtc3Vici5jJw0KIA0KICAgS0VSTkVMPWBz eXNjdGwgLW4ga2Vybi5ib290ZmlsZWANCkBAIC00MzMsNyArNDMzLDcgQEAN CiAJZXNhYw0KIAlzaGlmdA0KICAgZG9uZQ0KLSAgS0VSTkVMX0NQUEZMQUdT PSIke0tFUk5FTF9DUFBGTEFHU30gLUkuICRkZWZzJHtkZWZzOisgfSRmbGFn cyR7ZmxhZ3M6KyB9JGluY2wiDQorICBLRVJORUxfQ1BQRkxBR1M9IiR7S0VS TkVMX0NQUEZMQUdTfSAtSS91c3IvaW5jbHVkZSAtSS4gJGRlZnMke2RlZnM6 KyB9JGZsYWdzJHtmbGFnczorIH0kaW5jbCINCiAgIHRlc3RfS0VSTkVMX0NG TEFHUz0iJHtLRVJORUxfQ0ZMQUdTfSINCiAgIEtFUk5FTF9MRD0nbGQnDQog DQpAQCAtODY3LDYgKzg2Nyw3IEBADQogCQlzeXMvbXV0ZXguaAkJCVwNCiAJ CXN5cy9wYXJhbS5oCQkJXA0KIAkJc3lzL3ByY3RsLmgJCQlcDQorCQlzeXMv cHJpdi5oCQkJXA0KIAkJc3lzL3BvbGwuaAkJCVwNCiAJCXN5cy9wb29sLmgJ CQlcDQogCQlzeXMvcXVldWUuaAkJCVwNCkBAIC0xMzU2LDExICsxMzU3LDEz IEBADQogdWRldjJkZXYJCQkJCVwNCiBzbnByaW50ZgkJCQkJXA0KIHN1c2Vy X3VjcmVkCQkJCQlcDQorcHJpdl9jaGVjawkJCQkJXA0KIG5hbWVpX2hhc2gJ CQkJCVwNCiBub3N5cwkJCQkJCVwNCiBzeXNfbm9zeXMJCQkJCVwNCiBzeXNf bGttbm9zeXMJCQkJCVwNCiBjYWNoZV9wdXJnZXZmcwkJCQkJXA0KK2luc21u dHF1ZQkJCQkJXA0KIF0pDQogQUNfQ0hFQ0tfS0VSTkVMX0ZVTkMobWVtY3B5 LCBbMCwwLDBdKQ0KIA0KQEAgLTE0NDYsOCArMTQ0OSwxMiBAQA0KIEFDX0NI RUNLX0tFUk5FTF9WT1BfVA0KIEFDX0JTRF9GVU5DX1ZGU19PQkpFQ1RfQ1JF QVRFDQogQUNfQlNEX0ZVTkNfVk9QX0xPQ0sNCitBQ19CU0RfRlVOQ19WT1Bf VU5MT0NLDQorQUNfQlNEX0ZVTkNfVk9QX09QRU4NCiBBQ19CU0RfRlVOQ19W RlNfQlVTWQ0KK0FDX0JTRF9GVU5DX1ZGU19RVU9UQUNUTA0KIEFDX0JTRF9G VU5DX1ZHRVQNCitBQ19CU0RfRlVOQ19WTk9ERV9DUkVBVEVfVk9CSkVDVA0K IEFDX0JTRF9GVU5DX1NVU0VSDQogQUNfQlNEX0ZVTkNfVkZTX0dFVE5FV0ZT SUQNCiBBQ19CU0RfRlVOQ19MT0NLTUdSDQpJbmRleDogYXBwbC9mcy9mc19s b2NhbC5oDQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09DQpSQ1MgZmlsZTogL3N0 YWNrZW4tY3ZzL2FybGEvYXBwbC9mcy9mc19sb2NhbC5oLHYNCnJldHJpZXZp bmcgcmV2aXNpb24gMS4zOA0KZGlmZiAtdSAtcjEuMzggZnNfbG9jYWwuaA0K LS0tIGFwcGwvZnMvZnNfbG9jYWwuaAkxNiBNYXIgMjAwNiAxNDozODoyOCAt MDAwMAkxLjM4DQorKysgYXBwbC9mcy9mc19sb2NhbC5oCTE2IEZlYiAyMDA4 IDAzOjQ5OjUzIC0wMDAwDQpAQCAtNDMsNyArNDMsNiBAQA0KICNlbmRpZg0K ICNpbmNsdWRlIDxwYXJzZV91bml0cy5oPg0KICNpbmNsdWRlIDxubnBmcy9u bnBmc19kZWJ1Zy5oPg0KLSNpbmNsdWRlIDxubnBmcy9ubnBmc19kZWIuaD4N CiAjaW5jbHVkZSA8YXJsYWRlYi5oPg0KIA0KICNpbmNsdWRlIDx2ZXJzLmg+ DQpJbmRleDogY2YvYnNkLXZmcy1xdW90YWN0bC5tNA0KPT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PQ0KUkNTIGZpbGU6IGNmL2JzZC12ZnMtcXVvdGFjdGwubTQN CmRpZmYgLU4gY2YvYnNkLXZmcy1xdW90YWN0bC5tNA0KLS0tIC9kZXYvbnVs bAkxIEphbiAxOTcwIDAwOjAwOjAwIC0wMDAwDQorKysgY2YvYnNkLXZmcy1x dW90YWN0bC5tNAkxNiBGZWIgMjAwOCAwMzo0OTo1MyAtMDAwMA0KQEAgLTAs MCArMSwzOCBAQA0KK2RubA0KK2RubCAkSWQkDQorZG5sDQorDQorZG5sDQor ZG5sIEZpbmQgb3V0IGlmIFZGU19RVU9UQUNUTCBhY2NlcHRzIGEgdm9pZCAq IG9yIGEgY2FkZHJfdCBhcmd1bWVudC4NCitkbmwNCisNCitBQ19ERUZVTihb QUNfQlNEX0ZVTkNfVkZTX1FVT1RBQ1RMXSwgWw0KK0FDX0NBQ0hFX0NIRUNL KGlmIFZGU19RVU9UQUNUTCB0YWtlcyBjYWRkcl90IGFyZ3VtZW50LCBhY19j dl9mdW5jX3Zmc19xdW90YWN0bF9jYWRkciwNCitBQ19UUllfQ09NUElMRV9L RVJORUwoWw0KKyNpZmRlZiBIQVZFX1NZU19DREVGU19IDQorI2luY2x1ZGUg PHN5cy9jZGVmcy5oPg0KKyNlbmRpZg0KKyNpbmNsdWRlIDxzeXMvcGFyYW0u aD4NCisjaW5jbHVkZSA8c3lzL3R5cGVzLmg+DQorI2luY2x1ZGUgPHN5cy90 aW1lLmg+DQorI2luY2x1ZGUgPHN5cy91aW8uaD4NCisjaW5jbHVkZSA8c3lz L3Zub2RlLmg+DQorI2luY2x1ZGUgPHN5cy9tb3VudC5oPg0KKw0KK3Zmc19x dW90YWN0bF90IGZvb19xdW90YWN0bDsNCisNCitpbnQNCitmb29fcXVvdGFj dGwoc3RydWN0IG1vdW50ICptcCwgaW50IGNtZHMsIHVpZF90IHVpZCwgY2Fk ZHJfdCBhcmcsDQorICAgIHN0cnVjdCB0aHJlYWQgKnRkKQ0KK3sNCisNCisJ cmV0dXJuICgwKTsNCit9DQorXSxbXSwNCithY19jdl9mdW5jX3Zmc19xdW90 Y3RsX2NhZGRyPXllcywNCithY19jdl9mdW5jX3Zmc19xdW90YWN0bF9jYWRk cj1ubykpDQoraWYgdGVzdCAiJGFjX2N2X2Z1bmNfdmZzX3F1b3RhY3RsX2Nh ZGRyIiA9IHllczsgdGhlbg0KKwlBQ19ERUZJTkUoSEFWRV9WRlNfUVVPVEFD VExfQ0FERFIsIDEsDQorCVtkZWZpbmUgaWYgVkZTX1FVT1RBQ1RMIHRha2Vz IGEgY2FkZHJfdCBhcmd1bWVudF0pDQorZmkNCitdKQ0KSW5kZXg6IGNmL2Jz ZC12bm9kZS1jcmVhdGUtdm9iamVjdC5tNA0KPT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PQ0KUkNTIGZpbGU6IGNmL2JzZC12bm9kZS1jcmVhdGUtdm9iamVjdC5t NA0KZGlmZiAtTiBjZi9ic2Qtdm5vZGUtY3JlYXRlLXZvYmplY3QubTQNCi0t LSAvZGV2L251bGwJMSBKYW4gMTk3MCAwMDowMDowMCAtMDAwMA0KKysrIGNm L2JzZC12bm9kZS1jcmVhdGUtdm9iamVjdC5tNAkxNiBGZWIgMjAwOCAwMzo0 OTo1MyAtMDAwMA0KQEAgLTAsMCArMSwyOCBAQA0KK2RubA0KK2RubCAkSWQk DQorZG5sDQorDQorZG5sDQorZG5sIEZpbmQgb3V0IGlmIHZub2RlX2NyZWF0 ZV92b2JqZWN0KCkgdGFrZXMgb25lIGFyZ3VtZW50IG9yIHR3byBvbiBCU0Q7 IGlmDQorZG5sIHR3byB0aGVuIHdlIG5lZWQgdG8gdXNlIHZub2RlX2NyZWF0 ZV92b2JqZWN0X29mZigpIGluc3RlYWQuDQorZG5sDQorDQorQUNfREVGVU4o W0FDX0JTRF9GVU5DX1ZOT0RFX0NSRUFURV9WT0JKRUNUXSwgWw0KK0FDX0NB Q0hFX0NIRUNLKGlmIHZub2RlX2NyZWF0ZV92b2JqZWN0IHRha2VzIHRocmVl IGFyZ3VtZW50cywgYWNfY3ZfZnVuY192bm9kZV9jcmVhdGVfdm9iamVjdCwN CitBQ19UUllfQ09NUElMRV9LRVJORUwoWw0KKyNpZmRlZiBIQVZFX1NZU19D REVGU19IDQorI2luY2x1ZGUgPHN5cy9jZGVmcy5oPg0KKyNlbmRpZg0KKyNp bmNsdWRlIDxzeXMvcGFyYW0uaD4NCisjaW5jbHVkZSA8c3lzL3R5cGVzLmg+ DQorI2luY2x1ZGUgPHN5cy90aW1lLmg+DQorI2luY2x1ZGUgPHN5cy91aW8u aD4NCisjaW5jbHVkZSA8c3lzL3Zub2RlLmg+DQorXSxbdm5vZGVfY3JlYXRl X3ZvYmplY3QoMCwgMCwgMCldLA0KK2FjX2N2X2Z1bmNfdm5vZGVfY3JlYXRl X3ZvYmplY3RfdGhyZWVfYXJncz15ZXMsDQorYWNfY3ZfZnVuY192bm9kZV9j cmVhdGVfdm9iamVjdF90aHJlZV9hcmdzPW5vKSkNCitpZiB0ZXN0ICIkYWNf Y3ZfZnVuY192bm9kZV9jcmVhdGVfdm9iamVjdF90aHJlZV9hcmdzIiA9IHll czsgdGhlbg0KKwlBQ19ERUZJTkUoSEFWRV9USFJFRV9BUkdVTUVOVF9WTk9E RV9DUkVBVEVfVk9CSiwgMSwNCisJW2RlZmluZSBpZiB2bm9kZV9jcmVhdGVf dm9iamVjdCB0YWtlcyB0aHJlZSBhcmd1bWVudHNdKQ0KK2ZpDQorXSkNCklu ZGV4OiBjZi9ic2Qtdm9wLW9wZW4ubTQNCj09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT0NClJDUyBmaWxlOiBjZi9ic2Qtdm9wLW9wZW4ubTQNCmRpZmYgLU4gY2Yv YnNkLXZvcC1vcGVuLm00DQotLS0gL2Rldi9udWxsCTEgSmFuIDE5NzAgMDA6 MDA6MDAgLTAwMDANCisrKyBjZi9ic2Qtdm9wLW9wZW4ubTQJMTYgRmViIDIw MDggMDM6NDk6NTMgLTAwMDANCkBAIC0wLDAgKzEsMjggQEANCitkbmwNCitk bmwgJElkJA0KK2RubA0KKw0KK2RubA0KK2RubCBGaW5kIG91dCBpZiBWT1Bf T1BFTiB0YWtlcyBhIHN0cnVjdCBmaWxlIG9yIGFuIGludGVnZXIgZmluYWwg YXJndW1lbnQgb24NCitkbmwgRnJlZUJTRC4NCitkbmwNCisNCitBQ19ERUZV TihbQUNfQlNEX0ZVTkNfVk9QX09QRU5dLCBbDQorQUNfQ0FDSEVfQ0hFQ0so aWYgVk9QX09QRU4gdGFrZXMgYSBzdHJ1Y3QgZmlsZSBmaW5hbCBhcmd1bWVu dCwgYWNfY3ZfZnVuY192b3Bfb3Blbl9maWxlX2FyZywNCitBQ19UUllfQ09N UElMRV9LRVJORUwoWw0KKyNpZmRlZiBIQVZFX1NZU19DREVGU19IDQorI2lu Y2x1ZGUgPHN5cy9jZGVmcy5oPg0KKyNlbmRpZg0KKyNpbmNsdWRlIDxzeXMv cGFyYW0uaD4NCisjaW5jbHVkZSA8c3lzL3R5cGVzLmg+DQorI2luY2x1ZGUg PHN5cy90aW1lLmg+DQorI2luY2x1ZGUgPHN5cy91aW8uaD4NCisjaW5jbHVk ZSA8c3lzL3Zub2RlLmg+DQorXSxbVk9QX09QRU4oTlVMTCwgMCwgTlVMTCwg TlVMTCwgKHN0cnVjdCBmaWxlICopTlVMTCldLA0KK2FjX2N2X2Z1bmNfdm9w X29wZW5fZmlsZV9hcmc9eWVzLA0KK2FjX2N2X2Z1bmNfdm9wX29wZW5fZmls ZV9hcmc9bm8pKQ0KK2lmIHRlc3QgIiRhY19jdl9mdW5jX3ZvcF9vcGVuX2Zp bGVfYXJnIiA9IHllczsgdGhlbg0KKwlBQ19ERUZJTkUoSEFWRV9GSU5BTF9B UkdfRklMRV9WT1BfT1BFTiwgMSwNCisJW2RlZmluZSBpZiBWT1BfT1BFTiB0 YWtlcyBhIGZpbGUgZmluYWwgYXJndW1lbnRdKQ0KK2ZpDQorXSkNCkluZGV4 OiBjZi9ic2Qtdm9wLXVubG9jay5tNA0KPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PQ0KUkNTIGZpbGU6IGNmL2JzZC12b3AtdW5sb2NrLm00DQpkaWZmIC1OIGNm L2JzZC12b3AtdW5sb2NrLm00DQotLS0gL2Rldi9udWxsCTEgSmFuIDE5NzAg MDA6MDA6MDAgLTAwMDANCisrKyBjZi9ic2Qtdm9wLXVubG9jay5tNAkxNiBG ZWIgMjAwOCAwMzo0OTo1MyAtMDAwMA0KQEAgLTAsMCArMSw0NSBAQA0KK2Ru bA0KK2RubCAkSWQkDQorZG5sDQorDQorZG5sDQorZG5sIEZpbmQgb3V0IGlm IFZPUF9VTkxPQ0sgdGFrZXMgdHdvIG9yIHRocmVlIGFyZ3VtZW50cw0KK2Ru bA0KKw0KK0FDX0RFRlVOKFtBQ19CU0RfRlVOQ19WT1BfVU5MT0NLXSwgWw0K K0FDX0NBQ0hFX0NIRUNLKGlmIFZPUF9VTkxPQ0sgdGFrZXMgdHdvIGFyZ3Vt ZW50cywgYWNfY3ZfZnVuY192b3BfdW5sb2NrX3R3b19hcmdzLA0KK0FDX1RS WV9DT01QSUxFX0tFUk5FTChbDQorI2lmZGVmIEhBVkVfU1lTX0NERUZTX0gN CisjaW5jbHVkZSA8c3lzL2NkZWZzLmg+DQorI2VuZGlmDQorI2luY2x1ZGUg PHN5cy9wYXJhbS5oPg0KKyNpbmNsdWRlIDxzeXMvdHlwZXMuaD4NCisjaW5j bHVkZSA8c3lzL3RpbWUuaD4NCisjaW5jbHVkZSA8c3lzL3Vpby5oPg0KKyNp bmNsdWRlIDxzeXMvdm5vZGUuaD4NCitdLFtWT1BfVU5MT0NLKDAsIDApXSwN CithY19jdl9mdW5jX3ZvcF91bmxvY2tfdHdvX2FyZ3M9eWVzLA0KK2FjX2N2 X2Z1bmNfdm9wX3VubG9ja190d29fYXJncz1ubykpDQoraWYgdGVzdCAiJGFj X2N2X2Z1bmNfdm9wX3VubG9ja190d29fYXJncyIgPSB5ZXM7IHRoZW4NCisJ QUNfREVGSU5FKEhBVkVfVFdPX0FSR1VNRU5UX1ZPUF9VTkxPQ0ssIDEsDQor CVtkZWZpbmUgaWYgVk9QX1VOTE9DSyB0YWtlcyB0d28gYXJndW1lbnRzXSkN CitmaQ0KKw0KK0FDX0NBQ0hFX0NIRUNLKGlmIFZPUF9VTkxPQ0sgdGFrZXMg dGhyZWUgYXJndW1lbnRzLCBhY19jdl9mdW5jX3ZvcF91bmxvY2tfdGhyZWVf YXJncywNCitBQ19UUllfQ09NUElMRV9LRVJORUwoWw0KKyNpZmRlZiBIQVZF X1NZU19DREVGU19IDQorI2luY2x1ZGUgPHN5cy9jZGVmcy5oPg0KKyNlbmRp Zg0KKyNpbmNsdWRlIDxzeXMvcGFyYW0uaD4NCisjaW5jbHVkZSA8c3lzL3R5 cGVzLmg+DQorI2luY2x1ZGUgPHN5cy90aW1lLmg+DQorI2luY2x1ZGUgPHN5 cy91aW8uaD4NCisjaW5jbHVkZSA8c3lzL3Zub2RlLmg+DQorXSxbVk9QX1VO TE9DSygwLCAwLCAwKV0sDQorYWNfY3ZfZnVuY192b3BfdW5sb2NrX3RocmVl X2FyZ3M9eWVzLA0KK2FjX2N2X2Z1bmNfdm9wX3VubG9ja190aHJlZV9hcmdz PW5vKSkNCitpZiB0ZXN0ICIkYWNfY3ZfZnVuY192b3BfdW5sb2NrX3RocmVl X2FyZ3MiID0geWVzOyB0aGVuDQorCUFDX0RFRklORShIQVZFX1RIUkVFX0FS R1VNRU5UX1ZPUF9VTkxPQ0ssIDEsDQorCVtkZWZpbmUgaWYgVk9QX1VOTE9D SyB0YWtlcyB0aHJlZSBhcmd1bWVudHNdKQ0KK2ZpDQorXSkNCkluZGV4OiBj Zi90cnktY29tcGlsZS1rZXJuZWwubTQNCj09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT0NClJDUyBmaWxlOiAvc3RhY2tlbi1jdnMvYXJsYS9jZi90cnktY29tcGls ZS1rZXJuZWwubTQsdg0KcmV0cmlldmluZyByZXZpc2lvbiAxLjQNCmRpZmYg LXUgLXIxLjQgdHJ5LWNvbXBpbGUta2VybmVsLm00DQotLS0gY2YvdHJ5LWNv bXBpbGUta2VybmVsLm00CTEyIEZlYiAyMDA0IDE2OjI4OjE4IC0wMDAwCTEu NA0KKysrIGNmL3RyeS1jb21waWxlLWtlcm5lbC5tNAkxNiBGZWIgMjAwOCAw Mzo0OTo1MyAtMDAwMA0KQEAgLTgsNyArOCw3IEBADQogaWYgdGVzdCAiWCR7 S0VSTkVMX0NDfSIgIT0gIlgiOyB0aGVuDQogICBDQz0iJEtFUk5FTF9DQyIN CiBmaQ0KLUNGTEFHUz0iJENGTEFHUyAkdGVzdF9LRVJORUxfQ0ZMQUdTICRL RVJORUxfQ1BQRkxBR1MiDQorQ0ZMQUdTPSIkQ0ZMQUdTIC1JL3Vzci9pbmNs dWRlICR0ZXN0X0tFUk5FTF9DRkxBR1MgJEtFUk5FTF9DUFBGTEFHUyINCiBB Q19UUllfQ09NUElMRShbJDFdLCBbJDJdLCBbJDNdLCBbJDRdKQ0KIENGTEFH Uz0iJHNhdmVfQ0ZMQUdTIg0KIENDPSIkc2F2ZV9DQyINCkluZGV4OiBpbmNs dWRlL2Fmc3N5c2RlZnMuaA0KPT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQ0KUkNT IGZpbGU6IC9zdGFja2VuLWN2cy9hcmxhL2luY2x1ZGUvYWZzc3lzZGVmcy5o LHYNCnJldHJpZXZpbmcgcmV2aXNpb24gMS45DQpkaWZmIC11IC1yMS45IGFm c3N5c2RlZnMuaA0KLS0tIGluY2x1ZGUvYWZzc3lzZGVmcy5oCTkgQXVnIDIw MDQgMTM6NDI6MzIgLTAwMDAJMS45DQorKysgaW5jbHVkZS9hZnNzeXNkZWZz LmgJMTYgRmViIDIwMDggMDM6NDk6NTMgLTAwMDANCkBAIC00Miw2ICs0Miw3 IEBADQogICogZW50cnkgcG9pbnQgc3lzY2FsbHMuDQogICovDQogDQorI2lm IGRlZmluZWQoU3VuT1MpDQogI2lmIFN1bk9TID09IDQwDQogI2RlZmluZSBB RlNfU1lTQ0FMTAkzMQ0KICNlbmRpZg0KQEAgLTU2LDYgKzU3LDcgQEANCiAN CiAjaWYgU3VuT1MgPj0gNTgNCiAjZGVmaW5lIEFGU19TWVNDQUxMCTY1DQor I2VuZGlmDQogI2VuZGlmDQogDQogI2lmIGRlZmluZWQoX19ocHV4KQ0KSW5k ZXg6IG5ucGZzL2JzZC9ubnBmc19ibG9ja3MuYw0KPT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PQ0KUkNTIGZpbGU6IC9zdGFja2VuLWN2cy9hcmxhL25ucGZzL2Jz ZC9ubnBmc19ibG9ja3MuYyx2DQpyZXRyaWV2aW5nIHJldmlzaW9uIDEuMTAN CmRpZmYgLXUgLXIxLjEwIG5ucGZzX2Jsb2Nrcy5jDQotLS0gbm5wZnMvYnNk L25ucGZzX2Jsb2Nrcy5jCTI4IE1hciAyMDA3IDEyOjA1OjQ1IC0wMDAwCTEu MTANCisrKyBubnBmcy9ic2Qvbm5wZnNfYmxvY2tzLmMJMTYgRmViIDIwMDgg MDM6NDk6NTQgLTAwMDANCkBAIC0zMDYsOCArMzA2LDEzIEBADQogCQkgICAg ICBMRUFTRV9XUklURSk7DQogCSAgICANCiAJICAgIHJldCA9IG5ucGZzX2Js b2NrX2V4dGVuZF9pbnQobm9kZSwgdnAsIHApOw0KLQkgICAgDQorDQorI2lm ZGVmIEhBVkVfVFdPX0FSR1VNRU5UX1ZPUF9VTkxPQ0sNCisJICAgIFZPUF9V TkxPQ0sodnAsIDApOw0KKyNlbmRpZg0KKyNpZmRlZiBIQVZFX1RIUkVFX0FS R1VNRU5UX1ZPUF9VTkxPQ0sNCiAJICAgIFZPUF9VTkxPQ0sodnAsIDAsIHAp Ow0KKyNlbmRpZg0KIAkgICAgdm5fZmluaXNoZWRfd3JpdGUobXApOw0KIAl9 DQogI2Vsc2UNCkBAIC01MDAsNyArNTA1LDExIEBADQogI2VuZGlmDQogDQog I2lmZGVmIF9fRnJlZUJTRF9fDQorI2lmZGVmIEhBVkVfRklOQUxfQVJHX0ZJ TEVfVk9QX09QRU4NCisJZXJyb3IgPSBWT1BfT1BFTigqdnBwLCBmbW9kZSwg Y3JlZCwgcCwgTlVMTCk7DQorI2Vsc2UNCiAJZXJyb3IgPSBWT1BfT1BFTigq dnBwLCBmbW9kZSwgY3JlZCwgcCwgLTEpOw0KKyNlbmRpZg0KICNlbHNlDQog CWVycm9yID0gVk9QX09QRU4oKnZwcCwgZm1vZGUsIGNyZWQsIHApOw0KICNl bmRpZg0KSW5kZXg6IG5ucGZzL2JzZC9ubnBmc19jb21tb24tYnNkLmMNCj09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvc3RhY2tlbi1jdnMv YXJsYS9ubnBmcy9ic2Qvbm5wZnNfY29tbW9uLWJzZC5jLHYNCnJldHJpZXZp bmcgcmV2aXNpb24gMS4zMw0KZGlmZiAtdSAtcjEuMzMgbm5wZnNfY29tbW9u LWJzZC5jDQotLS0gbm5wZnMvYnNkL25ucGZzX2NvbW1vbi1ic2QuYwkyNCBP Y3QgMjAwNiAxNjozMzowMiAtMDAwMAkxLjMzDQorKysgbm5wZnMvYnNkL25u cGZzX2NvbW1vbi1ic2QuYwkxNiBGZWIgMjAwOCAwMzo0OTo1NCAtMDAwMA0K QEAgLTcyLDEwICs3MiwzMyBAQA0KICNlbmRpZiAvKiBOTlBGU19ERUJVRyAq Lw0KIA0KIGludA0KLW5ucGZzX3N1c2VyKGRfdGhyZWFkX3QgKnApDQorbm5w ZnNfcHJpdl9jaGVja19kZWJ1ZyhkX3RocmVhZF90ICpwKQ0KIHsNCiAjaWYg ZGVmaW5lZChfX0FQUExFX18pDQogICAgIHJldHVybiBwcm9jX3N1c2VyKHAp Ow0KKyNlbGlmIGRlZmluZWQoSEFWRV9LRVJORUxfUFJJVl9DSEVDSykNCisg ICAgcmV0dXJuIChwcml2X2NoZWNrKHAsIFBSSVZfTk5QRlNfREVCVUcpKTsN CisjZWxpZiBkZWZpbmVkKEhBVkVfS0VSTkVMX0tBVVRIX0NSRURfR0VUVUlE KQ0KKyAgICB1aWRfdCB1aWQgPSBrYXV0aF9jcmVkX2dldHVpZChwLT5sX3By b2MtPnBfY3JlZCk7DQorICAgIGlmICh1aWQgPT0gMCkNCisJcmV0dXJuIDA7 DQorICAgIHJldHVybiBFUEVSTTsNCisjZWxpZiBkZWZpbmVkKEhBVkVfS0VS TkVMX1NVU0VSX1VDUkVEKQ0KKyAgICByZXR1cm4gc3VzZXJfdWNyZWQgKG5u cGZzX3Byb2NfdG9fY3JlZChwKSk7DQorI2VsaWYgZGVmaW5lZChIQVZFX1RX T19BUkdVTUVOVF9TVVNFUikNCisgICAgcmV0dXJuIHN1c2VyIChubnBmc19w cm9jX3RvX2NyZWQocCksIE5VTEwpOw0KKyNlbHNlDQorICAgIHJldHVybiBz dXNlciAocCk7DQorI2VuZGlmDQorfQ0KKw0KK2ludA0KK25ucGZzX3ByaXZf Y2hlY2tfZmhsb29rdXAoZF90aHJlYWRfdCAqcCkNCit7DQorI2lmIGRlZmlu ZWQoX19BUFBMRV9fKQ0KKyAgICByZXR1cm4gcHJvY19zdXNlcihwKTsNCisj ZWxpZiBkZWZpbmVkKEhBVkVfS0VSTkVMX1BSSVZfQ0hFQ0spDQorICAgIHJl dHVybiAocHJpdl9jaGVjayhwLCBQUklWX1ZGU19HRVRGSCkpOw0KICNlbGlm IGRlZmluZWQoSEFWRV9LRVJORUxfS0FVVEhfQ1JFRF9HRVRVSUQpDQogICAg IHVpZF90IHVpZCA9IGthdXRoX2NyZWRfZ2V0dWlkKHAtPmxfcHJvYy0+cF9j cmVkKTsNCiAgICAgaWYgKHVpZCA9PSAwKQ0KSW5kZXg6IG5ucGZzL2JzZC9u bnBmc19tZXNzYWdlLmMNCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0NClJDUyBm aWxlOiAvc3RhY2tlbi1jdnMvYXJsYS9ubnBmcy9ic2Qvbm5wZnNfbWVzc2Fn ZS5jLHYNCnJldHJpZXZpbmcgcmV2aXNpb24gMS4xMDkNCmRpZmYgLXUgLXIx LjEwOSBubnBmc19tZXNzYWdlLmMNCi0tLSBubnBmcy9ic2Qvbm5wZnNfbWVz c2FnZS5jCTI4IE1hciAyMDA3IDEyOjA1OjQ2IC0wMDAwCTEuMTA5DQorKysg bm5wZnMvYnNkL25ucGZzX21lc3NhZ2UuYwkxNiBGZWIgMjAwOCAwMzo0OTo1 NSAtMDAwMA0KQEAgLTQ4Myw3ICs0ODMsNyBAQA0KIAkNCiAJbm5wZnNfYmxv Y2tfZnJlZV9hbGwoVk5PREVfVE9fWE5PREUodnApKTsNCiAJDQotI2lmIEhB VkVfS0VSTkVMX1ZHT05FTA0KKyNpZmRlZiBIQVZFX0tFUk5FTF9WR09ORUwN CiAJdmdvbmVsICh2cCwgcCk7DQogI2Vsc2UgLyogIWhhdmUgdmdvbmVsICov DQogCW5ucGZzX2ludGVybG9ja191bmxvY2soJnZwLT52X2ludGVybG9jayk7 DQpJbmRleDogbm5wZnMvYnNkL25ucGZzX25vZGUtYnNkLmMNCj09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvc3RhY2tlbi1jdnMvYXJsYS9u bnBmcy9ic2Qvbm5wZnNfbm9kZS1ic2QuYyx2DQpyZXRyaWV2aW5nIHJldmlz aW9uIDEuOTkNCmRpZmYgLXUgLXIxLjk5IG5ucGZzX25vZGUtYnNkLmMNCi0t LSBubnBmcy9ic2Qvbm5wZnNfbm9kZS1ic2QuYwkxNCBNYXIgMjAwNyAxNjo0 NDozMCAtMDAwMAkxLjk5DQorKysgbm5wZnMvYnNkL25ucGZzX25vZGUtYnNk LmMJMTYgRmViIDIwMDggMDM6NDk6NTUgLTAwMDANCkBAIC0xNjksNiArMTY5 LDE1IEBADQogICAgIGdlbmZzX25vZGVfaW5pdCgqdnBwLCAmbm5wZnNfZ2Vu ZnNvcHMpOw0KICNlbmRpZg0KIA0KKyNpZmRlZiBIQVZFX0tFUk5FTF9JTlNN TlRRVUUNCisgICAgZXJyb3IgPSBpbnNtbnRxdWUoKnZwcCwgTk5QRlNfVE9f VkZTKG5ucGZzcCkpOw0KKyAgICBpZiAoZXJyb3IpIHsNCisgICAgICBubnBm c19mcmVlKHJlc3VsdCwgc2l6ZW9mKCpyZXN1bHQpLCBNX05OUEZTX05PREUp Ow0KKyAgICAgICp2cHAgPSBOVUxMOw0KKyAgICAgIHJldHVybiBlcnJvcjsN CisgICAgfQ0KKyNlbmRpZg0KKw0KICByZXRyeToNCiAgICAgZXJyb3IgPSBu bnBmc19ub2RlX2ZpbmQobm5wZnNwLCBoYW5kbGUsICZjaGVjayk7DQogICAg IGlmIChlcnJvciA9PSBFTk9FTlQpIHsNCkBAIC03NTYsNyArNzY1LDcgQEAN CiAgKiBUaGUgcmVhbCBjaGFuZ2UgaXMgc3lzL2tlcm4vdmZzX2NhY2hlOjEu MjANCiAgKi8NCiANCi0jaWYgX19OZXRCU0RfVmVyc2lvbl9fID49IDEwNDEy MDAwMCB8fCBPcGVuQlNEID4gMjAwMjExDQorI2lmIChkZWZpbmVkKF9fTmV0 QlNEX1ZlcnNpb25fXykgJiYgX19OZXRCU0RfVmVyc2lvbl9fID49IDEwNDEy MDAwMCkgfHwgKGRlZmluZWQoT3BlbkJTRCkgJiYgT3BlbkJTRCA+IDIwMDIx MSkNCiAJaWYgKGNhY2hlX2xvb2t1cChkdnAsICZkdW1teSwgY25wKSAhPSAt MSkgew0KIAkgICAgbm5wZnNfdmZzX3VubG9jayhkdW1teSwgbm5wZnNfY25w X3RvX3Byb2MoY25wKSk7DQogCSAgICBwcmludGYgKCJOTlBGUyBQQU5JQyBX QVJOSU5HISBubnBmc19kbmxjX2VudGVyOiAlcyBhbHJlYWR5IGluIGNhY2hl XG4iLA0KQEAgLTgwMSw3ICs4MTAsNyBAQA0KICAgICBjbi0+Y25fbmFtZXB0 ciA9IChjaGFyICopbmFtZTsNCiAgICAgY24tPmNuX25hbWVsZW4gPSBzdHJs ZW4obmFtZSk7DQogICAgIGNuLT5jbl9mbGFncyAgID0gMDsNCi0jaWYgX19B UFBMRV9fDQorI2lmZGVmIF9fQVBQTEVfXw0KICAgICBjbi0+Y25faGFzaCA9 IDA7IC8qIExldCB0aGUgdmZzIGNvbXB1dGUgdGhlIGhhc2ggKi8NCiAjZWxp ZiBkZWZpbmVkKEhBVkVfS0VSTkVMX05BTUVJX0hBU0gpDQogICAgIHsNCklu ZGV4OiBubnBmcy9ic2Qvbm5wZnNfc3lzY2FsbHMtY29tbW9uLmMNCj09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvc3RhY2tlbi1jdnMvYXJs YS9ubnBmcy9ic2Qvbm5wZnNfc3lzY2FsbHMtY29tbW9uLmMsdg0KcmV0cmll dmluZyByZXZpc2lvbiAxLjg1DQpkaWZmIC11IC1yMS44NSBubnBmc19zeXNj YWxscy1jb21tb24uYw0KLS0tIG5ucGZzL2JzZC9ubnBmc19zeXNjYWxscy1j b21tb24uYwkyOCBNYXIgMjAwNyAxMjowNTo0NiAtMDAwMAkxLjg1DQorKysg bm5wZnMvYnNkL25ucGZzX3N5c2NhbGxzLWNvbW1vbi5jCTE2IEZlYiAyMDA4 IDAzOjQ5OjU1IC0wMDAwDQpAQCAtNDg0LDcgKzQ4NCw3IEBADQogCWlmICh2 aWNlX2lvY3RsLT5pbl9zaXplIDwgc2l6ZW9mKGludDMyX3QpKQ0KIAkgICAg cmV0dXJuIEVJTlZBTDsNCiAJDQotCWVycm9yID0gbm5wZnNfc3VzZXIocCk7 DQorCWVycm9yID0gbm5wZnNfcHJpdl9jaGVja19kZWJ1ZyhwKTsNCiAJaWYg KGVycm9yKQ0KIAkgICAgcmV0dXJuIGVycm9yOw0KIA0KSW5kZXg6IG5ucGZz L2JzZC9ubnBmc192ZnNvcHMtYnNkLmMNCj09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT0NClJDUyBmaWxlOiAvc3RhY2tlbi1jdnMvYXJsYS9ubnBmcy9ic2Qvbm5w ZnNfdmZzb3BzLWJzZC5jLHYNCnJldHJpZXZpbmcgcmV2aXNpb24gMS4xMDMN CmRpZmYgLXUgLXIxLjEwMyBubnBmc192ZnNvcHMtYnNkLmMNCi0tLSBubnBm cy9ic2Qvbm5wZnNfdmZzb3BzLWJzZC5jCTI3IE1hciAyMDA3IDEzOjI2OjU4 IC0wMDAwCTEuMTAzDQorKysgbm5wZnMvYnNkL25ucGZzX3Zmc29wcy1ic2Qu YwkxNiBGZWIgMjAwOCAwMzo0OTo1NiAtMDAwMA0KQEAgLTg0LDcgKzg0LDEx IEBADQogfQ0KIA0KIGludA0KKyNpZmRlZiBIQVZFX1ZGU19RVU9UQUNUTF9D QUREUg0KIG5ucGZzX3F1b3RhY3RsKHN0cnVjdCBtb3VudCAqbXAsIGludCBj bWQsIHVpZF90IHVpZCwgY2FkZHJfdCBhcmcsIGRfdGhyZWFkX3QgKnApDQor I2Vsc2UNCitubnBmc19xdW90YWN0bChzdHJ1Y3QgbW91bnQgKm1wLCBpbnQg Y21kLCB1aWRfdCB1aWQsIHZvaWQgKmFyZywgZF90aHJlYWRfdCAqcCkNCisj ZW5kaWYNCiB7DQogICAgIE5OUEZTREVCKFhERUJWRk9QUywgKCJubnBmc19x dW90YWN0bDogbXAgPSAlbHgsIGNtZCA9ICVkLCB1aWQgPSAldSwgIg0KIAkJ ICAgICAgICJhcmcgPSAlbHgsIHByb2MgPSAlbHhcbiIsIA0KQEAgLTIyNSw2 ICsyMjksNyBAQA0KICAgICByZXR1cm4gMDsNCiB9DQogDQorI2lmbmRlZiBI QVZFX1ZPUF9WUFRPRkgNCiBpbnQNCiBubnBmc192cHRvZmgoc3RydWN0IHZu b2RlICogdnAsDQogCSAgICAgc3RydWN0IGZpZCAqIGZocA0KQEAgLTIzNiw2 ICsyNDEsNyBAQA0KICAgICBOTlBGU0RFQihYREVCVkZPUFMsICgibm5wZnNf dnB0b2ZoXG4iKSk7DQogICAgIHJldHVybiBFT1BOT1RTVVBQOw0KIH0NCisj ZW5kaWYNCiANCiAjZW5kaWYgLyogIV9fQVBQTEVfXyAqLw0KIA0KQEAgLTMy MSw3ICszMjcsNyBAQA0KIA0KICAgICBOTlBGU0RFQihYREVCVkZPUFMsICgi bm5wZnNfZmhsb29rdXAgKG5ucGZzKVxuIikpOw0KIA0KLSAgICBlcnJvciA9 IG5ucGZzX3N1c2VyIChwcm9jKTsNCisgICAgZXJyb3IgPSBubnBmc19wcml2 X2NoZWNrX2ZobG9va3VwKHByb2MpOw0KICAgICBpZiAoZXJyb3IpDQogCXJl dHVybiBFUEVSTTsNCiANCkBAIC00ODQsNyArNDkwLDExIEBADQogCWdvdG8g b3V0Ow0KIA0KICNpZiBkZWZpbmVkKF9fRnJlZUJTRF9fKSAmJiBfX0ZyZWVC U0RfdmVyc2lvbiA+PSA1MDIwMDANCisjaWZkZWYgSEFWRV9GSU5BTF9BUkdf RklMRV9WT1BfT1BFTg0KKyAgICBlcnJvciA9IFZPUF9PUEVOKHZwLCBmbGFn cywgY3JlZCwgcHJvYywgZnApOw0KKyNlbHNlDQogICAgIGVycm9yID0gVk9Q X09QRU4odnAsIGZsYWdzLCBjcmVkLCBwcm9jLCBpbmRleCk7DQorI2VuZGlm DQogI2Vsc2UNCiAgICAgZXJyb3IgPSBWT1BfT1BFTih2cCwgZmxhZ3MsIGNy ZWQsIHByb2MpOw0KICNlbmRpZg0KSW5kZXg6IG5ucGZzL2JzZC9ubnBmc192 ZnNvcHMtZnJlZWJzZC5jDQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09DQpSQ1Mg ZmlsZTogL3N0YWNrZW4tY3ZzL2FybGEvbm5wZnMvYnNkL25ucGZzX3Zmc29w cy1mcmVlYnNkLmMsdg0KcmV0cmlldmluZyByZXZpc2lvbiAxLjM3DQpkaWZm IC11IC1yMS4zNyBubnBmc192ZnNvcHMtZnJlZWJzZC5jDQotLS0gbm5wZnMv YnNkL25ucGZzX3Zmc29wcy1mcmVlYnNkLmMJNiBNYXIgMjAwNyAxNjowMDo1 NyAtMDAwMAkxLjM3DQorKysgbm5wZnMvYnNkL25ucGZzX3Zmc29wcy1mcmVl YnNkLmMJMTYgRmViIDIwMDggMDM6NDk6NTYgLTAwMDANCkBAIC02MCw3ICs2 MCwxMiBAQA0KICAgICAudm9wX2RlZmF1bHQgPSAmZGVmYXVsdF92bm9kZW9w cywNCiAgICAgLnZvcF9sb29rdXAgPSBubnBmc19kZWFkX2xvb2t1cCwNCiAg ICAgLnZvcF9yZWNsYWltID0gbm5wZnNfZGVhZF9yZWNsYWltLA0KKyNpZmRl ZiBIQVZFX1ZPUF9MT0NLMQ0KKyAgICAudm9wX2xvY2sxID0gdm9wX3N0ZGxv Y2ssDQorI2VuZGlmDQorI2lmZGVmIEhBVkVfVk9QX0xPQ0sNCiAgICAgLnZv cF9sb2NrID0gdm9wX3N0ZGxvY2ssDQorI2VuZGlmDQogICAgIC52b3BfdW5s b2NrID0gdm9wX3N0ZHVubG9jaywNCiAgICAgLnZvcF9pc2xvY2tlZCA9IHZv cF9zdGRpc2xvY2tlZCwNCiB9Ow0KQEAgLTc2LDYgKzgxLDE1IEBADQogICAg IGlmIChlcnJvciA9PSAwKQ0KIAlOTlBGU19NQUtFX1ZST09UKCp2cHApOw0K IA0KKyNpZmRlZiBIQVZFX0tFUk5FTF9JTlNNTlRRVUUNCisgICAgLyogWFhY OiBQb3NzaWJseSBzaG91bGQgbG9jayB3aXRoIGxvY2ttZ3IgaGVyZS4gKi8N CisgICAgZXJyb3IgPSBpbnNtbnRxdWUoKnZwcCwgbXApOw0KKyAgICBpZiAo ZXJyb3IpIHsNCisgICAgICAqdnBwID0gTlVMTDsNCisgICAgICByZXR1cm4g ZXJyb3I7DQorICAgIH0NCisjZW5kaWYNCisNCiAgICAgbm5wZnNfdmZzX3dy aXRlbG9jaygqdnBwLCBubnBmc19jdXJwcm9jKCkpOw0KIA0KICAgICByZXR1 cm4gZXJyb3I7DQpAQCAtMTYzLDcgKzE3Nyw5IEBADQogICAgIC52ZnNfdmdl dCA9IG5ucGZzX3ZnZXRfZnJlZWJzZCwNCiAgICAgLnZmc19maHRvdnAgPSBu bnBmc19maHRvdnAsDQogICAgIC52ZnNfY2hlY2tleHAgPSBubnBmc19jaGVj a2V4cCwNCisjaWZuZGVmIEhBVkVfVk9QX1ZQVE9GSA0KICAgICAudmZzX3Zw dG9maCA9IG5ucGZzX3ZwdG9maCwNCisjZW5kaWYNCiAgICAgLnZmc19pbml0 ID0gbm5wZnNfaW5pdA0KIH07DQogLypWRlNfU0VUKG5ucGZzX3Zmc29wcywg YXJsYW5ucGZzZGV2LCAwKTsqLw0KSW5kZXg6IG5ucGZzL2JzZC9ubnBmc192 bm9kZW9wcy1ic2QuYw0KPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQ0KUkNTIGZp bGU6IC9zdGFja2VuLWN2cy9hcmxhL25ucGZzL2JzZC9ubnBmc192bm9kZW9w cy1ic2QuYyx2DQpyZXRyaWV2aW5nIHJldmlzaW9uIDEuMTYxDQpkaWZmIC11 IC1yMS4xNjEgbm5wZnNfdm5vZGVvcHMtYnNkLmMNCi0tLSBubnBmcy9ic2Qv bm5wZnNfdm5vZGVvcHMtYnNkLmMJNiBNYXIgMjAwNyAxNjowMDo1NyAtMDAw MAkxLjE2MQ0KKysrIG5ucGZzL2JzZC9ubnBmc192bm9kZW9wcy1ic2QuYwkx NiBGZWIgMjAwOCAwMzo0OTo1NyAtMDAwMA0KQEAgLTg1LDcgKzg1LDExIEBA DQogICAgIHJldCA9IG5ucGZzX29wZW5fY29tbW9uKGFwLT5hX3ZwLCBhcC0+ YV9tb2RlLCBjdHgpOw0KICNpZmRlZiBfX0ZyZWVCU0RfXw0KICAgICBpZiAo IXJldCkNCisjaWZkZWYgSEFWRV9USFJFRV9BUkdVTUVOVF9WTk9ERV9DUkVB VEVfVk9CSg0KKwl2bm9kZV9jcmVhdGVfdm9iamVjdChhcC0+YV92cCwNCisj ZWxzZQ0KIAl2bm9kZV9jcmVhdGVfdm9iamVjdF9vZmYoYXAtPmFfdnAsDQor I2VuZGlmDQogCQkJCSBubnBmc192YXR0cl9nZXRfc2l6ZSgmVk5PREVfVE9f WE5PREUoYXAtPmFfdnApLT5hdHRyKSwNCiAJCQkJIGFwLT5hX3RkKTsNCiAj ZW5kaWYNCkBAIC03OTMsMTYgKzc5NywyNSBAQA0KICNpZmRlZiBfX0ZyZWVC U0RfXw0KIA0KIGludA0KKyNpZmRlZiBIQVZFX1ZPUF9MT0NLMQ0KK25ucGZz X2xvY2sxKHN0cnVjdCB2b3BfbG9jazFfYXJncyAqIGFwKQ0KKyNlbHNlDQog bm5wZnNfbG9jayhzdHJ1Y3Qgdm9wX2xvY2tfYXJncyAqIGFwKQ0KKyNlbmRp Zg0KIHsgICAgICAgICAgICAgICANCiAgICAgc3RydWN0IHZub2RlICp2cCA9 IGFwLT5hX3ZwOw0KICAgICBpbnQgcmV0Ow0KIA0KICAgICBubnBmc19hc3Nl cnQodnApOw0KKyNpZmRlZiBIQVZFX1ZPUF9MT0NLMQ0KKyAgICBOTlBGU0RF QihYREVCVk5PUFMsICgibm5wZnNfbG9jazE6ICVseCwgZmxhZ3MgMHgleFxu IiwNCisJCQkgKHVuc2lnbmVkIGxvbmcpdnAsIGFwLT5hX2ZsYWdzKSk7DQor I2Vsc2UNCiAgICAgbm5wZnNfYXNzZXJ0KGFwLT5hX3RkKTsNCiAgICAgTk5Q RlNERUIoWERFQlZOT1BTLCAoIm5ucGZzX2xvY2s6ICVseCwgdGQgJXAsIGZs YWdzIDB4JXgsIG5sb2NrcyAlZFxuIiwNCiAJCQkgKHVuc2lnbmVkIGxvbmcp dnAsIE5OUEZTX0FQX1BST0MoYXApLCBhcC0+YV9mbGFncywNCiAJCQkgTk5Q RlNfQVBfUFJPQyhhcCktPnRkX2xvY2tzKSk7DQorI2VuZGlmDQogDQogICAg IG5ucGZzX2xrX2luZm8oIm5ucGZzX2xvY2sgYmVmb3JlIiwgdnApOw0KICAg ICByZXQgPSB2b3Bfc3RkbG9jayhhcCk7DQpAQCAtODE3LDE2ICs4MzAsMjYg QEANCiAgICAgc3RydWN0IHZub2RlICp2cCA9IGFwLT5hX3ZwOw0KICAgICBp bnQgcmV0Ow0KIA0KKyNpZmRlZiBIQVZFX1RXT19BUkdVTUVOVF9WT1BfVU5M T0NLDQorICAgIE5OUEZTREVCKFhERUJWTk9QUywNCisJICAgICAoIm5ucGZz X3VubG9jazogJWx4LCBmbGFncyAweCV4XG4iLCAodW5zaWduZWQgbG9uZyl2 cCwNCisJICAgICAgYXAtPmFfZmxhZ3MpKTsNCisjZWxzZQ0KICAgICBOTlBG U0RFQihYREVCVk5PUFMsDQogCSAgICAgKCJubnBmc191bmxvY2s6ICVseCwg dGQgJXAsIGZsYWdzIDB4JXgsIG5sb2NrcyAlZFxuIiwNCiAJICAgICAgKHVu c2lnbmVkIGxvbmcpdnAsIGFwLT5hX3RkLCBhcC0+YV9mbGFncywNCiAJICAg ICAgTk5QRlNfQVBfUFJPQyhhcCktPnRkX2xvY2tzKSk7DQorI2VuZGlmDQog ICAgIA0KICAgICBubnBmc19sa19pbmZvKCJubnBmc191bmxvY2sgYmVmb3Jl IiwgdnApOw0KICAgICByZXQgPSB2b3Bfc3RkdW5sb2NrKGFwKTsNCiANCisj aWZkZWYgSEFWRV9UV09fQVJHVU1FTlRfVk9QX1VOTE9DSw0KKyAgICBOTlBG U0RFQihYREVCVk5PUFMsICgibm5wZnNfdW5sb2NrOiByZXR1cm4gJWRcbiIs IHJldCkpOw0KKyNlbHNlDQogICAgIE5OUEZTREVCKFhERUJWTk9QUywgKCJu bnBmc191bmxvY2s6IHJldHVybiAlZCwgdGQgJXAsIG5sb2NrcyAlZFxuIiwN CiAJCQkgcmV0LCBOTlBGU19BUF9QUk9DKGFwKSwgTk5QRlNfQVBfUFJPQyhh cCktPnRkX2xvY2tzKSk7DQorI2VuZGlmDQogICAgIHJldHVybiByZXQ7DQog fQ0KIA0KQEAgLTEzMTMsNiArMTMzNiwyMyBAQA0KIH0NCiAjZW5kaWYNCiAN CisjaWZkZWYgSEFWRV9WT1BfVlBUT0ZIDQoraW50DQorbm5wZnNfdnB0b2Zo KHN0cnVjdCB2b3BfdnB0b2ZoX2FyZ3MgKmFwKQ0KKy8qDQorc3RydWN0IHZv cF92cHRvZmhfYXJncyB7DQorCXN0cnVjdCB2bm9kZW9wX2Rlc2MgKmFfZGVz YzsNCisJc3RydWN0IHZub2RlICphX3ZwOw0KKwlzdHJ1Y3QgZmlkICphX2Zo cDsNCit9Ow0KKyovDQorew0KKyAgICBOTlBGU0RFQihYREVCVk5PUFMsICgi bm5wZnNfdnB0b2ZoXG4iKSk7DQorDQorICAgIHJldHVybiBFT1BOT1RTVVBQ Ow0KK30NCisjZW5kaWYNCisNCiANCiANCiB2b3BfdCAqKm5ucGZzX3Zub2Rl b3BfcDsNCkBAIC0xNDY2LDcgKzE1MDYsMTIgQEANCiAJLnZvcF9ibWFwID0J CW5ucGZzX2JtYXAsDQogDQogCS52b3BfcG9sbCA9CQlubnBmc19wb2xsLA0K KyNpZmRlZiBIQVZFX1ZPUF9MT0NLMQ0KKwkudm9wX2xvY2sxID0JCW5ucGZz X2xvY2sxLA0KKyNlbmRpZg0KKyNpZmRlZiBIQVZFX1ZPUF9MT0NLDQogCS52 b3BfbG9jayA9CQlubnBmc19sb2NrLCAgDQorI2VuZGlmDQogCS52b3BfdW5s b2NrID0JCW5ucGZzX3VubG9jaywNCiAJLnZvcF9pc2xvY2tlZCA9CQlubnBm c19pc2xvY2tlZCwNCiAJLnZvcF9yZXZva2UgPQkJbm5wZnNfcmV2b2tlLA0K QEAgLTE2MDUsNiArMTY1MCw5IEBADQogI2VuZGlmDQogI2lmZGVmIEhBVkVf Vk9QX1BBVEhDT05GDQogICAgIHsmdm9wX3BhdGhjb25mX2Rlc2MsICh2b3Bf dCAqKSBubnBmc19wYXRoY29uZiB9LA0KKyNlbmRpZg0KKyNpZmRlZiBIQVZF X1ZPUF9WUFRPRkgNCisgICAgeyZ2b3BfdnB0b2ZoX2Rlc2MsICh2b3BfdCAq KSBubnBmc192cHRvZmggfSwNCiAjZW5kaWYNCiAgICAgeyhzdHJ1Y3Qgdm5v ZGVvcF9kZXNjICopIE5VTEwsIChpbnQgKCopICh2b2lkICopKSBOVUxMfQ0K IH07DQpJbmRleDogbm5wZnMvYnNkL25ucGZzX3Zub2Rlb3BzLWNvbW1vbi5j DQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09DQpSQ1MgZmlsZTogL3N0YWNrZW4t Y3ZzL2FybGEvbm5wZnMvYnNkL25ucGZzX3Zub2Rlb3BzLWNvbW1vbi5jLHYN CnJldHJpZXZpbmcgcmV2aXNpb24gMS4xMjENCmRpZmYgLXUgLXIxLjEyMSBu bnBmc192bm9kZW9wcy1jb21tb24uYw0KLS0tIG5ucGZzL2JzZC9ubnBmc192 bm9kZW9wcy1jb21tb24uYwkyOCBNYXIgMjAwNyAxMjowNTo0NiAtMDAwMAkx LjEyMQ0KKysrIG5ucGZzL2JzZC9ubnBmc192bm9kZW9wcy1jb21tb24uYwkx NiBGZWIgMjAwOCAwMzo0OTo1NyAtMDAwMA0KQEAgLTUwLDcgKzUwLDcgQEAN CiBzdGF0aWMgdm9pZA0KIG5ucGZzX2hhbmRsZV9zdGFsZShzdHJ1Y3Qgbm5w ZnNfbm9kZSAqeG4pDQogew0KLSNpZiBfX0FQUExFX18NCisjaWZkZWYgX19B UFBMRV9fDQogICAgIHN0cnVjdCB2bm9kZSAqdnAgPSBYTk9ERV9UT19WTk9E RSh4bik7DQogI2VuZGlmDQogDQpAQCAtMTQzOCw3ICsxNDM4LDkgQEANCiAg ICAgaW50IGVycm9yID0gMDsNCiAgICAgbm5wZnNfY3JlZCBjcmVkOw0KICAg ICBzdHJ1Y3Qgbm5wZnMgKm5ucGZzcCA9IE5OUEZTX0ZST01fVk5PREUodnAp Ow0KKyNpZmRlZiBIQVZFX1RIUkVFX0FSR1VNRU5UX1ZPUF9VTkxPQ0sNCiAg ICAgZF90aHJlYWRfdCAqcHJvYyA9IG5ucGZzX3Zmc19jb250ZXh0X3Byb2Mo Y3R4KTsNCisjZW5kaWYNCiANCiAgICAgTk5QRlNERUIoWERFQlZOT1BTLCAo Im5ucGZzX3JlYWRsaW5rXG4iKSk7DQogDQpJbmRleDogbm5wZnMvYnNkL2Jp bi9tbnRvcHRzLmgNCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0NClJDUyBmaWxl OiAvc3RhY2tlbi1jdnMvYXJsYS9ubnBmcy9ic2QvYmluL21udG9wdHMuaCx2 DQpyZXRyaWV2aW5nIHJldmlzaW9uIDEuNw0KZGlmZiAtdSAtcjEuNyBtbnRv cHRzLmgNCi0tLSBubnBmcy9ic2QvYmluL21udG9wdHMuaAkyOCBEZWMgMjAw NSAxNDozMjowMyAtMDAwMAkxLjcNCisrKyBubnBmcy9ic2QvYmluL21udG9w dHMuaAkxNiBGZWIgMjAwOCAwMzo0OTo1OCAtMDAwMA0KQEAgLTQ4LDcgKzQ4 LDkgQEANCiAjZGVmaW5lIE1PUFRfTk9BQ0NFU1NUSU1FCXsgImFjY2Vzc3Rp bWUiLAkxLCBNTlRfTk9BVElNRSB9DQogI2RlZmluZSBNT1BUX05PQVRJTUUJ CXsgImF0aW1lIiwJMSwgTU5UX05PQVRJTUUgfQ0KICNlbmRpZg0KKyNpZmRl ZiBNT1BUX05PREVWDQogI2RlZmluZSBNT1BUX05PREVWCQl7ICJkZXYiLAkx LCBNTlRfTk9ERVYgfQ0KKyNlbmRpZg0KICNkZWZpbmUgTU9QVF9OT0VYRUMJ CXsgImV4ZWMiLAkxLCBNTlRfTk9FWEVDIH0NCiAjZGVmaW5lIE1PUFRfTk9T VUlECQl7ICJzdWlkIiwJMSwgTU5UX05PU1VJRCB9DQogI2RlZmluZSBNT1BU X1JET05MWQkJeyAicmRvbmx5IiwJMCwgTU5UX1JET05MWSB9DQpAQCAtNzgs NiArODAsNyBAQA0KIA0KIC8qIFN0YW5kYXJkIG9wdGlvbnMgd2hpY2ggYWxs IG1vdW50cyBjYW4gdW5kZXJzdGFuZC4gKi8NCiANCisjaWZkZWYgTU9QVF9O T0RFVg0KICNkZWZpbmUgTU9QVF9TVERPUFRTCQkJCQkJCVwNCiAJTU9QVF9V U0VSUVVPVEEsCQkJCQkJCVwNCiAJTU9QVF9HUk9VUFFVT1RBLAkJCQkJCVwN CkBAIC04Niw1ICs4OSwxNCBAQA0KIAlNT1BUX05PRVhFQywJCQkJCQkJXA0K IAlNT1BUX05PU1VJRCwJCQkJCQkJXA0KIAlNT1BUX1JET05MWQ0KKyNlbHNl DQorI2RlZmluZSBNT1BUX1NURE9QVFMJCQkJCQkJXA0KKwlNT1BUX1VTRVJR VU9UQSwJCQkJCQkJXA0KKwlNT1BUX0dST1VQUVVPVEEsCQkJCQkJXA0KKwlN T1BUX0ZTVEFCX0NPTVBBVCwJCQkJCQlcDQorCU1PUFRfTk9FWEVDLAkJCQkJ CQlcDQorCU1PUFRfTk9TVUlELAkJCQkJCQlcDQorCU1PUFRfUkRPTkxZDQor I2VuZGlmDQogDQogdm9pZCBnZXRtbnRvcHRzIChjb25zdCBjaGFyICosIGNv bnN0IHN0cnVjdCBtbnRvcHQgKiwgaW50ICopOw0KSW5kZXg6IG5ucGZzL2Jz ZC9ubnBmcy9ubnBmc19jb21tb24uaA0KPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PQ0KUkNTIGZpbGU6IC9zdGFja2VuLWN2cy9hcmxhL25ucGZzL2JzZC9ubnBm cy9ubnBmc19jb21tb24uaCx2DQpyZXRyaWV2aW5nIHJldmlzaW9uIDEuMjIN CmRpZmYgLXUgLXIxLjIyIG5ucGZzX2NvbW1vbi5oDQotLS0gbm5wZnMvYnNk L25ucGZzL25ucGZzX2NvbW1vbi5oCTI0IE9jdCAyMDA2IDE2OjMzOjE0IC0w MDAwCTEuMjINCisrKyBubnBmcy9ic2Qvbm5wZnMvbm5wZnNfY29tbW9uLmgJ MTYgRmViIDIwMDggMDM6NDk6NTggLTAwMDANCkBAIC01OSw3ICs1OSw4IEBA DQogI2RlZmluZSBubnBmc19mcmVlKGEsIHNpemUsdCkgZnJlZShhLCB0KQ0K ICNlbmRpZiAvKiBOTlBGU19ERUJVRyAqLw0KIA0KLWludCBubnBmc19zdXNl cihkX3RocmVhZF90ICpwKTsNCitpbnQgbm5wZnNfcHJpdl9jaGVja19kZWJ1 ZyhkX3RocmVhZF90ICpwKTsNCitpbnQgbm5wZnNfcHJpdl9jaGVja19maGxv b2t1cChkX3RocmVhZF90ICpwKTsNCiANCiAjaWZuZGVmIEhBVkVfS0VSTkVM X01FTUNQWQ0KIHZvaWQgKg0KSW5kZXg6IG5ucGZzL2JzZC9ubnBmcy9ubnBm c19sb2NsLmgNCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAv c3RhY2tlbi1jdnMvYXJsYS9ubnBmcy9ic2Qvbm5wZnMvbm5wZnNfbG9jbC5o LHYNCnJldHJpZXZpbmcgcmV2aXNpb24gMS4xMTMNCmRpZmYgLXUgLXIxLjEx MyBubnBmc19sb2NsLmgNCi0tLSBubnBmcy9ic2Qvbm5wZnMvbm5wZnNfbG9j bC5oCTYgTWFyIDIwMDcgMTY6MDE6NTYgLTAwMDAJMS4xMTMNCisrKyBubnBm cy9ic2Qvbm5wZnMvbm5wZnNfbG9jbC5oCTE2IEZlYiAyMDA4IDAzOjQ5OjU4 IC0wMDAwDQpAQCAtMTIxLDYgKzEyMSw5IEBADQogI2lmZGVmIEhBVkVfU1lT X0tBVVRIX0gNCiAjaW5jbHVkZSA8c3lzL2thdXRoLmg+DQogI2VuZGlmDQor I2lmZGVmIEhBVkVfU1lTX1BSSVZfSA0KKyNpbmNsdWRlIDxzeXMvcHJpdi5o Pg0KKyNlbmRpZg0KIA0KICNpZmRlZiBIQVZFX01JU0NGU19HRU5GU19HRU5G U19IDQogI2luY2x1ZGUgPG1pc2Nmcy9nZW5mcy9nZW5mcy5oPg0KQEAgLTI5 Niw3ICsyOTksNyBAQA0KIHN0cnVjdCBnZW5mc19vcHMgbm5wZnNfZ2VuZnNv cHM7DQogI2VuZGlmDQogDQotI2lmIF9fTmV0QlNEX1ZlcnNpb25fXyA+PSAz OTkwMDE5MDAgLyogTmV0QlNEIDMuOTkuMTkgKi8NCisjaWYgZGVmaW5lZChf X05ldEJTRF9fKSAmJiBfX05ldEJTRF9WZXJzaW9uX18gPj0gMzk5MDAxOTAw IC8qIE5ldEJTRCAzLjk5LjE5ICovDQogdHlwZWRlZiBzdHJ1Y3Qga2F1dGhf Y3JlZCAqbm5wZnNfa2VybmVsX2NyZWQ7DQogI2RlZmluZSBubnBmc19jcmVk X2dldF91aWQoY3JlZCkga2F1dGhfY3JlZF9nZXR1aWQoY3JlZCkNCiAjZWxz ZQ0KSW5kZXg6IG5ucGZzL2JzZC9ubnBmcy9ubnBmc192ZnNvcHMtYnNkLmgN Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvc3RhY2tlbi1j dnMvYXJsYS9ubnBmcy9ic2Qvbm5wZnMvbm5wZnNfdmZzb3BzLWJzZC5oLHYN CnJldHJpZXZpbmcgcmV2aXNpb24gMS4yNQ0KZGlmZiAtdSAtcjEuMjUgbm5w ZnNfdmZzb3BzLWJzZC5oDQotLS0gbm5wZnMvYnNkL25ucGZzL25ucGZzX3Zm c29wcy1ic2QuaAk2IE1hciAyMDA3IDE2OjAxOjU2IC0wMDAwCTEuMjUNCisr KyBubnBmcy9ic2Qvbm5wZnMvbm5wZnNfdmZzb3BzLWJzZC5oCTE2IEZlYiAy MDA4IDAzOjQ5OjU4IC0wMDAwDQpAQCAtNzksNyArNzksMTEgQEANCiBubnBm c19yb290KHN0cnVjdCBtb3VudCAqbXAsIHN0cnVjdCB2bm9kZSAqKnZwcCk7 DQogDQogaW50DQorI2lmZGVmIEhBVkVfVkZTX1FVT1RBQ1RMX0NBRERSDQog bm5wZnNfcXVvdGFjdGwoc3RydWN0IG1vdW50ICptcCwgaW50IGNtZCwgdWlk X3QgdWlkLCBjYWRkcl90IGFyZywgZF90aHJlYWRfdCAqcCk7DQorI2Vsc2UN CitubnBmc19xdW90YWN0bChzdHJ1Y3QgbW91bnQgKm1wLCBpbnQgY21kLCB1 aWRfdCB1aWQsIHZvaWQgKmFyZywgZF90aHJlYWRfdCAqcCk7DQorI2VuZGlm DQogDQogaW50DQogbm5wZnNfc3RhdGZzKHN0cnVjdCBtb3VudCAqbXAsIG5u cGZzX3N0YXR2ZnMgKnNicCwgZF90aHJlYWRfdCAqcCk7DQpAQCAtMTE0LDYg KzExOCw3IEBADQogCSAgICAgc3RydWN0IHVjcmVkICoqIGNyZWRhbm9ucCk7 DQogI2VuZGlmDQogDQorI2lmbmRlZiBIQVZFX1ZPUF9WUFRPRkgNCiBpbnQN CiBubnBmc192cHRvZmgoc3RydWN0IHZub2RlICogdnAsDQogCSAgICAgc3Ry dWN0IGZpZCAqIGZocA0KQEAgLTEyMSw2ICsxMjYsNyBAQA0KIAkgICAgICxz aXplX3QgKiBmaWRzeg0KICNlbmRpZg0KIAkgICAgICk7DQorI2VuZGlmDQog DQogaW50DQogbm5wZnNfZGVhZF9sb29rdXAoc3RydWN0IHZvcF9sb29rdXBf YXJncyAqYXApOw0KSW5kZXg6IG5ucGZzL2ZyZWVic2QvRnJlZUJTRC1NYWtl ZmlsZQ0KPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PQ0KUkNTIGZpbGU6IC9zdGFj a2VuLWN2cy9hcmxhL25ucGZzL2ZyZWVic2QvRnJlZUJTRC1NYWtlZmlsZSx2 DQpyZXRyaWV2aW5nIHJldmlzaW9uIDEuNw0KZGlmZiAtdSAtcjEuNyBGcmVl QlNELU1ha2VmaWxlDQotLS0gbm5wZnMvZnJlZWJzZC9GcmVlQlNELU1ha2Vm aWxlCTYgTWFyIDIwMDcgMTI6NTQ6MDAgLTAwMDAJMS43DQorKysgbm5wZnMv ZnJlZWJzZC9GcmVlQlNELU1ha2VmaWxlCTE2IEZlYiAyMDA4IDAzOjQ5OjU4 IC0wMDAwDQpAQCAtMjksNiArMjksNyBAQA0KIGJzZHNyY2Rpcj0JJChzcmNk aXIpLy4uL2JzZA0KIA0KIENGTEFHUz0gLWcgXA0KKwktSS91c3IvaW5jbHVk ZSBcDQogCS1JJHsuQ1VSRElSfSBcDQogCS1JJHsuQ1VSRElSfS8uLi8uLi9p bmNsdWRlIFwNCiAJLUkkey5DVVJESVJ9Ly4uL2luY2x1ZGUgXA0KQEAgLTQ1 LDYgKzQ2LDYgQEANCiAuUEFUSDoJJChic2RzcmNkaXIpDQogDQogbm5wZnNf dm9wZGVmcy5oOiB2bm9kZV9pZi5oDQotCWF3ayAnL15zdHJ1Y3Qgdm9wX1th LXpdKl9hcmdzLyB7IHZvcD1zdWJzdHIoJCQyLDUsbGVuZ3RoKCQkMiktOSk7 IHByaW50ZigiI2RlZmluZSBIQVZFX1ZPUF8lcyAxXG4iLCB0b3VwcGVyKHZv cCkpOyBwcmludGYoIk5OUEZTX1ZPUF9ERUYoJXMpO1xuIiwgdm9wKTsgfScg dm5vZGVfaWYuaCA+ICR7LlRBUkdFVH0NCisJYXdrICcvXnN0cnVjdCB2b3Bf W2EtejAtOV0qX2FyZ3MvIHsgdm9wPXN1YnN0cigkJDIsNSxsZW5ndGgoJCQy KS05KTsgcHJpbnRmKCIjZGVmaW5lIEhBVkVfVk9QXyVzIDFcbiIsIHRvdXBw ZXIodm9wKSk7IHByaW50ZigiTk5QRlNfVk9QX0RFRiglcyk7XG4iLCB2b3Ap OyB9JyB2bm9kZV9pZi5oID4gJHsuVEFSR0VUfQ0KIAktbWtkaXIgbm5wZnMN CiAJdGVzdCAtZCBubnBmcyAmJiAoIHRlc3QgLWYgbm5wZnMvbm5wZnNfdm9w ZGVmcy5oIHx8IGxuIC1zIC4uL25ucGZzX3ZvcGRlZnMuaCBubnBmcy9ubnBm c192b3BkZWZzLmggKQ0K --621616949-433016993-1203135147=:93919 Content-Type: TEXT/PLAIN; CHARSET=us-ascii Content-ID: <20080216105116.M93919@fledge.watson.org> Content-Description: Content-Disposition: INLINE _______________________________________________ freebsd-afs@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-afs To unsubscribe, send any mail to "freebsd-afs-unsubscribe@freebsd.org" --621616949-433016993-1203135147=:93919--