Date: Thu, 17 Apr 2014 23:43:03 +1000 (EST) From: Bruce Evans <brde@optusnet.com.au> To: Dirk-Willem van Gulik <dirkx@webweaving.org> Cc: freebsd-bugs@freebsd.org, freebsd-gnats-submit@freebsd.org Subject: Re: bin/188715: int64 not handled right as arg on badsect(8), possible other issues lurking Message-ID: <20140417222618.O1829@besplex.bde.org> In-Reply-To: <201404170812.s3H8CYxd096189@cgiserv.freebsd.org> References: <201404170812.s3H8CYxd096189@cgiserv.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 17 Apr 2014, Dirk-Willem van Gulik wrote: >> Description: > Was trying to map out some bad blocks prior to temporarily read/empty 4 T= byte volume using =82badsect(8)=92 - and returing it. > > Was expecting to be able to put the sector # into badsect (e.g. 343263142= 4 from below FSCK output). > > This gave me a bit of an odd: > > =09badsect: 3432631424: Result too large badsect's ABI uses mknod() and dev_t so it can only work up to 32 bits. (Before 4.4BSD, dev_t was only 16 bits, so badsect only works up to 16 bits. That is a whole 65536 sectors, or 33MB with 512-blocks. This was almost enough in 1980. However, the limit is on fs-blocks (fragments), not on 512-blocks, so the limit can be expanded a bit. With the current default ffs block size of 32K, the fragment size is 4K, so the limit in 1984 would have been 256MB. However, the default fragment size was 512 until about 1995.) It also has bugs in its error checking, so it only works up to 31 bits on arches with 32-bit longs. The above is one of them. 3432631424 doesn't fit in 31 bits, but it does fit in 32 bits. The bugs in the error checking are more serious than I thought, since the limit is on 512-blocks. So on arches with 32-bit longs, the limit on the file system size is 2**31 * 2**9 =3D 1TB. With 4K-frags it should be 2**32 * 2**12 =3D 16TB, so badsect should actually work on your 4TB file system. It shouldn't be expected to work. 4TB disks are supposed to be replaced if they have a single uncorrectable error. There are related limits on file system size from the file system itself. ffs1 uses 32-bit signed block (fragment) numbers internally, so it shouldn'= t be affected by the 332-bit badsect ABI limit. > As the daddr_t seems to be a 64bit unsigned; I assumed that the: daddr_t is 64 bits signed. > > =09=09=09number =3D strtol(*argv, NULL, 0); > > was some legacy culprint - and changed it to a strtoll as the daddr_t you= are entering is an int 64. daddr_t used to be 32-bit signed, and badsect hasn't been maintained since before daddr_t became 64 bits. Even 1GB disks are supposed to be replaced if they have a single uncorrectable error. > =09=09=09number =3D strtoll(*argv, NULL, 0); Ugh. Long long should never be used. Use intmax_t. badsect's buggy error checking involves many type errors. It uses the system type daddr_t for 'number'. This is correct. But it also uses long, and in the above, long long, and in the clean version, intmax_t, for initializing 'number'. I neglected to fix this when I updated badsect from 16 bits to 32 bits in 1995. The update was incomplete. The result of strtoimax() should be assigned to a variable of type intmax_t. It shouldn't be assigned to 'number' before checking that it fits. The type errors continue with badsect abusing daddr_t for the disk block number. That was as correct as possible in 1994, and also in my fixes, because ffs also used this wrong type internally. ffs now uses ufs_daddr1_= t and ufs_daddr_t internally. daddr_t would work in badsect since it is 64 bits signed and only 32 bits unsigned is needed, but it is logically wrong. > That gets it past that point; only to segv out on: > > cg =3D dtog(fs, fsbn); > > =09/usr/include/ufs/ffs/fs.h:#define=09dtog(fs, d)=09((d) / (fs)->fs_fpg) > =09/usr/include/ufs/ffs/fs.h:#define=09dtogd(fs, d)=09((d) % (fs)->fs_fpg= ) > > a bit later. While fs is valid - it seems fs->fs_fpg returns as =820=92= =97 why is this ? Is geom too new ? Or is badsect too old/retired ? I don't see why that doesn't work. fsdb is the fs-block (frag) number, and the types are now large enough although logically wrong. Some of the differences for ffs2 are in macros, but the above macros are too simple to depend on the ffs version. Maybe libufs messes up the initialization of all of 'fs'. > aacd1: hard error cmd=3Dread 4246326690-4246326721 > . > > fsck(8):... > THE FOLLOWING DISK SECTORS COULD NOT BE READ: 3432631424, 3432631425, 343= 2631426, 3432631427, 3432631428, 3432631429, 3432631430, 3432631431, 343263= 1432, 3432631433, 3432631434, 3432631435, 3432631436, 3432631437, 343263143= 8, 3432631439, 3432631440, 3432631441, 3432631442, 3432631443, 3432631444, = 3432631445, 3432631446, 3432631447, 3432631448, 3432631449, 3432631450, 343= 2631451, 3432631452, 3432631453, 3432631454, 3432631455, Perhaps too many for badsect. I haven't used badsect recently, but spent a lot of time rearranging whole partitions on a laptop drive to avoid a 20GB area with a few bad sectors. The rest of the disk worked suprisingly well. Bruce From owner-freebsd-bugs@FreeBSD.ORG Thu Apr 17 14:10:01 2014 Return-Path: <owner-freebsd-bugs@FreeBSD.ORG> Delivered-To: freebsd-bugs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AB499C23 for <freebsd-bugs@smarthost.ysv.freebsd.org>; Thu, 17 Apr 2014 14:10:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8972511DE for <freebsd-bugs@smarthost.ysv.freebsd.org>; Thu, 17 Apr 2014 14:10:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3HEA1MR077948 for <freebsd-bugs@freefall.freebsd.org>; Thu, 17 Apr 2014 14:10:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3HEA1JW077947; Thu, 17 Apr 2014 14:10:01 GMT (envelope-from gnats) Date: Thu, 17 Apr 2014 14:10:01 GMT Message-Id: <201404171410.s3HEA1JW077947@freefall.freebsd.org> To: freebsd-bugs@FreeBSD.org Cc: From: Bruce Evans <brde@optusnet.com.au> Subject: Re: bin/188715: int64 not handled right as arg on badsect(8), possible other issues lurking X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Bruce Evans <brde@optusnet.com.au> List-Id: Bug reports <freebsd-bugs.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-bugs>, <mailto:freebsd-bugs-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-bugs/>; List-Post: <mailto:freebsd-bugs@freebsd.org> List-Help: <mailto:freebsd-bugs-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-bugs>, <mailto:freebsd-bugs-request@freebsd.org?subject=subscribe> X-List-Received-Date: Thu, 17 Apr 2014 14:10:01 -0000 The following reply was made to PR bin/188715; it has been noted by GNATS. From: Bruce Evans <brde@optusnet.com.au> To: Dirk-Willem van Gulik <dirkx@webweaving.org> Cc: freebsd-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org Subject: Re: bin/188715: int64 not handled right as arg on badsect(8), possible other issues lurking Date: Thu, 17 Apr 2014 23:43:03 +1000 (EST) This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-1210473500-1397742183=:1829 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Thu, 17 Apr 2014, Dirk-Willem van Gulik wrote: >> Description: > Was trying to map out some bad blocks prior to temporarily read/empty 4 T= byte volume using =82badsect(8)=92 - and returing it. > > Was expecting to be able to put the sector # into badsect (e.g. 343263142= 4 from below FSCK output). > > This gave me a bit of an odd: > > =09badsect: 3432631424: Result too large badsect's ABI uses mknod() and dev_t so it can only work up to 32 bits. (Before 4.4BSD, dev_t was only 16 bits, so badsect only works up to 16 bits. That is a whole 65536 sectors, or 33MB with 512-blocks. This was almost enough in 1980. However, the limit is on fs-blocks (fragments), not on 512-blocks, so the limit can be expanded a bit. With the current default ffs block size of 32K, the fragment size is 4K, so the limit in 1984 would have been 256MB. However, the default fragment size was 512 until about 1995.) It also has bugs in its error checking, so it only works up to 31 bits on arches with 32-bit longs. The above is one of them. 3432631424 doesn't fit in 31 bits, but it does fit in 32 bits. The bugs in the error checking are more serious than I thought, since the limit is on 512-blocks. So on arches with 32-bit longs, the limit on the file system size is 2**31 * 2**9 =3D 1TB. With 4K-frags it should be 2**32 * 2**12 =3D 16TB, so badsect should actually work on your 4TB file system. It shouldn't be expected to work. 4TB disks are supposed to be replaced if they have a single uncorrectable error. There are related limits on file system size from the file system itself. ffs1 uses 32-bit signed block (fragment) numbers internally, so it shouldn'= t be affected by the 332-bit badsect ABI limit. > As the daddr_t seems to be a 64bit unsigned; I assumed that the: daddr_t is 64 bits signed. > > =09=09=09number =3D strtol(*argv, NULL, 0); > > was some legacy culprint - and changed it to a strtoll as the daddr_t you= are entering is an int 64. daddr_t used to be 32-bit signed, and badsect hasn't been maintained since before daddr_t became 64 bits. Even 1GB disks are supposed to be replaced if they have a single uncorrectable error. > =09=09=09number =3D strtoll(*argv, NULL, 0); Ugh. Long long should never be used. Use intmax_t. badsect's buggy error checking involves many type errors. It uses the system type daddr_t for 'number'. This is correct. But it also uses long, and in the above, long long, and in the clean version, intmax_t, for initializing 'number'. I neglected to fix this when I updated badsect from 16 bits to 32 bits in 1995. The update was incomplete. The result of strtoimax() should be assigned to a variable of type intmax_t. It shouldn't be assigned to 'number' before checking that it fits. The type errors continue with badsect abusing daddr_t for the disk block number. That was as correct as possible in 1994, and also in my fixes, because ffs also used this wrong type internally. ffs now uses ufs_daddr1_= t and ufs_daddr_t internally. daddr_t would work in badsect since it is 64 bits signed and only 32 bits unsigned is needed, but it is logically wrong. > That gets it past that point; only to segv out on: > > cg =3D dtog(fs, fsbn); > > =09/usr/include/ufs/ffs/fs.h:#define=09dtog(fs, d)=09((d) / (fs)->fs_fpg) > =09/usr/include/ufs/ffs/fs.h:#define=09dtogd(fs, d)=09((d) % (fs)->fs_fpg= ) > > a bit later. While fs is valid - it seems fs->fs_fpg returns as =820=92= =97 why is this ? Is geom too new ? Or is badsect too old/retired ? I don't see why that doesn't work. fsdb is the fs-block (frag) number, and the types are now large enough although logically wrong. Some of the differences for ffs2 are in macros, but the above macros are too simple to depend on the ffs version. Maybe libufs messes up the initialization of all of 'fs'. > aacd1: hard error cmd=3Dread 4246326690-4246326721 > . > > fsck(8):... > THE FOLLOWING DISK SECTORS COULD NOT BE READ: 3432631424, 3432631425, 343= 2631426, 3432631427, 3432631428, 3432631429, 3432631430, 3432631431, 343263= 1432, 3432631433, 3432631434, 3432631435, 3432631436, 3432631437, 343263143= 8, 3432631439, 3432631440, 3432631441, 3432631442, 3432631443, 3432631444, = 3432631445, 3432631446, 3432631447, 3432631448, 3432631449, 3432631450, 343= 2631451, 3432631452, 3432631453, 3432631454, 3432631455, Perhaps too many for badsect. I haven't used badsect recently, but spent a lot of time rearranging whole partitions on a laptop drive to avoid a 20GB area with a few bad sectors. The rest of the disk worked suprisingly well. Bruce --0-1210473500-1397742183=:1829--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140417222618.O1829>