Date: Thu, 18 Dec 2014 17:52:10 -0800 From: John-Mark Gurney <jmg@funkthat.com> To: "Pokala, Ravi" <rpokala@panasas.com> Cc: "freebsd-geom@freebsd.org" <freebsd-geom@freebsd.org> Subject: Re: Converting LBAs to byte offsets through the GEOM stack Message-ID: <20141219015210.GY25139@funkthat.com> In-Reply-To: <D0B89F30.127DAE%rpokala@panasas.com> References: <D0B89F30.127DAE%rpokala@panasas.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Ravi Pokala wrote this message on Thu, Dec 18, 2014 at 23:11 +0000:
> When you issue a BIO, the requested byte offset (bio_offset) gets
> transformed by each layer of the GEOM stack as needed. If the bottom of
> the stack is a physical disk, g_disk_start() transforms the final offset
> to a device block address (bio_pblkno), which the disk device driver uses
> as the LBA.
>
> My question is this - is there a way to go in the other direction, from an
> LBA to a byte offset? For example, let's say I have a set of four drives
> which are configured as a RAID10:
>
> STRIPE: /dev/ada0p2 && /dev/ada1p2 => /dev/stripe/gs0
> STRIPE: /dev/ada2p2 && /dev/ada3p2 => /dev/stripe/gs1
> MIRROR: /dev/stripe/gs0 && /dev/stripe/gs1 => /dev/stripe/gm0
>
> I kick off a media scrub of the drive devices, to look for unreadable
> sectors. For the sake of saving bandwidth, I use the ATA_READ_VERIFY /
> ATA_READ_VERIFY48 commands (which read from the media, set the status and
> error bits, but don't transfer the data to the host). That requires
> talking directly to the drive, not the higher-level GEOMs, so I have to
> work in terms of LBAs.
Hmm. that'd be a nice to be able to expose via geom too...
> If I find an unreadable sector on one of the drives, I'd like to re-write
> the sector to heal it. I can do that by reading from the mirror; that will
> either pick the good side of the mirror in the first place, or will try
> and fail from the bad side, then failover to read from the good side.
> Either way, I end up with the proper data, and can re-write unreadable
> sector. The problem is, how do I calculate the byte offset in the mirror
> to read from?
>
> In the example above, since it's a relatively straightforward stack, I
> could do some math taking into account the LBA offsets for the GPT
> partitions, and the stripesize of the stripes, etc. That would work for
> this example, but it gets ugly fast if there are more complex transforms
> in the stack.
>
> It's easy enough to look at the partition table and say "LBA 12345 is in
> the range 1024 - 1048576, which is part of ada0p2". Going from there to
> "ada0p2 is part of gs0, which has a stripe interleave of 256KB" is more
> complicated. If there's something like GEOM_RAID3 in the mix - which has
> parity sectors which are not visible to the higher layers of the stack -
> then it gets uglier still.
>
> Is there a generic, supported way for doing this mapping? Or can someone
> point me in the right direction so, I can *create* a generic way for doing
> this, and submit it? :-)
I've only done this manually... It isn't too hard, as all the
partitioning schemes are simple offsets, and the stripe should be
regular...
The funny thing is that I hit a similar issue today myself!
The issue w/ manually mapping, is that you might loose a race w/ the
mirror when writing out new data... I was thinking it would be good for
gmirror to grow a mode that when it detects a pending sector or offline
sector, to figure out via some mapping, what data needed to be fixed,
and attempt to read/write the data back...
It would be interesting to add to geom the ability to notify the upper
layers about possible bad sectores, though this will take more work to
add...
My work to fix it today somewhat failed, in that I read from the gmirror
device, and when it hit the broken drive, returned a read error and
kicked the drive out of the mirror instead of fixing it (which I would
have preferred)...
Even having a simple mode that upon read error, would read from the other
drive and write back would be good... We'd need to have a way to say
that this drive is FAILING, but still usable...
--
John-Mark Gurney Voice: +1 415 225 5579
"All that I will do, has been done, All that I have, has not."
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20141219015210.GY25139>