Date: Tue, 13 Jul 1999 19:40:56 -0700 (PDT) From: Matthew Jacob <mjacob@feral.com> To: "Robert J. Adams" <radams@siscom.net> Cc: hardware@FreeBSD.ORG, stable@FreeBSD.ORG Subject: Re: Fibre Channel Controller [ LONG RESPONSE ] Message-ID: <Pine.BSF.4.05.9907131541540.9687-100000@semuta.feral.com> In-Reply-To: <121f01becd68$46c41120$3102fbd1@siscom.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 13 Jul 1999, Robert J. Adams wrote:
> Have you used these?
A lot. Not only for FreeBSD, but Solaris, Linux, OpenBSD, NetBSD, NT,
etc..
> If so, how does the FC route compare to say U2W SCSI?
> Something about a little piece of fiber running my drives makes me feel all
> warm and fuzzy.
I'd have to say it's a mixed experience at this time. The data rates for
U2W SCSI busses (and U3W which I'm getting a sample card for in about
a month), what with PCI latencies, are comparable to what you get for
1024Mbit Fibre Channel (the difference between 40 Mhz x 16 bits and 100MB
is not a lot, really). The drives for U2W or Fibre Channel are the same
(but with different interfaces), so if you're running a small setup, U2W
is probably a better choice (see below). Now, that's data rates.
What turns out to be somewhat significant (particularly for larger bus
configurations) is that the non-data transfer phases stuff on SCSI is the
same data rate as it's always been (async mode, varying between 2 && 8MB/s
depending on bus length)- Additionally, Request Sense is a completely
separate command. In Fibre Channel SCSI, command and device control
functions go out as a single FC packet, data moves as multiple packets,
and the response that includes status *and* sense data (as needed) goes as
one packet, and they all go at the same rate. This gives more headroom
that you don't get in parallel SCSI. A classic amusement to me about this
is an arrangement at NASA/Ames where differential SCSI is carried to a
fibre optic repeater and then over fibre optics to a building about a
kilometer away to tape drives in a silo (for offsite secondaries)- the
tape drives really do take the data bursts at a healthy 10Mhz, 16 bits
wide, but because command, mesage and status bytes, and inter-burst times,
all operate at async data rates, the effective data rate is about 5MB/s.
Whuf.
What you really get with Fibre Channel that you don't get with U2W is a
very large address space that configures a little bit more sanely than
parallel SCSI. With private loop (that's the best tested at this time),
you get 126 loop addresses, so you can put a lot of devices on one loop.
You can ask (as the FreeBSD driver does do) for a specific loop address,
but you'll just get what's available after arbitration. No more of this
"oops, two devices have the same ID, we're hosed!". Don't. Even. Think.
About. Mentioning. SCAM. Thank you.
On the other hand, the code that's currently in FreeBSD that trys to
manage loop address rearbitration where it's possible to come back after a
loop initialization ('coz joe bob rebooted his workstation) is not very
well tested yet. The Fabric support (which allows you to have, in theory,
an arbitrarily large disk address space, but in practice for the way the
current fabric management is being done allows only another 127 disks,
has a lot more wringing out to be done too).
You also get a bit more tolerant interconnect technology (whether you're
using FC Optical or FC copper) where cable distance pretty much becomes a
non-issue. But you also pay for this. For U2W you can expect to pay a tad
more for the disks && the controllers and LVD cabling, and whatever your
normal SCSI disk case costs are, but for Fibre Channel you should expect
to budget for FC hubs (at a minimum 2k$ for a ~10 port hub), beaucoup more
if you get a switch (for fabrics). The FC adapters are also more expensive
than SCSI, but FC disks are in fact often *cheaper* than LVD or Ultra
disks (why? I dunno- probably because the FC connector is a 40 pin SCA
rather than an 80 pin SCA or 68 pin high density.... and the actual
bus interface technology basically only has to manage 4 twisted pairs
plus grounds). The real killer here is that boxes to put your fibre
channel disks in are not only expensive, they're harder than hell to find,
particularly in unit 1 quantities.
So, to get back to your question: I'd have to say that FC is cool, and
will really be awesome over time, but it's just *barely* at the level you
could put it together in your own shop with. Caveat Emptor. Man, it's
painful to admit this, but it's true.
-matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.05.9907131541540.9687-100000>