Date: Fri, 5 Jun 2009 00:03:59 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: Alexander Motin <mav@mavhome.dp.ua> Cc: FreeBSD-Current <freebsd-current@freebsd.org>, freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration Message-ID: <200906050703.n5573x5Q071765@apollo.backplane.com> References: <4A254B45.8050800@mavhome.dp.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
:Hi.
:
:After replying to several similar questions about my ATA plans last
:time, I have decided to announce things I am working on now together
:with Scott Long.
:
:While learning FreeBSD ATA implementation, I have found, that it has
:numerous deep problems, from quite fuzzy APIs to different issues in
:device detection and error recovery. Also, as soon as this
:infrastructure was written many years ago, it has completely no support
:for any kind of command queuing, which is normal for the most of modern
:drives and controllers. Fixing all of this require many significant changes.
:
:Also, you may know, that SAS controllers and expanders allow attaching
:SATA devices and port multipliers to them, by transporting ATA commands
:...
:project of making CAM a system's universal infrastructure for both SCSI
:and ATA. This project is not about some kind of SCSI-to-ATA translation,
:used by some OS, like OpenBSD. It is about extending CAM, to equally
:support both SCSI and ATA worlds natively and integrate them as tight as
:possible.
:
:...
:Our code now lives in PERFORCE in scottl-camlock project branch. It is
:in early development, but we already have there working CAM driver for
:AHCI controller with command queuing and basic NCQ support, simple SATA
:bus management code and ATA disk driver. I am able now to boot my system
:and work from SATA drive on AHCI controller, using ATA disk driver,
:having command queuing and NCQ enabled, read and write disks with SATA
:ATAPI DVD-RW drive, using native SCSI CD driver. And all of that only
:with CAM, without using any part of ATA infrastructure.
:
:--
:Alexander Motin
The biggest issue with AHCI (and ATA) interfacing is that AHCI devices
attach either as DISK or ATAPI. A device which attaches as a DISK
does not typically support ATAPI commands (though it would be an
interesting experiment to see if some did). This means that no matter
what you do a SCSI<->ATA translation layer needs to do some significant
fake-ups for DISK attachments, similar to what OpenBSD does in their
SCSI<->ATA layer. ATA DISK attachments simply do not support enough
of the SCSI command set for direct integration into CAM (IMHO).
The second biggest issue is that it is really unclear to me how the
hell one probes an ATAPI device for NCQ support. The OpenBSD driver
only uses AHCI-NCQ for DISK attachments, where the NCQ support is
returned in the IDENTIFY command. I'm sure there must be a way to
probe an ATAPI device for NCQ support but I don't know what it is.
I don't think it is possible to get much cleaner then OpenBSD's AHCI
driver (/usr/src/sys/dev/pci/ahci.c in OpenBSD). There is also a
DragonFly port of the OpenBSD AHCI driver (/usr/src/sys/dev/disk/ahci
in DragonFly) which you may want to look at. You are already familiar
with OpenBSD's SCSI<->ATA layer (in /us/src/sys/dev/ata in OpenBSD).
The DragonFly port of the OpenBSD driver will be done in about a week,
and maybe a bit longer for the port-multiplier additions. It essentially
works now. I'm still working on hot-plug support (the OpenBSD driver
doesn't have it), some error reporting / SENSE issues, and CAM bus
rescan. The DFly port is a closer match to FreeBSD since we use busdma
and CAM. There are some API differences but far fewer verses the
OpenBSD driver.
It sounds like your own AHCI driver is well underway, though. Going
with a separate AHCI-only driver and then just using the ATA driver
to pick-up non-AHCI ATA ports is probably the correct way to go.
That is what we intend to do.
It's really amazing how *CLEAN* an AHCI-only driver is without all
that old ATA hardware interfaces to deal with. The entire DragonFly
driver is only 3700 lines of code.
--
A couple of notes on the OpenBSD AHCI driver. OpenBSD only allocates
a 24-entry PRDT (DMA chain) table per tag per port. The PRDT table can
be up to 65535 entries and should be large enough to at least
handle MAXPHYS transfers (56 entries would do the job). Their
ATAPI implementation does not appear to do compatibility translations
for INQUIRY, READ_6, or WRITE_6 (the DragonFly version does). OpenBSD's
port reset code also doesn't work perfectly, something I will be fixing
for hot plug support in the DragonFly port over the next week.
The OpenBSD driver does not have port multiplier support but adding
it to the DFly driver will be pretty easy... I just need some hardware
to test it with (it's on the way). Unfortunately the AHCI-1.0
specfication says it cannot be used for high-performance multi-disk
I/O because all parallel commands in operation are only allowed to go
to a single target at a time. i.e. you can't mix parallel commands
to different targets on the same port. That's a serious problem.
(Does anyone know if the AHCI-1.1 or 1.2 specifications do anything
about that?).
It is unclear to me from reading the specification as to whether
AHCI-NCQ is supported for SATA ATAPI attachments. If anyone has an
answer to that I'm looking for a way to probe the device's
max-commands for ATAPI. for DISKs the IDENTIFY command has the
necessary feature bits and information. I'm sure the host controller
supports it natively but the real question is how to probe
the capability on the attached device and whether the device(s)
support it.
Ultimately the best way to expand-out an AHCI interface is with
SCSI pass-through over ATAPI, assuming NCQ can be supported somehow.
The port-multiplier spec is badly broken (at least in Intel's AHCI-1.0
spec). It is a bit annoying, actually, I wouldn't have though that
Intel would have made such a basic mistake. All they had to do was
implement 4 bits in the FIS and the problem would have been solved.
Instead they have routing bits in a port register. Sigh.
-Matt
Matthew Dillon
<dillon@backplane.com>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200906050703.n5573x5Q071765>