Date: Mon, 21 Jan 2013 23:18:24 +0530 From: "Desai, Kashyap" <Kashyap.Desai@lsi.com> To: "Kenneth D. Merry" <ken@freebsd.org> Cc: "freebsd-scsi@freebsd.org" <freebsd-scsi@freebsd.org>, "jhb@freebsd.org" <jhb@freebsd.org>, "McConnell, Stephen" <Stephen.McConnell@lsi.com> Subject: RE: Max Queue depth of HBA limited to 256 ? Message-ID: <B2FD678A64EAAD45B089B123FDFC3ED75E68FD616E@inbmail01.lsi.com> In-Reply-To: <20130121170529.GA64188@nargothrond.kdm.org> References: <B2FD678A64EAAD45B089B123FDFC3ED75E68FD6158@inbmail01.lsi.com> <20130121170529.GA64188@nargothrond.kdm.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> -----Original Message-----
> From: Kenneth D. Merry [mailto:ken@freebsd.org]
> Sent: Monday, January 21, 2013 10:35 PM
> To: Desai, Kashyap
> Cc: freebsd-scsi@freebsd.org; McConnell, Stephen; Saxena, Sumit;
> jhb@freebsd.org
> Subject: Re: Max Queue depth of HBA limited to 256 ?
>=20
> On Mon, Jan 21, 2013 at 20:15:47 +0530, Desai, Kashyap wrote:
> > Hi,
> >
> > I was trying to check few things on LSI controller, where we have more
> than 256 queue depth support.
> > I added default maxtags in scsi/scsi_xpt.c as below. (Because I don't
> want mattags to restrict any outstanding commands the LSI HBA.
> >
> > {
> > /* Default tagged queuing parameters for all devices */
> > {
> > T_ANY, SIP_MEDIA_REMOVABLE|SIP_MEDIA_FIXED,
> > /*vendor*/"*", /*product*/"*", /*revision*/"*"
> > },
> > /*quirks*/0, /*mintags*/2, /*maxtags*/1024 <--- Default
> maxtags were 256. I increase it to 10234
> > },
> >
> >
> > LSI's SAS-HBA and MR-HBA can support more than 256 outstanding
> commands in Firmware. But due to some reason, I am not able to pump
> more than 256 outstanding commands to the HBA.
> >
> > I used "rawio -p 256 /dev/da1" and more /dev/dax in loop. I have
> sysctl parameter in Driver to display outstanding "FW commands". Max
> value for FW outstanding only goes up to 256.
> >
> > Also from some other mail thread Subject "mfi driver performance", I
> found that folks talk about tuning queue depth _but_ nobody discussed to
> increase it beyond 256. Is there any limitation in FreeBSD ?
> >
>=20
> As Jim pointed out, one thing to check is the values passed into
> cam_sim_alloc(). In the case of the mps(4) driver, the calculation is
> in mps_attach():
>=20
> sc->num_reqs =3D MIN(MPS_REQ_FRAMES, sc->facts->RequestCredit);
>=20
> What is reported for the RequestCredit on this particular adapter?
>=20
> The other question is, what does 'camcontrol tags daX -v' show when you
> are running the test?
Below is output of camcontrol tags da1 -v.
dhcp-135-24-192-127# camcontrol tags da13 -v
(pass13:mrsas0:0:13:0): dev_openings 1024
(pass13:mrsas0:0:13:0): dev_active 0
(pass13:mrsas0:0:13:0): devq_openings 1024
(pass13:mrsas0:0:13:0): devq_queued 0
(pass13:mrsas0:0:13:0): held 0
(pass13:mrsas0:0:13:0): mintags 2
(pass13:mrsas0:0:13:0): maxtags 1024
dhcp-135-24-192-127# camcontrol tags da1 -v
(pass1:mrsas0:0:1:0): dev_openings 1024
(pass1:mrsas0:0:1:0): dev_active 0
(pass1:mrsas0:0:1:0): devq_openings 1024
(pass1:mrsas0:0:1:0): devq_queued 0
(pass1:mrsas0:0:1:0): held 0
(pass1:mrsas0:0:1:0): mintags 2
(pass1:mrsas0:0:1:0): maxtags 1024
Value 1024 is hard coded for my testing. In MegaRaid controller and SAS-HBA=
Driver read max commands value from FW.=20
Similar to "RequestCredit".. Different FW has different value, but they ar=
e every time above 255.
When I run IOs dev_active stays in range of 0-255 only. See below output w=
hen I run IOs on /dev/da1 and /dev/da13. I expect total dev_openings should=
go beyond 255, which is not happening.
=20
dhcp-135-24-192-127# camcontrol tags da1 -v
(pass1:mrsas0:0:1:0): dev_openings 832
(pass1:mrsas0:0:1:0): dev_active 192
(pass1:mrsas0:0:1:0): devq_openings 832
(pass1:mrsas0:0:1:0): devq_queued 0
(pass1:mrsas0:0:1:0): held 0
(pass1:mrsas0:0:1:0): mintags 2
(pass1:mrsas0:0:1:0): maxtags 1024
dhcp-135-24-192-127# camcontrol tags da13 -v
(pass13:mrsas0:0:13:0): dev_openings 881
(pass13:mrsas0:0:13:0): dev_active 143
(pass13:mrsas0:0:13:0): devq_openings 881
(pass13:mrsas0:0:13:0): devq_queued 0
(pass13:mrsas0:0:13:0): held 0
(pass13:mrsas0:0:13:0): mintags 2
(pass13:mrsas0:0:13:0): maxtags 1024
Jim:
Below is my API call. I have hard code value "queue_depth" =3D 1024=20
sc->sim_0 =3D cam_sim_alloc(mrsas_action, mrsas_poll, "mrsas", sc,
device_get_unit(sc->mrsas_dev), &sc->sim_lock, queue_depth,
queue_depth, devq);
~ Kashyap
>=20
> Ken
> --
> Kenneth Merry
> ken@FreeBSD.ORG
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B2FD678A64EAAD45B089B123FDFC3ED75E68FD616E>