Date: Tue, 12 Mar 2019 10:21:09 -0700 (PDT) From: "Rodney W. Grimes" <freebsd@gndrsh.dnsmgr.net> To: Kashyap D Desai <kadesai@freebsd.org> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r345056 - head/sys/dev/mrsas Message-ID: <201903121721.x2CHL9O2015999@gndrsh.dnsmgr.net> In-Reply-To: <201903120924.x2C9Ow52017475@repo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> Author: kadesai > Date: Tue Mar 12 09:24:58 2019 > New Revision: 345056 > URL: https://svnweb.freebsd.org/changeset/base/345056 > > Log: > fw_outstanding"(outstanding IOs at firmware level) counter gets screwed up when R1 fastpath > writes are running. Some of the cases which are not handled properly in driver are: > > 1. With R1 fastpath supported, single write from CAM layer can consume 2 MPT frames > at driver/firmware level for fastpath qualification(if fw_outstanding < controller Queue Depth). > Due to this driver has to throttle IOs coming from CAM layer as well as second fastpath > write(of R1 write) against Adapter Queue Depth. > If "fw_outstanding" reaches to adapter queue depth, driver should return IOs from CAM layer with > device busy status.While allocating second MPT frame(corresponding to R1 FP write) also, driver > should ensure fw_outstanding should not exceed adapter QD. > > 2. For R1 fastpath writes completion, driver decrements "fw_oustanding" counter without > really returning MPT frame to free pool. It may cause IOs(with heavy IOs running, consuming whole > adapter Queue Depth) consuming MPT frames reserved for DCMDs(management commands) and > DCMDs(internal and sent by application) not getting MPT frame will start failing. > > Below is one test case to hit the issue described above- > 1. Run heavy IOs (outstanding IOs should hit adapter Queue Depth). > 2. Run management tool (Broadcom's storcli tool) querying adapter in loop (run command- "storcli64 /c0 show" in loop). > 3. Management tool's requests would start failing due to non-availability of free MPT frames as all frames would be consumed by IOs. > > Fix: Increment/decrement of "fw_outstanding" counter should be in sync with MPT frame get/return. > > Submitted by: Sumit Saxena <sumit.saxena@broadcom.com> > Reviewed by: Kashyap Desai <Kashyap.Desai@broadcom.com> > Approved by: Ken > MFC after: 3 days > Sponsored by: Broadcom Inc > Can we please keep commit messages formatted to <80 columns? Thanks, Rod -- Rod Grimes rgrimes@freebsd.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201903121721.x2CHL9O2015999>