Date: Sun, 6 Mar 2016 23:40:55 -0800 From: Scott Long <scottl@samsco.org> To: Slawa Olhovchenkov <slw@zxy.spb.ru> Cc: freebsd-stable@freebsd.org Subject: Re: kernel: mps0: Out of chain frames, consider increasing hw.mps.max_chains. Message-ID: <5B8DD95A-9FA0-4E16-85A1-87B54035B3F7@samsco.org> In-Reply-To: <20160307060407.GK11654@zxy.spb.ru> References: <20160306194555.GC94639@zxy.spb.ru> <0F0C78F4-6FE2-43BA-B503-AA04A79F2E70@samsco.org> <20160306212733.GJ11654@zxy.spb.ru> <DFC3C4CF-89D4-417C-AEBA-67F49F3EA1DE@samsco.org> <20160307060407.GK11654@zxy.spb.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Mar 6, 2016, at 10:04 PM, Slawa Olhovchenkov <slw@zxy.spb.ru> = wrote: >=20 > On Sun, Mar 06, 2016 at 06:20:06PM -0800, Scott Long wrote: >=20 >>=20 >>> On Mar 6, 2016, at 1:27 PM, Slawa Olhovchenkov <slw@zxy.spb.ru> = wrote: >>>=20 >>> On Sun, Mar 06, 2016 at 01:10:42PM -0800, Scott Long wrote: >>>=20 >>>> Hi, >>>>=20 >>>> The message is harmless, it's a reminder that you should tune the = kernel for your workload. When the message is triggered, it means that = a potential command was deferred, likely for only a few microseconds, = and then everything moved on as normal. =20 >>>>=20 >>>> A command uses anywhere from 0 to a few dozen chain frames per I/o, = depending on the size of the io. The chain frame memory is allocated at = boot so that it's always available, not allocated on the fly. When I = wrote this driver, I felt that it would be wasteful to reserve memory = for a worst case scenario of all large io's by default, so I put in this = deferral system with a console reminder to for tuning. =20 >>>>=20 >>>> Yes, you actually do have 900 io's outstanding. The controller = buffers the io requests and allows the system to queue up much more than = what sata disks might allow on their own. It's debatable if this is = good or bad, but it's tunable as well. >>>>=20 >>>> Anyways, the messages should not cause alarm. Either tune up the = chain frame count, or tune down the max io count. >>>=20 >>> I am don't know depends or not, but I see dramaticaly performance = drop >>> at time of this messages. >>>=20 >>=20 >> Good to know. Part of the performance drop might be because of the = slowness of printing to the console. >=20 > no, on console print may be one per minute >=20 The one-per-minute prints are by design. I should probably make it = print once and then increment a sysctl counter. >>> How I can calculate buffers numbers? >>=20 >> If your system is new enough to have mpsutil, please run it = =E2=80=98mpsutil >> show iocfacts=E2=80=99. >=20 > As I see mpsutil present only on -HEAD. > Can I compile it on 10-STABLE? >=20 Yes, I believe it should compile on 10, but I have not tried it = recently. >> If not, then boot your system with bootverbose and send me the = output. >=20 > I can do this day ago. >=20 >>> I am have very heavy I/O. >>=20 >> Out of curiosity, do you redefine MAXPHYS/DFLTPHYS in your kernel = config? >=20 > no >=20 >>> This allocated one for all controllers, or allocated for every = controller? >>=20 >> It=E2=80=99s per-controller. >>=20 >> I=E2=80=99ve thought about making the tuning be dynamic at runtime. = I >> implemented similar dynamic tuning for other drivers, but it seemed >> overly complex for low benefit. Implementing it for this driver >> would be possible but require some significant code changes. >=20 > What cause of chain_free+io_cmds_active << max_chains? > One cmd can use many chains? Yes. A request uses and active command, and depending on the size of = the I/O, it might use several chain frames. Scott
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5B8DD95A-9FA0-4E16-85A1-87B54035B3F7>