Date: Sun, 6 Mar 2016 13:10:42 -0800 From: Scott Long <scottl@samsco.org> To: Slawa Olhovchenkov <slw@zxy.spb.ru> Cc: freebsd-stable@freebsd.org Subject: Re: kernel: mps0: Out of chain frames, consider increasing hw.mps.max_chains. Message-ID: <0F0C78F4-6FE2-43BA-B503-AA04A79F2E70@samsco.org> In-Reply-To: <20160306194555.GC94639@zxy.spb.ru> References: <20160306194555.GC94639@zxy.spb.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, The message is harmless, it's a reminder that you should tune the kernel for= your workload. When the message is triggered, it means that a potential co= mmand was deferred, likely for only a few microseconds, and then everything m= oved on as normal. =20 A command uses anywhere from 0 to a few dozen chain frames per I/o, dependin= g on the size of the io. The chain frame memory is allocated at boot so tha= t it's always available, not allocated on the fly. When I wrote this driver= , I felt that it would be wasteful to reserve memory for a worst case scenar= io of all large io's by default, so I put in this deferral system with a con= sole reminder to for tuning. =20 Yes, you actually do have 900 io's outstanding. The controller buffers the i= o requests and allows the system to queue up much more than what sata disks m= ight allow on their own. It's debatable if this is good or bad, but it's tu= nable as well. Anyways, the messages should not cause alarm. Either tune up the chain fram= e count, or tune down the max io count. Scott Sent from my iPhone > On Mar 6, 2016, at 11:45 AM, Slawa Olhovchenkov <slw@zxy.spb.ru> wrote: >=20 > I am use 10-STABLE r295539 and LSI SAS2008. >=20 > mps0: <Avago Technologies (LSI) SAS2008> port 0x8000-0x80ff mem 0xdfc00000= -0xdfc03fff,0xdfb80000-0xdfbbffff irq 32 at device 0.0 on pci2 > mps0: Firmware: 15.00.00.00, Driver: 20.00.00.00-fbsd > mps0: IOCCapabilities: 185c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry= ,IR> > mps1: <Avago Technologies (LSI) SAS2008> port 0x7000-0x70ff mem 0xdf600000= -0xdf603fff,0xdf580000-0xdf5bffff irq 34 at device 0.0 on pci3 > mps1: Firmware: 17.00.01.00, Driver: 20.00.00.00-fbsd > mps1: IOCCapabilities: 185c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry= ,IR> > mps2: <Avago Technologies (LSI) SAS2008> port 0xf000-0xf0ff mem 0xfba00000= -0xfba03fff,0xfb980000-0xfb9bffff irq 50 at device 0.0 on pci129 > mps2: Firmware: 15.00.00.00, Driver: 20.00.00.00-fbsd > mps2: IOCCapabilities: 185c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry= ,IR> > mps3: <Avago Technologies (LSI) SAS2008> port 0xe000-0xe0ff mem 0xfb400000= -0xfb403fff,0xfb380000-0xfb3bffff irq 56 at device 0.0 on pci130 > mps3: Firmware: 15.00.00.00, Driver: 20.00.00.00-fbsd > mps3: IOCCapabilities: 185c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry= ,IR> >=20 > Some time ago I am see in log messages like this: >=20 > Mar 6 22:28:27 edge02 kernel: mps3: Out of chain frames, consider increas= ing hw.mps.max_chains. > Mar 6 22:28:28 edge02 kernel: mps1: Out of chain frames, consider increas= ing hw.mps.max_chains. > Mar 6 22:28:28 edge02 kernel: mps0: Out of chain frames, consider increas= ing hw.mps.max_chains. > Mar 6 22:29:39 edge02 kernel: mps0: Out of chain frames, consider increas= ing hw.mps.max_chains. > Mar 6 22:30:07 edge02 kernel: mps3: Out of chain frames, consider increas= ing hw.mps.max_chains. > Mar 6 22:30:09 edge02 kernel: mps1: Out of chain frames, consider increas= ing hw.mps.max_chains. >=20 > This is peak hour. I am try to monitoring: >=20 > root@edge02:/ # sysctl dev.mps | grep -e chain_free: -e io_cmds_active > dev.mps.3.chain_free: 70 > dev.mps.3.io_cmds_active: 901 > dev.mps.2.chain_free: 504 > dev.mps.2.io_cmds_active: 725 > dev.mps.1.chain_free: 416 > dev.mps.1.io_cmds_active: 896 > dev.mps.0.chain_free: 39 > dev.mps.0.io_cmds_active: 12 > root@edge02:/ # sysctl dev.mps | grep -e chain_free: -e io_cmds_active > dev.mps.3.chain_free: 412 > dev.mps.3.io_cmds_active: 572 > dev.mps.2.chain_free: 718 > dev.mps.2.io_cmds_active: 687 > dev.mps.1.chain_free: 211 > dev.mps.1.io_cmds_active: 906 > dev.mps.0.chain_free: 65 > dev.mps.0.io_cmds_active: 144 > root@edge02:/ # sysctl dev.mps | grep -e chain_free: -e io_cmds_active > dev.mps.3.chain_free: 500 > dev.mps.3.io_cmds_active: 629 > dev.mps.2.chain_free: 623 > dev.mps.2.io_cmds_active: 676 > dev.mps.1.chain_free: 251 > dev.mps.1.io_cmds_active: 907 > dev.mps.0.chain_free: 139 > dev.mps.0.io_cmds_active: 144 >=20 > [...] >=20 > root@edge02:/ # sysctl dev.mps | grep -e chain_free: -e io_cmds_active > dev.mps.3.chain_free: 1874 > dev.mps.3.io_cmds_active: 78 > dev.mps.2.chain_free: 1888 > dev.mps.2.io_cmds_active: 64 > dev.mps.1.chain_free: 1922 > dev.mps.1.io_cmds_active: 42 > dev.mps.0.chain_free: 1936 > dev.mps.0.io_cmds_active: 48 > root@edge02:/ # sysctl dev.mps | grep -e chain_free: -e io_cmds_active > dev.mps.3.chain_free: 1890 > dev.mps.3.io_cmds_active: 78 > dev.mps.2.chain_free: 1890 > dev.mps.2.io_cmds_active: 82 > dev.mps.1.chain_free: 1729 > dev.mps.1.io_cmds_active: 150 > dev.mps.0.chain_free: 1893 > dev.mps.0.io_cmds_active: 57 >=20 > What this mean? Why with allocated 2048 chains per controller I see 65 fre= e and 144 allocated? > How I got 976 active commands? I am use SATA HDD suported only 32 tags on N= CQ, with 8 ports > this maximum 256 outstanding commands per controller. >=20 > How I can resolve this issue? > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0F0C78F4-6FE2-43BA-B503-AA04A79F2E70>