Date: Fri, 30 Jan 2015 15:11:56 +0530 From: Sibananda Sahu <sibananda.sahu@avagotech.com> To: Scott Long <scott4long@yahoo.com> Cc: freebsd-scsi@freebsd.org Subject: RE: How to send 1MB I/O size in a single I/O request without split Message-ID: <923e4c2e65d29f2f39e0aa2f6d4ab38a@mail.gmail.com> In-Reply-To: <8B56B74C-7EBC-4D1B-89AB-46DA8ED05DD5@yahoo.com> References: <f94a31843fde43237d9aa13bbe543ddf@mail.gmail.com> <8B56B74C-7EBC-4D1B-89AB-46DA8ED05DD5@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hey Scoot, Thanks very much for your reply. >> - How can I get more sge count in an I/O request? >> >If you want to test multiple segments, I suggest leaving you system runnin= g >for a long time with multiple processes freeing and allocating memory so >that the system becomes >fragmented. You can also modify the bus_dma_tag >in your driver to specify a maximum segment size of 4k instead of >(presumably in your case) something larger. That will force >busdma to >stop merging adjacent segments. As you have said: "The busdma API will see that the allocation is contiguous and attempt to merge the contiguous segments. This is usually desirable since few segment= s reduces processing overhead in the driver and the hardware." If I will modify the bus_dma_tag in our driver to have a max segment of 4K then what is the performance impact? Thanks, Sibananda Sahu -----Original Message----- From: Scott Long [mailto:scott4long@yahoo.com] Sent: Friday, January 30, 2015 1:47 AM To: Sibananda Sahu Cc: freebsd-scsi@freebsd.org Subject: Re: How to send 1MB I/O size in a single I/O request without split > On Jan 29, 2015, at 11:56 AM, Sibananda Sahu > <sibananda.sahu@avagotech.com> wrote: > > Hi All, > > > > Recently we have added large I/O size of 1MB in our LSI controller and > for that we have implemented the same in driver. > > But I have observed that the large I/O that an application is able to > send is 128KB in one I/O request. > > > > I used the following command to send 1MB I/O: > > # ddpt if=3D/dev/da0 of=3D/dev/zero count=3D1 bs=3D1M > > > > But I have observed that the number of scatter gather elements per I/O > request always comes 1 and the I/O length comes as 128KB(max). > > [=E2=80=A6] > > > So my primary questions are: > > - How can I send a large I/O size of 1MB in a single I/O request > without any split? You answered this question already, you must redefine MAXPHYS. This can be done via a kernel compile option, and many users and companies already know how to do this. I plan to start a discussion on increasing the default size. > > - Why I am getting always 1 scatter gather element? > You are likely getting an allocation that is physically contiguous. This i= s especially true since you are using =E2=80=98dd=E2=80=99 from userland, and= the memory allocator in userland tries to useon 2MB superpages for allocations. The busdma API will see that the allocation is contiguous and attempt to merge the contiguous segments. This is usually desirable since few segments reduces processing overhead in the driver and the hardware. > - How can I get more sge count in an I/O request? > If you want to test multiple segments, I suggest leaving you system running for a long time with multiple processes freeing and allocating memory so that the system becomes fragmented. You can also modify the bus_dma_tag in your driver to specify a maximum segment size of 4k instead of (presumably in your case) something larger. That will force busdma to stop merging adjacent segments. Scott
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?923e4c2e65d29f2f39e0aa2f6d4ab38a>