Date: Thu, 10 Dec 2015 17:02:10 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: Max Gurtovoy <maxg@mellanox.com> Cc: freebsd-scsi@freebsd.org, Sagi Grimberg <sagig@mellanox.com>, Oren Duer <oren@mellanox.com>, Hans Petter Selasky <hanss@mellanox.com>, freebsd-geom@freebsd.org Subject: Re: splitting iovecs to bios Message-ID: <20151210150210.GY82577@kib.kiev.ua> In-Reply-To: <56696E03.8050202@mellanox.com> References: <56696E03.8050202@mellanox.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Dec 10, 2015 at 02:20:19PM +0200, Max Gurtovoy wrote: > Hi all, > > I'm developing an iSER (iSCSI extensions for RDMA) driver for FreeBSD > 11-Current and I encounter some weired behaviour while testing IOs with > iovcnt > 1. > I wrote a small test program that creates an iovec struct (let's say in > size of 4) and each iov_base starts from page begining and 4k len: > 0 iov_base 0xe25000 iov_len 4096 > 1 iov_base 0xe26000 iov_len 4096 > 2 iov_base 0xe27000 iov_len 4096 > 3 iov_base 0xe28000 iov_len 4096 > > I use readv/writev to send this iovec data. > I saw that my driver get 4 different bio requests even thought this > vector is aligned. This is surprising becasue in other OS I get only 1 bio. > I have noticed the the physio function in sys/kern/kern_physio.c is > praparing one bio for each iovec entry. > > is there a reason for this kind of implementation ? is there an option > to send this array using 1 bio (some flag) ? > We can improve performance if we send it in 1 bio instead of 4. > My driver supports BIO_UNMAPPED. There might be indeed a reason, it could be that some drivers expect blocking to be done by the userspace. The drivers could have some restrictions on transfer sizes and atomicity of transfer, which would be broken by the unconditional merge. I cannot give you an example of such driver, known block-aware drivers like sa(4) only require the bio size to be multiple of the basic block size. OTOH, I see no issue with adding a SI_PHYSIOMERGE flag and doing the merges for the driver in physio(), when unmapped request has consequtive iov elements ending and starting at the page boundary.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20151210150210.GY82577>