From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 30 18:20:11 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6C5C182D for ; Fri, 30 Jan 2015 18:20:11 +0000 (UTC) Received: from mail-we0-f169.google.com (exprod7og119.obsmtp.com [64.18.2.16]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E44B4355 for ; Fri, 30 Jan 2015 18:20:10 +0000 (UTC) Received: from mail-we0-f169.google.com ([74.125.82.169]) (using TLSv1) by exprod7ob119.postini.com ([64.18.6.12]) with SMTP ID DSNKVMvLUx49o3WJZPhOHKYMsHmX7gektDdW@postini.com; Fri, 30 Jan 2015 10:20:11 PST Received: by mail-we0-f169.google.com with SMTP id u56so28670659wes.0 for ; Fri, 30 Jan 2015 10:20:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=DWZ7l06K5A2HurH3wcKjDAQSp8LJnmF3tWclMFjgPnk=; b=iE2tWAZrlia33ALll9NJ7tmaygymeCRwhQR51mYrOeKH07ONv6mVABAST69CAEf9Hl q2rntITbZNjhb4jow7DnoQU3xZQHplsXNC7pZZ4xDtPSD+/qppt8Y5aDGwzrDHv4XQBh Dy9utraTq7l5IRAJioQpn413NSqm55X9ezFhTSTBoiGBbQzc0S8jOA36Cc9zQ5VWkAHm wZiG0uxJPcUGHK3481HeHBZUrLuCVEm+HA9Ymnvx443AdSXjC9Kv60sZIdrCiJe+1hF9 wRBm2BHzXpuzXtD/YKhh9trkt/hFc6vdewHUsgzM8Jt6QQ2LgALpQCLk9ZB2WX3UcNEZ H4qw== X-Gm-Message-State: ALoCoQm4UpnJdBHefvhePTmbK9fvA5Xl+ofIGDjyEdxE2yYZDc22YlqATOMPwtsZUQZ8TulbKcpiUQ0z4zniSQa2d+y/cdanhizjtUu3Km05alKcBMdTq31Qpl2E7xmn8T11Yw0lY6sAgnmHCtwfrcWmmNtRGFe/9Q== X-Received: by 10.194.143.12 with SMTP id sa12mr14809999wjb.101.1422642002813; Fri, 30 Jan 2015 10:20:02 -0800 (PST) X-Received: by 10.194.143.12 with SMTP id sa12mr14809964wjb.101.1422642002630; Fri, 30 Jan 2015 10:20:02 -0800 (PST) From: Sibananda Sahu References: <8B56B74C-7EBC-4D1B-89AB-46DA8ED05DD5@yahoo.com> <923e4c2e65d29f2f39e0aa2f6d4ab38a@mail.gmail.com> <840B5221-A5E5-4FBC-BA2A-F44EF4036C59@yahoo.com> In-Reply-To: <840B5221-A5E5-4FBC-BA2A-F44EF4036C59@yahoo.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQJ90Y9Ud/YDW7817ZAJGKXgu7SlNwGkTx3UAb2EBL4BaZa2BptXc+jA Date: Fri, 30 Jan 2015 23:50:02 +0530 Message-ID: <7a375283987fdbae4d4b1db58021c9e4@mail.gmail.com> Subject: RE: How to send 1MB I/O size in a single I/O request without split To: Scott Long Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jan 2015 18:20:11 -0000 Thanks Scott for your great explanations. I really appreciate your time that helped me much in understanding the OS behavior. Thanks, Sibananda -----Original Message----- From: Scott Long [mailto:scott4long@yahoo.com] Sent: Friday, January 30, 2015 11:12 PM To: Sibananda Sahu Cc: freebsd-scsi@freebsd.org Subject: Re: How to send 1MB I/O size in a single I/O request without split > On Jan 30, 2015, at 2:41 AM, Sibananda Sahu > wrote: > > Hey Scoot, > > Thanks very much for your reply. > >>> - How can I get more sge count in an I/O request? >>> > >> If you want to test multiple segments, I suggest leaving you system >> running for a long time with multiple processes freeing and >> allocating memory so that the system becomes >fragmented. You can >> also modify the bus_dma_tag in your driver to specify a maximum >> segment size of 4k instead of (presumably in your case) something >> larger. That will force >busdma to stop merging adjacent segments. > > As you have said: > > "The busdma API will see that the allocation is contiguous and attempt > to merge the contiguous segments. This is usually desirable since few > segments reduces processing overhead in the driver and the hardware." > > If I will modify the bus_dma_tag in our driver to have a max segment > of 4K then what is the performance impact? > Most drivers have a for() loop that iterates through the list of segments from busdma and stores them into SGE elements for the hardware to use. Obviously, fewer segments means fewer trips through this loop. It also means a smaller SGE list, which means less command data transferred over th= e PCI bus. Though I cannot speak for the architecture details of your hardware, often times the same optimizations apply to the hardware internally. With modern storage controllers that are doing millions of i/o= =E2=80=99s per second, these small improvements can add up. Scott