Date: Thu, 18 Aug 2016 01:39:46 +0300 From: Max Gurtovoy <maxg@mellanox.com> To: Ben RUBSON <ben.rubson@gmail.com> Cc: "freebsd-scsi@freebsd.org" <freebsd-scsi@freebsd.org>, Oren Duer <oren@mellanox.com>, Oded Shanoon <odeds@mellanox.com>, Meny Yossefi <menyy@mellanox.com>, Roy Shterman <roysh@mellanox.com>, Hans Petter Selasky <hanss@mellanox.com> Subject: Re: Help to make iSER working Message-ID: <8312894b-dce1-94c0-6faf-121cf2db58e5@mellanox.com> In-Reply-To: <6B335920-6818-4A44-A905-2425B4CFA812@gmail.com> References: <71E4886B-61C3-4091-A7FF-E5A4F59AEACE@gmail.com> <64d8bc44-b6a3-4c0a-6d13-f919bf26685a@mellanox.com> <41CCC244-A781-4087-A811-B33847EDEC01@gmail.com> <6a464f2c-c048-d29c-e602-c129b35a5d4a@mellanox.com> <B3B47B30-D253-4543-A813-1AEDD3EE2656@gmail.com> <a8b1cd70-3e96-2dd6-20ae-1bf3b315ff89@mellanox.com> <621BCC32-4612-410F-B658-D1086A7A3A09@gmail.com> <8207D6A0-3647-48B4-A4E0-3F1FCF3A6723@gmail.com> <beeb0b05-98a0-6c9b-cc37-fd9876477558@mellanox.com> <AM5PR0501MB2580F57429494795D97005E3C21F0@AM5PR0501MB2580.eurprd05.prod.outlook.com> <AC9F5113-0909-4B90-9E89-8841F4060E2F@gmail.com> <CB6301B8-7A37-45BF-A019-8CACA3D4219A@gmail.com> <CE602A11-DB42-43CE-8B06-EFE4B21938ED@gmail.com> <f38f10a7-af85-1a0e-08c0-cff992d0e5be@mellanox.com> <6B335920-6818-4A44-A905-2425B4CFA812@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
NP. I never tried it on FreeBSD but you can try to build the user space target TGT on it. Maybe you'll need some adaptations but should be easier that adding a new iser target driver from scratch. On 8/16/2016 1:11 PM, Ben RUBSON wrote: > Thank you very much Max for all your advices. > > These tests were really interesting as they demonstrated that iSER can easily improve my iSCSI storage infrastructure throughput. > > Unfortunately, for production purpose, I need FreeBSD as the target system. > I will then keep an eye on future FreeBSD releases related to iSER. > > Thank you again ! > > Ben > > > >> On 16 Aug 2016, at 00:12, Max Gurtovoy <maxg@mellanox.com> wrote: >> >> Good to see that you got a nice progress there. >> You can try the fio tool and see even better performance. >> You can also use Mellanox community site to see how to reach wire spead and leaverage your numbers. The iser posts are mostly for linux users but you can find it helpfull for your needs to. >> >> Thanks, >> Max. >> >> On 8/15/2016 12:11 AM, Ben RUBSON wrote: >>> >>>> On 13 Aug 2016, at 15:12, Ben RUBSON <ben.rubson@gmail.com> wrote: >>>> >>>> Benchmarks will of course follow. >>> >>> (also posted @freebsd-fs) >>> >>> I then used Linux TGT as a test target. >>> >>> ### Target : local disk throughput, one thread : >>> # dd if=/dev/da8 of=/dev/null bs=$((128*1024)) count=81920 >>> 10737418240 bytes (11 GB) copied, 21.3898 s, 502 MB/s >>> >>> ### Initiator : iscsi disk throughput, one thread : >>> # dd if=/dev/da8 of=/dev/null bs=$((128*1024)) count=81920 >>> 10737418240 bytes transferred in 34.938676 secs (307321843 bytes/sec) - 293 MB/s >>> >>> ### Initiator : iSER disk throughput, one thread : >>> # dd if=/dev/da8 of=/dev/null bs=$((128*1024)) count=81920 >>> 10737418240 bytes transferred in 20.371947 secs (527068838 bytes/sec) - 502 MB/s >>> >>> In my example, iSER allows to reach the raw disk throughput (on one thread), whereas iSCSI/TCP is a bottleneck. >>> >>> I also tested with a target RAM device : >>> >>> ### Initiator : iscsi RAM throughput, one thread : >>> # dd if=/dev/da8 of=/dev/null bs=$((128*1024)) count=81920 >>> 10737418240 bytes transferred in 24.194665 secs (443792803 bytes/sec) - 423 MB/s >>> >>> ### Initiator : iSER RAM throughput, one thread : >>> # dd if=/dev/da8 of=/dev/null bs=$((128*1024)) count=81920 >>> 10737418240 bytes transferred in 10.991094 secs (976919873 bytes/sec) - 932 MB/s >>> (just don't know what is the bottleneck here, of course not the RAM, and network on one thread is around 30GB/s) >>> >>>> On 11 Aug 2016, at 01:13, Max Gurtovoy <maxg@mellanox.com> wrote: >>>> >>>> Regarding plans for the target side, it's not on my plate for the near future (but obviously can be later on :)). >>>> I'm not aware of other contributers plans. >>> >>> >>> Can't wait to see this then :) >>> >>> Many thanks again ! >>> >>> Best regards, >>> >>> Ben >>> >>> >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8312894b-dce1-94c0-6faf-121cf2db58e5>