Date: Tue, 16 Aug 2016 12:11:04 +0200 From: Ben RUBSON <ben.rubson@gmail.com> To: Max Gurtovoy <maxg@mellanox.com> Cc: "freebsd-scsi@freebsd.org" <freebsd-scsi@freebsd.org>, Oren Duer <oren@mellanox.com>, Oded Shanoon <odeds@mellanox.com>, Meny Yossefi <menyy@mellanox.com>, Roy Shterman <roysh@mellanox.com>, Hans Petter Selasky <hanss@mellanox.com> Subject: Re: Help to make iSER working Message-ID: <6B335920-6818-4A44-A905-2425B4CFA812@gmail.com> In-Reply-To: <f38f10a7-af85-1a0e-08c0-cff992d0e5be@mellanox.com> References: <71E4886B-61C3-4091-A7FF-E5A4F59AEACE@gmail.com> <64d8bc44-b6a3-4c0a-6d13-f919bf26685a@mellanox.com> <41CCC244-A781-4087-A811-B33847EDEC01@gmail.com> <6a464f2c-c048-d29c-e602-c129b35a5d4a@mellanox.com> <B3B47B30-D253-4543-A813-1AEDD3EE2656@gmail.com> <a8b1cd70-3e96-2dd6-20ae-1bf3b315ff89@mellanox.com> <621BCC32-4612-410F-B658-D1086A7A3A09@gmail.com> <8207D6A0-3647-48B4-A4E0-3F1FCF3A6723@gmail.com> <beeb0b05-98a0-6c9b-cc37-fd9876477558@mellanox.com> <AM5PR0501MB2580F57429494795D97005E3C21F0@AM5PR0501MB2580.eurprd05.prod.outlook.com> <AC9F5113-0909-4B90-9E89-8841F4060E2F@gmail.com> <CB6301B8-7A37-45BF-A019-8CACA3D4219A@gmail.com> <CE602A11-DB42-43CE-8B06-EFE4B21938ED@gmail.com> <f38f10a7-af85-1a0e-08c0-cff992d0e5be@mellanox.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Thank you very much Max for all your advices. These tests were really interesting as they demonstrated that iSER can = easily improve my iSCSI storage infrastructure throughput. Unfortunately, for production purpose, I need FreeBSD as the target = system. I will then keep an eye on future FreeBSD releases related to iSER. Thank you again ! Ben > On 16 Aug 2016, at 00:12, Max Gurtovoy <maxg@mellanox.com> wrote: >=20 > Good to see that you got a nice progress there. > You can try the fio tool and see even better performance. > You can also use Mellanox community site to see how to reach wire = spead and leaverage your numbers. The iser posts are mostly for linux = users but you can find it helpfull for your needs to. >=20 > Thanks, > Max. >=20 > On 8/15/2016 12:11 AM, Ben RUBSON wrote: >>=20 >>> On 13 Aug 2016, at 15:12, Ben RUBSON <ben.rubson@gmail.com> wrote: >>>=20 >>> Benchmarks will of course follow. >>=20 >> (also posted @freebsd-fs) >>=20 >> I then used Linux TGT as a test target. >>=20 >> ### Target : local disk throughput, one thread : >> # dd if=3D/dev/da8 of=3D/dev/null bs=3D$((128*1024)) count=3D81920 >> 10737418240 bytes (11 GB) copied, 21.3898 s, 502 MB/s >>=20 >> ### Initiator : iscsi disk throughput, one thread : >> # dd if=3D/dev/da8 of=3D/dev/null bs=3D$((128*1024)) count=3D81920 >> 10737418240 bytes transferred in 34.938676 secs (307321843 bytes/sec) = - 293 MB/s >>=20 >> ### Initiator : iSER disk throughput, one thread : >> # dd if=3D/dev/da8 of=3D/dev/null bs=3D$((128*1024)) count=3D81920 >> 10737418240 bytes transferred in 20.371947 secs (527068838 bytes/sec) = - 502 MB/s >>=20 >> In my example, iSER allows to reach the raw disk throughput (on one = thread), whereas iSCSI/TCP is a bottleneck. >>=20 >> I also tested with a target RAM device : >>=20 >> ### Initiator : iscsi RAM throughput, one thread : >> # dd if=3D/dev/da8 of=3D/dev/null bs=3D$((128*1024)) count=3D81920 >> 10737418240 bytes transferred in 24.194665 secs (443792803 bytes/sec) = - 423 MB/s >>=20 >> ### Initiator : iSER RAM throughput, one thread : >> # dd if=3D/dev/da8 of=3D/dev/null bs=3D$((128*1024)) count=3D81920 >> 10737418240 bytes transferred in 10.991094 secs (976919873 bytes/sec) = - 932 MB/s >> (just don't know what is the bottleneck here, of course not the RAM, = and network on one thread is around 30GB/s) >>=20 >>> On 11 Aug 2016, at 01:13, Max Gurtovoy <maxg@mellanox.com> wrote: >>>=20 >>> Regarding plans for the target side, it's not on my plate for the = near future (but obviously can be later on :)). >>> I'm not aware of other contributers plans. >>=20 >>=20 >> Can't wait to see this then :) >>=20 >> Many thanks again ! >>=20 >> Best regards, >>=20 >> Ben >>=20 >>=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6B335920-6818-4A44-A905-2425B4CFA812>