From owner-freebsd-questions@FreeBSD.ORG Mon Nov 8 08:37:57 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66183106566C for ; Mon, 8 Nov 2010 08:37:57 +0000 (UTC) (envelope-from svein-listmail@stillbilde.net) Received: from mail.stillbilde.net (d80.iso100.no [81.175.61.195]) by mx1.freebsd.org (Postfix) with ESMTP id DDD3B8FC0A for ; Mon, 8 Nov 2010 08:37:56 +0000 (UTC) Received: from [127.0.0.1] (unknown [192.168.4.11]) (Authenticated sender: svein-listmail) by mail.stillbilde.net (Familien Skogens mail) with ESMTPSA id 8837622 for ; Mon, 8 Nov 2010 09:37:55 +0100 (CET) Message-ID: <4CD7B6E0.2010501@stillbilde.net> Date: Mon, 08 Nov 2010 09:37:52 +0100 From: "Svein Skogen (Listmail account)" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.11) Gecko/20101013 Lightning/1.0b2 Thunderbird/3.1.5 MIME-Version: 1.0 To: freebsd-questions@freebsd.org References: <340170.48770.qm@web113309.mail.gq1.yahoo.com> In-Reply-To: <340170.48770.qm@web113309.mail.gq1.yahoo.com> X-Enigmail-Version: 1.1.1 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig7E358C8745EC838ABC4580B7" Subject: Re: zfs performance issues with iscsi (istgt) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Nov 2010 08:37:57 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig7E358C8745EC838ABC4580B7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 08.11.2010 09:13, DJ wrote: >=20 > After scratching my head for a few weeks, I've decided to ask for some = help. >=20 > First, I've got two machines connected by gigabit ethernet, network per= formance is not a problem as I am able to substantially saturate the wire= when not using iscsi [say iperf] or ftp. Both systems are 8.1-RELENG. Th= ey are both multi-core, 8G of RAM.=20 >=20 > Symptoms: When doing writes (size relatively independent) from a client= to a server via iSCSI I seem to be > hitting a wall between 18-26MB/s of write. This can be repeated contin= uously whether doing a newfs on a 2TB iscsi volume or doing a dd from /de= v/zero to the iscsi target. I haven't compared read performance. What ori= ginally put me on to this was watching the newfs *fly* across the screen,= and then hang for several seconds, and then *fly* again, and > then pause.=20 I'll snip down this mail a little bit to ask some control questions (having recently had quite a wrestle with iSCSI myself). -Is jumbo frames involved? (and enabled on the initiator, all switches in between, and the target) -What's the number of PPS (some switches have PPS issues, which becomes painfully relevant for small block IO)? (I got rid of most of my problems when I replaced the Netgear GS724Tv3 switch with a Cisco SG-300)= -Are you running digests? -Do you have TSO/TSOv2 enabled at the endpoints? -Does top -HSC reveal anything? -Does systat -vmstat 1 reveal anything? -What's the ICMP (ping) roundtrip times between the initiator and target IPs? //Svein --=20 --------+-------------------+------------------------------- /"\ |Svein Skogen | svein@d80.iso100.no \ / |Solberg =D8stli 9 | PGP Key: 0xE5E76831 X |2020 Skedsmokorset | svein@jernhuset.no / \ |Norway | PGP Key: 0xCE96CE13 | | svein@stillbilde.net ascii | | PGP Key: 0x58CD33B6 ribbon |System Admin | svein-listmail@stillbilde.net Campaign|stillbilde.net | PGP Key: 0x22D494A4 +-------------------+------------------------------- |msn messenger: | Mobile Phone: +47 907 03 575 |svein@jernhuset.no | RIPE handle: SS16503-RIPE --------+-------------------+------------------------------- A: Because it fouls the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? ------------------------------------------------------------ Picture Gallery: https://gallery.stillbilde.net/v/svein/ ------------------------------------------------------------ --------------enig7E358C8745EC838ABC4580B7 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkzXtuQACgkQODUnwSLUlKSldgCgn74zkXjfsXT0MRfjurLG8PF5 KlgAn3ycNZSiblM9f3lb1qFNj93ckTd0 =U+m9 -----END PGP SIGNATURE----- --------------enig7E358C8745EC838ABC4580B7--