From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 15:10:54 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D470106566C for ; Mon, 4 Apr 2011 15:10:54 +0000 (UTC) (envelope-from linuxmail@4lin.net) Received: from mail.4lin.net (mail.4lin.net [46.4.210.97]) by mx1.freebsd.org (Postfix) with ESMTP id E1C0B8FC0C for ; Mon, 4 Apr 2011 15:10:53 +0000 (UTC) Received: from localhost (angelica.4lin.net [127.0.0.1]) by mail.4lin.net (Postfix) with ESMTP id A94612C6A8 for ; Mon, 4 Apr 2011 17:14:42 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.4lin.net Received: from mail.4lin.net ([127.0.0.1]) by localhost (mail.4lin.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lm6iyKajDogk for ; Mon, 4 Apr 2011 17:14:39 +0200 (CEST) Received: from [130.83.160.152] (pcdenny.rbg.informatik.tu-darmstadt.de [130.83.160.152]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.4lin.net (Postfix) with ESMTPSA id 4BAC820081 for ; Mon, 4 Apr 2011 17:14:39 +0200 (CEST) From: Denny Schierz To: freebsd-stable@freebsd.org Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-sTvcvfVti2aMO0NdY5kv" Date: Mon, 04 Apr 2011 17:10:38 +0200 Message-ID: <1301929838.26698.213.camel@pcdenny> Mime-Version: 1.0 X-Mailer: Evolution 2.30.3 Subject: 8.2: ISCSI: ISTGT a bit slow, I think X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 15:10:54 -0000 --=-sTvcvfVti2aMO0NdY5kv Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable hi, I testing the maximum throughput from ISCSI, but I've reached only ~50MB/s (dd if=3D/dev/zero of=3D/dev/da13 bs=3D1M count=3D2048) with crosso= ver 1Gb/s cabel and raw disk. Both machines are FreeBSD 8.2-stable with istgt and the Onboard ISCSI initiator=20 With ZFS as target we loose round about 8-10MB/s. istgt.conf =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D [global] Timeout 30 NopInInterval 20 DiscoveryAuthMethod Auto MaxSessions 32 MaxConnections 8 #FirstBurstLength 65536 MaxBurstLength 1048576 MaxRecvDataSegmentLength 262144 # maximum number of sending R2T in each connection # actual number is limited to QueueDepth and MaxCmdSN and ExpCmdSN # 0=3Ddisabled, 1-256=3Dimproves large writing MaxR2T 32 # iSCSI initial parameters negotiate with initiators # NOTE: incorrect values might crash MaxOutstandingR2T 16 DefaultTime2Wait 2 DefaultTime2Retain 60 MaxBurstLength 1048576 [....] [LogicalUnit4] Comment "40GB Disk (iqn.san.foo:40gb)" TargetName 40gb TargetAlias "Data 40GB" Mapping PortalGroup1 InitiatorGroup1 #AuthMethod Auto #AuthGroup AuthGroup2 UnitType Disk UnitInquiry "FreeBSD" "iSCSI Disk" "01234" "10000004" QueueDepth 32 LUN0 Storage /failover/bigPool/disk40gb 40960MB [LogicalUnit5] Comment "2TB Disk (iqn.san.foo:2tb)" TargetName 2tb=20 TargetAlias "Data 2TB" Mapping PortalGroup1 InitiatorGroup1 #AuthMethod Auto #AuthGroup AuthGroup2 UnitType Disk UnitInquiry "FreeBSD" "iSCSI Disk" "01235" "10000005" QueueDepth 32 LUN0 Storage /dev/da12 200480MB =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D The raw disks, itself reaches over 150-200MB/s with or without ZFS (raidz2) We have 4GB Ram and 4 x 3Ghz Xeon CPUs on board. I thought, we should reach over 80-100MB/s, so, ISTGT or the Initiator is a bis slow, I think. I've tested just in the moment with Ubuntu 10.10 Initiator and I've got round about 70>MB/s - or without ZFS - constant 80>MB/s, over a regular switched network. Is this the end what we could reach? 'Cause of TCP and ISCSI overhead? What we can't: enable Jumbo frames. Our switches (Cisco catalyst WS-X4515) doesn't support jumbo frames. I've tested Jumbo Frames (9k) over the crossover, but the performance was worse. Round about 20MB/s .... So, does anyone has some hints for me? :-) cu denny --=-sTvcvfVti2aMO0NdY5kv Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEABECAAYFAk2Z324ACgkQKlzhkqt9P+CH8QCglvQbgHu81wlVSeggbFN/R1cf rsUAnRAV/CNy8LUKp7aKFI+KEiXBM50W =PymT -----END PGP SIGNATURE----- --=-sTvcvfVti2aMO0NdY5kv--