From owner-freebsd-scsi@FreeBSD.ORG Mon Jan 18 02:17:59 2010 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EFE9F106566C for ; Mon, 18 Jan 2010 02:17:59 +0000 (UTC) (envelope-from dan@langille.org) Received: from nyi.unixathome.org (nyi.unixathome.org [64.147.113.42]) by mx1.freebsd.org (Postfix) with ESMTP id C92048FC13 for ; Mon, 18 Jan 2010 02:17:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id F328650830; Mon, 18 Jan 2010 02:17:58 +0000 (GMT) X-Virus-Scanned: amavisd-new at unixathome.org Received: from nyi.unixathome.org ([127.0.0.1]) by localhost (nyi.unixathome.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W5G-hxrd08bQ; Mon, 18 Jan 2010 02:17:58 +0000 (GMT) Received: from smtp-auth.unixathome.org (smtp-auth.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) by nyi.unixathome.org (Postfix) with ESMTPSA id 2B6B850823 ; Mon, 18 Jan 2010 02:17:58 +0000 (GMT) Message-ID: <4B53C4DA.9080008@langille.org> Date: Sun, 17 Jan 2010 21:18:02 -0500 From: Dan Langille User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: do I want ch0 or pass1? X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jan 2010 02:18:00 -0000 Please CC me on replies. I'm running into issues with hard-coding some devices (see recent post titled 'device.hints isn't setting what I want'). Associated with this issue is confusion over whether I want to use ch0 or pass1. I have these devices: at scbus1 target 0 lun 0 (ch0,pass1) at scbus1 target 5 lun 0 (sa1,pass2) My understanding: chio(1) will with ch0, whereas mtx(1) will work with pass1. Is this correct? More information/elaboration will help I'm sure. Why do I ask? I can get the tape changer and tape drive hardwired to ch0 and sa1 respectively. I cannot [yet] do the same with pass1. Thanks folks. From owner-freebsd-scsi@FreeBSD.ORG Mon Jan 18 02:18:43 2010 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 271C9106566B for ; Mon, 18 Jan 2010 02:18:43 +0000 (UTC) (envelope-from dan@langille.org) Received: from nyi.unixathome.org (nyi.unixathome.org [64.147.113.42]) by mx1.freebsd.org (Postfix) with ESMTP id F42218FC1A for ; Mon, 18 Jan 2010 02:18:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id A68C950830; Mon, 18 Jan 2010 02:18:42 +0000 (GMT) X-Virus-Scanned: amavisd-new at unixathome.org Received: from nyi.unixathome.org ([127.0.0.1]) by localhost (nyi.unixathome.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RVd-3jJiav7W; Mon, 18 Jan 2010 02:18:41 +0000 (GMT) Received: from smtp-auth.unixathome.org (smtp-auth.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) by nyi.unixathome.org (Postfix) with ESMTPSA id BF85750823 ; Mon, 18 Jan 2010 02:18:41 +0000 (GMT) Message-ID: <4B53C505.7050608@langille.org> Date: Sun, 17 Jan 2010 21:18:45 -0500 From: Dan Langille User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: device.hints isn't setting what I want X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jan 2010 02:18:43 -0000 Folks, [please CC me on replies] First, see also my post: do I want ch0 or pass1? I have an external tape library and an external tape drive. They are not always powered up. My goal: always get the same devices regardless of whether or not the tape library is powered on at boot. After booting, with the tape library powered on, I have these devices: # camcontrol devlist at scbus0 target 5 lun 0 (sa0,pass0) at scbus1 target 0 lun 0 (ch0,pass1) at scbus1 target 5 lun 0 (sa1,pass2) at scbus2 target 0 lun 0 (cd0,pass3) at scbus5 target 0 lun 0 (da0,pass4) In /boot/devices, I have added these entries: hint.scbus.1.at="ahc0" hint.scbus.0.at="ahc1" hint.scbus.2.at="acd0" hint.scbus.5.at="umass0" The first two lines ensure I always get the QUANTUM drive at sa0, and the DEC at sa1. That part works. The second two lines aren't doing as expected. If the external tape library is not powered up, I get: # camcontrol devlist at scbus0 target 5 lun 0 (sa0,pass0) at scbus3 target 0 lun 0 (cd0,pass1) at scbus7 target 0 lun 0 (da0,pass2) Then, after powering up, and doing a rescan all, I get: # camcontrol devlist at scbus0 target 5 lun 0 (sa0,pass0) at scbus1 target 0 lun 0 (pass3,ch0) at scbus1 target 5 lun 0 (pass4,sa1) at scbus3 target 0 lun 0 (cd0,pass1) at scbus7 target 0 lun 0 (da0,pass2) So... I could keep the unit powered on all the time, but that's really not what I want to do. Clues please? Full dmesg output at http://www.langille.org/tmp/dmesg.boot From owner-freebsd-scsi@FreeBSD.ORG Mon Jan 18 11:07:05 2010 Return-Path: Delivered-To: freebsd-scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B94A11065672 for ; Mon, 18 Jan 2010 11:07:05 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9F6C68FC2A for ; Mon, 18 Jan 2010 11:07:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o0IB75sn047661 for ; Mon, 18 Jan 2010 11:07:05 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o0IB75WG047659 for freebsd-scsi@FreeBSD.org; Mon, 18 Jan 2010 11:07:05 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 18 Jan 2010 11:07:05 GMT Message-Id: <201001181107.o0IB75WG047659@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-scsi@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-scsi@FreeBSD.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jan 2010 11:07:05 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/142351 scsi [mpt] LSILogic driver performance problems o kern/141934 scsi [cam] [patch] add support for SEAGATE DAT Scopion 130 o kern/140091 scsi [da] [patch] allow for da(4) large block transfer than o kern/138789 scsi [cam] [patch] cd(4) patch for drives/discs failing the o kern/134488 scsi [mpt] MPT SCSI driver probes max. 8 LUNs per device o amd64/132394 scsi [isp] - bad underruns with QLogic qla2300 and amd64 o kern/132250 scsi [ciss] ciss driver does not support more then 15 drive o kern/132206 scsi [mpt] system panics on boot when mirroring and 2nd dri p kern/130735 scsi [cam] [patch] pass M_NOWAIT to the malloc() call insid o kern/130621 scsi [mpt] tranfer rate is inscrutable slow when use lsi213 o kern/129602 scsi [ahd] ahd(4) gets confused and wedges SCSI bus o kern/128452 scsi [sa] [panic] Accessing SCSI tape drive randomly crashe o kern/128245 scsi [scsi] "inquiry data fails comparison at DV1 step" [re o kern/127927 scsi [isp] isp(4) target driver crashes kernel when set up o kern/124667 scsi [amd] [panic] FreeBSD-7 kernel page faults at amd-scsi o kern/123674 scsi [ahc] ahc driver dumping f kern/123666 scsi [aac] attach fails with Adaptec SAS RAID 3805 controll o sparc/121676 scsi [iscsi] iscontrol do not connect iscsi-target on sparc o kern/120487 scsi [sg] scsi_sg incompatible with scanners o kern/120247 scsi [mpt] FreeBSD 6.3 and LSI Logic 1030 = only 3.300MB/s o kern/119668 scsi [cam] [patch] certain errors are too verbose comparing o kern/114597 scsi [sym] System hangs at SCSI bus reset with dual HBAs o kern/110847 scsi [ahd] Tyan U320 onboard problem with more than 3 disks o kern/99954 scsi [ahc] reading from DVD failes on 6.x [regression] o kern/94838 scsi Kernel panic while mounting SD card with lock switch o o kern/92798 scsi [ahc] SCSI problem with timeouts o kern/90282 scsi [sym] SCSI bus resets cause loss of ch device o kern/76178 scsi [ahd] Problem with ahd and large SCSI Raid system o kern/74627 scsi [ahc] [hang] Adaptec 2940U2W Can't boot 5.3 s kern/61165 scsi [panic] kernel page fault after calling cam_send_ccb o kern/60641 scsi [sym] Sporadic SCSI bus resets with 53C810 under load o kern/60598 scsi wire down of scsi devices conflicts with config s kern/57398 scsi [mly] Current fails to install on mly(4) based RAID di o kern/52638 scsi [panic] SCSI U320 on SMP server won't run faster than o kern/44587 scsi dev/dpt/dpt.h is missing defines required for DPT_HAND o kern/40895 scsi wierd kernel / device driver bug o kern/39388 scsi ncr/sym drivers fail with 53c810 and more than 256MB m o kern/38828 scsi [dpt] [request] DPT PM2012B/90 doesn't work o kern/35234 scsi World access to /dev/pass? (for scanner) requires acce 39 problems total. From owner-freebsd-scsi@FreeBSD.ORG Mon Jan 18 11:13:44 2010 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A30310656A4 for ; Mon, 18 Jan 2010 11:13:44 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id 023CE8FC1A for ; Mon, 18 Jan 2010 11:13:43 +0000 (UTC) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1NWpYD-0007MV-Ps; Mon, 18 Jan 2010 13:13:41 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Karl Pielorz In-reply-to: <071FB5A0A2DC2EFA265933DE@HPQuadro64.dmpriest.net.uk> References: <071FB5A0A2DC2EFA265933DE@HPQuadro64.dmpriest.net.uk> Comments: In-reply-to Karl Pielorz message dated "Wed, 13 Jan 2010 10:58:26 +0000." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 18 Jan 2010 13:13:41 +0200 From: Daniel Braniss Message-ID: Cc: freebsd-scsi@freebsd.org Subject: Re: Starting out in iSCSI - recovery questions / options from target vanishing... X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jan 2010 11:13:44 -0000 > > Hi, > > I've been looking at setting up iSCSI on my home network. I have a windows > machine, which I've installed iSCSI target software on, and set it up. > > I'm using FreeBSD 7.2-STABLE - and I've setup iSCSI as an initiator on that. > > At the end of the day, I get the following logged when I've finished > setting it up: > > " > Jan 12 20:25:30 test kernel: da0 at iscsi0 bus 0 target 0 lun 0 > Jan 12 20:25:30 test kernel: da0: Fixed Direct Access > SCSI-4 device > Jan 12 20:26:21 test kernel: GEOM_LABEL: Label for provider da0s1 is > ufsid/4b4cdaed5db746bf. > " > > I can label, partition, format and mount da0 - and everything works fine, > which looks pretty successful. > > Until I 'break the connection' - either by restarting the windows iSCSI > target, or pulling the network cable. > > Whatever was in progress on da0 expectedly freezes. However, when the > connection is available again - nothing seems to re-attach, continue, or > error. > > I'd guess I may be being optimistic if I expected it to re-attach, and > continue without error - but is there any way of either avoiding the freeze > (or raining in the timeout) - or forcing things to try to reconnect? > > I don't mind getting errors, or trashing the iSCSI drive (it's only used > for shifting backups on to large drives on the windows machine) - but as it > is, the only way to recover is to restart the FreeBSD box. > > If someone can point me in the right direction, or explain what's "meant to > happen" that'd be great, > > Thanks, > > -Karl Hi karl, what version of iscsi_initiator are you using? if it's not 2.2.x then get it from: ftp://ftp.cs.huji.ac.il/users/danny/freebsd/iscsi-2.2.3.tar.gz From owner-freebsd-scsi@FreeBSD.ORG Mon Jan 18 13:34:34 2010 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7A42106566C for ; Mon, 18 Jan 2010 13:34:34 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 86AC48FC12 for ; Mon, 18 Jan 2010 13:34:34 +0000 (UTC) Received: from HPQuadro64.dmpriest.net.uk (HPQuadro64.dmpriest.net.uk [62.13.130.30]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/Kp) with ESMTP id o0IDXnKk014977 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Mon, 18 Jan 2010 13:33:50 GMT Date: Mon, 18 Jan 2010 13:33:41 +0000 From: Karl Pielorz To: Daniel Braniss Message-ID: <0ACD40B5B0E066D23EE36CA2@HPQuadro64.dmpriest.net.uk> In-Reply-To: References: <071FB5A0A2DC2EFA265933DE@HPQuadro64.dmpriest.net.uk> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: freebsd-scsi@freebsd.org Subject: Re: Starting out in iSCSI - recovery questions / options from target vanishing... X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jan 2010 13:34:35 -0000 --On 18 January 2010 13:13 +0200 Daniel Braniss wrote: > Hi karl, > what version of iscsi_initiator are you using? > if it's not 2.2.x then get it from: > ftp://ftp.cs.huji.ac.il/users/danny/freebsd/iscsi-2.2.3.tar.gz Hi, Thanks for the reply :) It's whatever version was in 7.2-STABLE as of around June (looking at that, I need to update lol - csup now running). Having messed around with it for some time now - I've found: - Setting it up works, and it actually 'works' so long as everything is accessible. - If the remote target dies - it does in fact appear to reconnect at what seems 30 second intervals (and, given time [sometimes minutes] will recover, if the target is available 'as it was' before) - If you cannot get the original target back, your pretty much doomed. The device on the FreeBSD side (e.g. da0) will hang on any I/O, and it never seems to give up. You can't stop iscontrol, nor kill it - nor can I find any way of telling the system to just 'give up' and lose the data. After varying amounts of time the system slowly grinds to a halt (probably as other processes get stacked up on the vacant da0 device), and you have to reboot the system. Attempting to kld_unload the iscsi_initiator module leads to a far faster reboot (typically a panic). Would the 2.2.3 files above change any of this? - Is there meant to be a 'clean break' or something you can do to forcibly detach missing iscsi targets from the system (so recover it) - losing any data not written? - preferably without rebooting the whole server? -Karl From owner-freebsd-scsi@FreeBSD.ORG Wed Jan 20 15:50:26 2010 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A8D561065676 for ; Wed, 20 Jan 2010 15:50:26 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 336038FC1C for ; Wed, 20 Jan 2010 15:50:25 +0000 (UTC) Received: from elsa.codelab.cz (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id CF15A19E046; Wed, 20 Jan 2010 16:50:24 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 7F36E19E023; Wed, 20 Jan 2010 16:50:22 +0100 (CET) Message-ID: <4B57263E.7040809@quip.cz> Date: Wed, 20 Jan 2010 16:50:22 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.7) Gecko/20100104 SeaMonkey/2.0.2 MIME-Version: 1.0 To: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: performance tuning of iSCSI and Dell MD3000i X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Jan 2010 15:50:26 -0000 Hi, [1] I am using iSCSI Initiator 2.2.3 (on 7-STABLE amd64) and Dell MD3000i as storage for server distributing downloads of large files (10-200MB) through Lighttpd daemon. Is there any tips for performance tuning? My iscsi.conf is: styx_storage_1 { initiatorname = iqn.2005-01.il.ac.huji.cs::dust.example.com TargetName = iqn.1984-05.com.dell:powervault.md3000i.60026b900042587b000000004ae58efc TargetAddress = 192.168.130.101:3260,1 tags = 64 } Lighttpd is serving about 200-300 client connections at speed 220Mb/s it is about 215Mb/s on iSCSI network interface to MD3000i. (iostat showed 100% busy of da0) Is it really maximum achievable speed on this type of iSCSI storage? (made of 4 SATA drives in RAID5) Or can it be better with some tuned settings? Have anyone experiences with MD3000i - if I add more disk drives in to RAID5, will it help? How much? [2] Is somewhere final rc script for iscsi initiator which can be used to mount filesystem from iSCSI target at boot time by standard rc procedure? Why is it not included in the base system if iscsi_initiator is there for a long time? [3] And one other thing, I got following error logged in messages after 2 days of running some performance tests: Jan 19 05:36:50 dust kernel: >>> so_recv: have a problem, error=0 Jan 19 05:36:50 dust kernel: isc_in: sp->soc=0xffffff0028f14000 Jan 19 05:36:50 dust kernel: 0] isc_in: terminated, flags=39f so_count=2 so_state=2000 error=2080 proc=0xffffff0005ab1910 Jan 19 05:36:50 dust kernel: 0] isc_in: pid=963750 s]i gin_alfeudl(l3f0e)atu Jan 19 05:36:50 dust kernel: r0e]: ifslca_gi=n0: Jan 19 05:36:50 dust kernel: dropped ISC_CON_RUNNING Jan 19 05:36:50 dust iscontrol[96375]: trapped signal 30 Jan 19 05:36:52 dust kernel: 0] i_send: len=492 ahs_len=0 ds_len=441 buf=444@0xffffff0007068000 Jan 19 05:36:52 dust kernel: 0] i_setopt: maxRecvDataSegmentLength=65536 Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.maXmitDataSegmentLength=65536 Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.maxBurstLength=131072 Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.targetAddress='192.168.130.101' Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.targetName='iqn.1984-05.com.dell:powervault.md3000i.60026b900042587b000000004ae58efc' Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.initiatorName='iqn.2005-01.il.ac.huji.cs::dust.example.com' Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.headerDigest='None' Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.dataDigest='None' Jan 19 05:36:52 dust kernel: 0] i_fullfeature: flag=2 Jan 19 05:36:52 dust kernel: 0] ism_restart: restart ... Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30e4 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30e5 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30e7 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30ea Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30eb Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30f3 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30f4 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3111 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3117 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3118 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3119 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a311a Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a311d Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a311e Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3120 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3121 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3123 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3124 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3126 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3127 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3128 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3129 Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a312a Jan 19 05:36:52 dust kernel: 0] ism_restart: restarted sn.cmd=0x30a312b lastcmd=0x30a30e4 Then transfer of files continued at slightly lower speed than before. Miroslav Lachman From owner-freebsd-scsi@FreeBSD.ORG Thu Jan 21 08:04:42 2010 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40FE5106566B for ; Thu, 21 Jan 2010 08:04:42 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id EF64D8FC0A for ; Thu, 21 Jan 2010 08:04:41 +0000 (UTC) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1NXs1v-0008L0-Nx; Thu, 21 Jan 2010 10:04:39 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Miroslav Lachman <000.fbsd@quip.cz> In-reply-to: <4B57263E.7040809@quip.cz> References: <4B57263E.7040809@quip.cz> Comments: In-reply-to Miroslav Lachman <000.fbsd@quip.cz> message dated "Wed, 20 Jan 2010 16:50:22 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 21 Jan 2010 10:04:39 +0200 From: Daniel Braniss Message-ID: Cc: freebsd-scsi@freebsd.org Subject: Re: performance tuning of iSCSI and Dell MD3000i X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Jan 2010 08:04:42 -0000 Hi Miroslav, > Hi, > > [1] > I am using iSCSI Initiator 2.2.3 (on 7-STABLE amd64) and Dell MD3000i as > storage for server distributing downloads of large files (10-200MB) > through Lighttpd daemon. > Is there any tips for performance tuning? > > My iscsi.conf is: > styx_storage_1 { > initiatorname = iqn.2005-01.il.ac.huji.cs::dust.example.com > TargetName = > iqn.1984-05.com.dell:powervault.md3000i.60026b900042587b000000004ae58efc > TargetAddress = 192.168.130.101:3260,1 > tags = 64 > } > > Lighttpd is serving about 200-300 client connections at speed 220Mb/s it > is about 215Mb/s on iSCSI network interface to MD3000i. (iostat showed > 100% busy of da0) > Is it really maximum achievable speed on this type of iSCSI storage? > (made of 4 SATA drives in RAID5) > Or can it be better with some tuned settings? > the limiting factor is mainly the network, I have no experience with the MD3000i, we have MD3000 and they are very fast, unfourtunately, I can't come up with real statistical values, since the raid+zfs are doing very aggresive caching and my benchmarks are useless. In any case, the values you are getting seem a bit low, I would first measure the speed on the server, if your net is GB, then you should expect something around 80-100 MB/s. > Have anyone experiences with MD3000i - if I add more disk drives in to > RAID5, will it help? How much? > > > [2] > Is somewhere final rc script for iscsi initiator which can be used to > mount filesystem from iSCSI target at boot time by standard rc > procedure? Why is it not included in the base system if iscsi_initiator > is there for a long time? I guess I should work on that :-( > > > [3] > And one other thing, I got following error logged in messages after 2 > days of running some performance tests: can you check the M3000i logs at that time? or else let me know what tests you are doing, so that i can try it here? > > Jan 19 05:36:50 dust kernel: >>> so_recv: have a problem, error=0 > Jan 19 05:36:50 dust kernel: isc_in: sp->soc=0xffffff0028f14000 > Jan 19 05:36:50 dust kernel: 0] isc_in: terminated, flags=39f so_count=2 > so_state=2000 error=2080 proc=0xffffff0005ab1910 > Jan 19 05:36:50 dust kernel: 0] isc_in: pid=963750 s]i gin_alfeudl(l3f0e)atu > Jan 19 05:36:50 dust kernel: r0e]: ifslca_gi=n0: > Jan 19 05:36:50 dust kernel: dropped ISC_CON_RUNNING > Jan 19 05:36:50 dust iscontrol[96375]: trapped signal 30 > Jan 19 05:36:52 dust kernel: 0] i_send: len=492 ahs_len=0 ds_len=441 > buf=444@0xffffff0007068000 > Jan 19 05:36:52 dust kernel: 0] i_setopt: maxRecvDataSegmentLength=65536 > Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.maXmitDataSegmentLength=65536 > Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.maxBurstLength=131072 > Jan 19 05:36:52 dust kernel: 0] i_setopt: > opt.targetAddress='192.168.130.101' > Jan 19 05:36:52 dust kernel: 0] i_setopt: > opt.targetName='iqn.1984-05.com.dell:powervault.md3000i.60026b900042587b000000004ae58efc' > Jan 19 05:36:52 dust kernel: 0] i_setopt: > opt.initiatorName='iqn.2005-01.il.ac.huji.cs::dust.example.com' > Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.headerDigest='None' > Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.dataDigest='None' > Jan 19 05:36:52 dust kernel: 0] i_fullfeature: flag=2 > Jan 19 05:36:52 dust kernel: 0] ism_restart: restart ... > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30e4 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30e5 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30e7 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30ea > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30eb > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30f3 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30f4 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3111 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3117 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3118 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3119 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a311a > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a311d > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a311e > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3120 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3121 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3123 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3124 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3126 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3127 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3128 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a3129 > Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a312a > Jan 19 05:36:52 dust kernel: 0] ism_restart: restarted sn.cmd=0x30a312b > lastcmd=0x30a30e4 > > Then transfer of files continued at slightly lower speed than before. it seems that there was a network glitch, but it recovered. > > Miroslav Lachman From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 22 02:05:27 2010 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF4221065670 for ; Fri, 22 Jan 2010 02:05:27 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 6C0AD8FC17 for ; Fri, 22 Jan 2010 02:05:27 +0000 (UTC) Received: from elsa.codelab.cz (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 0478B19E047; Fri, 22 Jan 2010 03:05:26 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 4167919E046; Fri, 22 Jan 2010 03:05:23 +0100 (CET) Message-ID: <4B5907E2.2060206@quip.cz> Date: Fri, 22 Jan 2010 03:05:22 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.7) Gecko/20100104 SeaMonkey/2.0.2 MIME-Version: 1.0 To: Daniel Braniss References: <4B57263E.7040809@quip.cz> In-Reply-To: Content-Type: multipart/mixed; boundary="------------010604060208080102020106" Cc: freebsd-scsi@freebsd.org Subject: Re: performance tuning of iSCSI and Dell MD3000i X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jan 2010 02:05:28 -0000 This is a multi-part message in MIME format. --------------010604060208080102020106 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Daniel Braniss wrote: > Hi Miroslav, >> Hi, >> >> [1] >> I am using iSCSI Initiator 2.2.3 (on 7-STABLE amd64) and Dell MD3000i as >> storage for server distributing downloads of large files (10-200MB) >> through Lighttpd daemon. >> Is there any tips for performance tuning? >> >> My iscsi.conf is: >> styx_storage_1 { >> initiatorname = iqn.2005-01.il.ac.huji.cs::dust.example.com >> TargetName = >> iqn.1984-05.com.dell:powervault.md3000i.60026b900042587b000000004ae58efc >> TargetAddress = 192.168.130.101:3260,1 >> tags = 64 >> } >> >> Lighttpd is serving about 200-300 client connections at speed 220Mb/s it >> is about 215Mb/s on iSCSI network interface to MD3000i. (iostat showed >> 100% busy of da0) >> Is it really maximum achievable speed on this type of iSCSI storage? >> (made of 4 SATA drives in RAID5) >> Or can it be better with some tuned settings? >> > > the limiting factor is mainly the network, I have no experience with the > MD3000i, > we have MD3000 and they are very fast, unfourtunately, I can't come up > with real statistical values, since the raid+zfs are doing very aggresive > caching and my benchmarks are useless. > In any case, the values you are getting seem a bit low, I would first measure > the speed on the server, if your net is GB, then you should expect something > around 80-100 MB/s. The network is gigabit. Internet connection is through 3Com 2928 switch, connection between server and MD3000i is crosslink with Jumbo Frames (9k) enabled on server and storage array too. Internet NIC: bce0: flags=8843 metric 0 mtu 1500 options=1bb ether 00:22:11:64:0d:ff inet AA.BB.CC.DD netmask 0xffffff80 broadcast AA.BB.CC.EE media: Ethernet autoselect (1000baseTX ) status: active iSCSI NIC: bce1: flags=8843 metric 0 mtu 9000 options=1bb ether 00:22:11:64:0e:01 inet 192.168.130.2 netmask 0xffffff00 broadcast 192.168.130.255 media: Ethernet autoselect (1000baseTX ) status: active I can achieve higher speed with sequential read test by dd command. With dd I can read about 680Mbps (85MB/s), but I think the high concurrency workload is more stressing disk than network. Is there some interesting sysctl which I can post you / tune network performance? >> Have anyone experiences with MD3000i - if I add more disk drives in to >> RAID5, will it help? How much? >> >> >> [2] >> Is somewhere final rc script for iscsi initiator which can be used to >> mount filesystem from iSCSI target at boot time by standard rc >> procedure? Why is it not included in the base system if iscsi_initiator >> is there for a long time? > > I guess I should work on that :-( Is the attached version the more recent one? I found it in the list http://lists.freebsd.org/pipermail/freebsd-stable/2008-July/043611.html Is it really necessary to run it after normal mount stage, so iSCSI devices must be defined in fstab.iscsi and fscked & mounted by rc.d/iscsi instead of standard system procedures? (I am just asking as I am not so skilled in rc.d scripting) Did you talk about it with some freebsd-rc@ guys? >> [3] >> And one other thing, I got following error logged in messages after 2 >> days of running some performance tests: > can you check the M3000i logs at that time? or else let me know > what tests you are doing, so that i can try it here? There is nothing in the Event Log of MD3000i in the time of this error. The test was relatively simple. There is Lighttpd runing on the server (with server.max-worker = 4 to have higher throughput) and I run http_load (www/http_load) from another machine connected to the same switch. http_load was ran in 4 instances with the following settings: http_load -parallel 70 -seconds 43200 download.url So it generates 280 parallel connections on to Lighttpd with URLs from download.url file. There is 24376 of lines (URLs to files) in this file. >> Jan 19 05:36:50 dust kernel:>>> so_recv: have a problem, error=0 >> Jan 19 05:36:50 dust kernel: isc_in: sp->soc=0xffffff0028f14000 >> Jan 19 05:36:50 dust kernel: 0] isc_in: terminated, flags=39f so_count=2 >> so_state=2000 error=2080 proc=0xffffff0005ab1910 >> Jan 19 05:36:50 dust kernel: 0] isc_in: pid=963750 s]i gin_alfeudl(l3f0e)atu >> Jan 19 05:36:50 dust kernel: r0e]: ifslca_gi=n0: >> Jan 19 05:36:50 dust kernel: dropped ISC_CON_RUNNING >> Jan 19 05:36:50 dust iscontrol[96375]: trapped signal 30 >> Jan 19 05:36:52 dust kernel: 0] i_send: len=492 ahs_len=0 ds_len=441 >> buf=444@0xffffff0007068000 >> Jan 19 05:36:52 dust kernel: 0] i_setopt: maxRecvDataSegmentLength=65536 >> Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.maXmitDataSegmentLength=65536 >> Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.maxBurstLength=131072 >> Jan 19 05:36:52 dust kernel: 0] i_setopt: >> opt.targetAddress='192.168.130.101' >> Jan 19 05:36:52 dust kernel: 0] i_setopt: >> opt.targetName='iqn.1984-05.com.dell:powervault.md3000i.60026b900042587b000000004ae58efc' >> Jan 19 05:36:52 dust kernel: 0] i_setopt: >> opt.initiatorName='iqn.2005-01.il.ac.huji.cs::dust.example.com' >> Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.headerDigest='None' >> Jan 19 05:36:52 dust kernel: 0] i_setopt: opt.dataDigest='None' >> Jan 19 05:36:52 dust kernel: 0] i_fullfeature: flag=2 >> Jan 19 05:36:52 dust kernel: 0] ism_restart: restart ... >> Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30e4 >> Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a30e5 [...] >> Jan 19 05:36:52 dust kernel: 0] iscsi_requeue: last=30a30e4 n=30a312a >> Jan 19 05:36:52 dust kernel: 0] ism_restart: restarted sn.cmd=0x30a312b >> lastcmd=0x30a30e4 >> >> Then transfer of files continued at slightly lower speed than before. > > it seems that there was a network glitch, but it recovered. It is connected with direct crosslink cable and there is not "link state changed to DOWN / UP" message for the NIC so I don't know what network problem can happened. Miroslav Lachman --------------010604060208080102020106 Content-Type: text/plain; name="rc.d_iscsi.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="rc.d_iscsi.txt" #!/bin/sh # PROVIDE: iscsi # REQUIRE: NETWORKING # BEFORE: DAEMON # KEYWORD: nojail shutdown # # Add the following lines to /etc/rc.conf to enable iscsi: # # iscsi_enable="YES" # iscsi_fstab="/etc/fstab.iscsi" . /etc/rc.subr . /cs/share/etc/rc.subr name=iscsi rcvar=`set_rcvar` command=/sbin/iscontrol iscsi_enable=${iscsi_enable:-"NO"} iscsi_fstab=${iscsi_fstab:-"/etc/fstab.iscsi"} iscsi_exports=${iscsi_exports:-"/etc/exports.iscsi"} iscsi_debug=${iscsi_debug:-0} start_cmd="iscsi_start" faststop_cmp="iscsi_stop" stop_cmd="iscsi_stop" start_precmd="iscontrol_precmd" iscontrol_prog=${iscontrol_prog:-"iscontrol"} iscontrol_log=${iscontrol_log:-"/var/log/$iscontrol_prog"} iscontrol_syslog=${iscontrol_syslog:-"644 3 100 * JC"} iscontrol_precmd() { setup_syslog "$iscontrol_prog" "$iscontrol_log" "$iscontrol_syslog" } iscsi_wait() { dev=$1 trap "echo 'wait loop cancelled'; exit 1" 2 count=0 while true; do if [ -c $dev ]; then break; fi if [ $count -eq 0 ]; then echo -n Waiting for ${dev}': ' fi count=$((${count} + 1)) if [ $count -eq 6 ]; then echo " Failed for dev=$dev" return 0 break fi echo -n '.' sleep 5; done echo "$dev ok." return 1 } iscsi_start() { # # load needed modules for m in iscsi_initiator geom_label; do kldstat -qm $m || kldload $m done sysctl debug.iscsi_initiator=$iscsi_debug # # start iscontrol for each target if [ -n "${iscsi_targets}" ]; then for target in ${iscsi_targets}; do ${command} ${rc_flags} -n ${target} done fi if [ -f "${iscsi_fstab}" ]; then while read spec file type opt t1 t2 do case ${spec} in \#*|'') ;; *) if iscsi_wait ${spec}; then break; fi echo type=$type spec=$spec file=$file fsck -p ${spec} && mkdir -p ${file} && mount ${spec} ${file} chmod 755 ${file} ;; esac done < ${iscsi_fstab} fi if [ -f "${iscsi_exports}" ]; then cat ${iscsi_exports} >> /etc/exports #/etc/rc.d/mountd reload kill -1 `cat /var/run/mountd.pid` fi } iscsi_stop() { echo 'iscsi stopping' while read spec file type opt t1 t2 do case ${spec} in \#*|'') ;; *) echo iscsi: umount $spec umount -fv $spec ;; esac done < ${iscsi_fstab} } load_rc_config $name run_rc_command "$1" --------------010604060208080102020106-- From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 22 03:56:51 2010 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E1871065670 for ; Fri, 22 Jan 2010 03:56:51 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (email.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 107058FC08 for ; Fri, 22 Jan 2010 03:56:50 +0000 (UTC) Received: by email.octopus.com.au (Postfix, from userid 1002) id 5B5265CB8E7; Fri, 22 Jan 2010 14:32:43 +1100 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: * X-Spam-Status: No, score=1.9 required=10.0 tests=ALL_TRUSTED, FH_DATE_PAST_20XX autolearn=no version=3.2.3 Received: from [10.1.50.60] (ppp121-44-233-216.lns20.syd7.internode.on.net [121.44.233.216]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id 5F0205CB8E9; Fri, 22 Jan 2010 14:32:37 +1100 (EST) Message-ID: <4B5921B2.1030906@modulus.org> Date: Fri, 22 Jan 2010 14:55:30 +1100 From: Andrew Snow User-Agent: Thunderbird 2.0.0.14 (X11/20080523) MIME-Version: 1.0 To: Miroslav Lachman <000.fbsd@quip.cz>, freebsd-scsi@freebsd.org References: <4B57263E.7040809@quip.cz> <4B5907E2.2060206@quip.cz> In-Reply-To: <4B5907E2.2060206@quip.cz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: performance tuning of iSCSI and Dell MD3000i X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jan 2010 03:56:51 -0000 Hi Miroslav, have you tried increasing your TCP/IP Window settings at both ends? In freebsd sysctl.conf: kern.ipc.maxsockbuf=1048576 net.inet.tcp.recvspace=262144 net.inet.tcp.sendspace=262144 I never had trouble hitting line rate of 1Gbps iSCSI with freebsd, but the biggest problem was not speed but what happens if the device times out or drops out, it causes FreeBSD to completely crash instead of degrading gracefully. - Andrew From owner-freebsd-scsi@FreeBSD.ORG Fri Jan 22 11:12:49 2010 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B6FD1065670 for ; Fri, 22 Jan 2010 11:12:49 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 0921D8FC0C for ; Fri, 22 Jan 2010 11:12:48 +0000 (UTC) Received: from elsa.codelab.cz (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id E92E819E023; Fri, 22 Jan 2010 12:12:46 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id DC6F819E019; Fri, 22 Jan 2010 12:12:44 +0100 (CET) Message-ID: <4B59882C.7030506@quip.cz> Date: Fri, 22 Jan 2010 12:12:44 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.7) Gecko/20100104 SeaMonkey/2.0.2 MIME-Version: 1.0 To: Andrew Snow References: <4B57263E.7040809@quip.cz> <4B5907E2.2060206@quip.cz> <4B5921B2.1030906@modulus.org> In-Reply-To: <4B5921B2.1030906@modulus.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-scsi@freebsd.org Subject: Re: performance tuning of iSCSI and Dell MD3000i X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jan 2010 11:12:49 -0000 Andrew Snow wrote: > > Hi Miroslav, > > have you tried increasing your TCP/IP Window settings at both ends? > > In freebsd sysctl.conf: > > kern.ipc.maxsockbuf=1048576 > net.inet.tcp.recvspace=262144 > net.inet.tcp.sendspace=262144 > > > I never had trouble hitting line rate of 1Gbps iSCSI with freebsd, but > the biggest problem was not speed but what happens if the device times > out or drops out, it causes FreeBSD to completely crash instead of > degrading gracefully. My current settings is: kern.ipc.maxsockbuf: 262144 net.inet.tcp.recvspace: 65536 net.inet.tcp.sendspace: 32768 net.inet.tcp.recvbuf_max: 262144 net.inet.tcp.recvbuf_inc: 16384 net.inet.tcp.recvbuf_auto: 1 net.inet.tcp.sendbuf_max: 262144 net.inet.tcp.sendbuf_inc: 8192 net.inet.tcp.sendbuf_auto: 1 I will try your recommended values and post results later this day. (but I don't know how can I change it on MD3000i side) Thank you for your reply Miroslav Lachman From owner-freebsd-scsi@FreeBSD.ORG Sat Jan 23 19:06:15 2010 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B220106566C for ; Sat, 23 Jan 2010 19:06:15 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id B1B528FC1D for ; Sat, 23 Jan 2010 19:06:14 +0000 (UTC) Received: from elsa.codelab.cz (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id E89FA19E047; Sat, 23 Jan 2010 20:06:12 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id CDA2B19E046; Sat, 23 Jan 2010 20:06:10 +0100 (CET) Message-ID: <4B5B48A3.6070306@quip.cz> Date: Sat, 23 Jan 2010 20:06:11 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.7) Gecko/20100104 SeaMonkey/2.0.2 MIME-Version: 1.0 To: Andrew Snow References: <4B57263E.7040809@quip.cz> <4B5907E2.2060206@quip.cz> <4B5921B2.1030906@modulus.org> In-Reply-To: <4B5921B2.1030906@modulus.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-scsi@freebsd.org Subject: Re: performance tuning of iSCSI and Dell MD3000i X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Jan 2010 19:06:15 -0000 Andrew Snow wrote: > > Hi Miroslav, > > have you tried increasing your TCP/IP Window settings at both ends? > > In freebsd sysctl.conf: > > kern.ipc.maxsockbuf=1048576 > net.inet.tcp.recvspace=262144 > net.inet.tcp.sendspace=262144 I tried the same test with above settings on the server and on the client (http_load) but the resulting speed is the same as before. Miroslav Lachman