From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 16:57:05 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3F541807 for ; Wed, 12 Dec 2012 16:57:05 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id A9D5A8FC0A for ; Wed, 12 Dec 2012 16:57:04 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id j13so859441lah.13 for ; Wed, 12 Dec 2012 08:57:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=K1rIU7iP6L4+Amh0a4OEVJeG6a05QAt/zFx2cUxT918=; b=ZLJkuA8sYN3OWa4r5yR98h8qMEOtDxaUkmEbHnXT2JJ6sOCDYAt+YGx7fyK3VKTHff Hb1m4ROKXoRgTI9EPxSrsdA74NdZKnz9RwaXpL9K8VXxIn2HdKfhoXtYFG6KIjnZYDQu PwkN7OVyxn4+uaKtaQhfx3nIPmBek30WBBzyWMttV59om/A0mZh/n+SscEIBDxuvfDFT fsVwVyK4S45LoK1acMGUCxlcSt/JQjtmqA89kdk2xGdIVPib7DGzyNBbbKNaMRalQ5kt /NqkZDukKsQM/7jLgudxw8zRDlZvwxl1d0KxPF7qxP/N5zAQ3zfYzaW/7BRGUAO+ucq7 9tTw== MIME-Version: 1.0 Received: by 10.152.148.129 with SMTP id ts1mr1741387lab.19.1355331423533; Wed, 12 Dec 2012 08:57:03 -0800 (PST) Received: by 10.112.61.33 with HTTP; Wed, 12 Dec 2012 08:57:03 -0800 (PST) Date: Wed, 12 Dec 2012 11:57:03 -0500 Message-ID: Subject: iSCSI vs. SMB with ZFS. From: Zaphod Beeblebrox To: FreeBSD Hackers Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 16:57:05 -0000 So... I have two machines. My Fileserver is a core-2-duo machine with FreeBSD-9.1-ish ZFS, istgt and samba 3.6. My workstation is windows 7 on an i7. Both have GigE and are connected directly via a managed switch with jumbo packets (specifically 9016) enabled. Both are using tagged vlan packets to the switch (if that matters at all). Some time ago, I created a 2T iSCSI disk on ZFS to serve the "Steam" directory (games) on my C drive as it was growing rather large. I've been quite happy with this. The performance of the iSCSI disk is about the same as the local disk for some operations --- faster for some, slower for others. The workstation has 12G of memory and it's my perception that iSCSI is heavily cached and that this enhances it's performance. The second launch of a game ... or the second switch into an area (ie: loading a specific piece of geometry again) is very fast. But this is imperfect. The iSCSI disk reserves all of it's space and the files on the disk are only accessible to the computer that mounts it. The most recent Steam update supported an easy way to put steam folders on other disks and partitions. I created another Steam folder on an SMB share from the same server and proceeded to move one of my games there. The performance on the SMB share is abysmal compared to the performance on the iSCSI share. At the very least, there seems to be little benifit to launching the same application twice --- which is most likely windows fault. I haven't done any major amount of tuning on the SMB share lately, but the last time I cared, it was setup reasonably... with TCPNODELAY and whatnot. I also notice that my copy of smbd runs with 1 thread (according to top) rather than the 11 threads that istgt uses. Does this breakdown of performance square with other's experiences? Will SMB always have significantly less performance than iSCSI coming from ZFS?