From owner-freebsd-stable@freebsd.org Tue Nov 24 14:31:17 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35946A3649A for ; Tue, 24 Nov 2015 14:31:17 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-wm0-x233.google.com (mail-wm0-x233.google.com [IPv6:2a00:1450:400c:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD03A1BF1 for ; Tue, 24 Nov 2015 14:31:16 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by wmuu63 with SMTP id u63so98931937wmu.0 for ; Tue, 24 Nov 2015 06:31:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lSdVAdOdNn5qPPqdmnjajVbUkKzBF5le5YmnFEeGY6k=; b=saGcVhQvZec7YgpOhCmfeiRhCYVJR8qR0knoSuh3hB3WDdVVlZnKTs0Y+MV1cMVRyW Z7mCO7rdQudG53+KdmXUvWiEVabhj2r1ym4w1HI5AsfbLO3E1Xy5dNRmqSGrTd78ml5f 4UMFI/Shd6sUhQlGmJQLXvXRF/rMATx/gcYPFpwjuYGlhed0JiOUOKO5irWygBjnZgGU YlbTwj1xiS4e77WeZgyUqJkroxgI4sCV5G4VDVMPVaVpKlgOT2/NO18sH4EOyS2MMSJO rI0xTuF9Kh0o62iniRIlo98sg9/0fuYA3T30hdlHm3sVmWI5ct4ZZYr/j80w832xAF9P WpuA== MIME-Version: 1.0 X-Received: by 10.28.195.10 with SMTP id t10mr21903938wmf.11.1448375475308; Tue, 24 Nov 2015 06:31:15 -0800 (PST) Received: by 10.194.192.33 with HTTP; Tue, 24 Nov 2015 06:31:15 -0800 (PST) In-Reply-To: References: Date: Tue, 24 Nov 2015 08:31:15 -0600 Message-ID: Subject: Re: ZFS - poor performance with "large" directories From: Adam Vande More To: Albert Cervin Cc: "freebsd-stable@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Nov 2015 14:31:17 -0000 On Tue, Nov 24, 2015 at 8:00 AM, Albert Cervin wrote: > Hi all, > > Please feel free to direct me to a list that is more suitable. > > We are trying to set up a fileserver solution for a web application that we > are building. This fileserver is running FreeBSD 10.2 and ZFS. Files are > written over CIFS with Samba running on the fileserver host. > > However, we are seeing en exponential decrease in performance to write to > the file server when the number of files in the directory grows (when it > goes up to ~6000 files it becomes unusable and the write time has gone from > a fraction of a second to ten seconds). > > We ran the same setup on a Linux machine with an ext4 file system which did > NOT suffer from this performance degradation. > I should hope not. ext4 vs zfs comparison isn't fair for either. > > Are these "holes" in write speed normal. Since this is the exact symptom we > are getting when the network writes start to be slow. > Totally normal. You'll want to reference: https://wiki.freebsd.org/ZFSTuningGuide In particular for that issue see: vfs.zfs.txg.timeout and tuning related to NFS. Performance is also heavily dependent on pool structure and io characteristics. For example, a pool of 3 2 disk mirrors is in general going to be much faster than 1 6 disk raidz2. -- Adam