From owner-freebsd-fs@FreeBSD.ORG Wed Jun 27 09:15:11 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 25AE0106564A for ; Wed, 27 Jun 2012 09:15:11 +0000 (UTC) (envelope-from lserinol@gmail.com) Received: from mail-yw0-f44.google.com (mail-yw0-f44.google.com [209.85.213.44]) by mx1.freebsd.org (Postfix) with ESMTP id D67548FC14 for ; Wed, 27 Jun 2012 09:15:10 +0000 (UTC) Received: by yhq56 with SMTP id 56so810195yhq.17 for ; Wed, 27 Jun 2012 02:15:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=DTdF7PyaN3q6g339Toe4oHuBKYbVZPqEhYOibfjHEHQ=; b=eujHgMfUvBbOGHw21sANbbPRPekQUNUAr5VgZkY1xFWqZtXUv6omtCrVEpoNNF9k0K iio7DY1N/gPNZ5PhyO6Yq9s3HMR1quH7vzYNCoJVK4mQm3DMxMBFEX+UBxctPlddBjH7 oBDtsqbnndSXEhVy3H5VSynKNI6xfu3MxGFzbjDUnXBGgPSmjBQXXebjbX7yGOFCU/6f +TENDV7OsOrnZzuO8sqpNisTUfkMBo2jOZJlrfufOf8fe5xwoGlDygTdyBVTasNwOfdK FfNE0HemDeGdi30t3gM3svrjxXswymcrbHcKe4Z1ejHNdD0lvXBI5J6U+6w9Q8njrOnP Sw4A== MIME-Version: 1.0 Received: by 10.101.18.16 with SMTP id v16mr6679465ani.79.1340788510425; Wed, 27 Jun 2012 02:15:10 -0700 (PDT) Received: by 10.146.103.7 with HTTP; Wed, 27 Jun 2012 02:15:10 -0700 (PDT) Date: Wed, 27 Jun 2012 12:15:10 +0300 Message-ID: From: Levent Serinol To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: ZFS stalls on Heavy I/O X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jun 2012 09:15:11 -0000 Hi, Under heavy I/O load we face freeze problems on ZFS volumes on both Freebsd 9 Release and 10 Current versions. Machines are HP servers (64bit) with HP Smart array 6400 raid controllers (with battery units). Every da device is a hardware raid5 where each one includes 9x300GB 10K SCSI hard drivers. Main of I/O pattern happens on local system except some small NFS I/O from some other servers (NFS lookup/getattr/ etc.). These servers are mail servers (qmail) with small I/O patterns (64K Read/Write). Below you can find procstat output on freeze time. write_limit is set to 200MB because of the huge amount of txg_wait_opens observed before. Every process stops on D state I think due to txg queue and other 2 queues are full. Is there any suggestion to fix the problem ? btw inject_compress is the main process injecting emails to user inboxes (databases). Also, those machines were running without problems on Linux/XFS filesystem. For a while ago, we started migration from Linux to Freebsd http://pastebin.com/raw.php?i=ic3YepWQ