From owner-freebsd-fs@freebsd.org Sun Jan 24 20:51:27 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F4647654 for ; Sun, 24 Jan 2016 20:51:27 +0000 (UTC) (envelope-from zebekias@gmail.com) Received: from mail-qg0-x22d.google.com (mail-qg0-x22d.google.com [IPv6:2607:f8b0:400d:c04::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C913198B for ; Sun, 24 Jan 2016 20:51:27 +0000 (UTC) (envelope-from zebekias@gmail.com) Received: by mail-qg0-x22d.google.com with SMTP id b35so95353020qge.0 for ; Sun, 24 Jan 2016 12:51:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:mime-version:subject :message-id:date:to; bh=K6lg8T5ZfwGrrn2lAThDnkKj1cxoPjTouojFFz3ZzRs=; b=eqwNlZlWSBeYWFd0I/dQ9ZjWNXaotOW4RuN/yZd606vzD7TDxDaGfJEr11092IYaUZ i6BVgpTvFf/iI2GGBJ8AdlbGjXtmOb1FExxcbCuWPScDtZ57OiCVSjo0+A0mcW73cB9P 6LLbrOSKpWLZ5eapqgunyxyggfKW7pVyoMDXdoAoX3KDQ9VL/q71RFLxiK1NFM8rMqhP FJZtXw7KxMZZNgcsdSmpm6r9gwIz8KhYDJB+X4qJxl7v/qZM1VxdWGnK3zqmfvVuDynm nam7fCXcsQfJZNXO+eSAe/FHIwzKUd2DL4XRuETiX10pZ2bEm080n6V+SGKi4q623hnG F0sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-type:content-transfer-encoding :mime-version:subject:message-id:date:to; bh=K6lg8T5ZfwGrrn2lAThDnkKj1cxoPjTouojFFz3ZzRs=; b=SjMWhC9MaHSdtGF8NhcZbapsKoURKf3telSwCxEpyoRupjUt5lvpT3xR8zEjV885hD hG1y7bYb75WQvHU3Nt2DAWaXOuu1Gedyd17CnYDjfURT6bnZPiGht76QzYAcYkCI7Rgs kifeZe6ecn1iL4f2c1C1N0q7lZ2qCcHceNp6Gwfg43UFs3YbBSe7MUdIp4ZnkIBO0Y54 /Jhk1EhPTpzd6jalOPPbMCLYCkHX2DEn7xKBADne/CS6RbQZRbFVFo/l+ypMdt6yrA+2 pErCb9mg2LsYUFMQAKjjDtnh6BMmUZUtTblhfocthxA/BYYlBd6R/OdfS+f4OfFExNG6 rp+Q== X-Gm-Message-State: AG10YOQFpjjud9f+z3u51O7pql2arJ7OsqfK2Pv5ENal6j7kMwAIvmOVRDS8+KxHlaLHkA== X-Received: by 10.140.31.197 with SMTP id f63mr17118368qgf.12.1453668686461; Sun, 24 Jan 2016 12:51:26 -0800 (PST) Received: from [192.168.1.101] (static-71-246-239-3.washdc.fios.verizon.net. [71.246.239.3]) by smtp.gmail.com with ESMTPSA id t187sm7419788qht.39.2016.01.24.12.51.24 for (version=TLSv1/SSLv3 cipher=OTHER); Sun, 24 Jan 2016 12:51:25 -0800 (PST) From: Kiriakos Georgiou Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) Subject: zfs - reboot after panic Message-Id: Date: Sun, 24 Jan 2016 15:51:24 -0500 To: freebsd-fs@freebsd.org X-Mailer: iPad Mail (13D15) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Jan 2016 20:51:27 -0000 Hello, I've been happily running a FreeBSD 10.1 fileserver (updated regularly via f= reebsd-update) with ZFS zvols served out via iscsi. Total RAM is 8G. For 2-= 3 months I regularly successfully run some heavy duty checksuming via unison= (it's kind of like rsync, but does bidirectional syncing). I recently upda= ted the OS to 10.1-RELEASE-p26, and soon after I started to consistently get= kernel panics when I run unison to sync certain files across my VMs: reboot after panic: I/O to pool 'zdata' appears to be hung on vdev guid ....= First I thought a bug must have been introduced, but after some googling I s= et my vfs.zfs.arc_max to 4G (50% of my ram) and hammered zfs via a massive u= nison sync. I monitored memory via top. ARC grew quickly to 4096M and stay= ed there. Free memory would slowly drop all the way to just under 1G but th= en it'd suddenly jump to over 2G, then slowly go down to 1G and jump to 2G a= gain so on. I suspect it was a RAM starvation issue that is now resolved by= tuning the ARC max. I've no idea why it started happening. I just wanted t= o share it with you. Does the "fix" make sense? Thanks, Kiriakos=