From owner-freebsd-stable@freebsd.org  Mon Nov 21 17:47:32 2016
Return-Path: <owner-freebsd-stable@freebsd.org>
Delivered-To: freebsd-stable@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8932C4C01F
 for <freebsd-stable@mailman.ysv.freebsd.org>;
 Mon, 21 Nov 2016 17:47:32 +0000 (UTC)
 (envelope-from petefrench@ingresso.co.uk)
Received: from constantine.ingresso.co.uk
 (ingresso-1-pt.tunnel.tserv1.lon2.ipv6.he.net [IPv6:2001:470:1f1c:411::2])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 85A9E80F
 for <freebsd-stable@freebsd.org>; Mon, 21 Nov 2016 17:47:32 +0000 (UTC)
 (envelope-from petefrench@ingresso.co.uk)
Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6]
 helo=dilbert.ingresso.co.uk)
 by constantine.ingresso.co.uk with esmtps
 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.87 (FreeBSD))
 (envelope-from <petefrench@ingresso.co.uk>) id 1c8sgj-0007Ex-KY
 for freebsd-stable@freebsd.org; Mon, 21 Nov 2016 17:47:29 +0000
Received: from petefrench by dilbert.ingresso.co.uk with local (Exim 4.87
 (FreeBSD)) (envelope-from <petefrench@ingresso.co.uk>)
 id 1c8sgj-0003Q2-It
 for freebsd-stable@freebsd.org; Mon, 21 Nov 2016 17:47:29 +0000
To: freebsd-stable@freebsd.org
Subject: Help! two machines ran out of swap and corrupted their zpools!
Message-Id: <E1c8sgj-0003Q2-It@dilbert.ingresso.co.uk>
From: Pete French <petefrench@ingresso.co.uk>
Date: Mon, 21 Nov 2016 17:47:29 +0000
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-stable>, 
 <mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable/>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
 <mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 21 Nov 2016 17:47:32 -0000

So, I am off sick and my colleagues decided to load test our set of five
servers excesively. All ran out of swap. So far so irritating, but whats has
happened is that twoof them now will not boot, as it appears the ZFS pool
they are booting from has become corrupted.

One starts to boot, then crases importing the root pool. The other doenst
even get that far with gptzfsboot saying it can't find the pool to boot from!

Now I can recover these, but I am a bit worried, that it got like this at
all, as I havent ever seen ZFS corrupt a pool like this. Anyone got any insights,
or suggstions as to how to stop it happening again ?

We are swapping to a separate partition, not to the pool by theway.

-pete.