From owner-freebsd-stable@freebsd.org Sun Jul 29 15:27:56 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5857F105966C for ; Sun, 29 Jul 2018 15:27:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com [IPv6:2a00:1450:4864:20::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AF56B8406A for ; Sun, 29 Jul 2018 15:27:55 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-x233.google.com with SMTP id j19-v6so8295472ljc.7 for ; Sun, 29 Jul 2018 08:27:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=0ZbBRHvw+ls75G/ZJVakwzAWOdA4iXQtzVGG06gffHo=; b=troW4VuLyLPYneUvW3eFstumh8MAlgTiJQPYMr+csc8lo5CyKcSDV3B0E3XaahzNgX oCnPF9BRTnsWvGNA8/CXuXwrcJxhU2GM4wjDf34c5TispAhHgDex/ugCJ93PaXZ4WFmM m2wtP5dlzI9EEGrYLZJ2AzdrSuiWaF/3LKpNi4/arqPJ32JUCEQJsqX4D4TSKOrsHxd8 3QBwTluTXvNfLva2JOGSmQx4qWReIUXYyv5BUUrAU92mNEe+00IyuMtwNY54QVBF5Q3O 977UGVARN9KlRtJxiZC3dHBqL2baDiXlayLPR6sGwQOhny76nScTHiRF3dfxcleae55/ 0dBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=0ZbBRHvw+ls75G/ZJVakwzAWOdA4iXQtzVGG06gffHo=; b=KRsysDunBDDwyYk68H6TESTmLc7PQTz/bddXWLNHKDWinlQ4DBeHODZV+F+5Gv9KZC badFeoYFMM+LZoZMWGcE+TMxgKesPZZ5rrJpX82DAdaEq1LPs/SAxP9Ifk5iXYXyLz9G dsjVndikVtF3+cvlT+GhrVbdvfQ/obxn9iRxSJD1tglAmT7D0NvpasSmGf3eza4lvFHK NWSrKmHPJeMzU9DbufxhUoXOeSaRogVgtAERQf4DOGgix/dYBpL5nB6O/7epc7X3zkd1 uRaqMM4iJ8s14zL1NB27x/CNOC8AQa42jxeRsZclwayq6G+YxelqcR/7V4V5cAXsJ/0n A1ww== X-Gm-Message-State: AOUpUlF/GcCUVIUYuKnRXml6M8PVJqWRQp8QWS1zogqEJt4L54TrGbtO wBJ6EXzxDIv66BoUzM+ceDe2ZK14IsJ4sFJad30= X-Google-Smtp-Source: AAOMgpciorSMuKaalZicwrg4cLPAU7uC1JuvE6buqTNfnCX5IrIZ9pGroJOZANSWVJNbzuAlr5GzCaleokqNg1W74/U= X-Received: by 2002:a2e:1b83:: with SMTP id c3-v6mr9928992ljf.0.1532878073344; Sun, 29 Jul 2018 08:27:53 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 2002:ab3:7851:0:0:0:0:0 with HTTP; Sun, 29 Jul 2018 08:27:52 -0700 (PDT) In-Reply-To: <5243B4C1-513D-41E8-89AF-60487DEC29C8@lassitu.de> References: <5243B4C1-513D-41E8-89AF-60487DEC29C8@lassitu.de> From: Alan Somers Date: Sun, 29 Jul 2018 09:27:52 -0600 X-Google-Sender-Auth: K7QZMxVzixrrtI_oe7IACVOI_6E Message-ID: Subject: Re: Booting off ZFS pool with failed ZIL/cache device To: Stefan Bethke Cc: FreeBSD Stable Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 15:27:56 -0000 On Sun, Jul 29, 2018 at 4:43 AM, Stefan Bethke wrote: > Folks, > > my ZIL/cache SSD apparently just died. Rebooting the system with the SATA > M.2 SSD hung, so I removed the card from the system. > > On powerup, loader acts normally, all four SATA disks (main Raid-Z1 > devices) are all probed successfully, but mount root fails: > > ZFS filesystem version: 5 > ZFS storage pool version: features support (5000) > Timecounters tick every 1.000 msec > ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 > ada0: ATA8-ACS SATA 3.x device > ada0: Serial Number Z5Q7K0RIFFRC > ada0: 600.000MB/s transfers (SATA 3.x, UDMA5, PIO 8192bytes) > ada0: Command Queueing enabled > ada0: 4769307MB (9767541168 512 byte sectors) > ada1 at ahcich2 bus 0 scbus2 target 0 lun 0 > ada1: ATA8-ACS SATA 3.x device > ada1: Serial Number Y5PIK0A2FFRC > ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) > ada1: Command Queueing enabled > ada1: 4769307MB (9767541168 512 byte sectors) > ada2 at ahcich3 bus 0 scbus3 target 0 lun 0 > ada2: ATA8-ACS SATA 3.x device > ada2: Serial Number 36D2K0VZFFRC > ada2: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) > ada2: Command Queueing enabled > ada2: 4769307MB (9767541168 512 byte sectors) > ada3 at ahcich4 bus 0 scbus4 target 0 lun 0 > ada3: ATA8-ACS SATA 3.x device > ada3: Serial Number Z5SDK0J3FFRC > ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) > ada3: Command Queueing enabled > ada3: 4769307MB (9767541168 512 byte sectors) > pass4 at ahciem0 bus 0 scbus6 target 0 lun 0 > pass4: SEMB S-E-S 2.00 device > Trying to mount root from zfs:p2/be/11 []... > GEOM_MIRROR: Device mirror/p2swap launched (4/4). > random: unblocking device. > Mounting from zfs:p2/be/11 failed with error 6; retrying for 3 more seconds > Mounting from zfs:p2/be/11 failed with error 6. > > Loader variables: > vfs.root.mountfrom=zfs:p2/be/11 > > Manual root filesystem specification: > : [options] > Mount using filesystem > and with the specified (optional) option list. > > eg. ufs:/dev/da0s1a > zfs:tank > cd9660:/dev/cd0 ro > (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /) > > ? List valid disk boot devices > . Yield 1 second (for background tasks) > Abort manual input > > mountroot> > > Is there an easy way to boot into single user mode and remove the > ZIL/cache devices that are not there anymore? Or do I need a USB key to > boot off of and zfs import the pool first? > > > Thanks, > Stefan > > -- > Stefan Bethke Fon +49 151 14070811 A missing L2ARC is no problem, but a missing ZIL may mean filesystem corruption. Hopefully the ZIL was empty when it failed. I would try booting from ZFS and removing the offending device. Good luck. -Alan