From owner-freebsd-fs@freebsd.org Fri Dec 28 10:18:49 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 481E41433E3B for ; Fri, 28 Dec 2018 10:18:49 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lj1-f195.google.com (mail-lj1-f195.google.com [209.85.208.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 32E3083E7F for ; Fri, 28 Dec 2018 10:18:48 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lj1-f195.google.com with SMTP id v1-v6so18363751ljd.0 for ; Fri, 28 Dec 2018 02:18:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=bpmOZszWdl/QYM8cV1avAWlM4aIcrupf54L2ZcU6F+Y=; b=KzcIxxCBFo0a9ft50egZBzZhbxo4UlVD3gvHChXCw8W9Kzd7zcPoK7i8qxQm6EAbLp Z85tsrGZH+EpE+2Pt6RT23fmzmu1hdngEJsGX6ueU3ZwnK/11W+c5iTyMQP1OOnT5CK9 DauXQTfGcBswU2+Zol9GG9AA2KsREJcSOcXMM8thAWdB27+rxv8g8C/x4hD9luaiNQ9h GhN8mJxkSvVMr1CmEcgrDPwaSVIrNBgolitUWgSbPyxBN6kFdBDGpmybTUTnUh8Ch/UC j5Z1HR8shcBRglPjHT/BblzYBl05gviMt3Zing7xjWjcbQK+qXYgjtxipjHnyRhyFyuI YvUA== X-Gm-Message-State: AJcUukc0DHy5HYu9fY79rKc0ugte9PRiQNgtY9lkY7zNvpl+r496VDzG FK2nb7k1648zJgMSixc6k1+tIRtI X-Google-Smtp-Source: ALg8bN67SIb8ShN6p4QxPkV6AQEREw23mL00kbySeB9skHzfIon2wU7TsH5VEBg29re6Ac9dgQA7iQ== X-Received: by 2002:a2e:2b11:: with SMTP id q17-v6mr15421179lje.25.1545992325826; Fri, 28 Dec 2018 02:18:45 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id k21-v6sm8339599ljc.15.2018.12.28.02.18.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Dec 2018 02:18:45 -0800 (PST) Subject: Re: Upgrade to 12-Release system not booting kernel panic To: Kaya Saman , freebsd-fs@freebsd.org References: <4a9526c1-2c46-eebe-b92d-d10e522d8941@gmail.com> From: Andriy Gapon Openpgp: preference=signencrypt Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata= xsFNBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABzR5BbmRyaXkgR2Fw b24gPGF2Z0BGcmVlQlNELm9yZz7CwZQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/ QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/ psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryM7BTQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+ 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+ 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L HgZY3zfKOqcAEQEAAcLBfAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp Message-ID: <9c5e215a-6cb0-2f97-4652-0bca8683e555@FreeBSD.org> Date: Fri, 28 Dec 2018 12:18:44 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <4a9526c1-2c46-eebe-b92d-d10e522d8941@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 32E3083E7F X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of agapon@gmail.com designates 209.85.208.195 as permitted sender) smtp.mailfrom=agapon@gmail.com X-Spamd-Result: default: False [-3.86 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.79)[-0.787,0]; FORGED_SENDER(0.30)[avg@FreeBSD.org,agapon@gmail.com]; FREEMAIL_TO(0.00)[gmail.com]; IP_SCORE(-1.06)[ipnet: 209.85.128.0/17(-3.70), asn: 15169(-1.51), country: US(-0.08)]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[avg@FreeBSD.org,agapon@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[FreeBSD.org]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[195.208.85.209.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[195.208.85.209.rep.mailspike.net : 127.0.0.17] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Dec 2018 10:18:49 -0000 On 16/12/2018 09:27, Kaya Saman wrote: > Hi, > > > I've been digging around a little on this one and it seems that there is some > ZFS wierdness going on. > > > My original install was an 11.2-RELEASE fresh install and upon an upgrade to > 12-RELEASE the system is having issues booting. > > > I have a ZFS root pool called zroot; by default this mounts at > zroot/ROOT/default and a few other non-root / bootable ZFS pools. > > > The system is a SuperMicro SC216 chassis with LSI non-RAID HBA. > > > I have the boot disks ada0 and ada1 plugged into the rear of the chassis and > directly into the systemboard which is also a SuperMicro. These drives are both > Samsung SSD's. > > The 22 drive slots at the front of the chassis are occupied by the other various > data pools. > > > So here is some strangeness... if I remove all 22 drives from the front, the > system boots fine but straight after boot goes into kernel panic mode and > reboots before I can even look at the error or get to the login prompt. > > > With the non-root pools installed at the BTX loader after scanning through all > the bios drives I get a bunch of: > > read 264 from ... to 0x...., error 0x10 errors > > then: > > ZFS: i/o error - all block copies unavailable > > ZFS: can't read MOS of pool I suspect that this is because of which disks the BIOS presents to the loader (and in what order). > after this the system simply hangs? > > > I have tried looking around but everything mentioning the MOS error is talking > about the root pool, a particularly good reference is here: > http://freebsd.1045724.x6.nabble.com/ZFS-i-o-error-in-recent-12-0-td6245865.html > > > In fact I did try to boot with a USB stick and go into Live mode then import all > the pools on the system. This works without any issue! The pools are fine the > data is there everything looks normal. > > - I also rebuilt the zpool.cache according to the link just incase there was > some kind of corruption there, however upon reboot I still get the same issue?? > > > Looking at a bug report with a kernel panic: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=220923#c17 > > > I have attempted to add: > > > kern.cam.scsi_delay="50000" > > kern.cam.boot_delay="50000" > > > into the /boot/loader.conf file but unfortunately the issue still continues :-( > > > I wonder if there is a way to tell to tell the system to only look at certain > drives for booting?? The boot code looks at all disks presented by the BIOS. It should look at the disk configured as a boot disk first. > There is this line in my loader.conf: > > vfs.root.mountfrom="zfs:zroot" > > > It maybe the wrong hunch I have but it seems like the system is looking for > "zroot" on all pools instead of the actual root pool hence the above errors?? You can't look for a pool on all pool, that sounds nonsensical. The correct way to say this is that the system looks for zroot on all disks. > Would anyone be able to suggest anything or have any ideas about how to get the > system back online and booting?? I think that you need to look at the BIOS / disk controller configuration. -- Andriy Gapon