From owner-freebsd-stable@freebsd.org Wed Dec 9 21:32:08 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 40A229D6D76 for ; Wed, 9 Dec 2015 21:32:08 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E2FB0193F for ; Wed, 9 Dec 2015 21:32:07 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id D60A422F288 for ; Wed, 9 Dec 2015 15:31:59 -0600 (CST) Subject: Re: Something in r291926 (and earlier) causes reboots during periodic daily (14 jails on system) To: freebsd-stable@freebsd.org References: <20151209090049.000003db@Leidinger.net> <5667E86F.3050906@quip.cz> <20151209105144.000035be@Leidinger.net> <3923D6D6-E907-4B6C-931A-12CD7B863415@ebureau.com> From: Karl Denninger Message-ID: <56689DAE.7030802@denninger.net> Date: Wed, 9 Dec 2015 15:31:26 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <3923D6D6-E907-4B6C-931A-12CD7B863415@ebureau.com> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms010307030109000509030109" X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Dec 2015 21:32:08 -0000 This is a cryptographically signed message in MIME format. --------------ms010307030109000509030109 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable I doubt very much that's involved; I have multiple production systems with dynamic_write_buffer turned on and I've yet to see a panic from periodic activity or anything like it. The only panic I've got open right now that's related to ZFS happens during a send/receive backup and it's able to be reproduced only with difficulty (it looks to be a problem in snapshot management as I've isolated it fairly well but haven't yet found the root cause) -- and it can be coaxed out of a bare "clean" -STABLE install with no ZFS patches in at all. (The codepath that results in the panic doesn't have a logical connection to that patch, but I tested without it anyway just to make sure.) On 12/9/2015 15:15, Dustin Wenz wrote: > Are you by chance using the ARC patch from this PR? > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D187594 > > Do you have a vfs.zfs.dynamic_write_buffer tunable defined, if so, what= is it set to? > > - .Dustin > > >> On Dec 9, 2015, at 3:51 AM, Alexander Leidinger wrote: >> >> On Wed, 09 Dec 2015 09:38:07 +0100 >> Miroslav Lachman <000.fbsd@quip.cz> wrote: >> >>> Alexander Leidinger wrote on 12/09/2015 09:00: >>>> Does this ring a bell for someone or any ideas before I try to hunt >>>> this down? >>> The same problem was reported yesterday on Stable >>> >>> Periodic jobs triggering panics in 10.1 and 10.2 >>> https://lists.freebsd.org/pipermail/freebsd-stable/2015-December/0838= 07.html >>> >>> Also ZFS with jails. >> The above mail has a backtrace involving ZFS. >> >> I'm running the periodic scripts serially now, 8 out of 14 already >> finished without issues. Smells like a concurrency issue. I would >> assume it's something introduced between 5 and 1 month ago and MFCed >> back to 10. >> >> Does this ring a bell for someone on fs@? >> >> Bye, >> Alexander. >> >> --=20 >> http://www.Leidinger.net Alexander@Leidinger.net: PGP 0xC773696B3BAC17= DC >> http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0xC773696B3BAC17= DC >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.o= rg" > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g" --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms010307030109000509030109 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNTEyMDkyMTMxMjZaME8GCSqGSIb3DQEJBDFCBEBO 1u2oQ6f8Whrpcc/u/egOgCnh7Q0kxJ4t0K0bhfxUvteNfs3nzHrb2hWMXBYMimJsA+63WypU JQVKFoq3yXfxMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAA5TYSsM3 F1smSXdhmKjraL7UKn43utILti83lX1WPyvA3kXsNbQER4LL0nI4MTg0DFfJ44jYDNtbP148 l6xq1ek5j2CmnpC2S4FfQUHigXvrltN9FZOdekPdQGnyr+DvEGY51IrpjvWnIcjyLfpky4TU GlC62Y8raAMqIH248YGnyHujNjlal/CKRmw1Z0478PEfp/7iPMmhbRgscDmWbU2BKpPQ2NOh 7Fz/WkIDEgJprrKTNIRqUtdqT1wlmbsZRvx4QGC3NfqSo79g72YzAaoJNZnt5rNxpusYXY6a YqWLUlN0DaRqwy9ZVufLgD69pPZu/xKz+3qhr0wZHDpf77MP9l9r/Jph64fWkYFjp9o9jols 14DKq/5D88ywdCOL/DC1hCq9wCRDjMH1+tfPM9/OwLY6jlKfVeTprqTawa3fUiBbwPEpKzj+ oOG1eSrOqxg1nTq//2GvtriDUWUYlCyecDHE7bRwfDhf2LdvsZbx0oaFdEz7RM8o3MM3/uFE GmjdtQQfoTpacJJBEN1CfWpOfiT2vS9m3g6vjYgFo8lAPJhhBfNjyjnkD6xTEDzgLd19mjUL fvm4DOpeXr4w/pfETVQ86/tP9Nc58OOR+U3CvMARwMuqtWNAKthpvUynbQWCWRVkCDDbVVe1 LM/O9n4L3H6rfAWPPfhXC/ohTuQAAAAAAAA= --------------ms010307030109000509030109--