Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 31 Jul 2015 04:47:42 -0500
From:      Karl Denninger <karl@denninger.net>
To:        freebsd-fs@freebsd.org
Subject:   Panic in ZFS during zfs recv (while snapshots being destroyed)
Message-ID:  <55BB443E.8040801@denninger.net>

next in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format.

--------------ms040002020007030900020609
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

I have an automated script that runs zfs send/recv copies to bring a
backup data set into congruence with the running copies nightly.  The
source has automated snapshots running on a fairly frequent basis
through zfs-auto-snapshot.

Recently I have started having a panic show up about once a week during
the backup run, but it's inconsistent.  It is in the same place, but I
cannot force it to repeat.

The trap itself is a page fault in kernel mode in the zfs code at
zfs_unmount_snap(); here's the traceback from the kvm (sorry for the
image link but I don't have a better option right now.)

I'll try to get a dump, this is a production machine with encrypted swap
so it's not normally turned on.

Note that the pool that appears to be involved (the backup pool) has
passed a scrub and thus I would assume the on-disk structure is ok.....
but that might be an unfair assumption.  It is always occurring in the
same dataset although there are a half-dozen that are sync'd -- if this
one (the first one) successfully completes during the run then all the
rest will as well (that is, whenever I restart the process it has always
failed here.)  The source pool is also clean and passes a scrub.

traceback is at http://www.denninger.net/kvmimage.png; apologies for the
image traceback but this is coming from a remote KVM.

I first saw this on 10.1-STABLE and it is still happening on FreeBSD
10.2-PRERELEASE #9 r285890M, which I updated to in an attempt to see if
the problem was something that had been addressed.


--=20
Karl Denninger
karl@denninger.net <mailto:karl@denninger.net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/

--------------ms040002020007030900020609
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGXzCC
BlswggRDoAMCAQICASkwDQYJKoZIhvcNAQELBQAwgZAxCzAJBgNVBAYTAlVTMRAwDgYDVQQI
EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM
TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG9w0BCQEWE0N1ZGEg
U3lzdGVtcyBMTEMgQ0EwHhcNMTUwNDIxMDIyMTU5WhcNMjAwNDE5MDIyMTU5WjBaMQswCQYD
VQQGEwJVUzEQMA4GA1UECBMHRmxvcmlkYTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEe
MBwGA1UEAxMVS2FybCBEZW5uaW5nZXIgKE9DU1ApMIICIjANBgkqhkiG9w0BAQEFAAOCAg8A
MIICCgKCAgEAuYRY+EB2mGtZ3grlVO8TmnEvduVFA/IYXcCmNSOC1q+pTVjylsjcHKBcOPb9
TP1KLxdWP+Q1soSORGHlKw2/HcVzShDW5WPIKrvML+Ry0XvIvNBu9adTiCsA9nci4Cnf98XE
hVpenER0qbJkBUOGT1rP4iAcfjet0lEgzPEnm+pAxv6fYSNp1WqIY9u0b1pkQiaWrt8hgNOc
rJOiLbc8CeQ/DBP6rUiQjYNO9/aPNauEtHkNNfR9RgLSfGUdZuOCmJqnIla1HsrZhA5p69Bv
/e832BKiNPaH5wF6btAiPpTr2sRhwQO8/IIxcRX1Vxd1yZbjYtJGw+9lwEcWRYAmoxkzKLPi
S6Zo/6z5wgNpeK1H+zOioMoZIczgI8BlX1iHxqy/FAvm4PHPnC8s+BLnJLwr+jvMNHm82QwL
J9hC5Ho8AnFU6TkCuq+P2V8/clJVqnBuvTUKhYMGSm4mUp+lAgR4L+lwIEqSeWVsxirIcE7Z
OKkvI7k5x3WeE3+c6w74L6PfWVAd84xFlo9DKRdU9YbkFuFZPu21fi/LmE5brImB5P+jdqnK
eWnVwRq+RBFLy4kehCzMXooitAwgP8l/JJa9VDiSyd/PAHaVGiat2vCdDh4b8cFL7SV6jPA4
k0MgGUA/6Et7wDmhZmCigggr9K6VQCx8jpKB3x1NlNNiaWECAwEAAaOB9DCB8TA3BggrBgEF
BQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNV
HRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIFoDALBgNVHQ8EBAMCBeAwLAYJYIZIAYb4QgENBB8W
HU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBTFHJQt6cloXBdG1Pv1
o2YgH+7lWTAfBgNVHSMEGDAWgBQkcZudhX383d29sMqSlAOh+tNtNTAdBgNVHREEFjAUgRJr
YXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcNAQELBQADggIBAE9/dxi2YqjCYYhiybp4GKcm
7tBVa/GLW+qcHPcoT4dqmqghlLz8+iUH+HCJjRQATVGyMEnvISOKFVHC6aZIG+Sg7J8bfS4+
fjKDi9smRH2VPPx3bV8+yFYRNroMGHaPHZB/Xctmmvc+PZ9O2W7rExgrODtxIOB3Zs6wkYf+
ty+9r1KmTHlV+rRHI6timH1uiyFE3cPi1taAEBxf0851cJV8k40PGF8G48ewnq8SY9sCf5cv
liXbpdgU+I4ND5BuTjg63WS32zuhLd1VSuH3ZC/QbcncMX5W3oLXmcQP5/5uTiBJy74kdPtG
MSZ9rXwZPwNxP/8PXMSR7ViaFvjUkf4bJlyENFa2PGxLk4EUzOuO7t3brjMlQW1fuInfG+ko
3tVxko20Hp0tKGPe/9cOxBVBZeZH/VgpZn3cLculGzZjmdh2fqAQ6kv9Z9AVOG1+dq0c1zt8
2zm+Oi1pikGXkfz5UJq60psY6zbX25BuEZkthO/qiS4pxjxb7gQkS0rTEHTy+qv0l3QVL0wa
NAT74Zaj7l5DEW3qdQQ0dtVieyvptg9CxkfQJE3JyBMb0zBj9Qhc5/hbTfhSlHzZMEbUuIyx
h9vxqFAmGzfB1/WfOKkiNHChkpPW8ZeH9yPeDBKvrgZ96dREHFoVkDk7Vpw5lSM+tFOfdyLg
xxhb/RZVUDeUMYIE4zCCBN8CAQEwgZYwgZAxCzAJBgNVBAYTAlVTMRAwDgYDVQQIEwdGbG9y
aWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBMTEMxHDAa
BgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG9w0BCQEWE0N1ZGEgU3lzdGVt
cyBMTEMgQ0ECASkwCQYFKw4DAhoFAKCCAiEwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAc
BgkqhkiG9w0BCQUxDxcNMTUwNzMxMDk0NzQyWjAjBgkqhkiG9w0BCQQxFgQUoZV3i0Rm8pZx
bjs2rpJJS1xtWhYwbAYJKoZIhvcNAQkPMV8wXTALBglghkgBZQMEASowCwYJYIZIAWUDBAEC
MAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzAN
BggqhkiG9w0DAgIBKDCBpwYJKwYBBAGCNxAEMYGZMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4G
A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl
bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND
dWRhIFN5c3RlbXMgTExDIENBAgEpMIGpBgsqhkiG9w0BCRACCzGBmaCBljCBkDELMAkGA1UE
BhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQ
Q3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqG
SIb3DQEJARYTQ3VkYSBTeXN0ZW1zIExMQyBDQQIBKTANBgkqhkiG9w0BAQEFAASCAgBurVIH
oJXsG/Cev7VX6GUtDIURBJTgPn0RMK2oj7NP+WJzHSE1Wk8BxBNK8ejAvmjNmt/0AXlObXwC
6ikTh2ZoEkKAjYlE/6sOG3vQPeZRwca/861lZtfxpV9OO76VbByif843AtsQPbG5rnJ16XIW
YIiD/WuzBuadX/DeT4xnqxonotbcRtVjNjEyx+XZReC7WDgnbz2DdzmBf8PumncI/nmPt8Q+
Xtc/sQ9XwlxjUHCC/urUTJKAEOHYfp/0253vMd5FzkaSDNAd/7Zsx8o6cw3LAw9v4vaudj2X
UOT8RiAzE2j3gGH3cQ1usWjB+K1ecLtW3aIvo54aswFqNtaFSdBeQc+oZGuxu4fP8p5n1rlQ
fSjpv/Dv8fUBVu17OyD8L2BwPKneeCs4rHZw+N7uEOW2YIO293NQhXf/zaaWpZ+5c7d6hoT6
8nv2JUsnsf4xwuelYNuxPys2Yq9RAqYE0VNu05btdXKlChJJHIaIXljtFIyOJTVyvdeRF6Hi
tgnVp8OszuXhveEif9RinXXwLN6+kahqnsGXZhoThFG2Y+sSeQnbsg4vNzJqDzXQR0MZYO0I
O4V9jYhvqt8ndzWhITj5m+CAsRvOXsDzWxItRu4bxse0tPVJ/O57D+wwz7gItXqewjFfriB2
XPFzoZpzcXt/FZayGFnTat1d9bJwnQAAAAAAAA==
--------------ms040002020007030900020609--





Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?55BB443E.8040801>