Date: Mon, 21 May 2012 23:36:40 +0200 From: Frank Bartels <freebsd@knarf.de> To: freebsd-fs@freebsd.org Cc: Martin Ranne <martin.ranne@kockumsonics.com>, 'Andriy Gapon' <avg@freebsd.org> Subject: Re: zpool import reboots computer Message-ID: <20120521213640.GC69425@server-king.de> In-Reply-To: <20120518175225.GA4735@server-king.de> References: <39C592E81AEC0B418EAD826FC1BBB09B25253F@mailgate> <4F1878AC.6060704@FreeBSD.org> <39C592E81AEC0B418EAD826FC1BBB09B25284B@mailgate> <4F1AC995.7050506@FreeBSD.org> <39C592E81AEC0B418EAD826FC1BBB09B255E15@mailgate> <4F1D75CD.6050000@FreeBSD.org> <39C592E81AEC0B418EAD826FC1BBB09B25607F@mailgate> <4F1DC398.3050502@FreeBSD.org> <39C592E81AEC0B418EAD826FC1BBB09B25CF08@mailgate> <20120518175225.GA4735@server-king.de>
next in thread | previous in thread | raw e-mail | index | archive | help
--9Ek0hoCL9XbhcSqy Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi freebsd-fs, my zpool is still not importable. :( I've got some tips during the last days, but none of them changed the error messages below in the second "screenshot". The zdb -lu output is in sync for all disks. I've replaced the boot mirror (FreeBSD 8.3-RELEASE-p1, gptzfsboot) with a plain FreeBSD 9.0-STABLE (gptboot, single disk, / on ufs) with the patched vdev_mirror.c and finally managed to get a crashdump while trying to import the zpool zdata using "zpool import -f -R /angus.zdata -d /dev/gpt zdata": https://www.server-king.de/download/zfs-crash-debug/backtrace.zdata.crash.t= xt The patch I've used is here: https://www.server-king.de/download/zfs-crash-debug/vdev_mirror.c.patch.txt Otherwise it would crash before the same way as with plain RELENG_8_3. Do you need any more information? Thanks for any help, Knarf P.S.: The latest changes seen in http://www.freebsd.org/cgi/cvsweb.cgi/src/cddl/contrib/opensolaris/lib/libz= fs/common/libzfs_dataset.c (SVN rev 235702) do not seem to be related with my problem, but I'll do a make world right now. On Fri, May 18, 2012 at 19:52:25 +0200, Frank Bartels wrote: > Hi freebsd-fs, >=20 > I have a similar problem like Martin. >=20 > It started a while ago with a broken zfs, I was no longer able to > delete some files on /home/ncvs: >=20 > Checking setuid files and devices: > find: /home/ncvs/del/efax/Attic/pkg-comment,v: Bad file descriptor > find: /home/ncvs/del/libsyncml/files: No such file or directory >=20 > Two days ago the machine started rebooting every two hours, directly > after syncing my local cvsup-server. >=20 > So I renamed the zfs /home/ncvs to /home/ncvs.del and tried to > destroy it including its snapshots. The machine crashed again and > now I'm unable to import the pool. >=20 > First I've seen this backtrace: >=20 > https://www.server-king.de/download/DSC02742.medium.JPG >=20 > Then I've added the three blocks above to vdev_mirror.c. It still > crashes, but the backtrace has changed: >=20 > https://www.server-king.de/download/DSC02744.medium.JPG >=20 > ... > calltrap > zio_checksum_verify > zio_execute > arc_read_nolock > arc_read > ... >=20 > This is FreeBSD 8.3-RELEASE-p1 amd64 on a Xeon X5650 with 24 GByte > RAM and 12 hard disks and 2 SSDs. >=20 > This is what I see with zpool import -d /dev/gpt >=20 > pool: zdata > id: 18141461787395278116 > state: ONLINE > action: The pool can be imported using its name or numeric identifier. > config: >=20 > zdata ONLINE > raidz2-0 ONLINE > gpt/zdata1.eli ONLINE > gpt/zdata0.eli ONLINE > gpt/zdata2.eli ONLINE > gpt/zdata3.eli ONLINE > gpt/zdata5.eli ONLINE > gpt/zdata4.eli ONLINE > gpt/zdata6.eli ONLINE > gpt/zdata8.eli ONLINE > gpt/zdata9.eli ONLINE > cache > gpt/zcache0.eli > gpt/zcache1.eli > spares > gpt/zdata7.eli > logs > mirror-1 ONLINE > gpt/zlog0.eli ONLINE > gpt/zlog1.eli ONLINE >=20 > I have no idea why I don't see zcaches and zdata7 as ONLINE. >=20 > If I use zpool import (without -d) I see dsk/gpt instead of gpt/ > on these three disks: >=20 > cache > dsk/gpt/zcache0.eli > dsk/gpt/zcache1.eli > spares > dsk/gpt/zdata7.eli >=20 > Do you have any idea what I can do? I've tried 9.0-RELEASE (LiveCD) > without success. Do you think using 8.3-STABLE or 9.0-STABLE could > cure my problem? >=20 > Thanks, > Knarf >=20 > On Wed, Jan 25, 2012 at 16:10:19 +0000, Martin Ranne wrote: > > Thank you everyone who have helped me with hacking zfs. We have now bee= n able to do an import of the pool and transfered all the data to another c= omputer. Next step is to see if we can quickly repair the pool or just dele= te it and make it new again. > > > > We hacked the functions vdev_mirror_child_select() and vdev_mirror_io_s= tart(). In vdev_mirror_io_start() we added the code below just after the mc= pointer was set in both loops. > > > > if (mc->mc_vd =3D=3D NULL) { > > (void) printf("mc->mc_vd is NULL. Child %i\n", c); > > continue; > > } > > > > In vdev_mirror_child_select(), we added the code below just after the m= c pointer was set. > > > > if (mc->mc_vd =3D=3D NULL) { > > (void) printf("mc->mc_vd is NULL. Child %i\n", c); > > mc->mc_tried =3D 1; > > mc->mc_skipped =3D 1; > > continue; > > } > > > > > > Best regards, > > > > Martin Ranne > > > > >On 2012-01-23 21:31, Andriy Gapon wrote: > > >>on 23/01/2012 20:33 Martin Ranne said the following: > > >>Have done some checking and found mc->mc_vd =3D=3D NULL in the functi= on vdev_mirror_io_start() where the while-loop is. > > >> > > >>while (children--) { > > >> mc =3D &mm->mm_child[c]; > > >> zio_nowait(zio_vdev_child_io(zio, zio->io_bp, > > >> mc->mc_vd, mc->mc_offset, zio->io_data, zio->io_size, > > >> zio->io_type, zio->io_priority, 0, > > >> vdev_mirror_child_done, mc)); > > >> c++; > > >>} > > >> > > >>if i set a break before it runs zio_nowait() it will still crash the = kernel. > > >>What can i check next for it to be able to continue? Is it possible t= o have it ignore the child where mc_vd is NULL? I am also looking into what= more I can do to debug it (adding code to print to console as i can not us= e kernel dumps). > > >> > > >Not sure. If by "set a break" you mean inserting a break statement, t= ry > > >continue instead. > > > > > > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --9Ek0hoCL9XbhcSqy Content-Type: application/x-pkcs7-signature Content-Disposition: attachment; filename="smime.p7s" Content-Transfer-Encoding: base64 MIIQkgYJKoZIhvcNAQcCoIIQgzCCEH8CAQExCzAJBgUrDgMCGgUAMAsGCSqGSIb3DQEHAaCC DcQwggY0MIIEHKADAgECAgEgMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYD VQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0 ZSBTaWduaW5nMSkwJwYDVQQDEyBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAe Fw0wNzEwMjQyMTAyNTVaFw0xNzEwMjQyMTAyNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UE ChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUg U2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xhc3MgMiBQcmltYXJ5IEludGVybWVkaWF0 ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDLKIVFnAEs+xny q6UzjCqgDcvQVe1dIoFnRsQPCFO+y92k8RK0Pn3MbQ2Gd+mehh9GBZ+36uUQA7Xj9AGM6wgP hEE34vKtfpAN5tJ8LcFxveDObCKrL7O5UT9WsnAZHv7OYPYSR68mdmnEnJ83M4wQgKO19b+R t8sPDAz9ptkQsntCn4GeJzg3q2SVc4QJTg/WHo7wF2ah5LMOeh8xJVSKGEmd6uPkSbj113yK Mm8vmNptRPmM1+YgmVwcdOYJOjCgFtb2sOP79jji8uhWR91xx7TpM1K3hv/wrBZwffrmmEpU euXHRs07JqCCvFh9coKF4UQZvfEg+x3/69xRCzb1AgMBAAGjggGtMIIBqTAPBgNVHRMBAf8E BTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUrlWDb+wxyrn3HfqvazHzyB3jrLsw HwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6 Ly93d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8v d3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wu Y29tL3Nmc2NhLmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIB FiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4IC AQA6qScNyNO0FpHvaZTQacVMXH33O51KyEKSRw3IvdQxRu31YR0ZDGdSfgSoOVDVMSBSdmfQ fdDInHPzV3LO5DwUXZ+lxjv7z3PO2OkfnFkvTXPfn6dxJ5rJveDsTsCPcJ/Kp6/+qN5g+J6D /SaYcFD018B6L42r0Z4VEBy36P4tjRtF14Ex10tl5tJFVKM16qWKQHbpjIgf73s49UB0CQ5l HT2DHKfq3oPfdNc5Mk93w1v4ryVb+qVrZIej8NsrWU+5r4O2IV91edDb/OtHFddZqHFFXKgS 79IHE/hwQ2LW7r3sTX7cDUCg+dfdwO8zeLxuwk2JF8crUoyrl66RGrRIhT8VoG/OJ1Y9uUlO av69V4cG8upi4ZG2l7JZFbcBFk91Wp+Payo5SuF61CmGFrZ386umkmpObtFacXda2O/bVoQ9 xHQrzoTc/0KZTWvlZCLK3Ke/vGYT9ZdW9lOjGsSFbXrlTA919L84iMK+48WGnvRWY28ZaVHp ql43AtEGhXze6iNCbEDACy+4hkQYOytAqDgcxAnQ937mYpeZFPyz/XK9QSt9VNFMuudWxZwD DDJKoQAoSG59Hou9lZ26UrK60nRdAQBmEPL8h2nuWgoPh++XVQld9yuhbsWa39Pck8/lcfz5 HUVGJF5mc/zk38iV7FDlF68puiryNq2KXHEpOTCCB4gwggZwoAMCAQICAhnSMA0GCSqGSIb3 DQEBBQUAMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UE CxMiU2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRD b20gQ2xhc3MgMiBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwHhcNMTExMjA3MDg0 MjU3WhcNMTMxMjA3MTgwNTMxWjCBjzEgMB4GA1UEDRMXNTg1MzA2LXQ2SXE3M3MwMDFWQTlV M3ExCzAJBgNVBAYTAkRFMQ8wDQYDVQQIEwZCYXllcm4xETAPBgNVBAcTCE11ZW5jaGVuMRYw FAYDVQQDEw1GcmFuayBCYXJ0ZWxzMSIwIAYJKoZIhvcNAQkBFhNrbmFyZkBjYW1lbG90LWVr LmRlMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA8N5hhEgqWbS/FRuLFDLqAVvV +y0aDtXj5D5p8hkNNm4jgukL8DoqRLG3lc/tP5wZQHfKLXSwHdujwH8xu4Vl3tNIr1OagZnT Vcf7qWlv9+EFoIf5wql3zveLKTLA49mSqI7MJN3mWNwSxDeUXXYGrUvqnoF0X5ihg0zRfmpz xb1nSU9l9X29ONJh9IyoXucG/6eRLyjUu6QAtGRNe7rZJAyt/fnYvCqgeqUOfNLl5BDCCdpM FNqy5EH1B9NJYe74TGV87+woiZf+Tp47Nkle5aUG0Tp81Zak3MS1qW8Q4Uk8jSELLMCfvIv+ MFPrW3e6qkJGsAO+xK26GlmCj+kJfQIDAQABo4ID7TCCA+kwCQYDVR0TBAIwADALBgNVHQ8E BAMCBLAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMB0GA1UdDgQWBBRtarO3vRru MRytTmEChbqEJq191jAfBgNVHSMEGDAWgBSuVYNv7DHKufcd+q9rMfPIHeOsuzBdBgNVHREE VjBUgRNrbmFyZkBjYW1lbG90LWVrLmRlgRNrbmFyZkBjYW1lbG90LWVrLmRlgQ5rbmFyZkBr bmFyZi5kZYEYZi5iYXJ0ZWxzQGhhcHB5LXBpeGVsLmRlMIICIQYDVR0gBIICGDCCAhQwggIQ BgsrBgEEAYG1NwECAjCCAf8wLgYIKwYBBQUHAgEWImh0dHA6Ly93d3cuc3RhcnRzc2wuY29t L3BvbGljeS5wZGYwNAYIKwYBBQUHAgEWKGh0dHA6Ly93d3cuc3RhcnRzc2wuY29tL2ludGVy bWVkaWF0ZS5wZGYwgfcGCCsGAQUFBwICMIHqMCcWIFN0YXJ0Q29tIENlcnRpZmljYXRpb24g QXV0aG9yaXR5MAMCAQEagb5UaGlzIGNlcnRpZmljYXRlIHdhcyBpc3N1ZWQgYWNjb3JkaW5n IHRvIHRoZSBDbGFzcyAyIFZhbGlkYXRpb24gcmVxdWlyZW1lbnRzIG9mIHRoZSBTdGFydENv bSBDQSBwb2xpY3ksIHJlbGlhbmNlIG9ubHkgZm9yIHRoZSBpbnRlbmRlZCBwdXJwb3NlIGlu IGNvbXBsaWFuY2Ugb2YgdGhlIHJlbHlpbmcgcGFydHkgb2JsaWdhdGlvbnMuMIGcBggrBgEF BQcCAjCBjzAnFiBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTADAgECGmRMaWFi aWxpdHkgYW5kIHdhcnJhbnRpZXMgYXJlIGxpbWl0ZWQhIFNlZSBzZWN0aW9uICJMZWdhbCBh bmQgTGltaXRhdGlvbnMiIG9mIHRoZSBTdGFydENvbSBDQSBwb2xpY3kuMDYGA1UdHwQvMC0w K6ApoCeGJWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL2NydHUyLWNybC5jcmwwgY4GCCsGAQUF BwEBBIGBMH8wOQYIKwYBBQUHMAGGLWh0dHA6Ly9vY3NwLnN0YXJ0c3NsLmNvbS9zdWIvY2xh c3MyL2NsaWVudC9jYTBCBggrBgEFBQcwAoY2aHR0cDovL2FpYS5zdGFydHNzbC5jb20vY2Vy dHMvc3ViLmNsYXNzMi5jbGllbnQuY2EuY3J0MCMGA1UdEgQcMBqGGGh0dHA6Ly93d3cuc3Rh cnRzc2wuY29tLzANBgkqhkiG9w0BAQUFAAOCAQEAnfH80+NVJo1IwpzN1F2iQjiEKTj828LG a9petNHG2ivFylhzZlN6N93ioZmJvsUBe5KviJ2Pr4EyalHj+1MfCh475YI+vv0owqMGFEjC evcOOmgnNdugfQ4eVEoRshaRdYyOvftG8K9RTGoHrnk1De9QJMAKvEZa8Rx1/4mb3VC+pSnl ny2EdkEU0sJ30U6xXs+Z/JbVZJZyUMm2uDerF4lEykWuJL6vuBKuL/HZnIHWs5Nmd8ikVdlZ PBY/ul1fFbe3ertA4Z9i5aswdWiDbHCGUuEtZuMtjZfR9xFnTMwebTMuzlWv4ccRJWjF34pc cppyA3yda5I6y+dtUmszMDGCApYwggKSAgEBMIGTMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UE ChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUg U2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xhc3MgMiBQcmltYXJ5IEludGVybWVkaWF0 ZSBDbGllbnQgQ0ECAhnSMAkGBSsOAwIaBQCggdgwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEH ATAcBgkqhkiG9w0BCQUxDxcNMTIwNTIxMjEzNjQwWjAjBgkqhkiG9w0BCQQxFgQUDNyNBCHv PeqRe6oebFTm0q/jZ0gweQYJKoZIhvcNAQkPMWwwajALBglghkgBZQMEASowCwYJYIZIAWUD BAEWMAsGCWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcN AwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwDQYJKoZIhvcNAQEBBQAEggEAUGWhIcs/ fSABany7JiVlyC0B+/REvdb13cA53V6CREGA9fuRaqA8HcaqvL8fABUK7QGgCMi+1j4meY4m HJtBT8UW8/KQOv6viNpylKtFn52Ul2Ax223FxQF1OlmzTAHXc2WVW2BPJ/hKgLOj0T8ScxOz u1jlX6TyuYOvw5dSbXmKfioSudVNXpUb897Iknrf12VUfOAaXowd8aoDCT15Gn7/PMG1HC6x rWADVdFK+9fcDx4qqK4FoNfw81XwooAT8BP1dBTyK0p1hAo4B9misado9K3x4KjMtfFBAY97 GgsFnNvQUev0p+EaJ6FkbnFuGXlE4xm9SuOwrMXR7/TMyg== --9Ek0hoCL9XbhcSqy--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120521213640.GC69425>