Date: Thu, 07 Mar 2013 22:47:12 -0600 From: Larry Rosenman <ler@lerctr.org> To: Steven Hartland <killing@multiplay.co.uk> Cc: tevans.uk@googlemail.com, freebsd-fs@freebsd.org, ronald-freebsd8@klop.yi.org Subject: Re: zfs send/recv invalid data Message-ID: <1c4687bbf52352119abbc7d12334cef1@webmail.lerctr.org> In-Reply-To: <2DCDD0136DE7498E9DA95D5CDA3F60CC@multiplay.co.uk> References: <7c93aef20a88cdbcca85739e67470dce@webmail.lerctr.org> <25894116c93a59dab1fd976b425c36d1@webmail.lerctr.org> <07b59d5d4b2a3dab1385b054eea4f2da@webmail.lerctr.org> <A3EA720372254541BCCB6C92DBA36A0E@multiplay.co.uk> <7619c6383449c7a316edb1cdffc98c54@webmail.lerctr.org> <alpine.BSF.2.00.1303052052300.52489@thebighonker.lerctr.org> <473BCD0A4A7A4E6AB08409A9B0C82363@multiplay.co.uk> <d7c08e218ef301f1202354d0b11a6742@webmail.lerctr.org> <CAFHbX1LC25pyOBUGJ-PFxzTOXvJPYOF15H0%2BfTk6qfCyT-Q8fA@mail.gmail.com> <6dcfb2284551025af3cf58703a2b5cdc@webmail.lerctr.org> <920990505611cd96a075c80d06691bb0@webmail.lerctr.org> <201303061857.r26IvLnc024186@higson.cam.lispworks.com> <fc81037894dbc014853ba8fed06e427f@webmail.lerctr.org> <9e133e088f6c3c3dafdc0a99eb7c48c1@webmail.lerctr.org> <218B0537E987442EAB8027EA478F8BE9@multiplay.co.uk> <e84db628ac7be2bea86d0f4fd8d47b36@webmail.lerctr.org> <5378B8D5A65A4F19BE1CF6C25A9CFE22@multiplay.co.uk> <ccbde3358460c03a46ed75ff1d8895f9@webmail.lerctr.org> <2DCDD0136DE7498E9DA95D5CDA3F60CC@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2013-03-07 13:08, Steven Hartland wrote: > ----- Original Message ----- From: "Larry Rosenman" <ler@lerctr.org> > To: "Steven Hartland" <killing@multiplay.co.uk> > Cc: "Martin Simmons" <martin@lispworks.com>; > <tevans.uk@googlemail.com>; <freebsd-fs@freebsd.org>; > <ronald-freebsd8@klop.yi.org> > Sent: Thursday, March 07, 2013 1:14 AM > Subject: Re: zfs send/recv invalid data > > >> On 2013-03-06 17:48, Steven Hartland wrote: >>> ----- Original Message ----- From: "Larry Rosenman" <ler@lerctr.org> >>> To: "Steven Hartland" <killing@multiplay.co.uk> >>> Cc: "Martin Simmons" <martin@lispworks.com>; >>> <tevans.uk@googlemail.com>; <freebsd-fs@freebsd.org>; >>> <ronald-freebsd8@klop.yi.org> >>> Sent: Wednesday, March 06, 2013 11:16 PM >>> Subject: Re: zfs send/recv invalid data >>> >>> >>>> On 2013-03-06 17:00, Steven Hartland wrote: >>>>> ----- Original Message ----- From: "Larry Rosenman" >>>>> <ler@lerctr.org> >>>>>> # cat send.stream | ssh root@home openssl md5 >>>>>> (stdin)= 9cd1d73ea8411f1c222bc90e7bea3d33 >>>>>> # uname -a >>>>>> FreeBSD thebighonker.lerctr.org 8.3-STABLE FreeBSD 8.3-STABLE #54 >>>>>> r247820: Mon Mar 4 18:08:11 CST 2013 >>>>>> root@thebighonker.lerctr.org:/usr/obj/usr/src/sys/THEBIGHONKER >>>>>> amd64 >>>>>> # >>>>>> # exec csh >>>>>> root@borg:/home/ler # pwd >>>>>> /home/ler >>>>>> root@borg:/home/ler # ssh tbh "cat >>>>>> /home/ler/public_html/ZFS_RECV/send.stream | ssh >>>>>> root@home.lerctr.org openssl md5" >>>>>> (stdin)= 9cd1d73ea8411f1c222bc90e7bea3d33 >>>>>> root@borg:/home/ler # uname -a >>>>>> FreeBSD borg.lerctr.org 10.0-CURRENT FreeBSD 10.0-CURRENT #124 >>>>>> r247826: Mon Mar 4 19:59:08 CST 2013 >>>>>> root@borg.lerctr.org:/usr/obj/usr/src/sys/BORG-DTRACE amd64 >>>>>> root@borg:/home/ler # >>>>> When you run a command from ssh it will run as the target users >>>>> shell >>>>> so for csh if there is something bad in .cshrc or iirc .profile >>>>> then >>>>> you may get something output purely by logging in. >>>>> So ensure you shell config's for the user on the target machine >>>>> are >>>>> clean. >>>>> You can confirm this by comparing the output from the following >>>>> two commands run from the sending machine:- >>>>> md5 send.stream >>>>> cat send.stream | ssh root@home.lerctr.org "cat" |md5 >>>>> If these two commands produce different md5's then the login >>>>> shell for root on home.lerctr.org is outputing something >>>>> even when there is no terminal. >>>>> You should be able to see what the shell is outputting using:- >>>>> echo -n '' | ssh root@home.lerctr.org "cat" > 1.log >>>>> After running this 1.log should be 0 bytes if its not your >>>>> shell is definitely your problem and I'd start by looking >>>>> at .cshrc and .profile for this user, or even the globals >>>>> for these in /etc/ >>>>> >>>> I've done that, and get identical md5's..... >>>> I'm at a loss. >>> I believe what you ran was similar but not identical, if you >>> check the quoting subtly different. >>> I'm not sure that will have any effect but both of the above >>> are worth running just to confirm. >>> Regards >>> Steve >> $ ^DConnection to thebighonker.lerctr.org closed. >> $ ssh root@tbh >> Last login: Tue Mar 5 06:51:46 2013 from cpe-72-182-19-1 >> FreeBSD 8.3-STABLE (THEBIGHONKER) #54 r247820: Mon Mar 4 18:08:11 >> CST 2013 >> Welcome to FreeBSD! >> root@thebighonker:~ # pwd >> /root >> root@thebighonker:~ # md5 /home/ler/public_html/ZFS_RECV/send >> send-file-snap-list send.stream send.uname >> root@thebighonker:~ # md5 /home/ler/public_html/ZFS_RECV/send.stream >> MD5 (/home/ler/public_html/ZFS_RECV/send.stream) = >> 9cd1d73ea8411f1c222bc90e7bea3d33 >> root@thebighonker:~ # cat /home/ler/public_html/ZFS_RECV/send.stream >> |ssh root@home.lerctr.org "cat"|md5 >> 9cd1d73ea8411f1c222bc90e7bea3d33 >> root@thebighonker:~ # >> >> looks like it's clean..... > > So to summarise, just to make sure I've got everything clear so far:- > 1. sending the stream over ssh from the source -> target results in > the same md5 (from the test above) yes > 2. sending the stream over ssh from source -> target piped into > zfs recv fails yes, sometimes now (I need to see what happens from cron(8) tonight) > 3. scp'ing the stream file from the source to the target and then > importing it via zfs recv works > yes > Yes / No? > > If so this is very strange indeed. > > I'd like to confirm that adding the zfs recv to the picture isn't > changing > the dynamic such that the stdin results are changing and thus > eliminating > ssh as the source of any corruption to do this can you test:- > md5 send.stream > cat send.stream | ssh root@home.lerctr.org "tee tea.stream | zfs recv > <flags>" > ssh root@home.lerctr.org "md5 tea.stream" > Will try this with tonights stream(s) if it fails. appreciate..... > Regards > Steve > > ================================================ > This e.mail is private and confidential between Multiplay (UK) Ltd. > and the person or entity to whom it is addressed. In the event of > misdirection, the recipient is prohibited from using, copying, > printing or otherwise disseminating it or any information contained in > it. > > In the event of misdirection, illegible or incomplete transmission > please telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1c4687bbf52352119abbc7d12334cef1>