Date: Thu, 24 Jan 2008 13:40:40 +0200 From: John Hay <jhay@meraka.org.za> To: "M. Warner Losh" <imp@bsdimp.com> Cc: freebsd-arm@FreeBSD.ORG, des@FreeBSD.ORG Subject: Re: sshd broken on arm? Message-ID: <20080124114039.GF79134@zibbi.meraka.csir.co.za> In-Reply-To: <20080118.145436.-1540399028.imp@bsdimp.com> References: <20080118185634.GA28843@zibbi.meraka.csir.co.za> <20080118.120152.-345488389.imp@bsdimp.com> <20080118191638.GA30155@zibbi.meraka.csir.co.za> <20080118.145436.-1540399028.imp@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Jan 18, 2008 at 02:54:36PM -0700, M. Warner Losh wrote: > In message: <20080118191638.GA30155@zibbi.meraka.csir.co.za> > John Hay <jhay@meraka.org.za> writes: > : On Fri, Jan 18, 2008 at 12:01:52PM -0700, M. Warner Losh wrote: > : > In message: <20080118185634.GA28843@zibbi.meraka.csir.co.za> > : > John Hay <jhay@meraka.org.za> writes: > : > : On Fri, Jan 18, 2008 at 10:17:47AM -0700, M. Warner Losh wrote: > : > : > In message: <4790D750.4060702@errno.com> > : > : > Sam Leffler <sam@errno.com> writes: > : > : > : John Hay wrote: > : > : > : > On Thu, Jan 17, 2008 at 12:58:54PM +0200, John Hay wrote: > : > : > : > > : > : > : >> Hi Guys, > : > : > : >> > : > : > : >> I just did a new build using RELENG_7 for the arm (Avila boards) and then > : > : > : >> found that I cannot ssh into them. The sshd crash with a bus error just > : > : > : >> after you entered your username and password. My build of mid November > : > : > : >> did not do it. Anybody got ideas? > : > : > : >> > : > : > : >> The last part of "sshd -Dddd" on the arm board looks like this: > : > : > : >> > : > : > : >> debug1: server_input_channel_req: channel 0 request pty-req reply 0 > : > : > : >> debug1: session_by_channel: session 0 channel 0 > : > : > : >> debug1: session_input_channel_req: session 0 req pty-req > : > : > : >> debug1: Allocating pty. > : > : > : >> debug3: mm_request_send entering: type 25 > : > : > : >> debug3: monitor_read: checking request 25 > : > : > : >> debug3: mm_answer_pty entering > : > : > : >> debug1: session_new: init > : > : > : >> debug1: session_new: session 0 > : > : > : >> debug3: mm_pty_allocate: waiting for MONITOR_ANS_PTY > : > : > : >> debug3: mm_request_receive_expect entering: type 26 > : > : > : >> debug3: mm_request_receive entering > : > : > : >> debug3: mm_request_send entering: type 26 > : > : > : >> ssh_mm_receive_fd: recvmsg: expected received 1 got 0 > : > : > : >> debug1: do_cleanup > : > : > : >> debug1: PAM: cleanup > : > : > : >> Bus error (core dumped) > : > : > : >> debug3: PAM: sshpam_thread_cleanup entering > : > : > : >> > : > : > : > > : > : > : > Ok, I found the problem. It looks like something changed and now the > : > : > : > alignment for the char tmp[...] array in monitor_fdpass.c:mm_send_fd > : > : > : > and monitor_fdpass.c:mm_receive_fd is different and the arm processors > : > : > : > do not like it. Attached is my quick fix. > : > : > : > > : > : > : > One question that I have is if we should just fix all of these "problems" > : > : > : > or should something be changed so that these things are aligned again? In > : > : > : > the last month or two I have come across quite a few of these things that > : > : > : > used to work on the arm and now do not anymore because of alignment > : > : > : > changes. > : > : > : > > : > : > : > (I have cc'ed des@ because his name pitch up a lot in the openssh cvs logs. > : > : > : > :-) > : > : > : > > : > : > : > : > : > : This used to work fine so the problem is elsewhere. Sounds like a > : > : > : toolchain or header change is the root cause. > : > : > > : > : > Or some subtle change in the kernel that isn't using the macro (or is > : > : > now and didn't used to be). > : > : > : > : Hmmm Just to make sure that I'm on the right page. On FreeBSD ARM one > : > : is not supposed to be able to access unaligned memory? Ie. an int that > : > : does not start on an address that is a multiple of 4. > : > : > : > : In a C function if you have something like "char tmp[4]", can you assume > : > : that the compiler will align it on a 4 byte boundary or can it do it on > : > : a byte boundary? > : > : > : > : If one cannot access unaligned ints and char arrays are not int aligned, > : > : then we were just lucky that the code worked at some stage. > : > > : > You are correct. The fact that it seemed to work meant that we were > : > either getting lucky before, or there was some critical code on the > : > kernel side that has accidentally been removed... > : > : I don't think the kernel will make a difference. The bus error happened > : on line 64 of openssh/monitor_fdpass.c. That is before the kernel was > : called (sendmsg())... Except if the kernel aligned the stack differently. > > OK. Maybe the right fix is to add '__aligned(4)' to the declaration? > > : > : > John, I don't suppose you'd have time for a binary search? > : > : > : > : I'll see what I can do, but it will be slow going. > : > > : > Bad sshd with kernels going back in time should be sufficient... Ok, it turned out that the problem/change was not in the kernel. Old and new kernels work equally well/bad. On December 2, the default optimisation for the ARMs was changed from -O2 to -O. That is it. If I compile libsshd with -O2, it works and if I compile it with -O, it does not. Just adding __aligned(4) like Warner suggested also fix the problem. Index: crypto/openssh/monitor_fdpass.c =================================================================== RCS file: /home/ncvs/src/crypto/openssh/monitor_fdpass.c,v retrieving revision 1.1.1.7 diff -u -r1.1.1.7 monitor_fdpass.c --- crypto/openssh/monitor_fdpass.c 10 Nov 2006 16:38:34 -0000 1.1.1.7 +++ crypto/openssh/monitor_fdpass.c 24 Jan 2008 11:32:43 -0000 @@ -49,7 +49,7 @@ char ch = '\0'; ssize_t n; #ifndef HAVE_ACCRIGHTS_IN_MSGHDR - char tmp[CMSG_SPACE(sizeof(int))]; + char tmp[CMSG_SPACE(sizeof(int))] __aligned(4); struct cmsghdr *cmsg; #endif @@ -94,7 +94,7 @@ char ch; int fd; #ifndef HAVE_ACCRIGHTS_IN_MSGHDR - char tmp[CMSG_SPACE(sizeof(int))]; + char tmp[CMSG_SPACE(sizeof(int))] __aligned(4); struct cmsghdr *cmsg; #endif So what should we do? John -- John Hay -- John.Hay@meraka.csir.co.za / jhay@FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080124114039.GF79134>