From owner-freebsd-fs@FreeBSD.ORG Thu Aug 30 12:07:15 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 50B061065677 for ; Thu, 30 Aug 2012 12:07:15 +0000 (UTC) (envelope-from attila.bogar@linguamatics.com) Received: from mail.linguamatics.com (mail.linguamatics.com [188.39.80.203]) by mx1.freebsd.org (Postfix) with ESMTP id AD7AD8FC1D for ; Thu, 30 Aug 2012 12:07:14 +0000 (UTC) Received: from [10.252.10.232] (random.linguamatics.com [10.252.10.232]) by mail.linguamatics.com (Postfix) with ESMTPSA id B55C2EFB450; Thu, 30 Aug 2012 13:07:06 +0100 (BST) Message-ID: <503F576A.7030102@linguamatics.com> Date: Thu, 30 Aug 2012 13:07:06 +0100 From: =?UTF-8?B?QXR0aWxhIEJvZ8Ohcg==?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20120714 Thunderbird/14.0 MIME-Version: 1.0 To: Rick Macklem References: <487642909.1254184.1346246949938.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <487642909.1254184.1346246949938.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@FreeBSD.org Subject: KRPC layer: NFS requests are not processed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2012 12:07:15 -0000 Hi Rick, On 29/08/12 14:29, Rick Macklem wrote: > I'm an old fashioned printf() guy, so that's what I do. If it appears > that it is the upcall to the userland gssd that generates a failed > reply, then the problem is most likely the userland Kerberos > libraries. (I'd probably cram a pile of printf()s in the rpcsec_gss > and kgssapi sources, especially at the kgssapi upcalls to the userland > gssd, to try and figure out where the "authentication failure" is > occurring.) I've already emailed this person w.r.t. making sure he > isn't seeing the "non-atomic export update" problem. Since mountd > deletes all exports and then re-adds them non-atomically, the security > flavours allowed will be messed up whenever mountd gets a HUP signal. > (This happens automagically whenever a "mount" of a file system > happens on the NFS server, since "mount" sends "mountd" a HUP signal.) > If it is the userland Kerberos libraries, I'd try installing a vanilla > up to date Heimdal distro (it usually installs easily in somewhere > like /usr/heimdal) and then I'd rebuild gssd.c linked to those > libraries and see if I could get that to work. Long ago, to get > kerberized NFS to work on OpenBSD, I'd statically link the gssd I had > for it (very different than the FreeBSD gssd) to freshly built Heimdal > libraries. rick Thanks for your hints. I done all you suggested and now I see, that the problem lies on an upper level. I changed the subject according to this. I'm debugging now NFSv4 What I can see so far: - the "old" RHEL5 linux clients open 2 NFS TCP connection per mount request and it's successful - the "new" RHEL6 non-working or partly-working linux clients open at least 4 NFS TCP connection per mount request - I have two identically setup boxes (a vm and a bare metal) and the bare metal fails to mount all the time - the bare metal is a faster machine Furthermore: ./fs/nfsserver/nfs_nfsdkrpc.c: in nfsrvd_addsock(): before the out: label the svc_reg()'s are called. BUT same file: nfssvc_program() is not called for these TCP connections This could be a timing, locking, parallel race condition, rpc parsing or some other issue. I'm aware mountd HUP bug, it's an annoying one but I think we can rule this out for this case. In this thread http://lists.freebsd.org/pipermail/freebsd-stable/2012-August/069052.html I learned, that - the freebsd team doesn't want these patches to hanging around for years in the PR database without committing them - the freebsd committers and you do the volunteering for free and I think this is great thing which should be appreciated by everyone using freebsd - the closed ticket rate is around 1/3 rate of the opened ones in the kern category - the freebsd team is urges if users come up with patches as everyone does this in his/her spare time It is a fact that commiting a verified patch we kwow is much less effort than to to diagnose the problem and to come up with the patch. Ignoring the patches (effort of their userbase) for multiple years is not a fair attitude from the freebsd team even if they are resource limited. In other words if verified patches cannot be get committed into the stable means that freebsd is an OS with a growing number of bugs. From my point of view: - I currently have a freebsd-stable with some patches which I need to maintain during every upgrade. - I do not like this approach and I suppose the people experiencing the same problem and patching their system with the same patches do not like either. - the people who can't find these bugs in the pr database for the same problem are the most frustrated ones, because they feel, that freebsd is no good for them I'm happy to put effort into fixing this bug and possibly come up with a patch or diagnose the specific problem. What I'm certainly not happy to do and I won't do is to spend my time gratuitously with my patch hanging around in the PR database for years without being commited. I'd like to kindly ask you and other people involved in the freebsd project: how could patches be included in the -stable? Kind regards, Attila