From owner-freebsd-current@FreeBSD.ORG Wed Feb 1 13:17:33 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51EC2106566B for ; Wed, 1 Feb 2012 13:17:33 +0000 (UTC) (envelope-from se@freebsd.org) Received: from nm30-vm0.bullet.mail.sp2.yahoo.com (nm30-vm0.bullet.mail.sp2.yahoo.com [98.139.91.238]) by mx1.freebsd.org (Postfix) with SMTP id 2DF3F8FC0A for ; Wed, 1 Feb 2012 13:17:33 +0000 (UTC) Received: from [98.139.91.67] by nm30.bullet.mail.sp2.yahoo.com with NNFMP; 01 Feb 2012 13:05:20 -0000 Received: from [208.71.42.214] by tm7.bullet.mail.sp2.yahoo.com with NNFMP; 01 Feb 2012 13:05:20 -0000 Received: from [127.0.0.1] by smtp225.mail.gq1.yahoo.com with NNFMP; 01 Feb 2012 13:05:20 -0000 X-Yahoo-Newman-Id: 511382.55942.bm@smtp225.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: NXn8dXQVM1lmhCJo2mppF44h0NzKQJHYA9381sCce4B7bel KCPsVhUwhV_o1l89uR6xBzFxVcj55cTKIaRrXSNrQNr3Y_PZ1b151mpkMXhN 1zZRCj_wDG8BiGSP1UOLkzSnw.c4i.c.zQVGbBZulCwElYJ7_HPycRqH1rjZ YIrQODSRA2hzB22cdryOviE9ecuO508doVXmZY0FHZnuXGmh080Dw7k.HaWh nmxDMV59AzFil9N6Mgh4nJ.q2WykY80_xPEn9DJdOcqhETaW18S0U2kqntle 7phNHn8iWdb2.y.5dN.miMpjY9gPd6XpbsW4wIGskMGcTABqVD_h4ApiW5KH ot6W5pryABEM1wFy3TB43lobuwgqgyexc3CFA3gK572oh8KW00q40gcCwNR1 NjQ58_gSYqMOlkD1Rbwoy7cOmZ38KgA-- X-Yahoo-SMTP: iDf2N9.swBDAhYEh7VHfpgq0lnq. Received: from [192.168.119.22] (se@81.173.148.171 with plain) by smtp225.mail.gq1.yahoo.com with SMTP; 01 Feb 2012 05:05:19 -0800 PST Message-ID: <4F293892.7080504@freebsd.org> Date: Wed, 01 Feb 2012 14:05:22 +0100 From: Stefan Esser User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0) Gecko/20111222 Thunderbird/9.0.1 MIME-Version: 1.0 To: freebsd-current@freebsd.org References: <4F287338.8000002@zedat.fu-berlin.de> <4F28814D.2030804@b1c1l1.com> In-Reply-To: <4F28814D.2030804@b1c1l1.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: "O. Hartmann" Subject: Re: using nscd (ldap) makes passwd/group disappearing while installing ports X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2012 13:17:33 -0000 Am 01.02.2012 01:03, schrieb Benjamin Lee: > What's going on is: > > 1) The port checks if the group exists > 2) nscd caches that the group does not exist in its negative cache > 3) pw(8) creates the group then checks if it exists > 4) nscd returns the negative cache entry (group does not exist) > > This causes pw(8) to error since it expects the group that it just > created to exist. I had suggested before, that nscd be changed to only rely on cached negative entries, if they have repeatedly been seen in the underlying files/data-bases. E.g. only consider negative entries valid after they have been repeatedly stored into the cache (I'd think 3 times are a reasonable number). That way, the first lookup of an account or group will lead to an entry with count=1, which is found during cache lookup but which is ignored due to a too low "verification count". If the negative lookup is repeated 3 times within some reasonable time window (say 60 seconds), then it is to be considered verified. This will make the above sequence of queries and modifications of the passwd or group databases work with caching. This concept does not protect against scenarios where the negative cache entry is made active by several queries (e.g. manual checking for the presence of an account or group before the software installation repeats these tests). That is the reason to ask for more than 2 negative replies (3, perhaps better 5) before a negative cache entry is trusted. The main purpose of the cache (reduced latency for positive queries, limited load due to negative queries) will still be maintained. >> I also have this error very often when rebuilding/updating or even >> installing cups when "nscd" is enabled. A simple restart of nscd helps >> in most cases, most times I need to disable "cache" tag in >> /etc/nsswitch.conf, then everything runs smooth. >> >> Well, this behaviour is since a couple of years now, occurs sporadic. I >> have had in FreeBSD 7, 8, 9 and I see it in 10. What is it? >> >> I like the cache facility, since in domains with a lot of users >> searching LDAP takes some time and caching help keeping traffic and >> latency short. But the namservice caching mechanism seems to be >> unreliable. What is up there? > > You should put "files" before "cache" in /etc/nsswitch.conf, e.g.: > > group: files cache ldap > passwd: files cache ldap > > The problem is that tools that modify the passwd and group files, like > pw(8), don't invalidate nscd's negative cache entries when making > changes. You point out an alternative to making negative entries trusted only after they have been repeatedly entered into the cache: Tools that are used to modify the passwd/group databases might signal their changes to nscd. They could either purge the modified caches for the current or for all users, or they could just clear the single affected entry. In each case, nscd needs to re-fetch the modified (and possibly all other cached) entries. A simple implementation of such invalidation could be to invoke "nscd -I" after each modification performed by "pw". Still I think that the reduced trust in negative entries that have not been repeatedly tested is the best solution. I had looked into implementing such a logic in the cache, a few months back. It took more effort than I had hoped due to the way the cache is implemented, but I still think it should be possible without major changes to nscd. Regards, STefan