Date: Sun, 6 Oct 2024 22:13:58 +0200 From: Marek Zarychta <zarychtam@plan-b.pwste.edu.pl> To: David Cross <david@crossfamilyweb.com> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Review D38047 ... and then there was one.... Message-ID: <5235bcad-4ff9-4aa1-97ac-30766e114cef@plan-b.pwste.edu.pl> In-Reply-To: <5FCA5CA0-7F07-44A7-95A3-672AB8C2C6A1@crossfamilyweb.com> References: <6bfd6c61-38aa-4038-b54b-6c17b5b69ada@plan-b.pwste.edu.pl> <5FCA5CA0-7F07-44A7-95A3-672AB8C2C6A1@crossfamilyweb.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format.
--------------aFGhZ5Q0xWRThRteF60eofjP
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
W dniu 6.10.2024 o 22:04, David Cross pisze:
> Here’s the thing. The current implementation of nscd DOESN’T WORK at all. There is a symbol that nscd exports that libc is supposed to use as a flag to bypass lookups for nscd itself. But that symbol isn’t exported right.
>
> You will need to recompile libc and nscd. (I just do a buildworld to make sure i get everything as there are makefile changes related to the aforementioned symbol changes.
Yes, without world installed this patched nscd won't even start:
[host] /usr/src# service nscd start
Starting nscd.
limits: setrlimit pipebuf: Invalid argument
/etc/rc.d/nscd: WARNING: failed to start nscd
> And then after that make sure to check getgroupentries too
The number of groups is much lower, so the whole difference is like 0.01
vs 0.02 s, but yes, lookup is 100% faster when nscd is not running
(regardless to the state of the application of the patch).
>
>> On Oct 6, 2024, at 3:57 PM, Marek Zarychta<zarychtam@plan-b.pwste.edu.pl> wrote:
>>
>> W dniu 6.10.2024 o 20:35, David E. Cross pisze:
>>> Please, love to get some eyes on this. As it stands nscd is completely useless for LDAP for getgroupmembership (and really ANY implementation that defines a specific implementation of getgroupmembership, since it will then bypass the non-existent NSCD version). Additionally it fixes bugs with negative caching as well as increases thread safety.
>> Thank you for this patch. I am not competent to review this code, but can test it. Really, our nscd with LDAP is a nightmare. I have set filters to narrow lookups, but with full directory, when nscd is runnig I have have such timings:
>>
>> [host] ~# /usr/bin/time getent passwd > /dev/null
>> 0.62 real 0.06 user 0.15 sys
>> [host] ~# /usr/bin/time getent passwd > /dev/null
>> 0.47 real 0.07 user 0.12 sys
>> [host] ~# /usr/bin/time getent passwd > /dev/null
>> 0.46 real 0.04 user 0.15 sys
>>
>> After stopping nscd service:
>>
>> [host] ~# /usr/bin/time getent passwd > /dev/null
>> 0.15 real 0.03 user 0.06 sys
>> [host] ~# /usr/bin/time getent passwd > /dev/null
>> 0.16 real 0.01 user 0.08 sys
>>
>> Unfortunately, with this patch applied there is no much improvement:
>>
>> [host] ~# /usr/bin/time getent passwd > /dev/null
>> 0.65 real 0.03 user 0.19 sys
>> [host] ~# /usr/bin/time getent passwd > /dev/null
>> 0.48 real 0.02 user 0.22 sys
>> [host] ~# /usr/bin/time getent passwd > /dev/null
>> 0.43 real 0.06 user 0.12 sys
>>
>> The test were run on most recent stable/14 with net/nss-pam-ldapd as a Name Service Switch module for LDAP lookup.
>>
>> --
>> Marek Zarychta
>>
>
>
--
Marek Zarychta
--------------aFGhZ5Q0xWRThRteF60eofjP
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">W dniu 6.10.2024 o 22:04, David Cross
pisze:<br>
</div>
<blockquote type="cite"
cite="mid:5FCA5CA0-7F07-44A7-95A3-672AB8C2C6A1@crossfamilyweb.com">
<pre wrap="" class="moz-quote-pre">Here’s the thing. The current implementation of nscd DOESN’T WORK at all. There is a symbol that nscd exports that libc is supposed to use as a flag to bypass lookups for nscd itself. But that symbol isn’t exported right.
You will need to recompile libc and nscd. (I just do a buildworld to make sure i get everything as there are makefile changes related to the aforementioned symbol changes.
</pre>
</blockquote>
<p>Yes, without world installed this patched nscd won't even start:</p>
<p></p>
<p>[host] /usr/src# service nscd start<br>
Starting nscd.<br>
limits: setrlimit pipebuf: Invalid argument<br>
/etc/rc.d/nscd: WARNING: failed to start nscd<br>
<span style="white-space: pre-wrap">
</span></p>
<blockquote type="cite"
cite="mid:5FCA5CA0-7F07-44A7-95A3-672AB8C2C6A1@crossfamilyweb.com">
<pre wrap="" class="moz-quote-pre">
And then after that make sure to check getgroupentries too</pre>
</blockquote>
<p>The number of groups is much lower, so the whole difference is
like 0.01 vs 0.02 s, but yes, lookup is 100% faster when nscd is
not running (regardless to the state of the application of the
patch).<br>
</p>
<blockquote type="cite"
cite="mid:5FCA5CA0-7F07-44A7-95A3-672AB8C2C6A1@crossfamilyweb.com">
<pre wrap="" class="moz-quote-pre">
</pre>
<blockquote type="cite">
<pre wrap="" class="moz-quote-pre">On Oct 6, 2024, at 3:57 PM, Marek Zarychta <a class="moz-txt-link-rfc2396E" href="mailto:zarychtam@plan-b.pwste.edu.pl"><zarychtam@plan-b.pwste.edu.pl></a> wrote:
W dniu 6.10.2024 o 20:35, David E. Cross pisze:
</pre>
<blockquote type="cite">
<pre wrap="" class="moz-quote-pre">Please, love to get some eyes on this. As it stands nscd is completely useless for LDAP for getgroupmembership (and really ANY implementation that defines a specific implementation of getgroupmembership, since it will then bypass the non-existent NSCD version). Additionally it fixes bugs with negative caching as well as increases thread safety.
</pre>
</blockquote>
<pre wrap="" class="moz-quote-pre">
Thank you for this patch. I am not competent to review this code, but can test it. Really, our nscd with LDAP is a nightmare. I have set filters to narrow lookups, but with full directory, when nscd is runnig I have have such timings:
[host] ~# /usr/bin/time getent passwd > /dev/null
0.62 real 0.06 user 0.15 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
0.47 real 0.07 user 0.12 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
0.46 real 0.04 user 0.15 sys
After stopping nscd service:
[host] ~# /usr/bin/time getent passwd > /dev/null
0.15 real 0.03 user 0.06 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
0.16 real 0.01 user 0.08 sys
Unfortunately, with this patch applied there is no much improvement:
[host] ~# /usr/bin/time getent passwd > /dev/null
0.65 real 0.03 user 0.19 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
0.48 real 0.02 user 0.22 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
0.43 real 0.06 user 0.12 sys
The test were run on most recent stable/14 with net/nss-pam-ldapd as a Name Service Switch module for LDAP lookup.
--
Marek Zarychta
</pre>
</blockquote>
<pre wrap="" class="moz-quote-pre">
</pre>
</blockquote>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
Marek Zarychta</pre>
</body>
</html>
--------------aFGhZ5Q0xWRThRteF60eofjP--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5235bcad-4ff9-4aa1-97ac-30766e114cef>