From nobody Sun Oct 6 20:13:58 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XMD5M05ZVz5YTx5 for ; Sun, 06 Oct 2024 20:14:15 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (plan-b.pwste.edu.pl [IPv6:2001:678:618::40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "plan-b.pwste.edu.pl", Issuer "GEANT OV RSA CA 4" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XMD5L52y3z4j4t for ; Sun, 6 Oct 2024 20:14:14 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Authentication-Results: mx1.freebsd.org; none Received: from [192.168.7.70] (dom.potoki.eu [62.133.140.50]) (authenticated bits=0) by plan-b.pwste.edu.pl (8.18.1/8.17.2) with ESMTPSA id 496KDwLb002051 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Sun, 6 Oct 2024 22:13:59 +0200 (CEST) (envelope-from zarychtam@plan-b.pwste.edu.pl) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=plan-b.pwste.edu.pl; s=plan-b-mailer; t=1728245639; bh=UKO0GUtrbT2R8Ti/Gw/bSO59eC9e6yUqtW2m5+//P/U=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=U0ASO9ncNFvZQcmKDI9BFA+EnDOw7WtOfNywdIZVbM+KKcCMZ2KWGukxK4h2bpAYL WAr8IdTRGVJU8EfYJJYNlN/QDVolm/0pz5cDB+AUhkhUvbad95y7CV7E+qm0WQcMEa DUYrifbyz61kOEYqOcom+Fo/h4B11TVl/CMYNfw6+lCswRk7Y8HxQ1fdRQa3KTrFMh Hi2hU81Rugsn4Bko8yrm+rK8559fv3ga5T5A+kdaxJlHjHwUUjpLp3HPN4l0Y/wMJC E2/nMycQZ/lraOZwcyQBYOy86RQvNjCmtPg32bqz05CBI7NOrPtOAmeOa1dX+MRM+Q 7pgD0YQaL2zwg== X-Authentication-Warning: plan-b.pwste.edu.pl: Host dom.potoki.eu [62.133.140.50] claimed to be [192.168.7.70] Content-Type: multipart/alternative; boundary="------------aFGhZ5Q0xWRThRteF60eofjP" Message-ID: <5235bcad-4ff9-4aa1-97ac-30766e114cef@plan-b.pwste.edu.pl> Date: Sun, 6 Oct 2024 22:13:58 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Review D38047 ... and then there was one.... To: David Cross Cc: FreeBSD Hackers References: <6bfd6c61-38aa-4038-b54b-6c17b5b69ada@plan-b.pwste.edu.pl> <5FCA5CA0-7F07-44A7-95A3-672AB8C2C6A1@crossfamilyweb.com> Content-Language: en-US From: Marek Zarychta Autocrypt: addr=zarychtam@plan-b.pwste.edu.pl; keydata= xsBNBFfi3cMBCADLecMTFXad4uDXqv3eRuB4qJJ8G9tzzFezeRnnwxOsPdytW5ES2z1ibSrR IsiImx6+PTqrAmXpTInxAi7yiZGdSiONRI4CCxKY9d1YFiNYT/2WyNXCekm9x29YeIU7x0JB Llbz0f/9HC+styBIu2H+PY/X98Clzm110CS+n/b9l1AtiGxTiVFj7/uavYAKxH6LNWnbkuc5 v8EVNc7NkEcl5h7Z9X5NEtzDxTOiBIFQ/kOT7LAtkYUPo1lqLeOM2DtWSXTXQgXl0zJI4iP1 OAu4qQYm2nXwq4b2AH9peknelvnt1mpfgDCGSKnhc26q6ibTfMwydp+tvUtQIQYpA6b9ABEB AAHNN01hcmVrIFphcnljaHRhIChQbGFuLWIpIDx6YXJ5Y2h0YW1AcGxhbi1iLnB3c3RlLmVk dS5wbD7CwHcEEwEIACEFAlfi4LkCGwMFCwkIBwIGFQgJCgsCBBYCAwECHgECF4AACgkQHZW8 vIFppoJXdgf8D9X3VRFSNaR9lthSx/+uqas17J3FJKBo1xMQsC2a+44vzNvYJSuPGLLJ+LW2 HPVazjP/BWZJbxOYpliY4zxNRU0YCp0BLIVLibc//yax+mE42FND/+NiIZhqJscl6MLPrSwo sIwXec4XYkldkyqW/xBbBYXoIkBqdKB9j5j42Npy1IV/RizOSdmvTWY27ir8e/yGMR1RLr4F 8P5K3OWTdlGy2H2F/3J8bIPBLG6FpaIyLQw4dHSx8V02PYqDxK1cNo2kAOnU8PnZL/AGuMOH iv3MN1VYL8ehcmpBBsrZGebQJxrjY2/5IaTSgp9xHYT70kshuU6Qb97vk1mOjNZxgc7ATQRX 4t3DAQgA10h6RCXuBLMHxq5B8X/ZIlj9sgLoeyfRdDZEc9rT2KUeUJVHDsbvOFf4/7F1ovWY hJbA6GK/LUZeHHTjnbZcH1uDYQeHly4UOLxeEvhGoz4JhS2C7JzN/uRnwbdOAUbJr8rUj/IY a7gk906rktsc/Ldrxrxh7O6WO0JCh2XO/p4pDfEwwB37g4xHprSab28ECYJ9JMbtA8Sy4M55 g3+GQ28FvSlGnx48OoGXU2BZdc1vZKSQmNOlikB+9/hDX8zdYWVfDaX1TLQ8Ib4+xTUmapza mV/bxIsaZRBw+jFjLQHhTbIMfPEU+4mxFDvTdbKPruKPqVf1ydgMnPZWngowdwARAQABwsBf BBgBCAAJBQJX4t3DAhsMAAoJEB2VvLyBaaaC6qkIAJs9sDPqrqW0bYoRfzY6XjDWQ59p9tJi v8aogxacQNCfAu+WkJ8PNVUtC1dlVcG5NnZ80gXzd1rc8ueIvXlvdanUt/jZd8jbb3gaDbK3 wh1yMCGBl/1fOJTyEGYv1CRojv97KK89KP5+r8x1P1iHcSrunlDNqGxTMydNCwBH23QcOM+m u4spKnJ/s0VRBkw3xoKBZfZza6fTQ4gTpAipjyk7ldOGBV+PvkKATdhK2yLwuWXhKbg/GRlD 1r5P0gxzSqfV4My+KJuc2EDcrqp1y0wOpE1m9iZqCcd0fup5f7HDsYlLWshr7NQl28f6+fQb sylq/j672BHXsdeqf/Ip9V4= In-Reply-To: <5FCA5CA0-7F07-44A7-95A3-672AB8C2C6A1@crossfamilyweb.com> X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:206006, ipnet:2001:678:618::/48, country:PL] X-Rspamd-Queue-Id: 4XMD5L52y3z4j4t X-Spamd-Bar: ---- This is a multi-part message in MIME format. --------------aFGhZ5Q0xWRThRteF60eofjP Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit W dniu 6.10.2024 o 22:04, David Cross pisze: > Here’s the thing. The current implementation of nscd DOESN’T WORK at all. There is a symbol that nscd exports that libc is supposed to use as a flag to bypass lookups for nscd itself. But that symbol isn’t exported right. > > You will need to recompile libc and nscd. (I just do a buildworld to make sure i get everything as there are makefile changes related to the aforementioned symbol changes. Yes, without world installed this patched nscd won't even start: [host] /usr/src# service nscd start Starting nscd. limits: setrlimit pipebuf: Invalid argument /etc/rc.d/nscd: WARNING: failed to start nscd > And then after that make sure to check getgroupentries too The number of groups is much lower, so the whole difference is like 0.01 vs 0.02 s, but yes, lookup is 100% faster when nscd  is not running (regardless to the state of the  application of  the patch). > >> On Oct 6, 2024, at 3:57 PM, Marek Zarychta wrote: >> >> W dniu 6.10.2024 o 20:35, David E. Cross pisze: >>> Please, love to get some eyes on this. As it stands nscd is completely useless for LDAP for getgroupmembership (and really ANY implementation that defines a specific implementation of getgroupmembership, since it will then bypass the non-existent NSCD version). Additionally it fixes bugs with negative caching as well as increases thread safety. >> Thank you for this patch. I am not competent to review this code, but can test it. Really, our nscd with LDAP is a nightmare. I have set filters to narrow lookups, but with full directory, when nscd is runnig I have have such timings: >> >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.62 real 0.06 user 0.15 sys >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.47 real 0.07 user 0.12 sys >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.46 real 0.04 user 0.15 sys >> >> After stopping nscd service: >> >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.15 real 0.03 user 0.06 sys >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.16 real 0.01 user 0.08 sys >> >> Unfortunately, with this patch applied there is no much improvement: >> >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.65 real 0.03 user 0.19 sys >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.48 real 0.02 user 0.22 sys >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.43 real 0.06 user 0.12 sys >> >> The test were run on most recent stable/14 with net/nss-pam-ldapd as a Name Service Switch module for LDAP lookup. >> >> -- >> Marek Zarychta >> > > -- Marek Zarychta --------------aFGhZ5Q0xWRThRteF60eofjP Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
W dniu 6.10.2024 o 22:04, David Cross pisze:
Here’s the thing. The current implementation of nscd DOESN’T WORK at all. There is a symbol that nscd exports that libc is supposed to use as a flag to bypass lookups  for nscd itself. But that symbol isn’t exported right. 

You will need to recompile libc and nscd. (I just do a buildworld to make sure i get everything as there are makefile changes related to the aforementioned symbol changes.

Yes, without world installed this patched nscd won't even start:

[host] /usr/src# service nscd start
Starting nscd.
limits: setrlimit pipebuf: Invalid argument
/etc/rc.d/nscd: WARNING: failed to start nscd

And then after that make sure to check getgroupentries too

The number of groups is much lower, so the whole difference is like 0.01 vs 0.02 s, but yes, lookup is 100% faster when nscd  is not running (regardless to the state of the  application of  the patch).



On Oct 6, 2024, at 3:57 PM, Marek Zarychta <zarychtam@plan-b.pwste.edu.pl> wrote:

W dniu 6.10.2024 o 20:35, David E. Cross pisze:

Please, love to get some eyes on this.  As it stands nscd is completely useless for LDAP for getgroupmembership (and really ANY implementation that defines a specific implementation of getgroupmembership, since it will then bypass the non-existent NSCD version).  Additionally it fixes bugs with negative caching as well as increases thread safety.

Thank you for this patch. I am not competent to review this code, but can test it. Really, our nscd with LDAP is a nightmare. I have set filters to narrow lookups, but with full directory, when  nscd is runnig I have have such timings:

[host] ~# /usr/bin/time getent passwd > /dev/null
        0.62 real         0.06 user         0.15 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.47 real         0.07 user         0.12 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.46 real         0.04 user         0.15 sys

After stopping nscd service:

[host] ~# /usr/bin/time getent passwd > /dev/null
        0.15 real         0.03 user         0.06 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.16 real         0.01 user         0.08 sys

Unfortunately, with this patch applied there is no much improvement:

[host] ~# /usr/bin/time getent passwd > /dev/null
        0.65 real         0.03 user         0.19 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.48 real         0.02 user         0.22 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.43 real         0.06 user         0.12 sys

The test were run on most recent stable/14 with net/nss-pam-ldapd as a Name Service Switch module for LDAP lookup.

--
Marek Zarychta







-- 
Marek Zarychta
--------------aFGhZ5Q0xWRThRteF60eofjP--