From nobody Sun Oct 6 18:35:18 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XM9vQ56vhz5YPCR for ; Sun, 06 Oct 2024 18:35:30 +0000 (UTC) (envelope-from david@crossfamilyweb.com) Received: from mail.dcrosstech.com (syn-024-097-005-251.biz.spectrum.com [24.97.5.251]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail.dcrosstech.com", Issuer "DCrossTech.com LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XM9vP6Q0Pz4T5M for ; Sun, 6 Oct 2024 18:35:29 +0000 (UTC) (envelope-from david@crossfamilyweb.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of david@crossfamilyweb.com designates 24.97.5.251 as permitted sender) smtp.mailfrom=david@crossfamilyweb.com; dmarc=none X-Virus-Scanned: amavisd-new at dcrosstech.com Received: from [10.1.12.130] (d130.office.dcrosstech.com [10.1.12.130]) (authenticated bits=0) by mail.dcrosstech.com (8.15.2/8.15.2) with ESMTPSA id 496IZIZL021697 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO) for ; Sun, 6 Oct 2024 18:35:19 GMT (envelope-from david@crossfamilyweb.com) X-Authentication-Warning: mail.priv.dcrosstech.com: Host d130.office.dcrosstech.com [10.1.12.130] claimed to be [10.1.12.130] Message-ID: <21941f7f-ce32-e277-a565-b1db3b3841ab@crossfamilyweb.com> Date: Sun, 6 Oct 2024 14:35:18 -0400 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.15.0 Content-Language: en-US To: FreeBSD Hackers From: "David E. Cross" Subject: Review D38047 ... and then there was one.... Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.20 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:11351, ipnet:24.97.0.0/16, country:US]; MIME_TRACE(0.00)[0:+]; FREEFALL_USER(0.00)[david]; RCVD_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; HAS_XAW(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[crossfamilyweb.com]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4XM9vP6Q0Pz4T5M X-Spamd-Bar: --- As I have been prodding about open reviews, there is now only one that hasn't had any action.  One that is complete and in main (OMGYES), and one that is at the finish line (I think). That leaves just D38047 ( https://reviews.freebsd.org/D38047 ) ... Submitted Jan 13, **2023**, had initial peer review requests, I made all requested changes (or explained why I didn't)  Made my last substantive changes Aug 4, **2023**, I have since then semi-regularly updated with fresh rebases, but no other changes. Please, love to get some eyes on this.  As it stands nscd is completely useless for LDAP for getgroupmembership (and really ANY implementation that defines a specific implementation of getgroupmembership, since it will then bypass the non-existent NSCD version).  Additionally it fixes bugs with negative caching as well as increases thread safety. I've been running this successfully with LDAP for years now. From nobody Sun Oct 6 19:56:27 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XMCjQ43qvz5YTKP for ; Sun, 06 Oct 2024 19:56:58 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (plan-b.pwste.edu.pl [IPv6:2001:678:618::40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "plan-b.pwste.edu.pl", Issuer "GEANT OV RSA CA 4" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XMCjQ0j02z4f12 for ; Sun, 6 Oct 2024 19:56:57 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Authentication-Results: mx1.freebsd.org; none Received: from [192.168.7.70] (dom.potoki.eu [62.133.140.50]) (authenticated bits=0) by plan-b.pwste.edu.pl (8.18.1/8.17.2) with ESMTPSA id 496JuRqb002276 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Sun, 6 Oct 2024 21:56:27 +0200 (CEST) (envelope-from zarychtam@plan-b.pwste.edu.pl) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=plan-b.pwste.edu.pl; s=plan-b-mailer; t=1728244589; bh=grVoWHr1/vITyq3ZFk0pt9mBrjCWL20rvd0TnAXhT0k=; h=Date:Subject:To:References:From:In-Reply-To; b=PuJOWbIEyseJNXWlwKYksa0Sy68DBD7RHENxEy3Z7AwFH5NWSK2xQJkoMds1VJlsh NMcu9545bZalzR5a8yrKyf18z5SQB213ORi1Dmg4zG/q10638oEzNkMXfAtlzq0Kpi 8R9p159DzREcObgugR4vcsVNUr7ILlCgWeAjWx/s02UQQaVEdaikrAC7Y0yegT+kCd +pL04MSPP8tuA38MsqmEtLxHlF9R8Vec28TiGnhxW3o0Rgm+QGCVNbb+eLOENWP79k nvB7TM/uwplpNNBOhxFbOZI22r/Sado67GU6k2ZDxAlmUUWAkdKEbCBlTlVtsJnUZN pbg2X382xAghg== X-Authentication-Warning: plan-b.pwste.edu.pl: Host dom.potoki.eu [62.133.140.50] claimed to be [192.168.7.70] Message-ID: <6bfd6c61-38aa-4038-b54b-6c17b5b69ada@plan-b.pwste.edu.pl> Date: Sun, 6 Oct 2024 21:56:27 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Review D38047 ... and then there was one.... To: "David E. Cross" , FreeBSD Hackers References: <21941f7f-ce32-e277-a565-b1db3b3841ab@crossfamilyweb.com> Content-Language: en-US From: Marek Zarychta Autocrypt: addr=zarychtam@plan-b.pwste.edu.pl; keydata= xsBNBFfi3cMBCADLecMTFXad4uDXqv3eRuB4qJJ8G9tzzFezeRnnwxOsPdytW5ES2z1ibSrR IsiImx6+PTqrAmXpTInxAi7yiZGdSiONRI4CCxKY9d1YFiNYT/2WyNXCekm9x29YeIU7x0JB Llbz0f/9HC+styBIu2H+PY/X98Clzm110CS+n/b9l1AtiGxTiVFj7/uavYAKxH6LNWnbkuc5 v8EVNc7NkEcl5h7Z9X5NEtzDxTOiBIFQ/kOT7LAtkYUPo1lqLeOM2DtWSXTXQgXl0zJI4iP1 OAu4qQYm2nXwq4b2AH9peknelvnt1mpfgDCGSKnhc26q6ibTfMwydp+tvUtQIQYpA6b9ABEB AAHNN01hcmVrIFphcnljaHRhIChQbGFuLWIpIDx6YXJ5Y2h0YW1AcGxhbi1iLnB3c3RlLmVk dS5wbD7CwHcEEwEIACEFAlfi4LkCGwMFCwkIBwIGFQgJCgsCBBYCAwECHgECF4AACgkQHZW8 vIFppoJXdgf8D9X3VRFSNaR9lthSx/+uqas17J3FJKBo1xMQsC2a+44vzNvYJSuPGLLJ+LW2 HPVazjP/BWZJbxOYpliY4zxNRU0YCp0BLIVLibc//yax+mE42FND/+NiIZhqJscl6MLPrSwo sIwXec4XYkldkyqW/xBbBYXoIkBqdKB9j5j42Npy1IV/RizOSdmvTWY27ir8e/yGMR1RLr4F 8P5K3OWTdlGy2H2F/3J8bIPBLG6FpaIyLQw4dHSx8V02PYqDxK1cNo2kAOnU8PnZL/AGuMOH iv3MN1VYL8ehcmpBBsrZGebQJxrjY2/5IaTSgp9xHYT70kshuU6Qb97vk1mOjNZxgc7ATQRX 4t3DAQgA10h6RCXuBLMHxq5B8X/ZIlj9sgLoeyfRdDZEc9rT2KUeUJVHDsbvOFf4/7F1ovWY hJbA6GK/LUZeHHTjnbZcH1uDYQeHly4UOLxeEvhGoz4JhS2C7JzN/uRnwbdOAUbJr8rUj/IY a7gk906rktsc/Ldrxrxh7O6WO0JCh2XO/p4pDfEwwB37g4xHprSab28ECYJ9JMbtA8Sy4M55 g3+GQ28FvSlGnx48OoGXU2BZdc1vZKSQmNOlikB+9/hDX8zdYWVfDaX1TLQ8Ib4+xTUmapza mV/bxIsaZRBw+jFjLQHhTbIMfPEU+4mxFDvTdbKPruKPqVf1ydgMnPZWngowdwARAQABwsBf BBgBCAAJBQJX4t3DAhsMAAoJEB2VvLyBaaaC6qkIAJs9sDPqrqW0bYoRfzY6XjDWQ59p9tJi v8aogxacQNCfAu+WkJ8PNVUtC1dlVcG5NnZ80gXzd1rc8ueIvXlvdanUt/jZd8jbb3gaDbK3 wh1yMCGBl/1fOJTyEGYv1CRojv97KK89KP5+r8x1P1iHcSrunlDNqGxTMydNCwBH23QcOM+m u4spKnJ/s0VRBkw3xoKBZfZza6fTQ4gTpAipjyk7ldOGBV+PvkKATdhK2yLwuWXhKbg/GRlD 1r5P0gxzSqfV4My+KJuc2EDcrqp1y0wOpE1m9iZqCcd0fup5f7HDsYlLWshr7NQl28f6+fQb sylq/j672BHXsdeqf/Ip9V4= In-Reply-To: <21941f7f-ce32-e277-a565-b1db3b3841ab@crossfamilyweb.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:206006, ipnet:2001:678:618::/48, country:PL] X-Rspamd-Queue-Id: 4XMCjQ0j02z4f12 X-Spamd-Bar: ---- W dniu 6.10.2024 o 20:35, David E. Cross pisze: > Please, love to get some eyes on this.  As it stands nscd is > completely useless for LDAP for getgroupmembership (and really ANY > implementation that defines a specific implementation of > getgroupmembership, since it will then bypass the non-existent NSCD > version).  Additionally it fixes bugs with negative caching as well as > increases thread safety. Thank you for this patch. I am not competent to review this code, but can test it. Really, our nscd with LDAP is a nightmare. I have set filters to narrow lookups, but with full directory, when  nscd is runnig I have have such timings: [host] ~# /usr/bin/time getent passwd > /dev/null         0.62 real         0.06 user         0.15 sys [host] ~# /usr/bin/time getent passwd > /dev/null         0.47 real         0.07 user         0.12 sys [host] ~# /usr/bin/time getent passwd > /dev/null         0.46 real         0.04 user         0.15 sys After stopping nscd service: [host] ~# /usr/bin/time getent passwd > /dev/null         0.15 real         0.03 user         0.06 sys [host] ~# /usr/bin/time getent passwd > /dev/null         0.16 real         0.01 user         0.08 sys Unfortunately, with this patch applied there is no much improvement: [host] ~# /usr/bin/time getent passwd > /dev/null         0.65 real         0.03 user         0.19 sys [host] ~# /usr/bin/time getent passwd > /dev/null         0.48 real         0.02 user         0.22 sys [host] ~# /usr/bin/time getent passwd > /dev/null         0.43 real         0.06 user         0.12 sys The test were run on most recent stable/14 with net/nss-pam-ldapd as a Name Service Switch module for LDAP lookup. -- Marek Zarychta From nobody Sun Oct 6 20:04:01 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XMCvK30pdz5YT6C for ; Sun, 06 Oct 2024 20:05:33 +0000 (UTC) (envelope-from david@crossfamilyweb.com) Received: from mail.dcrosstech.com (syn-024-097-005-251.biz.spectrum.com [24.97.5.251]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail.dcrosstech.com", Issuer "DCrossTech.com LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XMCvJ5NNRz4gPK for ; Sun, 6 Oct 2024 20:05:32 +0000 (UTC) (envelope-from david@crossfamilyweb.com) Authentication-Results: mx1.freebsd.org; none X-Virus-Scanned: amavisd-new at dcrosstech.com Received: from smtpclient.apple (211.sub-174-231-53.myvzw.com [174.231.53.211]) (authenticated bits=0) by mail.dcrosstech.com (8.15.2/8.15.2) with ESMTPSA id 496K4nje031524 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Sun, 6 Oct 2024 20:05:12 GMT (envelope-from david@crossfamilyweb.com) X-Authentication-Warning: mail.priv.dcrosstech.com: Host 211.sub-174-231-53.myvzw.com [174.231.53.211] claimed to be smtpclient.apple Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: David Cross List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (1.0) Subject: Re: Review D38047 ... and then there was one.... Date: Sun, 6 Oct 2024 16:04:01 -0400 Message-Id: <5FCA5CA0-7F07-44A7-95A3-672AB8C2C6A1@crossfamilyweb.com> References: <6bfd6c61-38aa-4038-b54b-6c17b5b69ada@plan-b.pwste.edu.pl> Cc: FreeBSD Hackers In-Reply-To: <6bfd6c61-38aa-4038-b54b-6c17b5b69ada@plan-b.pwste.edu.pl> To: Marek Zarychta X-Mailer: iPhone Mail (22A3370) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:11351, ipnet:24.97.0.0/16, country:US] X-Rspamd-Queue-Id: 4XMCvJ5NNRz4gPK X-Spamd-Bar: ---- Here=E2=80=99s the thing. The current implementation of nscd DOESN=E2=80=99T= WORK at all. There is a symbol that nscd exports that libc is supposed to u= se as a flag to bypass lookups for nscd itself. But that symbol isn=E2=80=99= t exported right.=20 You will need to recompile libc and nscd. (I just do a buildworld to make su= re i get everything as there are makefile changes related to the aforementio= ned symbol changes.=20 And then after that make sure to check getgroupentries too > On Oct 6, 2024, at 3:57=E2=80=AFPM, Marek Zarychta wrote: >=20 > =EF=BB=BFW dniu 6.10.2024 o 20:35, David E. Cross pisze: >> Please, love to get some eyes on this. As it stands nscd is completely u= seless for LDAP for getgroupmembership (and really ANY implementation that d= efines a specific implementation of getgroupmembership, since it will then b= ypass the non-existent NSCD version). Additionally it fixes bugs with negat= ive caching as well as increases thread safety. >=20 > Thank you for this patch. I am not competent to review this code, but can t= est it. Really, our nscd with LDAP is a nightmare. I have set filters to nar= row lookups, but with full directory, when nscd is runnig I have have such t= imings: >=20 > [host] ~# /usr/bin/time getent passwd > /dev/null > 0.62 real 0.06 user 0.15 sys > [host] ~# /usr/bin/time getent passwd > /dev/null > 0.47 real 0.07 user 0.12 sys > [host] ~# /usr/bin/time getent passwd > /dev/null > 0.46 real 0.04 user 0.15 sys >=20 > After stopping nscd service: >=20 > [host] ~# /usr/bin/time getent passwd > /dev/null > 0.15 real 0.03 user 0.06 sys > [host] ~# /usr/bin/time getent passwd > /dev/null > 0.16 real 0.01 user 0.08 sys >=20 > Unfortunately, with this patch applied there is no much improvement: >=20 > [host] ~# /usr/bin/time getent passwd > /dev/null > 0.65 real 0.03 user 0.19 sys > [host] ~# /usr/bin/time getent passwd > /dev/null > 0.48 real 0.02 user 0.22 sys > [host] ~# /usr/bin/time getent passwd > /dev/null > 0.43 real 0.06 user 0.12 sys >=20 > The test were run on most recent stable/14 with net/nss-pam-ldapd as a Nam= e Service Switch module for LDAP lookup. >=20 > -- > Marek Zarychta >=20 From nobody Sun Oct 6 20:13:58 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XMD5M05ZVz5YTx5 for ; Sun, 06 Oct 2024 20:14:15 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (plan-b.pwste.edu.pl [IPv6:2001:678:618::40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "plan-b.pwste.edu.pl", Issuer "GEANT OV RSA CA 4" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XMD5L52y3z4j4t for ; Sun, 6 Oct 2024 20:14:14 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Authentication-Results: mx1.freebsd.org; none Received: from [192.168.7.70] (dom.potoki.eu [62.133.140.50]) (authenticated bits=0) by plan-b.pwste.edu.pl (8.18.1/8.17.2) with ESMTPSA id 496KDwLb002051 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Sun, 6 Oct 2024 22:13:59 +0200 (CEST) (envelope-from zarychtam@plan-b.pwste.edu.pl) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=plan-b.pwste.edu.pl; s=plan-b-mailer; t=1728245639; bh=UKO0GUtrbT2R8Ti/Gw/bSO59eC9e6yUqtW2m5+//P/U=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=U0ASO9ncNFvZQcmKDI9BFA+EnDOw7WtOfNywdIZVbM+KKcCMZ2KWGukxK4h2bpAYL WAr8IdTRGVJU8EfYJJYNlN/QDVolm/0pz5cDB+AUhkhUvbad95y7CV7E+qm0WQcMEa DUYrifbyz61kOEYqOcom+Fo/h4B11TVl/CMYNfw6+lCswRk7Y8HxQ1fdRQa3KTrFMh Hi2hU81Rugsn4Bko8yrm+rK8559fv3ga5T5A+kdaxJlHjHwUUjpLp3HPN4l0Y/wMJC E2/nMycQZ/lraOZwcyQBYOy86RQvNjCmtPg32bqz05CBI7NOrPtOAmeOa1dX+MRM+Q 7pgD0YQaL2zwg== X-Authentication-Warning: plan-b.pwste.edu.pl: Host dom.potoki.eu [62.133.140.50] claimed to be [192.168.7.70] Content-Type: multipart/alternative; boundary="------------aFGhZ5Q0xWRThRteF60eofjP" Message-ID: <5235bcad-4ff9-4aa1-97ac-30766e114cef@plan-b.pwste.edu.pl> Date: Sun, 6 Oct 2024 22:13:58 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Review D38047 ... and then there was one.... To: David Cross Cc: FreeBSD Hackers References: <6bfd6c61-38aa-4038-b54b-6c17b5b69ada@plan-b.pwste.edu.pl> <5FCA5CA0-7F07-44A7-95A3-672AB8C2C6A1@crossfamilyweb.com> Content-Language: en-US From: Marek Zarychta Autocrypt: addr=zarychtam@plan-b.pwste.edu.pl; keydata= xsBNBFfi3cMBCADLecMTFXad4uDXqv3eRuB4qJJ8G9tzzFezeRnnwxOsPdytW5ES2z1ibSrR IsiImx6+PTqrAmXpTInxAi7yiZGdSiONRI4CCxKY9d1YFiNYT/2WyNXCekm9x29YeIU7x0JB Llbz0f/9HC+styBIu2H+PY/X98Clzm110CS+n/b9l1AtiGxTiVFj7/uavYAKxH6LNWnbkuc5 v8EVNc7NkEcl5h7Z9X5NEtzDxTOiBIFQ/kOT7LAtkYUPo1lqLeOM2DtWSXTXQgXl0zJI4iP1 OAu4qQYm2nXwq4b2AH9peknelvnt1mpfgDCGSKnhc26q6ibTfMwydp+tvUtQIQYpA6b9ABEB AAHNN01hcmVrIFphcnljaHRhIChQbGFuLWIpIDx6YXJ5Y2h0YW1AcGxhbi1iLnB3c3RlLmVk dS5wbD7CwHcEEwEIACEFAlfi4LkCGwMFCwkIBwIGFQgJCgsCBBYCAwECHgECF4AACgkQHZW8 vIFppoJXdgf8D9X3VRFSNaR9lthSx/+uqas17J3FJKBo1xMQsC2a+44vzNvYJSuPGLLJ+LW2 HPVazjP/BWZJbxOYpliY4zxNRU0YCp0BLIVLibc//yax+mE42FND/+NiIZhqJscl6MLPrSwo sIwXec4XYkldkyqW/xBbBYXoIkBqdKB9j5j42Npy1IV/RizOSdmvTWY27ir8e/yGMR1RLr4F 8P5K3OWTdlGy2H2F/3J8bIPBLG6FpaIyLQw4dHSx8V02PYqDxK1cNo2kAOnU8PnZL/AGuMOH iv3MN1VYL8ehcmpBBsrZGebQJxrjY2/5IaTSgp9xHYT70kshuU6Qb97vk1mOjNZxgc7ATQRX 4t3DAQgA10h6RCXuBLMHxq5B8X/ZIlj9sgLoeyfRdDZEc9rT2KUeUJVHDsbvOFf4/7F1ovWY hJbA6GK/LUZeHHTjnbZcH1uDYQeHly4UOLxeEvhGoz4JhS2C7JzN/uRnwbdOAUbJr8rUj/IY a7gk906rktsc/Ldrxrxh7O6WO0JCh2XO/p4pDfEwwB37g4xHprSab28ECYJ9JMbtA8Sy4M55 g3+GQ28FvSlGnx48OoGXU2BZdc1vZKSQmNOlikB+9/hDX8zdYWVfDaX1TLQ8Ib4+xTUmapza mV/bxIsaZRBw+jFjLQHhTbIMfPEU+4mxFDvTdbKPruKPqVf1ydgMnPZWngowdwARAQABwsBf BBgBCAAJBQJX4t3DAhsMAAoJEB2VvLyBaaaC6qkIAJs9sDPqrqW0bYoRfzY6XjDWQ59p9tJi v8aogxacQNCfAu+WkJ8PNVUtC1dlVcG5NnZ80gXzd1rc8ueIvXlvdanUt/jZd8jbb3gaDbK3 wh1yMCGBl/1fOJTyEGYv1CRojv97KK89KP5+r8x1P1iHcSrunlDNqGxTMydNCwBH23QcOM+m u4spKnJ/s0VRBkw3xoKBZfZza6fTQ4gTpAipjyk7ldOGBV+PvkKATdhK2yLwuWXhKbg/GRlD 1r5P0gxzSqfV4My+KJuc2EDcrqp1y0wOpE1m9iZqCcd0fup5f7HDsYlLWshr7NQl28f6+fQb sylq/j672BHXsdeqf/Ip9V4= In-Reply-To: <5FCA5CA0-7F07-44A7-95A3-672AB8C2C6A1@crossfamilyweb.com> X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:206006, ipnet:2001:678:618::/48, country:PL] X-Rspamd-Queue-Id: 4XMD5L52y3z4j4t X-Spamd-Bar: ---- This is a multi-part message in MIME format. --------------aFGhZ5Q0xWRThRteF60eofjP Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit W dniu 6.10.2024 o 22:04, David Cross pisze: > Here’s the thing. The current implementation of nscd DOESN’T WORK at all. There is a symbol that nscd exports that libc is supposed to use as a flag to bypass lookups for nscd itself. But that symbol isn’t exported right. > > You will need to recompile libc and nscd. (I just do a buildworld to make sure i get everything as there are makefile changes related to the aforementioned symbol changes. Yes, without world installed this patched nscd won't even start: [host] /usr/src# service nscd start Starting nscd. limits: setrlimit pipebuf: Invalid argument /etc/rc.d/nscd: WARNING: failed to start nscd > And then after that make sure to check getgroupentries too The number of groups is much lower, so the whole difference is like 0.01 vs 0.02 s, but yes, lookup is 100% faster when nscd  is not running (regardless to the state of the  application of  the patch). > >> On Oct 6, 2024, at 3:57 PM, Marek Zarychta wrote: >> >> W dniu 6.10.2024 o 20:35, David E. Cross pisze: >>> Please, love to get some eyes on this. As it stands nscd is completely useless for LDAP for getgroupmembership (and really ANY implementation that defines a specific implementation of getgroupmembership, since it will then bypass the non-existent NSCD version). Additionally it fixes bugs with negative caching as well as increases thread safety. >> Thank you for this patch. I am not competent to review this code, but can test it. Really, our nscd with LDAP is a nightmare. I have set filters to narrow lookups, but with full directory, when nscd is runnig I have have such timings: >> >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.62 real 0.06 user 0.15 sys >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.47 real 0.07 user 0.12 sys >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.46 real 0.04 user 0.15 sys >> >> After stopping nscd service: >> >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.15 real 0.03 user 0.06 sys >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.16 real 0.01 user 0.08 sys >> >> Unfortunately, with this patch applied there is no much improvement: >> >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.65 real 0.03 user 0.19 sys >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.48 real 0.02 user 0.22 sys >> [host] ~# /usr/bin/time getent passwd > /dev/null >> 0.43 real 0.06 user 0.12 sys >> >> The test were run on most recent stable/14 with net/nss-pam-ldapd as a Name Service Switch module for LDAP lookup. >> >> -- >> Marek Zarychta >> > > -- Marek Zarychta --------------aFGhZ5Q0xWRThRteF60eofjP Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
W dniu 6.10.2024 o 22:04, David Cross pisze:
Here’s the thing. The current implementation of nscd DOESN’T WORK at all. There is a symbol that nscd exports that libc is supposed to use as a flag to bypass lookups  for nscd itself. But that symbol isn’t exported right. 

You will need to recompile libc and nscd. (I just do a buildworld to make sure i get everything as there are makefile changes related to the aforementioned symbol changes.

Yes, without world installed this patched nscd won't even start:

[host] /usr/src# service nscd start
Starting nscd.
limits: setrlimit pipebuf: Invalid argument
/etc/rc.d/nscd: WARNING: failed to start nscd

And then after that make sure to check getgroupentries too

The number of groups is much lower, so the whole difference is like 0.01 vs 0.02 s, but yes, lookup is 100% faster when nscd  is not running (regardless to the state of the  application of  the patch).



On Oct 6, 2024, at 3:57 PM, Marek Zarychta <zarychtam@plan-b.pwste.edu.pl> wrote:

W dniu 6.10.2024 o 20:35, David E. Cross pisze:

Please, love to get some eyes on this.  As it stands nscd is completely useless for LDAP for getgroupmembership (and really ANY implementation that defines a specific implementation of getgroupmembership, since it will then bypass the non-existent NSCD version).  Additionally it fixes bugs with negative caching as well as increases thread safety.

Thank you for this patch. I am not competent to review this code, but can test it. Really, our nscd with LDAP is a nightmare. I have set filters to narrow lookups, but with full directory, when  nscd is runnig I have have such timings:

[host] ~# /usr/bin/time getent passwd > /dev/null
        0.62 real         0.06 user         0.15 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.47 real         0.07 user         0.12 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.46 real         0.04 user         0.15 sys

After stopping nscd service:

[host] ~# /usr/bin/time getent passwd > /dev/null
        0.15 real         0.03 user         0.06 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.16 real         0.01 user         0.08 sys

Unfortunately, with this patch applied there is no much improvement:

[host] ~# /usr/bin/time getent passwd > /dev/null
        0.65 real         0.03 user         0.19 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.48 real         0.02 user         0.22 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.43 real         0.06 user         0.12 sys

The test were run on most recent stable/14 with net/nss-pam-ldapd as a Name Service Switch module for LDAP lookup.

--
Marek Zarychta







-- 
Marek Zarychta
--------------aFGhZ5Q0xWRThRteF60eofjP-- From nobody Sun Oct 6 20:17:01 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XMD8q6HLnz5YVV5 for ; Sun, 06 Oct 2024 20:17:15 +0000 (UTC) (envelope-from dcrosstech@gmail.com) Received: from mail-qt1-x82b.google.com (mail-qt1-x82b.google.com [IPv6:2607:f8b0:4864:20::82b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XMD8q0q7Fz4k3j for ; Sun, 6 Oct 2024 20:17:15 +0000 (UTC) (envelope-from dcrosstech@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-qt1-x82b.google.com with SMTP id d75a77b69052e-4582a0b438aso31707991cf.0 for ; Sun, 06 Oct 2024 13:17:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728245834; x=1728850634; darn=freebsd.org; h=to:in-reply-to:cc:references:message-id:date:subject:mime-version :from:content-transfer-encoding:from:to:cc:subject:date:message-id :reply-to; bh=mtBmG92uzu4Z7Y6yIiDsLsqEFwl312eTIHJexqcQUiw=; b=LgiyQH+XEgDLom1lB/yXqxssnQWez7mb5t7tK+FF2Jah5xEm1XGQW9o3Lx4hfUgXEM AiPXtUtV/0MZ8ZVx1YWw8PGkkOUGwPxJc/B8+1is9HLxsJdnnvEu1qe6oZGllkYzexoo tSgcjN5xpMBQoyYuWdwbeQu3AePZihLVNQz41vWYsIe/jn6EQAuniKUHFMbbtvPBxrLr LH4G+Z91KvCqJR7jilIlb3UaRGdxF/qU29+ooQ1k5o4jL3gzuKJDzBivK2yypqe6qF3F IsksZQk57nDiYNkAJiGxZYTXVVRg3fj39zSss2m8rhL7/5wxRhm4h4K1obNa37r2isEY JEzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728245834; x=1728850634; h=to:in-reply-to:cc:references:message-id:date:subject:mime-version :from:content-transfer-encoding:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mtBmG92uzu4Z7Y6yIiDsLsqEFwl312eTIHJexqcQUiw=; b=NPgh5pBuZtRCzjZXtaYZzCsvghzSycDFTANbL7scVXsFI7UPCWGu5FqKLXNQwowh9Q 3btyxOkKp1H6MbslqyD6LhpiTnppCtYz2HlquDWZSFDW9BLZGM0z2LxQDDYGWSnjrKKL 0pvgGqBwhhcU+97D89rrb0KTaZw76MGFmnibuPAoKq5FzAoWqwRF4oDaXtXqvfGegUvb MjcQLfKGiF37Mej1rJRbqGRyRjWyc+y0xChuObowe/ExkAsg30kUIwSHcyjYtdw8PGpT eUv4Y13c05xueqbFRfwlO40BXEH8fXQkJsrRHXUUp5srLNWc18knJK0Kx1GBRI0l4E1e F5SA== X-Forwarded-Encrypted: i=1; AJvYcCVzl0A+XuA987kAHPWjq5YSHrFpAEuw+TCv/CdjevvmA6A16T9wjXQ0Js0oJgSeS2SS7Brpvp9V6IikPLvolMU=@freebsd.org X-Gm-Message-State: AOJu0YxIsurpPaFczrJt7KhLyctVkZSvCXN69Q28MZcQdtqZg7QNL6J/ trPLXXgmTMsaAwizao2yC8xmOSMUoCkxwO18NLa853o+dbqN6h/5Ado9sQ== X-Google-Smtp-Source: AGHT+IHCKT/V1/JyUozQXgOzFtxXNlxOcyIUk8X4AAVcGyi/x/cVNmrEv4lUXpnwSejl4ooQL8CQkw== X-Received: by 2002:ac8:58d0:0:b0:458:14c8:3ed5 with SMTP id d75a77b69052e-45d9ba454d3mr164522271cf.22.1728245834201; Sun, 06 Oct 2024 13:17:14 -0700 (PDT) Received: from smtpclient.apple ([2600:1017:b0c7:642f:dd00:ec27:f3c8:336c]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45da75ed2e6sm19708741cf.76.2024.10.06.13.17.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 06 Oct 2024 13:17:13 -0700 (PDT) Content-Type: multipart/alternative; boundary=Apple-Mail-697F7FFB-CB5F-46F7-86CF-38BA14C80E2C Content-Transfer-Encoding: 7bit From: David Cross List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (1.0) Subject: Re: Review D38047 ... and then there was one.... Date: Sun, 6 Oct 2024 16:17:01 -0400 Message-Id: References: <5235bcad-4ff9-4aa1-97ac-30766e114cef@plan-b.pwste.edu.pl> Cc: David Cross , FreeBSD Hackers In-Reply-To: <5235bcad-4ff9-4aa1-97ac-30766e114cef@plan-b.pwste.edu.pl> To: Marek Zarychta X-Mailer: iPhone Mail (22A3370) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4XMD8q0q7Fz4k3j X-Spamd-Bar: ---- --Apple-Mail-697F7FFB-CB5F-46F7-86CF-38BA14C80E2C Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hmm= . It shouldn=E2=80=99t fail in that way regardless. Which version of freebsd= are you running on?

On O= ct 6, 2024, at 4:14=E2=80=AFPM, Marek Zarychta <zarychtam@plan-b.pwste.ed= u.pl> wrote:

=EF=BB=BF =20 =20 =20
W dniu 6.10.2024 o 22:04, David Cros= s pisze:
Here=E2=80=99s the thing. The c=
urrent implementation of nscd DOESN=E2=80=99T WORK at all. There is a symbol=
 that nscd exports that libc is supposed to use as a flag to bypass lookups =
 for nscd itself. But that symbol isn=E2=80=99t exported right.=20

You will need to recompile libc and nscd. (I just do a buildworld to make su=
re i get everything as there are makefile changes related to the aforementio=
ned symbol changes.=20

Yes, without world installed this patched nscd won't even start:

[host] /usr/src# service nscd start
Starting nscd.
limits: setrlimit pipebuf: Invalid argument
/etc/rc.d/nscd: WARNING: failed to start nscd

And then after that make sure t=
o check getgroupentries too

The number of groups is much lower, so the whole difference is like 0.01 vs 0.02 s, but yes, lookup is 100% faster when nscd  is= not running (regardless to the state of the  application of = the patch).


      

        
On Oct 6, 2024, at 3:57=E2=80=
=AFPM, Marek Zarychta <zarychtam@plan-b.pwste.edu.pl> wrote:

=EF=BB=BFW dniu 6.10.2024 o 20:35, David E. Cross pisze:


        

          
Please, love to get some ey=
es on this.  As it stands nscd is completely useless for LDAP for getgroupme=
mbership (and really ANY implementation that defines a specific implementati=
on of getgroupmembership, since it will then bypass the non-existent NSCD ve=
rsion).  Additionally it fixes bugs with negative caching as well as increas=
es thread safety.


        

        
Thank you for this patch. I a=
m not competent to review this code, but can test it. Really, our nscd with L=
DAP is a nightmare. I have set filters to narrow lookups, but with full dire=
ctory, when  nscd is runnig I have have such timings:

[host] ~# /usr/bin/time getent passwd > /dev/null
        0.62 real         0.06 user         0.15 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.47 real         0.07 user         0.12 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.46 real         0.04 user         0.15 sys

After stopping nscd service:

[host] ~# /usr/bin/time getent passwd > /dev/null
        0.15 real         0.03 user         0.06 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.16 real         0.01 user         0.08 sys

Unfortunately, with this patch applied there is no much improvement:

[host] ~# /usr/bin/time getent passwd > /dev/null
        0.65 real         0.03 user         0.19 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.48 real         0.02 user         0.22 sys
[host] ~# /usr/bin/time getent passwd > /dev/null
        0.43 real         0.06 user         0.12 sys

The test were run on most recent stable/14 with net/nss-pam-ldapd as a Name S=
ervice Switch module for LDAP lookup.

--
Marek Zarychta



      

      



    

    


    

    
--=20
Marek Zarychta

 =20


=

--Apple-Mail-697F7FFB-CB5F-46F7-86CF-38BA14C80E2C--

From nobody Sun Oct  6 22:36:28 2024
X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XMHFn0wdxz5Ydt3
	for ; Sun, 06 Oct 2024 22:36:45 +0000 (UTC)
	(envelope-from marklmi@yahoo.com)
Received: from sonic314-20.consmr.mail.gq1.yahoo.com (sonic314-20.consmr.mail.gq1.yahoo.com [98.137.69.83])
	(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
	(Client did not present a certificate)
	by mx1.freebsd.org (Postfix) with ESMTPS id 4XMHFk4Qgbz46gp
	for ; Sun,  6 Oct 2024 22:36:42 +0000 (UTC)
	(envelope-from marklmi@yahoo.com)
Authentication-Results: mx1.freebsd.org;
	dkim=pass header.d=yahoo.com header.s=s2048 header.b=Tq98QpPM;
	spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com;
	dmarc=pass (policy=reject) header.from=yahoo.com
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1728254200; bh=BaNolPGmCQrclujp1F6OsFnFlBixUgA1ZYJ8TFe5hxc=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=Tq98QpPMvgT1j0e8vwiiQRLEIhNkqHXWJja6a+st9RRiConvlWxxyiLV0eDJrBGv+H/TVjOL6SZw3cSFTzzKjm+BDF0+1Rd6T+S4LilF+pSL0oxz/+9apLkHSd/1btkaZccDe6RhHMb7PyCBaZJXApFLFF+kuhEKu9587XwT2pWEqf1SOfmlGhbd/9UFlNyRMiBeE62KuckEtvn/l6kTHA7BqYuibuzXr/MbuWML3rAlLFklqz9iH58coxPY8XW8S2EwbKvK4E9Mq/NPHRCzXsUCEfPIgjFlDc5eurQ3nPKsDw/lMX9BTGlEUjqjdHLeLqc2aE76X9osBHwPDHOAag==
X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1728254200; bh=AeVD4aIR1jQ3hXV0OMmm0gtcM1Wb6DmKvHYdU1VbWqy=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=qiODthpvv4KeIUPETEs8FqjlNj0y9c6CLZRIc3giLBwCXq0tmsjeM60t80hUnyaLKLyIvdCqgI6XsmWF7o2bEzJamRCq5Ba2/Wledl4FE3zQ2pfuO2IHHxid0gTJzgu6RjndTPzJZRHOR5KH3FYh0qgn4GT2P0cCVgOH+m+H4lK3SByEEIHBsHbx6Q0k6JsBC33WPeJjTp/zeS6X0qKlDNxZAOf5lWwPVmcz3RVdEiXIbOWCoEVINzOGdoDKbVZDu4JFtWScrG8n4frZ8QGXmNlRxuoR2JA+GK6n+Y0WY63XniPI2KG5sOSp6sPDys6UZaJtHQBvsFLsyObf83TvCw==
X-YMail-OSG: rMkaFzcVM1n4mYvxhFFlUVhPcz0Pd30b.yfX3sc0SQPvRngk3s6lA1g9Y7K_17O
 Jgjh.oHZXF1S8aruKRh6u76bmr7KM__rss6CIyTfgcnNyCnes0Y_HtDxGtnPtFwa.vLx0hNQtCt1
 Fh2Plp9BjDZKHyJJeamB.X.BscBjht0jyL8Cf4_Z76wUuot11ZShFDdE28j0s._Nw2BtT.glz9lW
 xBxrxMAmW2ruVGB.SPt6D6KGLQqDdSppZqNVEaT7K70TM2AC5AR3UZPSARM9ZVv8fynRNxBYp4xp
 GROZKuQD64ogJgbZuE6tKZ882C2kzhYyEfyUzwom9vas65dvZjW8lH4W2LpBoGv9NFtPDkgowcAj
 oSDgEUwuBjZ6iT.UZmhAioZ9OHg.4IOgF00K3KaAoHgCxULJQodA3wrJ99XHaP_dxr2AcWANceyE
 fy.CYBrozs7WzuJxypCRdU19egZ9t5EabJP94MA7tLcYQcutJpl73lJ.QRZzuBR3kHpDqhxzRDh2
 .GvYhfW.PVYBsZPauSohnq_34F2F3AN5G0Af7DdTYjINQRQEDckOFtoSHUVIfknd8E5Et_oMEKtF
 Ln0O4F8gj_lfBw8.cSsZfXFUxpDrPqP7dJxNoeT2UpT0Rxy3AT3Dfv8ilySZfTWK1l5S33CnoTyz
 3I.xvOBCJqEnpKLtSWdIJJizgD_eSpOBMQk.0qGaQJ1Jy0mdaHOU9THznfzuC9j8eFjMW0tAFDEa
 OXwbvQJ4UM8Ah46NOy9acV3r1pYt2LfOX9hr6yl3qZPEtTi9QRtTZcJem6Nu3eeY4Kr0laZnTg71
 aOctsagU4roRc7OmCfd.mHivM9sIt6zNthonTD604yf2w7mP7UMafwN5qA4ycHN2Ut.q6DPeptl7
 oKpnyzMTt.vxsTzsLalTdnwzdcKFTo8QlK.nx5KNSNbNIeyHGxbAbV7sgvRlF5n_rNNUAByVUyHC
 JciYEwRfkF3TUaS8ahOAXoSIc7Oe9SIgWi.Oe77y3.CDAqelwICWIDoJkqFchqSQZq_63GG5W7JL
 GWMvXWql3j_uG4Cv7J5joVvY5wvLmXxMRB7zTWtVyRXgMyUUmybL1fQmwlGLN3MSKZ8brlHoJNBu
 xUqzUiFYf9h.wIDYsFt87gJtnAXTh7whuBH1cmazRsWqAFgcMWnvGgCRkyS_JUsLa68uj8piwySQ
 S7_DXuJToZSdnZ.sWGK7PT.yIlDbLaWvWAGPjPCVqmO0ayJiaJKChS5Yyr0SzcHD4JlIzi8ThmB1
 po45E_jazzMyYIRR15WgkvAIsfvRdW7qrSzPnuPL8M.JfJBXGgG.ZL.9O81dOvYmpab3O6CgsuWv
 AtWucffx_P7hsRgtK3VTXkyP8tWeyOa1cVpDh994nU6XSX5rVkW8g8am7XG3ODqiRlIy7NkBv0xS
 UuxI8HilvR_q2PhjA68M3YSPN3cNmMq4gY1o3OcJcz5lqZgAPUAxQjnzT4jRBCtGZEOEvajBouJs
 zz4lxlBtSvQXl812gVJKvJ5RRgDH1XaAkPrYhsJV.k8uzGnXrU4GvxI1R1fjmwNJsl8W8Sy9.AEw
 tQThAfYFXcdIIBPEiE5oR7fIXJcLW7w1f_nBoMedTzUpjbFQXtdjigVHDHFiovb90Swxmu3MmUtl
 rQT5_v7pZLBq59KBi_YKfOMyh2lmGO3Qbw5EkPOrkrtGQgqeaE0lmqSgUIbWaDZ983g8Cxud9llC
 anwr6I6DnegATgsAVYt1X1kN5ZWje6ZjAU16JOYz6osEbcoQqySljoM6VsGaOQAGunphiTnnInNY
 iNf7Ud.5D_Dd0_.AxJRImEwNtcYErqug2C75iuDz.p3iXONsCXDu_nI7iSHZjYzX00UzZNV9ACMp
 zN6506cpObpC9_kYrgsVrOQtIUt_9vgKm9H7lzLl2jzst3O2C7LrG4sgQSxsnA_jDoWGNMIsUcFb
 OWIo1jtVczRX3D44qvcc3LtbLM3QkmHULymsUXvZubMWnQGMxPnptWzeUWj_efjN3cZLq8hngGIE
 v2yLSC.xKkk5H9sygUwGvGI2aOfXuByVuthDKVsR80ACkhZWseUAZqSOrSyeuGh82KXCAz9wgRNZ
 0e5niK9Vc4B443WEm_IqcJDo.Kkd0fy6onjau.vB6Mv9zbufMGMJE27DJ2nYG73kLTsQZw.GIcPx
 znbkrcigFGrdmDoyKMzP6AGvX_LDFpQX5dEG53AIcvakeSjqYWjG_W1q8A_dWhAx.CSOFJJ9hyeP
 3wAOlV1WT8L6J1kS7XQZcwdME6GpJ1SuxYNwuXVTJ7GsAKofIT7Gwr4hFrAb9e4iIDWyb5BloZTW
 p3MCIEQ1mcuh12bRm
X-Sonic-MF: 
X-Sonic-ID: 526f58ee-d24f-458e-ae3b-cf98ff36d752
Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Sun, 6 Oct 2024 22:36:40 +0000
Received: by hermes--production-gq1-5d95dc458-5j27b (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 9dd4251eb222fe9b07ef2cefb4b90595;
          Sun, 06 Oct 2024 22:36:39 +0000 (UTC)
From: Mark Millard 
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
List-Id: Technical discussions relating to FreeBSD 
List-Archive: https://lists.freebsd.org/archives/freebsd-hackers
List-Help: 
List-Post: 
List-Subscribe: 
List-Unsubscribe: 
Sender: owner-freebsd-hackers@FreeBSD.org
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\))
Subject: Re: Review D38047 ... and then there was one....
Message-Id: <43F7106E-C5C5-4467-9B72-1D7C51E5430B@yahoo.com>
Date: Sun, 6 Oct 2024 15:36:28 -0700
To: Marek Zarychta ,
 freebsd-hackers 
X-Mailer: Apple Mail (2.3776.700.51)
References: <43F7106E-C5C5-4467-9B72-1D7C51E5430B.ref@yahoo.com>
X-Spamd-Result: default: False [-4.00 / 15.00];
	NEURAL_HAM_MEDIUM(-1.00)[-1.000];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	NEURAL_HAM_SHORT(-1.00)[-0.999];
	DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject];
	R_SPF_ALLOW(-0.20)[+ptr:yahoo.com];
	R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048];
	MIME_GOOD(-0.10)[text/plain];
	FROM_HAS_DN(0.00)[];
	TO_DN_ALL(0.00)[];
	RCVD_TLS_LAST(0.00)[];
	RCPT_COUNT_TWO(0.00)[2];
	MIME_TRACE(0.00)[0:+];
	ARC_NA(0.00)[];
	DWL_DNSWL_NONE(0.00)[yahoo.com:dkim];
	FREEMAIL_FROM(0.00)[yahoo.com];
	DKIM_TRACE(0.00)[yahoo.com:+];
	ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	TO_MATCH_ENVRCPT_SOME(0.00)[];
	RCVD_COUNT_TWO(0.00)[2];
	FROM_EQ_ENVFROM(0.00)[];
	FREEMAIL_ENVFROM(0.00)[yahoo.com];
	MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org];
	APPLE_MAILER_COMMON(0.00)[];
	MID_RHS_MATCH_FROM(0.00)[];
	RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.83:from];
	RCVD_IN_DNSWL_NONE(0.00)[98.137.69.83:from]
X-Rspamd-Queue-Id: 4XMHFk4Qgbz46gp
X-Spamd-Bar: ---

Marek Zarychta  wrote on
Date: Sun, 06 Oct 2024 20:13:58 UTC :

> W dniu 6.10.2024 o 22:04, David Cross pisze:
> > Here=E2=80=99s the thing. The current implementation of nscd =
DOESN=E2=80=99T WORK at all. There is a symbol that nscd exports that =
libc is supposed to use as a flag to bypass lookups for nscd itself. But =
that symbol isn=E2=80=99t exported right.
> >
> > You will need to recompile libc and nscd. (I just do a buildworld to =
make sure i get everything as there are makefile changes related to the =
aforementioned symbol changes.
>=20
> Yes, without world installed this patched nscd won't even start:
>=20
> [host] /usr/src# service nscd start
> Starting nscd.
> limits: setrlimit pipebuf: Invalid argument
> /etc/rc.d/nscd: WARNING: failed to start nscd
. . .

This note is only about the "limits: setrlimit pipebuf:
Invalid argument" notice.

The main [so: 15] pipebuf related commits were done during
2024-Sep-20 UTC. If one has a kernel that predates those but
a world for which limits now tries to use the new pipebuf
material, the result is messages like that:

limits: setrlimit pipebuf: Invalid argument

(or related such messages).


For reference for main [so: 15]:

Fri, 20 Sep 2024
    . . .=20
    =E2=80=A2 git: 3458bbd39778 - main - kernel: add RLIMIT_PIPEBUF =
Konstantin Belousov=20
    =E2=80=A2 git: 54a8d1fbbf65 - main - getrlimit(2): document =
RLIMIT_PIPEBUF Konstantin Belousov=20
    =E2=80=A2 git: a4c04958f526 - main - libutil: support RLIMIT_PIPEBUF =
Konstantin Belousov=20
    =E2=80=A2 git: 5d92f20c7d31 - main - bin/sh: support RLIMIT_PIPEBUF =
Konstantin Belousov=20
    =E2=80=A2 git: f54f41403d14 - main - usr.bin/limits: support =
RLIMIT_PIPEBUF Konstantin Belousov=20
    =E2=80=A2 git: b029e29e0d8b - main - login.conf: add a placeholder =
for the pipebuf limit Konstantin Belousov=20
    =E2=80=A2 git: 80133d678ecb - main - procstat: support =
RLIMIT_PIPEBUF Konstantin Belousov=20
    =E2=80=A2 git: 8ae779832c6f - main - privs: add PRIV_PIPEBUF =
Konstantin Belousov=20
    =E2=80=A2 git: 7672cbef2c1e - main - pipes: reserve configured =
percentage of buffers zone to superuser Konstantin Belousov
    . . .=20
    =E2=80=A2 git: d6074f73af5c - main - pipe: use pipe subsystem KVA =
counter instead of pipe_map size Konstantin Belousov=20
    =E2=80=A2 git: 40769168a5ee - main - pipespace_new(): decrease =
uidinfo pipebuf usage if reservation check failed Konstantin Belousov
    . . .=20
    =E2=80=A2 git: a52b30ff98cd - main - sys_pipe: consistently use =
cr_ruidinfo for accounting of pipebuf Konstantin Belousov=20
    =E2=80=A2 git: af96ccc6a508 - main - uifree(9): report non-zero =
values for all shared resources Konstantin Belousov=20
    =E2=80=A2 git: 2c1963d46335 - main - procfs rlimit: handle pipebuf =
Konstantin Belousov=20
    =E2=80=A2 git: c84d8db0ab3d - main - procfs: ensure that =
RLIMIT_IDENT is properly updated when a limit is added Konstantin =
Belousov

The combination of an older kernel and a newer world will not be
nicely behaved when any non-kernel code from the above ends up
involved.


stable/14 has now also had the commits:

Sat, 05 Oct 2024
    =E2=80=A2 git: 1508dce2502d - stable/14 - procfs: ensure that =
RLIMIT_IDENT is properly updated when a limit is added Konstantin =
Belousov=20
    . . .
    =E2=80=A2 git: b7eecc86c3bd - stable/14 - kernel: add RLIMIT_PIPEBUF =
Konstantin Belousov=20
    =E2=80=A2 git: d20f0dae2f97 - stable/14 - getrlimit(2): document =
RLIMIT_PIPEBUF Konstantin Belousov=20
    =E2=80=A2 git: a03f7c040ce7 - stable/14 - libutil: support =
RLIMIT_PIPEBUF Konstantin Belousov=20
    =E2=80=A2 git: d5ed8778bf3b - stable/14 - bin/sh: support =
RLIMIT_PIPEBUF Konstantin Belousov=20
    =E2=80=A2 git: 25902860b270 - stable/14 - usr.bin/limits: support =
RLIMIT_PIPEBUF Konstantin Belousov=20
    =E2=80=A2 git: 524b9810de6a - stable/14 - login.conf: add a =
placeholder for the pipebuf limit Konstantin Belousov=20
    =E2=80=A2 git: 6600090e678e - stable/14 - procstat: support =
RLIMIT_PIPEBUF Konstantin Belousov=20
    =E2=80=A2 git: fd9babb1b85f - stable/14 - privs: add PRIV_PIPEBUF =
Konstantin Belousov=20
    =E2=80=A2 git: d532d9926ee7 - stable/14 - pipes: reserve configured =
percentage of buffers zone to superuser Konstantin Belousov=20
    =E2=80=A2 git: 6536b979b856 - stable/14 - pipe: use pipe subsystem =
KVA counter instead of pipe_map size Konstantin Belousov=20
    =E2=80=A2 git: a8c663bb4261 - stable/14 - pipespace_new(): decrease =
uidinfo pipebuf usage if reservation check failed Konstantin Belousov=20
    =E2=80=A2 git: c15b2e046e8c - stable/14 - sys_pipe: consistently use =
cr_ruidinfo for accounting of pipebuf Konstantin Belousov=20
    . . .
    =E2=80=A2 git: fc9070bf1d16 - stable/14 - procfs rlimit: handle =
pipebuf Konstantin Belousov=20
    . . .

Again, the combination of an older kernel and a newer world will not be
nicely behaved.

=3D=3D=3D
Mark Millard
marklmi at yahoo.com