From nobody Wed Nov 8 19:29:24 2023 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SQZsJ3fRdz4ynsl for ; Wed, 8 Nov 2023 19:29:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SQZsJ1ZQ1z3dnk for ; Wed, 8 Nov 2023 19:29:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699471764; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ut6u/iCg6zyAuvpxNyGpX0NoRvWAl202lLQz4T+Dak0=; b=e5OVfxTM5iq0BMB/+t2TrcEUOWLmwECep4uiLSEZPkqeHTr2guYC9SJUV6M95WpD6N82pg ox6iaW6bxaUqijNYt/inx4iF6v6YO86bbnLRSEno5WwdQ+00LbS7rDV1PRT8Swidrj5+6p CV8i/HBS7w9hHsERe4XotwtH7+GqPBWMXM3YLhKvn6fRByE49SHUG1zTGcqF40SHtSFptw gaJpyGzuwmHgX53J3swa1K+qfVlwqtpB15Gnmlc9JBEwALje1NxGMNvM0Es4QZdjowJQQj reuzApgEYM5E9ssKFrxvcPx/aG4V7l16ERCXWR+waOxSwGWzRyfSHpBz22/VOA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1699471764; a=rsa-sha256; cv=none; b=vq2iE+6D+AGLgshDrVasmMfSBuhvw/aUVmgGbelJ5kgj1b2XFqygXWBl3Bbizu1O4v1GEq X0QlTgmXjWnnwBciIm2dL2Bo42t2+dYbG4KJZMIK4kv4SlbcpYDj9hGL7+H5x3mY9ZSj+h TZe0LbP40XjND0gsWTeci6KcENA6UIHWlsO96wH+aTexzod1KiiDFFWwSkoa1P1HP9QE66 DkAWOctZq9ocjt+xD0fSbYm1/2VbK8bWla+lbx7OPpAo0DVKpPhGIAHwcyN/f44OveRw0n bviiaj9XgzV0QHEXnnznPXLvG0ktMCpmpMOtlT2WEXDWLMkzahi5wU1FIhyQMg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SQZsJ0dWJzlSt for ; Wed, 8 Nov 2023 19:29:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3A8JTOBU036992 for ; Wed, 8 Nov 2023 19:29:24 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3A8JTOBx036991 for bugs@FreeBSD.org; Wed, 8 Nov 2023 19:29:24 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 274968] rpc_clnt_create: The dg_cv variable uses absurdly too much memory Date: Wed, 08 Nov 2023 19:29:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: asomers@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D274968 Bug ID: 274968 Summary: rpc_clnt_create: The dg_cv variable uses absurdly too much memory Product: Base System Version: 14.0-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: bin Assignee: bugs@FreeBSD.org Reporter: asomers@FreeBSD.org The clng_dt_create function allocates global variables named dg_cv and dg_fd_locks, which are then used by the rest of the routines in clnt_dg.c.= =20 They are never freed, and live for the life of the process. These variables are arrays indexed by file descriptor. To ensure that they have enough spa= ce, they are sized according to RLIMIT_NOFILE. The problem is that resource li= mit can be very, very, big. On my production servers, it's autoscaled to 22608= 720. So any process that does _anything_ involving NIS must allocate 259 MiB ju= st for these variables. The exact same mistake is made in clnt_vc.c, with the vc_fd_locks and vc_cv variables. These add up to about 984 MB for every sshd process on my system (I'm not s= ure how 259 gets multiplied to 984, but valgrind --tool=3Dmassif does show that= all the memory is coming from clnt_dg_create). A few hundred of those sshd processes and my entire server falls over. Even with sshd rate limiting, a= bout half of my server's physical RAM is used just for this one stupid array variable. Stupidly, it appears that there's no need for a huge array. Instead, the condvar could've been part of the CLIENT structure. Even stupider, this variable _never_ gets used in some simple applications = like getgrouplist. So we allocate it, fault in every page, and then never use i= t.=20 :facepalm: --=20 You are receiving this mail because: You are the assignee for the bug.=