From owner-freebsd-stable@freebsd.org Fri Feb 5 10:54:11 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 469D1544846 for ; Fri, 5 Feb 2021 10:54:11 +0000 (UTC) (envelope-from freebsd-stable@gomor.org) Received: from onyphe.fr (super1.onyphe.io [54.36.107.100]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DXC360LRbz4r9K for ; Fri, 5 Feb 2021 10:54:09 +0000 (UTC) (envelope-from freebsd-stable@gomor.org) Received: (qmail 10758 invoked by uid 0); 5 Feb 2021 10:54:07 -0000 Received: from unknown (HELO www.onyphe.io) (172.16.6.254) by smtpout.jail with SMTP; 5 Feb 2021 10:54:07 -0000 MIME-Version: 1.0 Date: Fri, 05 Feb 2021 11:54:07 +0100 From: GomoR To: John Baldwin Cc: freebsd-stable@freebsd.org Subject: Re: Suspected mbuf leak with Nginx + sendfile + TLS in 12.2-STABLE In-Reply-To: <069535216479ce00859e4bcbf499f8a1@gomor.org> References: <9c56bfda-725c-9c2a-9db3-4599abcfeaa0@FreeBSD.org> <069535216479ce00859e4bcbf499f8a1@gomor.org> User-Agent: Roundcube Webmail/1.4.8 Message-ID: <8f02057bee5e8196644e85bbe7f8b31e@gomor.org> X-Sender: freebsd-stable@gomor.org Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DXC360LRbz4r9K X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-stable@gomor.org designates 54.36.107.100 as permitted sender) smtp.mailfrom=freebsd-stable@gomor.org X-Spamd-Result: default: False [-3.30 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:54.36.107.100]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[gomor.org]; RBL_DBL_DONT_QUERY_IPS(0.00)[54.36.107.100:from]; SPAMHAUS_ZRD(0.00)[54.36.107.100:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.999]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:16276, ipnet:54.36.0.0/16, country:FR]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-stable]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 10:54:11 -0000 On 2021-02-05 09:11, GomoR wrote: >> The first step I would do if possible would be to bisect between the >> last >> known working version and the version that is known to be broken to >> determine which commit introduced the problem. One thing that could >> help >> here is to see if you can reproduce the problem using a 12.2 kernel on >> a >> 12.1 world + ports. If you can, then you can limit your bisecting to >> just >> building new kernels which will make that process quicker. We have reinstalled from scratch our system with FreeBSD 12.1-RELEASE. We then have installed just enough of our software stack to reproduce the issue. No problem with a stock 12.1-RELEASE kernel, but problem arise after installkernel with the latest 12.2-STABLE. We then turned off all our customizations, including some specific sysctl.conf values. The bug didn't triggered. After dissecting our sysctl values, the faulty one has been identified: kern.ipc.maxsockbuf=157286400 This value is 75 times the default value (2097152). Restoring the default value fixes the issue. After some tests, the bug is triggered starting somewhere to 64 times the default value (134217728). There was no issue with this setting in 12.1-RELEASE, but there is in 12.2-RELEASE. Do you have some insights onto why it causes that mbuf problems? In the meantime, we have our solution, but we are willing to help identify if that's a kernel bug or just a real bad idea to set maxsockbuf to such a high value. Best regards,