From owner-freebsd-virtualization@freebsd.org Thu May 27 04:08:03 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DB49463167B for ; Thu, 27 May 2021 04:08:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FrDnH5cNhz3kfx for ; Thu, 27 May 2021 04:08:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id BE8E963167A; Thu, 27 May 2021 04:08:03 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BD15B631902 for ; Thu, 27 May 2021 04:08:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FrDnH4dWmz3kZh for ; Thu, 27 May 2021 04:08:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7E16C1D726 for ; Thu, 27 May 2021 04:08:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 14R483GC054441 for ; Thu, 27 May 2021 04:08:03 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 14R483U5054440 for virtualization@FreeBSD.org; Thu, 27 May 2021 04:08:03 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 168298] VirtualBox using AIO on a zvol crashes Date: Thu, 27 May 2021 04:08:00 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: tbr@acm.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Works As Intended X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2021 04:08:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D168298 Tom Rushworth changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tbr@acm.org --- Comment #46 from Tom Rushworth --- I may have found a clue towards the root cause of bug 168298 (and the dupli= cate 212128) where virtualbox-ose locks up or crashes doing heavy AIO. I found the location in the source code where virtualbox picks up the sysctl value: vfs.aio.max_aio_per_proc but could not find any place in the code that actually used it to make any = sort of decision. The virtualbox AIO code would attempt to grow the local numbe= r of allowed AIO requests whenever it got EAGAIN, without any limit. When the attempt to grow went past the sysctl value so that the attempt failed, bad things happened :). I have attached a tiny but very ugly patch that stops the attempt to grow t= he local resources when the number of such resources meets the AIO sysctl valu= e.=20 The patch makes my virtualbox version 6.1.20 running on FreeBSD 13.0 stop crashng even when the AIO sysctl values are left at the default installed values. It also (when adjusted for line number differences) makes my 5.2.34 virtualbox running on FreeBSD 12.1 with AIO stable. Both versions of virtualbox had a fairly repeatable crash or lockup with a Linux Mint 20 VM = when I tried to upgrade the kernel and headers. After applying the patch I was = able to upgrade the VMs with no problems and no changes to the AIO sysctl values. I do not suggest this patch as any sort of final solution :). Someone far = more knowledgable about the virtualbox code than I (since the first time I looke= d at it was 2 weeks ago) should use it as a starting point to make the same deci= sion about growing the number of local resources for AIO requests in a virtualbox consistent way (and style :) ). The patch would do as a workaround while we all wait for the virtualbox code gurus to improve it or come up with a bett= er alternative. --=20 You are receiving this mail because: You are the assignee for the bug.=