From owner-freebsd-virtualization@freebsd.org Sun Apr 28 04:41:20 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47FB81571D44 for ; Sun, 28 Apr 2019 04:41:20 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B0D1583B09 for ; Sun, 28 Apr 2019 04:41:19 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=Message-ID:Subject:To:From:Date:In-Reply-To; bh=W/lfflfF1874GaVYTQ4wz1pvNnjmE4ebzGOFV9EHnLE=; b=hzapJ7TYH5Hs6r2jfr5VA7u/4K PmifB3JWQqt671qpg6pDTHj0XM4pp87GVKm9AuhWeajSOIeYn35x5YaIoWYPwEySS9fTY9wjSQWWg 6ukWZ/4QeofoZS0UTgJ7XR1nICqm8Nn7IQtUlbtGNYGNo+63m1s8fbgT115sXCiOiQ3E=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hKbcs-000C6i-2D for freebsd-virtualization@freebsd.org; Sun, 28 Apr 2019 11:41:18 +0700 Date: Sun, 28 Apr 2019 11:41:18 +0700 From: Victor Sudakov To: freebsd-virtualization@freebsd.org Subject: [vm-bhyve] shutting down VMs and kern.init_shutdown_timeout Message-ID: <20190428044118.GA46284@admin.sibptus.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="RnlQjJ0d97Da+TV1" Content-Disposition: inline X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.4 (2019-03-13) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2019 04:41:20 -0000 --RnlQjJ0d97Da+TV1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Dear Colleagues, The shutdown of vm-bhyve (and probably other bhyve-based VM managers) is commanded by "service vm shutdown." From man init(8) I've learned that "if the /etc/rc.shutdown does not terminate within 120 seconds, init will terminate it. The timeout can be configured via the sysctl variable kern.init_shutdown_timeout." Does that mean that any VM is supposed to shut down within 2 minutes in a default FreeBSD host system (in reality probably even less), or else the bhyve process will be mercilessly killed?=20 This is completely unsuitable for Windows guests who may take a considerable longer time to shutdown. Even in my FreeBSD guests I occasionally find the message that "root was not properly dismounted." Can something be done about this? What about CBSD, FreeNAS and other more specialized FreeBSD-based systems, what do they do about VM shutdown? Or I may be assuming incorrectly about the 2 minutes timeout. Please correct me then. --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --RnlQjJ0d97Da+TV1 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcxS7uAAoJEA2k8lmbXsY0aBwIAJixTcKJlxv3AK5MvmT1h+ZR vAzA6h1MKS3+4ZO/Fs9Vil5/tA4ITF+ONpCLnHqdYUS9//HabmZEMlNclCD0crbs V3T5mGny+GsoSQGXwl95ohQH4tojXcKf9cqZhTRIKl0xS90Z1CXiSKNBQR5iYYPh 7sV/VR+ZgeAE3eNlHmBzHeUyVrndtYJavN3zbnOVtIH9Gl+y/dUXi1ci0QbZ8VDp IRm8ItFOXNMu6J4VxljHKBjzvI/N6z/WQqUzgadmcVM6OpTut2vBc3+x/YfnRwg5 2LQIMyc/TenhpjYu4lmjzXJXpl/Llp0XtjX97PPIulMXsSK5fb+AQw98V4YQaYc= =ZbKa -----END PGP SIGNATURE----- --RnlQjJ0d97Da+TV1-- From owner-freebsd-virtualization@freebsd.org Mon Apr 29 07:45:25 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 874891583F08 for ; Mon, 29 Apr 2019 07:45:25 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1057869D62 for ; Mon, 29 Apr 2019 07:45:18 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10]) by gate2.intern.punkt.de with ESMTP id x3T7jB2E032213; Mon, 29 Apr 2019 09:45:11 +0200 (CEST) Received: from [217.29.44.36] ([217.29.44.36]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id x3T7jB3E090307; Mon, 29 Apr 2019 09:45:11 +0200 (CEST) (envelope-from hausen@punkt.de) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: [vm-bhyve] shutting down VMs and kern.init_shutdown_timeout From: "Patrick M. Hausen" In-Reply-To: <20190428044118.GA46284@admin.sibptus.ru> Date: Mon, 29 Apr 2019 09:45:11 +0200 Cc: freebsd-virtualization@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <1DE7A90B-E3AB-4F1E-9A3A-981B18751F8E@punkt.de> References: <20190428044118.GA46284@admin.sibptus.ru> To: Victor Sudakov X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 1057869D62 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of hausen@punkt.de designates 217.29.33.131 as permitted sender) smtp.mailfrom=hausen@punkt.de X-Spamd-Result: default: False [-2.89 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:217.29.32.0/20]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[punkt.de]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[mailin.pluspunkthosting.de,mailin.pluspunkthosting.de]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[131.33.29.217.list.dnswl.org : 127.0.10.0]; NEURAL_HAM_SHORT(-0.84)[-0.842,0]; IP_SCORE(-0.24)[ipnet: 217.29.32.0/20(-0.66), asn: 16188(-0.53), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16188, ipnet:217.29.32.0/20, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 07:45:25 -0000 Hi all, > Am 28.04.2019 um 06:41 schrieb Victor Sudakov : > "if the /etc/rc.shutdown does not terminate within 120 seconds, init = will > terminate it. The timeout can be configured via the sysctl variable > kern.init_shutdown_timeout.=E2=80=9C What=E2=80=99s keeping you from increasing that timeout for your particular requirements? IMHO 2 minutes are a reasonable default for a system that does not run hypervisor VMs - which is probably the vast majority. Anyone with the knowledge to implement a virtualized environment can simply set that to a more suitable value, can=E2=80=99t he/she? Kind regards, Patrick --=20 punkt.de GmbH Internet - Dienstleistungen - Beratung Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100 76133 Karlsruhe info@punkt.de http://punkt.de AG Mannheim 108285 Gf: Juergen Egeling From owner-freebsd-virtualization@freebsd.org Mon Apr 29 08:14:01 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 988F41584A70 for ; Mon, 29 Apr 2019 08:14:01 +0000 (UTC) (envelope-from ml@netfence.it) Received: from soth.netfence.it (net-2-44-121-52.cust.vodafonedsl.it [2.44.121.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mailserver.netfence.it", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1970E6AE43 for ; Mon, 29 Apr 2019 08:13:57 +0000 (UTC) (envelope-from ml@netfence.it) Received: from guardian.ventu (88-147-119-21.v4.ngi.it [88.147.119.21] (may be forged)) (authenticated bits=0) by soth.netfence.it (8.15.2/8.15.2) with ESMTPSA id x3T8Di5S093252 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 29 Apr 2019 10:13:46 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: soth.netfence.it: Host 88-147-119-21.v4.ngi.it [88.147.119.21] (may be forged) claimed to be guardian.ventu Subject: Re: [vm-bhyve] shutting down VMs and kern.init_shutdown_timeout To: freebsd-virtualization@freebsd.org References: <20190428044118.GA46284@admin.sibptus.ru> <1DE7A90B-E3AB-4F1E-9A3A-981B18751F8E@punkt.de> From: Andrea Venturoli Message-ID: <306069c2-1e7c-9f95-107d-adb0724d5e0e@netfence.it> Date: Mon, 29 Apr 2019 10:13:44 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <1DE7A90B-E3AB-4F1E-9A3A-981B18751F8E@punkt.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.83 X-Rspamd-Queue-Id: 1970E6AE43 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of ml@netfence.it designates 2.44.121.52 as permitted sender) smtp.mailfrom=ml@netfence.it X-Spamd-Result: default: False [-2.82 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.989,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:2.44.121.52]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_NONE(0.00)[]; MX_GOOD(-0.01)[mailserver.netfence.it]; NEURAL_HAM_SHORT(-0.09)[-0.094,0]; DMARC_NA(0.00)[netfence.it]; IP_SCORE(-0.42)[ip: (-1.23), ipnet: 2.44.0.0/16(-0.62), asn: 30722(-0.32), country: IT(0.05)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:30722, ipnet:2.44.0.0/16, country:IT]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 08:14:01 -0000 On 2019-04-29 09:45, Patrick M. Hausen wrote: > What’s keeping you from increasing that timeout for your > particular requirements? IMHO 2 minutes are a reasonable > default for a system that does not run hypervisor VMs - > which is probably the vast majority. > > Anyone with the knowledge to implement a virtualized > environment can simply set that to a more suitable value, > can’t he/she? The problem is: what is a reasonable value? Shutting down Windows can take literally hours if there are pending updates; that's of course unacceptable if you are running other services and need them up again in a reasonable time. Solution, of course, is not shutting down VMs, but suspending them. IIRC that is coming in bhyve in the future. bye av. From owner-freebsd-virtualization@freebsd.org Mon Apr 29 08:24:10 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D2311584D21 for ; Mon, 29 Apr 2019 08:24:10 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 709DB6B248 for ; Mon, 29 Apr 2019 08:24:09 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=CRrzj0Dy177uRw+ZyoniL8Mo6aYPqv7N4BXRTrspCYk=; b=fHW3AfcV4SJxddOXGVPfY8RGUk fjMOLJV/vLFVZ/spwo27hiqeNOTXBJmRM40OVICKR/4fXfPJPt2KNoi0ktR+Nw+GqsV1iARdSfrJn jF8Fz0STLAGZz8ILmJlUEc4m7cVJx4Sp3qO1Q5Ad8b3g7sYTlEmTyVYF66oPctK9yR8w=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hL1a4-000Ly2-9R for freebsd-virtualization@freebsd.org; Mon, 29 Apr 2019 15:24:08 +0700 Date: Mon, 29 Apr 2019 15:24:08 +0700 From: Victor Sudakov To: freebsd-virtualization@freebsd.org Subject: Re: [vm-bhyve] shutting down VMs and kern.init_shutdown_timeout Message-ID: <20190429082408.GA84142@admin.sibptus.ru> References: <20190428044118.GA46284@admin.sibptus.ru> <1DE7A90B-E3AB-4F1E-9A3A-981B18751F8E@punkt.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="SUOF0GtieIMvvwua" Content-Disposition: inline In-Reply-To: <1DE7A90B-E3AB-4F1E-9A3A-981B18751F8E@punkt.de> X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.4 (2019-03-13) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 08:24:10 -0000 --SUOF0GtieIMvvwua Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Patrick M. Hausen wrote: >=20 > > Am 28.04.2019 um 06:41 schrieb Victor Sudakov : > > "if the /etc/rc.shutdown does not terminate within 120 seconds, init wi= ll > > terminate it. The timeout can be configured via the sysctl variable > > kern.init_shutdown_timeout.=E2=80=9C > What=E2=80=99s keeping you from increasing that timeout for your > particular requirements?=20 Is that really necessary? Am I correct assuming that any VM is supposed to shut down within 2 minutes or else the bhyve processes will be killed? Or is this kern.init_shutdown_timeout only about the rc.shutdown script itself and not its children? > IMHO 2 minutes are a reasonable > default for a system that does not run hypervisor VMs - > which is probably the vast majority. >=20 > Anyone with the knowledge to implement a virtualized > environment=20 The lack of information about this caveat in https://wiki.freebsd.org/bhyve, man vm-bhyve, https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/virtualization-h= ost-bhyve.html and other sources suggests that this particular knowledge may be limited. Other hypervisors (non-bhyve) probably wait for all VMs to stop. > can simply set that to a more suitable value, > can=E2=80=99t he/she? Yes if they know about it. There should be a warning somewhere in the documentation if we want to be serious about FreeBSD as a hypervisor. Or better still, a warning that rcshutdown_timeout (if set) and kern.init_shutdown_timeout can be too low when running vm-bhyve and such is desirable. --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --SUOF0GtieIMvvwua Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcxrSoAAoJEA2k8lmbXsY0sKYH/AzVoUXQjHtobAe+SNn68pNT Fe0TdV6b4kr0kIVaGs8b5rT3EijiPx6HCsN/j9WWDGgpcgWSMcyZBSePKtHqtrpV ZZ5mrt8vnvfARJhvbENwU+0TMnCfIW4EJmSDx6j+G5MIBOhS8VMPIl2KZo1AxDWV CX4fLkgL1mVNoOk/UcFqDBMRUWpfafeFB9WWRyTNR9BSdZdi6jM4NQLUApCH42ia pjf98KNhWt+4DEI60QbvMn6mRqXkSHatEnH5QN0arMSSeXMTzbgz2c/kESXzkxKf ykTkcF4BHFbILeAhkbuYtfV8BLIGeX4spokh88jMXKQxGnZjRwTCHGboYzbKKw8= =nxTs -----END PGP SIGNATURE----- --SUOF0GtieIMvvwua-- From owner-freebsd-virtualization@freebsd.org Mon Apr 29 08:53:23 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B670C1585715 for ; Mon, 29 Apr 2019 08:53:23 +0000 (UTC) (envelope-from subbsd@gmail.com) Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com [IPv6:2a00:1450:4864:20::12e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FE716C0E8 for ; Mon, 29 Apr 2019 08:53:22 +0000 (UTC) (envelope-from subbsd@gmail.com) Received: by mail-lf1-x12e.google.com with SMTP id v1so7345152lfg.5 for ; Mon, 29 Apr 2019 01:53:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=cfe88GrRboIAz9LF0bTfD8cWoEUS57jL9slvz+tspQ0=; b=V/xMSYNxv0i0hYST4zhsQVMmIyEGopucCt+5JE07Ovsx41IRB844ZKZIRBsDhgYqp5 u4gJgHsZT13et/5+y6kwn0DfB9uK4SRLVaMoSR7HvB9V+0n/ePgWVaJixBFsM8NStfBx ATt9MEM9AGVdVyEIhYwq8UCKVAP7S/YrraFegPgDA5lA1iaKJqwazLelCoHROHMoRj3I wok7Zv0qavU0pGIe9Hxoto1h9CY1e5iJa+/2sInlICwXxG8PmtARQziH4GUnsAlXm0jW 7Rd3QrwnuUE3KuE3HAJtQqzyWixQF52cvbPRg96sqU2GAuBw2akETqBpwwi4J1GbYkNc wm1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=cfe88GrRboIAz9LF0bTfD8cWoEUS57jL9slvz+tspQ0=; b=auSogyUJhvj7NYF2m4DpubMXqWKI3tt0/qoR6uVypIpkuimMKU/KeKfctSN5R4DzNo MXS0esaqj5ZAd6Pv7R8BsBZRGF+6m5pDQJoGzswjN4NQDxLidyBvtsDei0+orhcm6++S IZGmGpMlCU36zNCZzB1xh1G77F6k6152dhlB9hYF9LA4PAZZhhiEltkyreXRRQk7Sn5t duhf8wQChlSdjwUy4sGsM082vxPI0ItbtGHBtl3ll8kq2Pu+jjwCXxz9QQQvybZ3cCrk yX+mQvrIRR/fjIwrWPbKb1xufcxXpOLmFRLWCt2R1DRbCcwTwN0OSIjivT88xHd5DoI9 hbZQ== X-Gm-Message-State: APjAAAVbR4KTx9lgtc1QEjUVieX6Bl5ycm7yX1InP1zt1DP5oKAUKsxI BWfWEVSTpSqDQiozq5VQlkr7JiNTQMn5BXHopDDiuz1x X-Google-Smtp-Source: APXvYqzRDmn5+B0OzDRPrOt+M+cXARP3XmdcLfeMhnIy5QjxET/mmymEyPojx3lIZK7vaLtpNwKhEpo+DmMv+XEQKdA= X-Received: by 2002:ac2:4355:: with SMTP id o21mr33529540lfl.123.1556528001102; Mon, 29 Apr 2019 01:53:21 -0700 (PDT) MIME-Version: 1.0 References: <20190428044118.GA46284@admin.sibptus.ru> In-Reply-To: <20190428044118.GA46284@admin.sibptus.ru> From: Subbsd Date: Mon, 29 Apr 2019 11:53:09 +0300 Message-ID: Subject: Re: [vm-bhyve] shutting down VMs and kern.init_shutdown_timeout To: Victor Sudakov Cc: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 8FE716C0E8 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=V/xMSYNx; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of subbsd@gmail.com designates 2a00:1450:4864:20::12e as permitted sender) smtp.mailfrom=subbsd@gmail.com X-Spamd-Result: default: False [-6.78 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[e.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_SHORT(-0.95)[-0.948,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-2.82)[ip: (-9.43), ipnet: 2a00:1450::/32(-2.32), asn: 15169(-2.27), country: US(-0.06)]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 08:53:24 -0000 On Sun, Apr 28, 2019 at 7:41 AM Victor Sudakov wrote: > Can something be done about this? What about CBSD, FreeNAS and other > more specialized FreeBSD-based systems, what do they do about VM > shutdown? > CBSD offers to reassign these settings during the initialization process ( as well as rcshutdown_timeout ): https://github.com/cbsd/cbsd/issues/374 I think others bhyve manager can do the same. From owner-freebsd-virtualization@freebsd.org Mon Apr 29 14:29:55 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 865921590FC6 for ; Mon, 29 Apr 2019 14:29:55 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D60A84B70 for ; Mon, 29 Apr 2019 14:29:55 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by mail-io1-xd34.google.com with SMTP id y6so9116865ior.5 for ; Mon, 29 Apr 2019 07:29:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=aFlI5HX/kqrNOAYhfHscIlLGGmFWb4JE7kKMSURK0+0=; b=LtVZE1ozKgJMP7oWrOln+MshjCnaOfUDshjPmPGzBICVHXWGOF4DUi9LnxQmBXbebS Okehkr/ao3WaRYU0ehTgvXfzzE44D1bDSiuE2oTa5MIyGPsK1zGNpysshvCo2vHBTqxf nnJVe8eHK/iFC7ADdnShYah7AxxzAG24clfZwIsfOfCKnQ/J5LlDa00KCqYj8aIJVcUL KF7xW8uS2Ujj8mDapkkp4YhT9DJJG21ksEH/ia3iIFcCxnfDCcR9Vgq/Sdbn8YbzugRb 6O1Ud8i8uSE7Q3zgg0ZH5boSVjquSJclFmSEZQIsChvJ/Pah5kmt/M7iB8J+LaExAX8q AHBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=aFlI5HX/kqrNOAYhfHscIlLGGmFWb4JE7kKMSURK0+0=; b=rwqBryXKa0kQXbkBVUtq+hW+ZSJUJlp1UDYswEip4AixuhRY0ymsxDVvoSp93I9Uz1 Ek6Rh449fWKcETtCwrCHMUy6Kn3rMNiUMrlA2CAWVOU/j/HnFXjdZnliN430F9Ts4Yg+ Ub4zDG+r5IcHyjgSVT06mCUhUnmTkD4YrgV6o5lwMd2hPhajXtDr5ZaHRsBfiGXPmY3X CxpPxz3vN8YA3Oq6Du4JSp5IL7c+XNwZ3X1GRVdB4LgBE/oQ9WFjPktywGcUlfc/74sq MBXdjAStttswZ3tsDxoMZNi7zeihDHa9Ni8Sag7LyNOnls+qKlIj1XY4dhBqUC1oN5jG LtKQ== X-Gm-Message-State: APjAAAV8C3HQgBW6tMH02boBvZA5PenxDNgaodH0JD1LevyuUY5N5ZUi D5j2NgRi2xSaLH4LI+J8J9MOfe1fYzSd3P95K4vO8iad X-Google-Smtp-Source: APXvYqyIhgHdRH3l/PNioPVmiAIeYdC//K+EwzrWc0mu8vMeOw2pQMgRUh0UDgShaL2Uqjxj9f9zTbkb0eIPv+Bb3tU= X-Received: by 2002:a5d:9f4c:: with SMTP id u12mr1268991iot.72.1556548194287; Mon, 29 Apr 2019 07:29:54 -0700 (PDT) MIME-Version: 1.0 References: <20190428044118.GA46284@admin.sibptus.ru> <1DE7A90B-E3AB-4F1E-9A3A-981B18751F8E@punkt.de> <20190429082408.GA84142@admin.sibptus.ru> In-Reply-To: <20190429082408.GA84142@admin.sibptus.ru> From: Adam Date: Mon, 29 Apr 2019 09:29:42 -0500 Message-ID: Subject: Re: [vm-bhyve] shutting down VMs and kern.init_shutdown_timeout To: Victor Sudakov Cc: FreeBSD virtualization X-Rspamd-Queue-Id: 1D60A84B70 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.97 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.97)[-0.970,0] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 14:29:55 -0000 On Mon, Apr 29, 2019 at 3:24 AM Victor Sudakov wrote: > Patrick M. Hausen wrote: > > > > > Am 28.04.2019 um 06:41 schrieb Victor Sudakov : > > > "if the /etc/rc.shutdown does not terminate within 120 seconds, init > will > > > terminate it. The timeout can be configured via the sysctl variable > > > kern.init_shutdown_timeout.=E2=80=9C > > > > What=E2=80=99s keeping you from increasing that timeout for your > > particular requirements? > > Is that really necessary? Am I correct assuming that any VM is supposed > to shut down within 2 minutes or else the bhyve processes will be > killed? Or is this kern.init_shutdown_timeout only about the rc.shutdown > script itself and not its children? > FreeBSD is tuned as a general purpose server OS. If you find the defaults don't meet your needs, you can change them. That's why tunables exist. > > IMHO 2 minutes are a reasonable > > default for a system that does not run hypervisor VMs - > > which is probably the vast majority. > > > > Anyone with the knowledge to implement a virtualized > > environment > > The lack of information about this caveat in > https://wiki.freebsd.org/bhyve, man vm-bhyve, > > https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/virtualization= -host-bhyve.html > and other sources suggests that this particular knowledge may be > limited. Other hypervisors (non-bhyve) probably wait for all VMs to stop. > RH implemented this relatively recently. https://bugzilla.redhat.com/show_bug.cgi?id=3D1334982 > can simply set that to a more suitable value, > > can=E2=80=99t he/she? > > Yes if they know about it. There should be a warning somewhere in the > documentation if we want to be serious about FreeBSD as a hypervisor. > > Or better still, a warning that rcshutdown_timeout (if set) and > kern.init_shutdown_timeout can be too low when running vm-bhyve and > such is desirable. > Why don't you simply write a wrapper script which calls shutdown after shutting down guests? I doubt many admins on any hypervisor are gratuitously calling shutdown on host with important guests running. --=20 Adam From owner-freebsd-virtualization@freebsd.org Mon Apr 29 15:32:53 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 630EA159433E for ; Mon, 29 Apr 2019 15:32:53 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6D2AD890CB for ; Mon, 29 Apr 2019 15:32:51 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=/J+0zUolWwxLCCpVHio0iPe9PLyGmlwinE6JafeDnO0=; b=MogugUam0m0xs4/C803cmxyZmq VngdBzCsltjDKGS2h/VJ7zepARpTyEfRS2i7Qpd0nX+FbD2O4/hBro164a4Ir0qDFgdpH9CL3G+y8 oRGPebtlT85k+s6kOKY8Vhl9oeUgbrQVk8ut4hQuSGW5fgRXnag/xlVr3CMo92SBQOZM=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hL8Gw-000OKa-Ny for freebsd-virtualization@freebsd.org; Mon, 29 Apr 2019 22:32:50 +0700 Date: Mon, 29 Apr 2019 22:32:50 +0700 From: Victor Sudakov To: FreeBSD virtualization Subject: Re: [vm-bhyve] shutting down VMs and kern.init_shutdown_timeout Message-ID: <20190429153250.GA93256@admin.sibptus.ru> References: <20190428044118.GA46284@admin.sibptus.ru> <1DE7A90B-E3AB-4F1E-9A3A-981B18751F8E@punkt.de> <20190429082408.GA84142@admin.sibptus.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Q68bSM7Ycu6FN28Q" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.4 (2019-03-13) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 15:32:53 -0000 --Q68bSM7Ycu6FN28Q Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Adam wrote: >=20 > > Patrick M. Hausen wrote: > > > > > > > Am 28.04.2019 um 06:41 schrieb Victor Sudakov : > > > > "if the /etc/rc.shutdown does not terminate within 120 seconds, init > > will > > > > terminate it. The timeout can be configured via the sysctl variable > > > > kern.init_shutdown_timeout.=E2=80=9C > > > > > > > What=E2=80=99s keeping you from increasing that timeout for your > > > particular requirements? > > > > Is that really necessary? Am I correct assuming that any VM is supposed > > to shut down within 2 minutes or else the bhyve processes will be > > killed? Or is this kern.init_shutdown_timeout only about the rc.shutdown > > script itself and not its children? > > >=20 > FreeBSD is tuned as a general purpose server OS. If you find the defaults > don't meet your needs, you can change them. That's why tunables exist. Adam, if you know the answer to my direct question above, please answer it. I also would like to know if there are any adverse effects of increasing kern.init_shutdown_timeout.=20 > > > IMHO 2 minutes are a reasonable > > > default for a system that does not run hypervisor VMs - > > > which is probably the vast majority. > > > > > > Anyone with the knowledge to implement a virtualized > > > environment > > > > The lack of information about this caveat in > > https://wiki.freebsd.org/bhyve, man vm-bhyve, > > > > https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/virtualizati= on-host-bhyve.html > > and other sources suggests that this particular knowledge may be > > limited. Other hypervisors (non-bhyve) probably wait for all VMs to sto= p. > > >=20 > RH implemented this relatively recently. >=20 > https://bugzilla.redhat.com/show_bug.cgi?id=3D1334982 Good for them. We need to raise awareness too, so I've created this PR https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D237646 >=20 > > can simply set that to a more suitable value, > > > can=E2=80=99t he/she? > > > > Yes if they know about it. There should be a warning somewhere in the > > documentation if we want to be serious about FreeBSD as a hypervisor. > > > > Or better still, a warning that rcshutdown_timeout (if set) and > > kern.init_shutdown_timeout can be too low when running vm-bhyve and > > such is desirable. > > >=20 > Why don't you simply write a wrapper script which calls shutdown after > shutting down guests? =20 A lot of things can be done to mitigate this problem, as soon as you are aware of it. > I doubt many admins on any hypervisor are > gratuitously calling shutdown on host with important guests running. Well, after I started noticing "WARNING: / was not properly dismounted" in my guest OS's logs, after the perfectly correct shutdown of the host, I stopped calling shutdown on host, and started thinking why this was happening. But not before.=20 --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --Q68bSM7Ycu6FN28Q Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcxxkiAAoJEA2k8lmbXsY0yuwH/2SmdD1dU/5jqfx5K9qNntOW RT9dWeXrdNXI3zoS2Uu/t1yNQpJApvB2ay3svceq4GK4DEKAYbtoJd1Ib6EVCeeX mlbQFk0keSFiXg0jKjtGTJ13LL4+hJLG6zpNgNoxAv/HwOiPk1PXjT7rA0AyqF18 PgVyZYHREY1OBzWcUlTguKbmjxzrnD+W1drHrJhH53hH1DkOzrMXvOQT6QVFwBup KOE8OI0HgFkO0pYOfLR/xmRg+zENVG26TPCeZnulND/XRAOqjJ/lfB5tBRxO0ujR eF75wswPAlL1hjp0d7nkVRPQnjGOrM67kYHiwye9enR/tnuJwpGtbOuNPh2WcdY= =ONRq -----END PGP SIGNATURE----- --Q68bSM7Ycu6FN28Q-- From owner-freebsd-virtualization@freebsd.org Thu May 2 18:52:03 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6AB06159E8F3 for ; Thu, 2 May 2019 18:52:03 +0000 (UTC) (envelope-from Udayan.Bapat@netapp.com) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-eopbgr770041.outbound.protection.outlook.com [40.107.77.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AD66B6D98C for ; Thu, 2 May 2019 18:52:01 +0000 (UTC) (envelope-from Udayan.Bapat@netapp.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netapp.onmicrosoft.com; s=selector1-netapp-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S3RSRLFiaKQcw9/YLBr7/sJZRCpUif60Uwpup3GtdHg=; b=ZLs+Bw0DNefZGy78BZUN1e92XzPszY60jY7nF92kWTrcZtJvxKC+tkqhmT7yBhp/HYkcFY33W0u/SgB2BqEkcuFCvNd7igLg0gwlqITlQ4OPbT7nPI6HFOCd3xssA59ORD8/9E+x94vrNbCmpTfyEx3Ubr+2iaLBbITru2IXTTU= Received: from BN7PR06MB6051.namprd06.prod.outlook.com (20.176.31.148) by BN7PR06MB5217.namprd06.prod.outlook.com (20.176.29.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1835.14; Thu, 2 May 2019 18:51:58 +0000 Received: from BN7PR06MB6051.namprd06.prod.outlook.com ([fe80::15b8:895e:5491:e4e2]) by BN7PR06MB6051.namprd06.prod.outlook.com ([fe80::15b8:895e:5491:e4e2%7]) with mapi id 15.20.1856.008; Thu, 2 May 2019 18:51:58 +0000 From: "Bapat, Udayan" To: "freebsd-virtualization@freebsd.org" Subject: BSD11 vs BSD12 KVM performance Thread-Topic: BSD11 vs BSD12 KVM performance Thread-Index: AQHVARgeQBBS4Dr1OEWHI3GrvmsxdA== Date: Thu, 2 May 2019 18:51:58 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [216.240.30.5] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 146d267e-6e01-4019-60f7-08d6cf2f4150 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(2017052603328)(7193020); SRVR:BN7PR06MB5217; x-ms-traffictypediagnostic: BN7PR06MB5217: x-ms-exchange-purlcount: 2 x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0025434D2D x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(366004)(396003)(346002)(136003)(39860400002)(199004)(189003)(53754006)(66066001)(76116006)(54896002)(86362001)(9326002)(478600001)(2906002)(6306002)(33656002)(7736002)(82746002)(71200400001)(5660300002)(99286004)(4744005)(6116002)(73956011)(3846002)(91956017)(66946007)(66556008)(66476007)(64756008)(36756003)(2501003)(66446008)(2351001)(316002)(14444005)(83716004)(8676002)(256004)(14454004)(6506007)(53936002)(5640700003)(26005)(102836004)(6486002)(476003)(486006)(8936002)(25786009)(6436002)(2616005)(6512007)(72206003)(81156014)(68736007)(71190400001)(6916009)(81166006)(186003); DIR:OUT; SFP:1101; SCL:1; SRVR:BN7PR06MB5217; H:BN7PR06MB6051.namprd06.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: netapp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: BEjkjgResLQlW8Kyn2nBW3IbRjD+tFgst04D0FHJti1GQGK215qiXwJ4oFbpvRrv+KSxaYJmYPCnhR8wmAvMQSF5uQSDvHJWFYy2qIpuGjiWi43VZcI744xk5i6upwz5mN+C1iRu0KnerSKpDH01ogEO+YRToEAZ/7SFvmGwDU6pPw4gm4iIYXRmoa3s7iiz8/1LG0tBiNjLL50Rvw0ja4s8MiJ+eiRyY9nTO2jcK1s9/sHo7epT0KqRF6EjmjgDl27hFcdIBqfy+VFQOw/2QM9hgJKuqhYy3SrFy+LRaMy2vadzzbqpbHDhLdlbIUx9e1CYWZKvwWykpi6ncQwtOqaLkbmbPd0QksKR4wAK1EnmY+u5jJynNID2pwnI35lJIFzwS+7+cBy+eOiIonAUHyeN8kuJubQxIMh4zD1jSkQ= MIME-Version: 1.0 X-OriginatorOrg: netapp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 146d267e-6e01-4019-60f7-08d6cf2f4150 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 May 2019 18:51:58.2984 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4b0911a0-929b-4715-944b-c03745165b3a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR06MB5217 X-Rspamd-Queue-Id: AD66B6D98C X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=netapp.onmicrosoft.com header.s=selector1-netapp-com header.b=ZLs+Bw0D; spf=pass (mx1.freebsd.org: domain of Udayan.Bapat@netapp.com designates 40.107.77.41 as permitted sender) smtp.mailfrom=Udayan.Bapat@netapp.com X-Spamd-Result: default: False [-4.33 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[netapp.onmicrosoft.com:s=selector1-netapp-com]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[netapp.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-0.99)[ipnet: 40.64.0.0/10(-2.66), asn: 8075(-2.23), country: US(-0.06)]; MX_GOOD(-0.01)[cached: mx1.netapp.iphmx.com]; DKIM_TRACE(0.00)[netapp.onmicrosoft.com:+]; RCVD_IN_DNSWL_NONE(0.00)[41.77.107.40.list.dnswl.org : 127.0.3.0]; NEURAL_HAM_SHORT(-0.93)[-0.931,0]; TO_DN_EQ_ADDR_ALL(0.00)[]; MIME_BASE64_TEXT(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:40.64.0.0/10, country:US]; MID_RHS_MATCH_FROM(0.00)[] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 18:52:03 -0000 SGkgQWxsLA0KDQpJIHdvcmsgaW4gcGVyZm9ybWFuY2UgdGVhbSB3aXRoIE5ldEFwcCwgSW5jLg0K DQpEdXJpbmcgcXVhbGlmaWNhdGlvbiBwcm9jZXNzIG9mIEJTRDExIGFzIGEgZ3Vlc3QsIHdlIG9i c2VydmVkIHRoYXQgdGhlIGNvc3Qgb2YgcmR0c2MgY2FsbCBoYXMgbWFnbmlmaWVkIGJ5IDEweCBj b21wYXJlZCB0byBCU0QxMC4gV2UgYXJlIHVzaW5nIHFlbXUtMi4xMCB0aGF0IGNhbWUgYXMgYSBk ZWZhdWx0IHBhY2thZ2UgaW4gUkg3LjQuIEl0IGFwcGVhcnMgdGhhdCBCU0QxMSB3YXMgcmVsZWFz ZWQgYmVmb3JlIHFlbXUtMi4xMCBzbyBJIGFtIG5vdCBzdXJlIGlmIHdlIGhhdmUgYW55IHN1cHBv cnRhYmlsaXR5IGlzc3VlIGFuZCBJIHdvdWxkIG5lZWQgdG8gdXBncmFkZSBRRU1VIHZlcnNpb24u ICBJbiB3aGF0IHdheSwgSSBjYW4gZG8gbW9yZSBkZWJ1Z2dpbmcgYXJvdW5kIHRoaXM/IFdlIGFy ZSBhbHNvIGV4cGVyaW1lbnRpbmcgd2l0aCBFU1ggYW5kIHdlIGRpZG7igJl0IHNlZSB0aGlzIGlz c3VlIHRoZXJlLiBJIGFsc28gY291bGQgbm90IGZpbmQgYW55IHBlcmZvcm1hbmNlIGJlbmNobWFy a2luZyBhcm91bmQgQlNEK1FFTVUvS1ZNLg0KDQpUaGFuayB5b3UsDQpVZGF5YW4gQmFwYXQNClNv ZnR3YXJlIEVuZ2luZWVyaW5nLCBQZXJmb3JtYW5jZSBBbmFseXNpcw0KTmV0QXBwLCBJbmMuDQo= From owner-freebsd-virtualization@freebsd.org Sat May 4 10:11:52 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14EF41589157 for ; Sat, 4 May 2019 10:11:52 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4019282E7E for ; Sat, 4 May 2019 10:11:44 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id x44ABfGB094100; Sat, 4 May 2019 12:11:42 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id D3A95682; Sat, 4 May 2019 12:11:41 +0200 (CEST) Subject: Re: BSD11 vs BSD12 KVM performance To: "Bapat, Udayan" , "freebsd-virtualization@freebsd.org" References: From: Harry Schmalzbauer Organization: OmniLAN Message-ID: Date: Sat, 4 May 2019 12:11:41 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Sat, 04 May 2019 12:11:42 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-Rspamd-Queue-Id: 4019282E7E X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of freebsd@omnilan.de designates 2a00:e10:2800::a130 as permitted sender) smtp.mailfrom=freebsd@omnilan.de X-Spamd-Result: default: False [-6.80 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[omnilan.de]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[mx0.gentlemail.de]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-3.51)[ip: (-9.20), ipnet: 2a00:e10:2800::/64(-4.67), asn: 25074(-3.68), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:25074, ipnet:2a00:e10:2800::/64, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 10:11:52 -0000 Am 02.05.2019 um 20:51 schrieb Bapat, Udayan: > Hi All, > > I work in performance team with NetApp, Inc. > > During qualification process of BSD11 as a guest, we observed that the cost of rdtsc call has magnified by 10x compared to BSD10. We are using qemu-2.10 that came as a default package in RH7.4. It appears that BSD11 was released before qemu-2.10 so I am not sure if we have any supportability issue and I would need to upgrade QEMU version. In what way, I can do more debugging around this? We are also experimenting with ESX and we didn’t see this issue there. I also could not find any performance benchmarking around BSD+QEMU/KVM. Hello, please clarify whether it is about FreeBSD-10 vs. FreeBSD-11, like the body tells, or FreeBSD-11 vs. FreeBSD-12, like the topic tells. Can't help in any case regading the QMEU question, unfortunately. -harry From owner-freebsd-virtualization@freebsd.org Sat May 4 13:44:27 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 736D5158F0B7 for ; Sat, 4 May 2019 13:44:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 092CC8B08B for ; Sat, 4 May 2019 13:44:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id C1CDF158F0B6; Sat, 4 May 2019 13:44:26 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF778158F0B4 for ; Sat, 4 May 2019 13:44:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 467C48B088 for ; Sat, 4 May 2019 13:44:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 74F111C167 for ; Sat, 4 May 2019 13:44:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x44DiPBY061777 for ; Sat, 4 May 2019 13:44:25 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x44DiPGJ061774 for virtualization@FreeBSD.org; Sat, 4 May 2019 13:44:25 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 212782] [bhyve] Increasing VM_MAXCPU to >= 26 will result in ACPI MADT table corruption Date: Sat, 04 May 2019 13:44:23 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: rgrimes@FreeBSD.org X-Bugzilla-Flags: mfc-stable11+ mfc-stable12+ X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 13:44:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212782 --- Comment #6 from commit-hook@freebsd.org --- A commit references this bug: Author: rgrimes Date: Sat May 4 13:43:49 UTC 2019 New revision: 347124 URL: https://svnweb.freebsd.org/changeset/base/347124 Log: MFC: r346715: Acpi MADT table correction for VM_MAXCPU > 21 The bhyve acpi MADT table was given a static space of 256 (0x100) bytes, this is enough space to allow VM_MAXCPU to be 21, this patch changes that so VM_MAXCPU can be of arbitrary value and not overflow the space by actually calculating the space needed for the table. PR: 212782 Reviewed by: Patrick Mooney Approved by: bde (mentor, implicit), jhb (maintainer) Differential Revision: https://reviews.freebsd.org/D18815 Changes: _U stable/11/ stable/11/usr.sbin/bhyve/acpi.c _U stable/12/ stable/12/usr.sbin/bhyve/acpi.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Sat May 4 13:49:02 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75CBE158F1E3 for ; Sat, 4 May 2019 13:49:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 085F78B16C for ; Sat, 4 May 2019 13:49:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id BD6AC158F1E2; Sat, 4 May 2019 13:49:01 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC29F158F1E1 for ; Sat, 4 May 2019 13:49:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48D2B8B167 for ; Sat, 4 May 2019 13:49:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 871F71C16A for ; Sat, 4 May 2019 13:49:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x44Dn0PT066626 for ; Sat, 4 May 2019 13:49:00 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x44Dn0IP066624 for virtualization@FreeBSD.org; Sat, 4 May 2019 13:49:00 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 212782] [bhyve] Increasing VM_MAXCPU to >= 26 will result in ACPI MADT table corruption Date: Sat, 04 May 2019 13:49:00 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: rgrimes@FreeBSD.org X-Bugzilla-Flags: mfc-stable11+ mfc-stable12+ X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 13:49:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212782 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Progress |Closed Resolution|--- |FIXED --=20 You are receiving this mail because: You are on the CC list for the bug.=