From nobody Wed Jul 21 14:06:42 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 15C4212BCFC8 for ; Wed, 21 Jul 2021 14:06:52 +0000 (UTC) (envelope-from merlyn@geeks.org) Received: from mail.geeks.org (jacobs.geeks.org [204.153.247.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GVHSp4hfNz4jDq for ; Wed, 21 Jul 2021 14:06:50 +0000 (UTC) (envelope-from merlyn@geeks.org) Received: from mail.geeks.org (localhost [127.0.0.1]) by after-clamsmtpd.geeks.org (Postfix) with ESMTP id 9B90812F16 for ; Wed, 21 Jul 2021 09:06:42 -0500 (CDT) Received: by mail.geeks.org (Postfix, from userid 1003) id 8E31412F15; Wed, 21 Jul 2021 09:06:42 -0500 (CDT) Date: Wed, 21 Jul 2021 09:06:42 -0500 From: Doug McIntyre To: FreeBSD Stable Mailing List Subject: Re: CPU hot-plug and RAM hot-add in virtual machines Message-ID: References: <4336a1bf-d826-dba3-9ec1-9b48cf7cd177@quip.cz> <70033628-bc3a-24d2-4c65-9a3b9c1c66d5@FreeBSD.org> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Virus-Scanned: ClamAV using ClamSMTP X-Rspamd-Queue-Id: 4GVHSp4hfNz4jDq X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of merlyn@geeks.org designates 204.153.247.1 as permitted sender) smtp.mailfrom=merlyn@geeks.org X-Spamd-Result: default: False [-3.30 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[204.153.247.1:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[geeks.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[204.153.247.1:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7753, ipnet:204.153.244.0/22, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-ThisMailContainsUnwantedMimeParts: N On Wed, Jul 21, 2021 at 02:06:07PM +0200, Miroslav Lachman wrote: > I don't need that badly but it is another problem our customers have > with FreeBSD over Linux. One of them has about 50 VM, mostly Linux, some > Windows and 8 FreeBSD. I'm also a cloud provider. Again, I've maybe had one customer ever request/use it. (Although I'm much further away from the front lines now-a-days). There's a reason it is disabled by default, in that enabling it also disables vNUMA, which gives up to an 8% CPU hit accross the board in all your VMs that you have Hot-Add enabled on. Also, hot-add memory really trashes the page tables on OSs that support it. I would not be surprised that memory utilization efficiency becomes much worse when hot-adding memory because they have to suddenly deal with more than they started with. (I couldn't find studies on this, probably because it is so little utilized). So, if people want it in the first place, they are going to have to shut down the VM to enable it. And then it will have the CPU hit when enabled. (It also does some weird stuff with pass-through USB connections, but that probably isn't too much an issue in a cloud environment). Surely, they have regular patch windows that CPU/memory can be added in those windows? I wouldn't tout a feature that virtually nobody uses as a plus for an OS, especially with downsides such as it'll make it run slower in general for the same level of CPU.