From owner-freebsd-virtualization@FreeBSD.ORG Wed Nov 12 22:09:01 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6BC07D78; Wed, 12 Nov 2014 22:09:01 +0000 (UTC) Received: from mail2.openmailbox.org (mail2.openmailbox.org [62.4.1.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3D7ABF33; Wed, 12 Nov 2014 22:08:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail2.openmailbox.org (Postfix) with ESMTP id CAA102026F8; Wed, 12 Nov 2014 23:08:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=openmailbox.org; h=user-agent:message-id:references:in-reply-to:subject:subject :from:from:date:date:content-transfer-encoding:content-type :content-type:mime-version:received:received; s=openmailbox; t= 1415830121; bh=Xl2SE0LTxE1MbZ5EScwcYsX20LtfbpVcp6rl/X1zw/E=; b=j 4kFOxiZNY2IS6O0wmeBrk8HZN7XH+SdroOOfX5lv1ibPHuyB7UUwodZFfRKoBfRk 8MjrW/vynkzb6PwiLzjHQbqryn8SokJjWU4YZLEdR5c5r/6l6za+c+ZpE3RE2znr H4f9GUbkEpS/0N1pn4wfAQWJ5d18QLFzK7nJRYtx50= X-Virus-Scanned: at openmailbox.org Received: from mail2.openmailbox.org ([62.4.1.33]) by localhost (mail.openmailbox.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p-ceXJpBeVpW; Wed, 12 Nov 2014 23:08:41 +0100 (CET) Received: from www.openmailbox.org (localhost [127.0.0.1]) by mail2.openmailbox.org (Postfix) with ESMTP id 1F1D02020BC; Wed, 12 Nov 2014 23:08:40 +0100 (CET) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 13 Nov 2014 00:08:40 +0200 From: Tinker To: Allan Jude Subject: Re: How hard is BHyVe's 16 vCPU limit, is it configurable under any =?UTF-8?Q?circumstance=3F?= In-Reply-To: <5463D746.4030803@freebsd.org> References: <8e39523570b667d223091adc3791c9e2@openmailbox.org> <5463D746.4030803@freebsd.org> Message-ID: <49cc2695179830c899aef87b7a8b72d8@openmailbox.org> X-Sender: tinkr@openmailbox.org User-Agent: Roundcube Webmail/1.0.2 Cc: owner-freebsd-virtualization@freebsd.org, freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Nov 2014 22:09:01 -0000 On 2014-11-12 23:55, Allan Jude wrote: > On 2014-11-12 16:39, tinkr@openmailbox.org wrote: >> Hi! >> >> In order justify giving energy to BHyVe, I need to know if it's >> "future-proof" in that the 16 vCPU limit can be increased? >> >> Please let the world know if BHyVe's 16 vCPU limit can be lifted in >> some >> way, by configuration, patch, etc. (and if you want to, why this limit >> is in place by default today). >> >> Thanks!! >> Tinker >> >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to >> "freebsd-virtualization-unsubscribe@freebsd.org" > > You can increase the limit by editing sys/amd64/include/vmm.h > > #define VM_MAXCPU 16 > > From what I've been told, things scale badly above 24 CPUs. They plan > to > solve this issue, but have not yet. If you system has enough cores to > support using more than 16 per VM, you can modify the file and > recompile > the kernel and use as many CPUs as you want, but not much testing has > happened with bigger numbers. Dear Allan, Thank you very much for responding. Aha, great! Only for completeness, do you have any particular idea about * When the scaling above 24 vCPU:s will be optimized, like approx how many years away is it (like 1 or more than 1)?, and * What the technological reason for the scaling is, is it that somehow the BHyVe instances on the different cores need to inter-communicate, for instance that all disk and network IO is done via one single core currently? In all cases, your response is great news, as your baseline answer that it's doable and only a question of optimization and tweaking of present functionality - Thanks again! Tinker