From owner-freebsd-virtualization@freebsd.org Sun Dec 11 07:20:34 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 412C2C7060B for ; Sun, 11 Dec 2016 07:20:34 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 03C939C9 for ; Sun, 11 Dec 2016 07:20:33 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 272362126636 for ; Sun, 11 Dec 2016 17:20:14 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 1DB0F280996 for ; Sun, 11 Dec 2016 17:20:14 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id hFWPKTloKZlY for ; Sun, 11 Dec 2016 17:20:14 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (c-67-180-92-13.hsd1.ca.comcast.net [67.180.92.13]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 57A4C280502; Sun, 11 Dec 2016 17:20:12 +1000 (AEST) Subject: Re: Debian 8 CPU stall To: Rajil Saraswat References: <9c9e83a5-16c6-0ab5-0ac4-af0a54430706@freebsd.org> <6cff5bf2-9654-8627-83c4-6ab48ee763b5@gmail.com> From: Peter Grehan Cc: freebsd-virtualization@FreeBSD.org Message-ID: <56863fec-d8f8-2601-811f-511cd199e82f@freebsd.org> Date: Sat, 10 Dec 2016 23:20:28 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <6cff5bf2-9654-8627-83c4-6ab48ee763b5@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.2 cv=YJDv8VOx c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=5eVCmCvhg37cu/pjidAGzw==:17 a=N659UExz7-8A:10 a=n5n_aSjo0skA:10 a=MK8az4FAIlLI1K8yN9IA:9 a=pILNOxqGKmIA:10 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 07:20:34 -0000 Hi Rajil, > No the system is not oversubscribed. I have a 11 vCPU (1 on debian and 1 > on ubuntu) on a 24 core machine. The debian jail is running x2go and an > ssh server for remote access, so the I/O shouldnt be an issue. The > ubuntu jail doesnt give out any warning messages though. ... > I use vm-bhyve for managing the jails which looks like this for debian8: > > guest="linux" > loader="grub" > cpu=1 > memory=2048M > network0_type="virtio-net" > network0_switch="lannetwork" > disk0_type="virtio-blk" > disk0_name="/dev/zvol/vmpool/os2" > disk0_dev="custom" > passthru0="2/0/0" > passthru1="2/0/1" Looks like a possibly lost interrupt. Can you check to see if this system is using APIC virtualization ? # sysctl -a hw.vmm.vmx.cap ... hw.vmm.vmx.cap.posted_interrupts: 1 hw.vmm.vmx.cap.virtual_interrupt_delivery: 1 (these will be zero if APIC virtualization not in use) If so, one thing to try is disabling these and see if that helps. This can be done by setting the tunable hw.vmm.vmx.use_apic_vid to 1 (e.g. in /boot/loader.conf, or unloading vmm.ko, setting the value with kenv, and the reloading vmm.ko) later, Peter.