Date: Wed, 28 Nov 2012 17:42:51 +0200 From: Andriy Gapon <avg@FreeBSD.org> To: Alex Chistyakov <alexclear@gmail.com> Cc: "freebsd-emulation@freebsd.org" <freebsd-emulation@FreeBSD.org>, Alexander Motin <mav@FreeBSD.org> Subject: Re: VirtualBox 4.2.4 on FreeBSD 9.1-PRERELEASE problem: VMs behave very different when pinned to different cores Message-ID: <50B630FB.4050403@FreeBSD.org> In-Reply-To: <CA%2Bkq2xtxwC%2BU15C6pO=tVQcDqFk07Zqivrt_9=fwzn5hMaHi%2BA@mail.gmail.com> References: <CA%2Bkq2xvh3j5CM7UzRVfXCeLhHwpTY%2B_M7dCJx0c27NtV8EVJwg@mail.gmail.com> <CAE-m3X1UPsy%2Bwbqm_02JpXMr-UO3m7N6z_ZwY2HNo4GL0YUi1w@mail.gmail.com> <CA%2Bkq2xva61m_bHdzBZM2TYL5z7XiohvkxsYWtOyoBwQkpyvp0A@mail.gmail.com> <50AFAD05.1050604@FreeBSD.org> <CA%2Bkq2xv%2BU4ZnfK=1js4PRaNpTNdW-y-G50GV4%2BMVP0LugBf1pQ@mail.gmail.com> <50B25C17.20208@FreeBSD.org> <CA%2Bkq2xvjDa1BeuzPUuH99bgriEA-GJH36AZGiqScKSo4QZmHDg@mail.gmail.com> <CA%2Bkq2xtxwC%2BU15C6pO=tVQcDqFk07Zqivrt_9=fwzn5hMaHi%2BA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
on 27/11/2012 02:45 Alex Chistyakov said the following: > On Mon, Nov 26, 2012 at 11:10 AM, Alex Chistyakov <alexclear@gmail.com> wrote: >> A DTrace result for "bad" core is at https://gist.github.com/4145782 >> and a result for "good" core is at https://gist.github.com/4146946 > > Okay, I have finally found a way to utilize PMP and got some > stacktraces for "good" and "bad" VM runs. > The "good" results are here: https://gist.github.com/4151328 and the > "bad" ones are here: https://gist.github.com/4151343 > I was sampling for 160 seconds both times, a sample per a second. I > used a script titled "Full technology demonstration" on > http://poormansprofiler.org/ so numbers on the left are total counts > of repeating stacktraces. > As you can see the second result has "select,select,listenerRun,??,??" > and "select,select,clientInput,??,??" samples so I guessed they can be > related to VNC/VRDE. I disabled VRDE and was able to boot the VM on > the "bad" core without major problems then. > But CPU usage remained quite high on the "bad" core and ping was very > unstable too so this was not the end of the story. > I collected PMP stats for the "bad" core with VRDE disabled: > https://gist.github.com/4151530 > A lot of time seems to be spent in VMMR3HwAccRunGC call. Well, this > function is just a bridge for the guest code, so I'm not sure how to > interpret this fact. I am inclined to interpret both this data and the DTrace data as some sort of churn in the guest or in userland part of VirtualBox. Is the problem easily reproducible with different kinds of guests? Do you have any unusual configuration settings for the VM (especially timer related)? What kind of guest do you run? I feel tempted to draw some connection between this problem and the TSC skew, but I am not sure what it could be. >From the Dtrace profiling, here is where all the extra time gets spent: 12777 100783 VBoxHeadless 0xffffffff81f6477c 0xffffffff81f70bd4 vboxdrv.ko`supdrvIOCtlFast+0x8b vboxdrv.ko`VBoxDrvFreeBSDIOCtl+0xa5 kernel`devfs_ioctl_f+0x7a kernel`kern_ioctl+0x106 kernel`sys_ioctl+0xfd kernel`amd64_syscall+0x334 kernel`0xffffffff80745e07 10317 As far as I understand supdrvIOCtlFast is used to pass guest code for execution. Could you please also run the following script (in the bad case)? fbt::supdrvIOCtlFast:entry { @counts[curthread->td_oncpu, args[1], args[0], args[2], args[3]] = count(); } END { trunc(@counts, 40); printa(@counts); } -- Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50B630FB.4050403>