From owner-freebsd-emulation@FreeBSD.ORG Fri Dec 5 19:29:29 2014 Return-Path: Delivered-To: vbox@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35E23EED for ; Fri, 5 Dec 2014 19:29:29 +0000 (UTC) Received: from mail-ob0-x22b.google.com (mail-ob0-x22b.google.com [IPv6:2607:f8b0:4003:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F0C54B5F for ; Fri, 5 Dec 2014 19:29:28 +0000 (UTC) Received: by mail-ob0-f171.google.com with SMTP id uz6so1038163obc.16 for ; Fri, 05 Dec 2014 11:29:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vopenhouse.ca; s=google; h=mime-version:date:message-id:subject:from:to:content-type; bh=AbUA1kfFQTWwEDOVDY57jluuL7Y59QALbaILB+Ad1MU=; b=xXJ34oHMV/tn5pHm68dBw2piFNCJdy+EziM08658Q+TnXFr7VWFqTc524NS4339h+w JaRtf9Evb6kP4+sDXsWqf1OICf2hvnlwUEF2rNjybDtrIt5JisMS2GLbphnbb1lEil6v W1HKkK6lFegWH2EQuV852t4DYmTdfFnnepGYM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=AbUA1kfFQTWwEDOVDY57jluuL7Y59QALbaILB+Ad1MU=; b=jrDXFqvqXtQ3QZIBMQgOdGZ+8Y93D+tigbJ5ChY79Wugcvv6LjGf/0Olgfj8snf+E1 ho6hlJk/0vkA/6jF/KTNaWUTYkzO0NXkhPBcnPn6qOcerckJLDk9eJC1g0Tefoljw1No ZHOuJjcrYoICaLs8OZ4gjszGIFIADPiZM0Jl7Xjb3uZ0Cxtz8sq14vJjy+/qRgXIIzuo dtfc+QM+kDqoTyjbME/QBdbtSBNn28bXfv5NFgK7/Aw8JEUEaO7ITMIcNnjo3XsslokC bSU7N94U+HyxU4xy3K1s/vikpZsfWOqV2lr9EF6o9fEqU2ORmZQSUlp8jcUAaOtN0KQi WZlw== X-Gm-Message-State: ALoCoQl7aWxDxHmRXM89T4wk7LKHlgsZxDfaJk8LSjG2pDG2X+Yf1MpkHP8XN7auS+4SemXKjBRm MIME-Version: 1.0 X-Received: by 10.202.214.80 with SMTP id n77mr11077448oig.9.1417807768014; Fri, 05 Dec 2014 11:29:28 -0800 (PST) Received: by 10.76.180.73 with HTTP; Fri, 5 Dec 2014 11:29:27 -0800 (PST) X-Originating-IP: [207.6.79.235] Date: Fri, 5 Dec 2014 11:29:27 -0800 Message-ID: Subject: VBoxHeadless seg fault From: Ron Thomas To: vbox@FreeBSD.org X-Mailman-Approved-At: Fri, 05 Dec 2014 19:53:45 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-emulation@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Development of Emulators of other operating systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Dec 2014 19:29:29 -0000 I'm trying to understand what is happening when this segfault occurs. From the user perspective it happens randomly. In the code ptr is incremented by the size of VRDEORDERCODE which I think pushes it out of bounds causing the other values to go out of bounds as well? Any help would be appreciated. Ron #uname -a FreeBSD myhostname.someplace.com 9.2-RELEASE-p12 FreeBSD 9.2-RELEASE-p12 #0: Mon Sep 15 18:46:46 UTC 2014 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 # gdb `which VBoxHeadless` VBoxHeadless.core.save-1 (gdb) bt #0 0x0000000808801f67 in VNCServerImpl::VRDEUpdate (hServer=0x808064500, uScreenId=0, pvUpdate=0x824186562, cbUpdate=1702105) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/ExtPacks/VNC/VBoxVNC.cpp:823 #1 0x000000080616693f in ConsoleVRDPServer::SendUpdate (this=0x805813f00, uScreenId=0, pvUpdate=0x824186562, cbUpdate=1702105) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/Main/src-client/ConsoleVRDPServer.cpp:3795 #2 0x00000008061755a5 in Display::displayVBVAUpdateProcess (pInterface=0x80bb89cb8, uScreenId=0, pCmd=0x824186562, cbCmd=1702105) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/Main/src-client/DisplayImpl.cpp:4770 #3 0x00000008129264de in vbvaFlushProcess (uScreenId=0, pVGAState=0x8072f0100, pPartialRecord=0x80b317b90, pVBVA=0x82413f000) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/Devices/Graphics/DevVGA_VBVA.cpp:380 #4 0x00000008129268ac in vbvaFlush (pVGAState=0x8072f0100, pCtx=0x80b317b48) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/Devices/Graphics/DevVGA_VBVA.cpp:462 #5 0x000000081292691d in VBVAUpdateDisplay (pVGAState=0x8072f0100) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/Devices/Graphics/DevVGA_VBVA.cpp:2492 #6 0x000000081285a679 in vgaPortUpdateDisplay (pInterface=0x807300e10) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/Devices/Graphics/DevVGA.cpp:4541 #7 0x000000080617d160 in Display::displayRefreshCallback (pInterface=0x80bb89cb8) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/Main/src-client/DisplayImpl.cpp:3872 #8 0x0000000812859f97 in vgaTimerRefresh (pDevIns=0x8072f0000, pTimer=0x8006ce430, pvUser=0x8072f0100) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/Devices/Graphics/DevVGA.cpp:5211 #9 0x00000008066e7c93 in tmR3TimerQueueRun (pVM=0x80067e000, pQueue=0x8006b8890) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/VMM/VMMR3/TM.cpp:2072 #10 0x00000008066f5dd9 in TMR3TimerQueuesDo (pVM=0x80067e000) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/VMM/VMMR3/TM.cpp:1990 #11 0x000000080670da99 in vmR3HaltGlobal1Halt (pUVCpu=0x80ac02180, fMask=540, u64Now=3630149027607711) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/VMM/VMMR3/VMEmt.cpp:685 #12 0x000000080670d25b in VMR3WaitHalted (pVM=0x80067e000, pVCpu=0x80069e000, fIgnoreInterrupts=true) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/VMM/VMMR3/VMEmt.cpp:1104 #13 0x00000008065ea394 in EMR3ExecuteVM (pVM=0x80067e000, pVCpu=0x80069e000) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/VMM/VMMR3/EM.cpp:2562 #14 0x000000080670f66a in vmR3EmulationThreadWithId (ThreadSelf=0x808093100, pUVCpu=0x80ac02180, idCpu=1) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/VMM/VMMR3/VMEmt.cpp:235 #15 0x000000080670f8ac in vmR3EmulationThread (ThreadSelf=0x808093100, pvArgs=0x80ac02180) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/VMM/VMMR3/VMEmt.cpp:62 #16 0x0000000801d88d78 in rtThreadMain (pThread=0x808093100, NativeThread=34494825472, pszThreadName=0x8080939d8 "EMT-1") at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/Runtime/common/misc/thread.cpp:712 #17 0x0000000801e6416d in rtThreadNativeMain (pvArgs=0x808093100) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/Runtime/r3/posix/thread-posix.cpp:324 #18 0x000000080082ad34 in pthread_getprio () from /lib/libthr.so.3 #19 0x0000000000000000 in ?? () (gdb) frame 0 #0 0x0000000808801f67 in VNCServerImpl::VRDEUpdate (hServer=0x808064500, uScreenId=0, pvUpdate=0x824186562, cbUpdate=1702105) at /usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.20/src/VBox/ExtPacks/VNC/VBoxVNC.cpp:823 823 instance->mFrameBuffer[destx] = instance->mScreenBuffer[srcx + 2]; (gdb) list + 818 // RGB to BGR 819 for (srcx = srcy, destx = desty; 820 destx < desty + order->w * VNC_SIZEOFRGBA; 821 srcx += bpp, destx += VNC_SIZEOFRGBA) 822 { 823 instance->mFrameBuffer[destx] = instance->mScreenBuffer[srcx + 2]; 824 instance->mFrameBuffer[destx + 1] = instance->mScreenBuffer[srcx + 1]; 825 instance->mFrameBuffer[destx + 2] = instance->mScreenBuffer[srcx]; 826 } 827 } (gdb) info locals srcx = 14680064 desty = 14419848 width = 1280 bpp = 4 joff = 479202 srcy = 14419848 destx = 14680064 ptr = 0x82418656e
instance = (VNCServerImpl *) 0x808064500 order = (VRDEORDERHDR *) 0x824186562