From owner-freebsd-questions@FreeBSD.ORG Fri Mar 30 17:12:43 2012 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 933B9106566B; Fri, 30 Mar 2012 17:12:43 +0000 (UTC) (envelope-from feld@feld.me) Received: from feld.me (unknown [IPv6:2607:f4e0:100:300::2]) by mx1.freebsd.org (Postfix) with ESMTP id 574898FC08; Fri, 30 Mar 2012 17:12:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=feld.me; s=blargle; h=In-Reply-To:Message-Id:From:Mime-Version:Date:References:Subject:Cc:To:Content-Type; bh=HNh4+lFCp7YHA6nqBBsBT3YjTTPmBu3RxFmFbbgPmAU=; b=Ha/oAjvHKRwM0jiQ14d0MzyGWYYhEYJJxMidcVtX1HkJOdZ2uqxgfXJIsMGTQK/2OGFV3tL5w04Un0Dxnru5PxRgT/a/zW6vBV7CT6MWsH/+n9icuK0hhLQvYWIrDIxn; Received: from localhost ([127.0.0.1] helo=mwi1.coffeenet.org) by feld.me with esmtp (Exim 4.77 (FreeBSD)) (envelope-from ) id 1SDfNR-0008f8-5t; Fri, 30 Mar 2012 12:12:42 -0500 Received: from feld@feld.me by mwi1.coffeenet.org (Archiveopteryx 3.1.4) with esmtpa id 1333127555-20726-20725/5/37; Fri, 30 Mar 2012 17:12:35 +0000 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-hackers@freebsd.org, freebsd-questions@freebsd.org References: <201203300027.q2U0RVZS085304@aurora.sol.net> <201203301444.q2UEilmj097567@aurora.sol.net> <201203301653.q2UGrAEY098765@aurora.sol.net> Date: Fri, 30 Mar 2012 12:12:35 -0500 Mime-Version: 1.0 From: Mark Felder Message-Id: In-Reply-To: <201203301653.q2UGrAEY098765@aurora.sol.net> User-Agent: Opera Mail/11.62 (FreeBSD) X-SA-Score: -1.5 Cc: Joe Greco Subject: Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Mar 2012 17:12:43 -0000 On Fri, 30 Mar 2012 11:53:10 -0500, Joe Greco wrote: > On the same vmdk files? "Deleting the VM" makes it sound like not. Fresh new VMDK files every time, and always thick provisioned. > None of the other VM's, even the VM's that had been abused in this > horribly insensitive manner of being placed on intolerably slow iSCSI, > developed this condition. We've seen similar results. Baffling how VMs you know are worse off never develop this condition. > There are dozens of other VM's running on these hosts, alongside the > one that was exhibiting this behaviour. > The VM continued to exhibit this behaviour even after having been moved > onto a different ESXi platform and architecture (Opteron->Xeon). > For the problem to "follow" the VM in this manner, and afflict *only* > the one VM, strongly suggests that it is something that is contained > within the VM files that constitute this VM. That is consistent with > the observation that the problem arose at a point where the VM is > known to have had all those files moved from one location to a dodgy > location. We were hoping that was the explanation as well, but rebuilding the VM entirely from scratch on a new host and seeing the crash come back was a big blow to morale. :( > That's why I believe the evidence points to corruption of some sort. > Of course, your case makes this all interesting. For the last year I've been convinced it's something hidden inside ESXi's I/O virtualization layer that happens to trigger on only certain VMs.