From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 4 04:37:06 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 121D9648; Thu, 4 Jun 2015 04:37:06 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9F410123C; Thu, 4 Jun 2015 04:37:05 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: by wgez8 with SMTP id z8so24121766wge.0; Wed, 03 Jun 2015 21:37:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=pCr3PjEExDUQp/gYK8FdIGaMIEPl6ezwYocd9FTb8p4=; b=vHmG6egGm8o+QDwVzlnv8jP66dC6GnYTdb6mfbREnNYEthokozapffVjeV2D3wO9la HR822z4S7btQf3EUTWejQywmdPFSkK8fNymRAVyqlz8w8pa0TNR25H8AR/w2DpKGrB/m VXsLQYWe5bl1M3oRB8tsLpFTmedJFkvPZj/mwnKTcwZN+F1KYiVQe4+vqNml4Bmbqx/w 2b2CGG91COJnNrOTcld148Rb1MxQNG7QFYTIRLRuln6dhJ9dMErfNnBaa9jEZHBty1DE dYuTimHZVbwlQYsTG9vnKcNSoFqef100urNK39yGSayTkV3w75L+ODQWslLAT9D/gGy9 5Q3w== MIME-Version: 1.0 X-Received: by 10.180.98.103 with SMTP id eh7mr3636115wib.75.1433392624114; Wed, 03 Jun 2015 21:37:04 -0700 (PDT) Received: by 10.27.52.18 with HTTP; Wed, 3 Jun 2015 21:37:04 -0700 (PDT) In-Reply-To: <556ED071.5030009@FreeBSD.org> References: <556D9005.4020802@FreeBSD.org> <556DDDA9.6090005@FreeBSD.org> <556ED071.5030009@FreeBSD.org> Date: Wed, 3 Jun 2015 21:37:04 -0700 Message-ID: Subject: Re: bhyve: corrupting zfs pools? From: Neel Natu To: Andriy Gapon Cc: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jun 2015 04:37:06 -0000 Hi Andriy, On Wed, Jun 3, 2015 at 3:01 AM, Andriy Gapon wrote: > On 03/06/2015 00:40, Neel Natu wrote: >> Perhaps, but I wasn't able to repro this. I tried your recipe to repro >> on two systems running -current at r283917. >> >> - Intel Sandybridge server: Xeon E52650 with 8 cores/16 threads and 64GB memory >> - hdd1/hdd2 on UFS >> - hdd1/hdd2 on ZFS >> >> - AMD Opteron server: Opteron 6230 with 8 cores and 16GB memory >> - hdd1/hdd2 on UFS >> >> Can you provide some more details about your setup? I can then try to >> repro on a system similar to your setup. >> >> - uname -a > > FreeBSD trant 11.0-CURRENT FreeBSD 11.0-CURRENT #134 r283188+ab4f83f(devel): Fri > May 22 15:55:27 EEST 2015 avg@trant:/usr/obj/usr/src/sys/TRANT amd64 > > This is not a pure FreeBSD, there are some local changes, but none to vmm or VM. > >> - sysctl hw.model > > hw.model: AMD Athlon(tm) II X2 250 Processor > >> - sysctl hw.ncpu > > hw.ncpu: 2 > >> - sysctl hw.physmem > > hw.physmem: 8029335552 > >> - sysctl hw.vmm > > hw.vmm.npt.pmap_flags: 507 > hw.vmm.svm.num_asids: 64 > hw.vmm.svm.disable_npf_assist: 0 > hw.vmm.svm.features: 15 > hw.vmm.svm.vmcb_clean: 959 > hw.vmm.vmx.vpid_alloc_failed: 0 > hw.vmm.vmx.posted_interrupt_vector: -1 > hw.vmm.vmx.cap.posted_interrupts: 0 > hw.vmm.vmx.cap.virtual_interrupt_delivery: 0 > hw.vmm.vmx.cap.invpcid: 0 > hw.vmm.vmx.cap.monitor_trap: 0 > hw.vmm.vmx.cap.unrestricted_guest: 0 > hw.vmm.vmx.cap.pause_exit: 0 > hw.vmm.vmx.cap.halt_exit: 0 > hw.vmm.vmx.initialized: 0 > hw.vmm.vmx.cr4_zeros_mask: 0 > hw.vmm.vmx.cr4_ones_mask: 0 > hw.vmm.vmx.cr0_zeros_mask: 0 > hw.vmm.vmx.cr0_ones_mask: 0 > hw.vmm.ept.pmap_flags: 0 > hw.vmm.vrtc.flag_broken_time: 1 > hw.vmm.ppt.devices: 0 > hw.vmm.iommu.initialized: 0 > hw.vmm.bhyve_xcpuids: 4 > hw.vmm.topology.cpuid_leaf_b: 1 > hw.vmm.topology.cores_per_package: 1 > hw.vmm.topology.threads_per_core: 1 > hw.vmm.create: beavis > hw.vmm.destroy: beavis > hw.vmm.force_iommu: 0 > hw.vmm.trace_guest_exceptions: 0 > hw.vmm.ipinum: 251 > hw.vmm.halt_detection: 1 > >> - host filesystem underlying hdd1 and hdd2 > > It's ZFS. > Ok, there are some differences in our systems. The interesting ones are number of ASIDs (64 versus 65536), flush-by-asid capability, vmcb-clean capability and the number of cores. I was able to mimic all of these on my Opteron but still wasn't able to reproduce the issue. I am going to get a Sempron tomorrow which belongs to the same processor family as the Athlon II so hoping that it is easier to repro. BTW does this happen consistently on your system? best Neel > Thank you! > > -- > Andriy Gapon