From owner-freebsd-virtualization@freebsd.org Fri Mar 13 03:51:45 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1BC742731B6 for ; Fri, 13 Mar 2020 03:51:45 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: from mail-qt1-x835.google.com (mail-qt1-x835.google.com [IPv6:2607:f8b0:4864:20::835]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48dsFX6HZyz4JxF; Fri, 13 Mar 2020 03:51:44 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: by mail-qt1-x835.google.com with SMTP id n5so6464394qtv.7; Thu, 12 Mar 2020 20:51:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=MqYQEfkEY7sCb9UYsuT4drPXIBgSgnfKLT5H6kv2J3k=; b=HvKjRbSSB2V1VG3Z1N9VRVvm2dsBqNrb/fat0n5lKYhQUbxESXpuM8PCrKXjwg+7Da eCF+pkZk6xi11LMaFPUA5Jp50EeShQ3r1Gzz64X7ICpGq5h7+y9nVEY+r3mEoTMtoijc QIodVsmETRgytsxxlCbjMVaM0rvjjbrYnYdBdgcPQ4Fd23KFGinlbw2c/PZiZMl6yadZ /J9HI3ePYrk2eYQxYs7MQVwqIBvc+MM9utOSUthcyEMkki2vZl4gnSj8VcWa9vxrz3N0 QyOf94WITE+ueRdRidXSQTgyJ2X/xf2HqrygKGFMPkdU2GCokjP1YaPIVQAaj5GWdOul XbtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=MqYQEfkEY7sCb9UYsuT4drPXIBgSgnfKLT5H6kv2J3k=; b=P52DDbYDdyUhiyCAIZ0PpVV73MKEUGRuqnAOc2/WHhQKO0xtedIiJlhYrpD5zmfU8F 7A1bwvuSpcVkCiuxC+jmWNn2EF6mk0lNCKFjlGqYifEpXyDDkyYWhJr+6Zc6RkiXLQkz sf1dIykJrACJeulVkvR1eg8tN7+HjVURm+u/Yd9rtKqXMZCjWPVLCRBb966N5qmUlL74 BaADkiTa9AESj6T4YZmgyPJncMwwfVmrdTi4XYH6O2j7XkCsfA3Xryf5r335zeyjNIOp KIgUNYzZwpRdJPgW2yPxMi3pWrYulruOounLN1jksYC77qDtHCi9lNktKpVT567jLs3R Wo4Q== X-Gm-Message-State: ANhLgQ1lmWM6K3SeWln5EgTTbUS550ntnXSC2NCvU0H9x4Kk6+DsGVXL t0tpJl+nvPrwse3+/qVMIG/zVbJBzSF9/Q0c79oEufex598= X-Google-Smtp-Source: ADFU+vsuulc2TMHk83AZeDwks7tbeUSKKaIeyfCuu+E2EeiJAj47478wK3Y2mifKvSYM1IX23H2VkkR8IRFvj8oDMdo= X-Received: by 2002:ac8:5448:: with SMTP id d8mr10290480qtq.205.1584071502702; Thu, 12 Mar 2020 20:51:42 -0700 (PDT) MIME-Version: 1.0 References: <8357fd6d-d02d-76bb-712a-e5c330ecf434@freebsd.org> In-Reply-To: <8357fd6d-d02d-76bb-712a-e5c330ecf434@freebsd.org> From: Wanpeng Qian Date: Fri, 13 Mar 2020 12:51:21 +0900 Message-ID: Subject: Re: Passthru a VF to Windows guest cause BSOD, chelsio T520BT, SR-IOV To: Peter Grehan Cc: FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 48dsFX6HZyz4JxF X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Mar 2020 03:51:45 -0000 Hi Peter > Can you try disabling APIC acceleration (which should be available on > that Xeon E5) ? This can be done on the fly using kenv and > unloading/reloading vmm.ko (or loader.conf and rebooting) After disabling tpr_shadowing, the Windows 10 1909 client boots without the IRQL_NOT_LESS_OR_EQUAL BSoD, (Windows 10 is very slow without tpr shadowing) But the driver didn't work. With Windows 10 built in driver, device status is : Windows is still setting up the class configuration for this device. (Code 56) When installing chelsio driver, cause BSoD, which is SYSTEM_THREAD_EXCEPTION_NOT_HANDLE (chnetx64.sys) I also try Windows 2012, same as Windows 10, BSoD while installing driver. And Windows 7 Pro, the chelsio driver didn't have a valid digial signed, not working. Thanks Qian On Fri, Mar 13, 2020 at 11:06 AM Peter Grehan wrote: > > > When I pass the same VF1 to Windows 10 guest, ver 1909, the guest got > > BSOD during installation. I also try to add VF1 post installation. > > also got BSOD. > > the message is IRQL_NOT_LESS_OR_EQUAL. > > I also try Windows server 2012, during installation of Chelsio Driver, BSOD too. > > > > Any suggestions? driver issue or bhyve issue? > > Can you try disabling APIC acceleration (which should be available on > that Xeon E5) ? This can be done on the fly using kenv and > unloading/reloading vmm.ko (or loader.conf and rebooting) > > kenv hw.vmm.vmx.use_tpr_shadowing=0 > kldunload vmm > kldload vmm > > later, > > Peter. >