From owner-freebsd-virtualization@freebsd.org Fri Jun 14 19:54:00 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8D03515AFF71 for ; Fri, 14 Jun 2019 19:54:00 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9FDD176C1D for ; Fri, 14 Jun 2019 19:53:59 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by mail-io1-xd2f.google.com with SMTP id u19so8300921ior.9 for ; Fri, 14 Jun 2019 12:53:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=kuqwOcEV7Qk6m75Z/ILROioHpnSiYZXwAyrEujjwC4c=; b=gm94JZObqfhsCkhaKDp4mO2Gu0EDx02TZg7nXorIvqeeT8Ynmit1yYjv4M9/LkosEC YfnTK0250wIxjmpvMT0ssQP0YAiCEBcQpL97n6tpnSgFrrIqjXHub2KJdWsgvzicLoUy u5c3HGGeyOz5ECDPy5TIkZRwkyYqN4kvfuq7uShGnEybrIQ+QAH4JxZddKhf4zOqUSDP mqD3s9b9kv4an+Kw841PpBpg+YwPLBpebxUkDqTYzeuysVkoDAX4irb7i9pBqITkBT7a cVi8+VSQpuaUON33DAIwebJb8RFwFefHjmmQiuduotxfUSIpN2K1Ms3/efTizAUnlQRx Vq3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=kuqwOcEV7Qk6m75Z/ILROioHpnSiYZXwAyrEujjwC4c=; b=WsYMSY6P7NSjHta1H4aYGncmiMzyHsD2Jck6CVS5yUUAcgnWuogRzb43qLPuH+rBT/ h/bDHI/Jch7f6ik0hjFgYEpWBp8bT2/pjg9IXtAzC/078/UNbvUU7QYwHzwytg0lPOpl MvOPe3jcbSXISDEREgFl6Ptc0lVtsvtJ+9aKM7FWEuLgcf1Q7keiH+BbbVEtnGKawFaI Q30BfdPDtV8hN4XBGUelxEnD3er7v4XpTKv8ENC08w6r57vHIzB1O2ayDNHTaVJpuwFO 6vtyIGcmpVn+hVc0INZQmx5YghDjDA/jYyFE6ggKdb9gfnR9FScqef9vUKVdGFCV3vr6 vkCA== X-Gm-Message-State: APjAAAVnbzrM2lIwoePP0qHaGlhtPy4hEz0ExnQrXjenKKaQJe5Yst9l +QZRnX/IA86v2Pk+4YVLw2+Xj/8n5dGVtrt2ov7MaqKZ X-Google-Smtp-Source: APXvYqyQrhYAb4m+z4TAWsulkPutwj5+r/209CkK1RyiS8Z//qNMqwOR2wAbNYKLT0DyF06H/whbht+nyxEOiiO2eHo= X-Received: by 2002:a5d:9ad6:: with SMTP id x22mr10833388ion.136.1560542038175; Fri, 14 Jun 2019 12:53:58 -0700 (PDT) MIME-Version: 1.0 From: Adam Date: Fri, 14 Jun 2019 14:53:46 -0500 Message-ID: Subject: Bhyve GPU passthrough To: FreeBSD virtualization X-Rspamd-Queue-Id: 9FDD176C1D X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=gm94JZOb; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of amvandemore@gmail.com designates 2607:f8b0:4864:20::d2f as permitted sender) smtp.mailfrom=amvandemore@gmail.com X-Spamd-Result: default: False [-6.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[f.2.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(-2.87)[ip: (-8.84), ipnet: 2607:f8b0::/32(-3.16), asn: 15169(-2.31), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.83)[-0.828,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Jun 2019 19:54:00 -0000 There was a thread here a while back about some adventurer trying to passthrough an Nvidia GPU which ultimately did not work. Unfortunately that still seems to be the case. I'm using 12.x and I could not get it to work in FreeBSD, Linux, or Windows guest. Win10 wouldn't even boot with it passed through. Linux wouldn't load the driver(could have been me) and FreeBSD guest loaded the driver, but I could not do anything useful with the card. For example, nvidia-smi -q reported No devices were found. Here is the guest view: vgapci0@pci0:0:1:0: class=0x030000 card=0x77101558 chip=0x1f5010de rev=0xa1 hdr=0x00 vendor = 'NVIDIA Corporation' device = 'TU106BM [GeForce RTX 2070 Mobile]' class = display subclass = VGA bar [10] = type Memory, range 32, base 0xc0000000, size 16777216, enabled bar [14] = type Prefetchable Memory, range 64, base 0xd000000000, size 268435456, enabled bar [1c] = type Prefetchable Memory, range 64, base 0xc2000000, size 33554432, enabled bar [24] = type I/O Port, range 32, base 0x2000, size 128, enabled cap 01[60] = powerspec 3 supports D0 D3 current D0 cap 05[68] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[78] = PCI-Express 2 legacy endpoint max data 128(256) FLR RO NS link x16(x16) speed 2.5(8.0) ASPM disabled(L0s/L1) I believe in earlier discussion there was objection to overcoming Nvidia's virtualization restrictions, but I think since then both vmware and kvm have implemented workarounds to allow for this. Is anyone one currently working on this or needs some testing done? -- Adam