From owner-freebsd-virtualization@freebsd.org Thu Jan 12 07:29:44 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DCC05CACFB4 for ; Thu, 12 Jan 2017 07:29:44 +0000 (UTC) (envelope-from soralx@cydem.org) Received: from smtp.triumf.ca (smtp.triumf.ca [142.90.100.188]) by mx1.freebsd.org (Postfix) with ESMTP id CAB6D1D5F for ; Thu, 12 Jan 2017 07:29:44 +0000 (UTC) (envelope-from soralx@cydem.org) Received: from mscad14 (mscad14.triumf.ca [142.90.115.36]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.triumf.ca (Postfix) with ESMTP id 5F8C9F805 for ; Wed, 11 Jan 2017 23:29:43 -0800 (PST) Date: Wed, 11 Jan 2017 23:29:42 -0800 From: To: Subject: Re: Issues with GTX960 on CentOS7 using bhyve PCI passthru (FreeBSD 11-RC2) Message-ID: <20170111232942.1e3cc7cc@mscad14> In-Reply-To: <20170111183729.6bd82398@mscad14> References: <20170111014544.70670784@mscad14> <201701111041.v0BAfOoV043390@pdx.rh.CN85.dnsmgr.net> <20170111183729.6bd82398@mscad14> X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.29; amd64-portbld-freebsd9.3) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jan 2017 07:29:45 -0000 > This gives me the idea to try a different driver version in Linux... Tried the same driver version in Linux as in FreeBSD. The driver seems to talk to the card now, but not sure whether I can call this progress: [ 0.536988] PCI host bridge to bus 0000:00 [ 0.537291] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.537776] pci_bus 0000:00: root bus resource [io 0x0d00-0x1fff window] [ 0.538248] pci_bus 0000:00: root bus resource [io 0x2000-0x211f window] [ 0.538722] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xc40fffff window] [ 0.539244] pci_bus 0000:00: root bus resource [mem 0x3400000000-0x340c0fffff window] [ 0.539791] pci_bus 0000:00: root bus resource [bus 00] [ 0.540204] pci 0000:00:00.0: [1275:1275] type 00 class 0x060000 [ 0.540402] pci 0000:00:00.0: reg 0x30: [mem 0x00000000-0x000007ff pref] [ 0.540557] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.540826] pci 0000:00:01.0: reg 0x30: [mem 0x00000000-0x000007ff pref] [ 0.540923] pci 0000:00:02.0: [1af4:1001] type 00 class 0x010000 [ 0.541052] pci 0000:00:02.0: reg 0x10: [io 0x2000-0x203f] [ 0.541090] pci 0000:00:02.0: reg 0x14: [mem 0xc0000000-0xc0001fff] [ 0.541273] pci 0000:00:02.0: reg 0x30: [mem 0x00000000-0x000007ff pref] [ 0.541442] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 [ 0.541568] pci 0000:00:03.0: reg 0x10: [io 0x2040-0x205f] [ 0.541605] pci 0000:00:03.0: reg 0x14: [mem 0xc0002000-0xc0003fff] [ 0.541786] pci 0000:00:03.0: reg 0x30: [mem 0x00000000-0x000007ff pref] [ 0.541992] pci 0000:00:04.0: [10de:0dd8] type 00 class 0x030000 [ 0.542136] pci 0000:00:04.0: reg 0x10: [mem 0xc2000000-0xc3ffffff] [ 0.542198] pci 0000:00:04.0: reg 0x14: [mem 0x3400000000-0x3407ffffff 64bit pref] [ 0.542259] pci 0000:00:04.0: reg 0x1c: [mem 0x3408000000-0x340bffffff 64bit pref] [ 0.542302] pci 0000:00:04.0: reg 0x24: [io 0x2080-0x20ff] [ 0.542346] pci 0000:00:04.0: reg 0x30: [mem 0xf6000000-0xf607ffff pref] [ 0.549031] vgaarb: setting as boot device: PCI:0000:00:04.0 [ 0.549430] vgaarb: device added: PCI:0000:00:04.0,decodes=io+mem,owns=io+mem,locks=none [ 0.549995] vgaarb: loaded [ 0.550190] vgaarb: bridge control possible 0000:00:04.0 [ 0.616082] pci 0000:00:04.0: can't claim BAR 6 [mem 0xf6000000-0xf607ffff pref]: no compatible bridge window [ 0.616775] pci 0000:00:04.0: BAR 6: assigned [mem 0xc0080000-0xc00fffff pref] [ 0.617281] pci 0000:00:01.0: BAR 6: assigned [mem 0xc0004000-0xc00047ff pref] [ 0.617789] pci 0000:00:02.0: BAR 6: assigned [mem 0xc0004800-0xc0004fff pref] [ 0.618303] pci 0000:00:03.0: BAR 6: assigned [mem 0xc0005000-0xc00057ff pref] [ 0.618807] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.618808] pci_bus 0000:00: resource 5 [io 0x0d00-0x1fff window] [ 0.618809] pci_bus 0000:00: resource 6 [io 0x2000-0x211f window] [ 0.618810] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xc40fffff window] [ 0.618811] pci_bus 0000:00: resource 8 [mem 0x3400000000-0x340c0fffff window] [ 1.669308] nvidia: module verification failed: signature and/or required key missing - tainting kernel [ 1.676499] AVX2 version of gcm_enc/dec engaged. [ 1.676844] nvidia 0000:00:04.0: can't derive routing for PCI INT A [ 1.676845] nvidia 0000:00:04.0: PCI INT A: no GSI [ 1.676904] vgaarb: device changed decodes: PCI:0000:00:04.0,olddecodes=io+mem,decodes=none:owns=io+mem [ 1.676983] nvidia-nvlink: Nvlink Core is being initialized, major device number 248 [ 1.676991] NVRM: loading NVIDIA UNIX x86_64 Kernel Module 367.57 Mon Oct 3 20:37:01 PDT 2016 [ 1.683125] AES CTR mode by8 optimization enabled [ 1.687576] [drm] Initialized drm 1.1.0 20060810 [ 1.706991] nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms 367.57 Mon Oct 3 20:32:57 PDT 2016 [ 1.708732] [drm] [nvidia-drm] [GPU ID 0x00000004] Loading driver After starting Xorg: [ 23.762260] divide error: 0000 [#1] SMP [ 23.762271] Modules linked in: nvidia_uvm(POE) mac_hid 8250_fintek ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear nvidia_drm(POE) nvidia_modeset(POE) drm_kms_helper crct10dif_pclmul syscopyarea sysfillrect crc32_pclmul ghash_clmulni_intel sysimgblt fb_sys_fops drm aesni_intel aes_x86_64 lrw nvidia(POE) gf128mul glue_helper ablk_helper cryptd fjes [ 23.762273] CPU: 2 PID: 1423 Comm: Xorg Tainted: P OE 4.4.0-59-generic #80-Ubuntu [ 23.762273] Hardware name: BHYVE, BIOS 1.00 03/14/2014 [ 23.762274] task: ffff880005129c00 ti: ffff880006b08000 task.ti: ffff880006b08000 [ 23.762373] RIP: 0010:[] [] _nv008359rm+0xdb/0x150 [nvidia] [ 23.762374] RSP: 0018:ffff880006b0b990 EFLAGS: 00010246 [ 23.762374] RAX: 00000000ffffffff RBX: 0000000000000000 RCX: 00000000ffffffff [ 23.762375] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000001 [ 23.762375] RBP: ffff880006062e78 R08: 0000000000000020 R09: ffff880006062e60 [ 23.762375] R10: 0000000058772da3 R11: 0000000000000000 R12: ffff880039484008 [ 23.762375] R13: 0000000000000000 R14: ffff880006046008 R15: 0000000000000200 [ 23.762376] FS: 00007fa89f280a00(0000) GS:ffff88003fd00000(0000) knlGS:0000000000000000 [ 23.762377] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 23.762377] CR2: 00007f7151f3a760 CR3: 000000003875c000 CR4: 00000000000406e0 [ 23.762378] Stack: [ 23.762379] 0000000000010000 ffff880039484008 ffff880035538008 ffff880006046008 [ 23.762380] ffff88003a170408 ffffffffc0256d39 0000000000000009 ffff88000618a008 [ 23.762380] ffff880035538008 ffff880039484008 ffff88000670e808 ffffffffc024e74d [ 23.762380] Call Trace: [ 23.762460] [] ? _nv009265rm+0xa69/0xeb0 [nvidia] [ 23.762539] [] ? _nv009268rm+0x7d/0x300 [nvidia] [ 23.762633] [] ? _nv011447rm+0xa7/0x280 [nvidia] [ 23.762700] [] ? _nv011662rm+0x3b/0x60 [nvidia] [ 23.762765] [] ? _nv016842rm+0x9b8/0xad0 [nvidia] [ 23.762831] [] ? _nv000833rm+0x32c/0x6a0 [nvidia] [ 23.762896] [] ? rm_init_adapter+0x6a/0x100 [nvidia] [ 23.762929] [] ? nv_open_device+0x186/0x620 [nvidia] [ 23.762962] [] ? nvidia_open+0x14d/0x2f0 [nvidia] [ 23.762995] [] ? nvidia_frontend_open+0x58/0xa0 [nvidia] [ 23.762997] [] ? chrdev_open+0xbf/0x1b0 [ 23.762998] [] ? do_dentry_open+0x1ff/0x310 [ 23.762999] [] ? cdev_put+0x30/0x30 [ 23.763000] [] ? vfs_open+0x54/0x80 [ 23.763002] [] ? may_open+0x5b/0xf0 [ 23.763003] [] ? path_openat+0x1b7/0x1330 [ 23.763004] [] ? simple_xattr_get+0x2f/0xb0 [ 23.763005] [] ? do_filp_open+0x91/0x100 [ 23.763006] [] ? __alloc_fd+0x46/0x190 [ 23.763007] [] ? do_sys_open+0x138/0x2a0 [ 23.763008] [] ? mntput+0x24/0x40 [ 23.763009] [] ? path_put+0x1e/0x30 [ 23.763010] [] ? SyS_open+0x1e/0x20 [ 23.763011] [] ? entry_SYSCALL_64_fastpath+0x16/0x71 [ 23.763019] Code: 14 48 89 d8 48 c1 e8 1d 48 85 c0 75 73 8d 14 dd 00 00 00 00 3b 55 04 76 03 8b 55 04 41 89 56 2c 8b 75 08 8d 4c 32 ff 31 d2 89 c8 f6 41 89 46 78 89 c1 44 89 f8 0f af 45 14 41 0f af c5 41 c1 [ 23.763100] RIP [] _nv008359rm+0xdb/0x150 [nvidia] [ 23.763100] RSP [ 23.763102] ---[ end trace ceee1eeaca570dad ]--- -- [SorAlx] ridin' VN2000 Classic LT