From nobody Fri Jun 27 07:17:27 2025 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4bT6MK1Rj5z60PJy for ; Fri, 27 Jun 2025 07:17:33 +0000 (UTC) (envelope-from Stephan.Althaus@Duedinghausen.eu) Received: from mo4-p05-ob.smtp.rzone.de (mo4-p05-ob.smtp.rzone.de [81.169.146.181]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "*.smtp.rzone.de", Issuer "Telekom Security ServerID OV Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4bT6MH1wnrz3p1N for ; Fri, 27 Jun 2025 07:17:30 +0000 (UTC) (envelope-from Stephan.Althaus@Duedinghausen.eu) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hoewweken.de header.s=strato-dkim-0002 header.b=h0dIVfmL; dkim=pass header.d=hoewweken.de header.s=strato-dkim-0003 header.b=etzp3Qf3; dkim=pass header.d=duedinghausen.eu header.s=default header.b=Hhbc7VTS; spf=none (mx1.freebsd.org: domain of Stephan.Althaus@Duedinghausen.eu has no SPF policy when checking 81.169.146.181) smtp.mailfrom=Stephan.Althaus@Duedinghausen.eu; dmarc=pass (policy=reject) header.from=duedinghausen.eu; arc=pass ("strato.com:s=strato-dkim-0002:i=1") ARC-Seal: i=1; a=rsa-sha256; t=1751008648; cv=none; d=strato.com; s=strato-dkim-0002; b=A2OdwOR6bfKIr5iADSGEEciOwX7ANZlO2BZ0m05WMJpsVrY5s8/+R/fzyC2WGL7eIz 6FKWqrpfo/X2hGzZHq5DuZx58XH85GF/CkbKgPgULILw/nvqfrwwaBU7B9kZyb4CBzWR Tao4nth901ekEvu6QDoTIQMYqyZu4uc2x3sHn3XfdP8PONWJFvx2sNePO0hXOhLAqXMc dOYhABqqN/Mwmf1xDr4dr32907Cl8Xb/bcYCNuIkgilJ9us9zjTuc42yW6FPZVx1DVkG AQ+6ifXrjsQyqG0j51SOQfCXlSVRrUl86D3SqApJPLxjo+xIIPzlnbiGidR9kRuaYO78 Csyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1751008648; s=strato-dkim-0002; d=strato.com; h=In-Reply-To:From:References:To:Subject:Date:Message-ID:Cc:Date:From: Subject:Sender; bh=4o4ZonFSBxCSFKpix37sRDiNss2LmyDS3c0RmXZ6kus=; b=GJcGSby46MBiG9AA8LO96COQNuRJcjPbPyihYjGWX8sRqFMJEtdgn2ILChV+TDMtfi ZYno/b0NBGZCh53RfqSXnwQcFTnM1nYJkw4ea1vvebR+GjZAH5ydNz8Y73xuTpkseFGI Lm3faqnJ6mbYkuMgXdmzzdE+c4N/tAeN73wA+nZDqCfmMBZACXXDyg6GB9Q4qwCHMw4K 4y51KMygYx9JXiEcw/9uG7V1KXCF8SsQ4k2mx9qYXS6GLvo6vb1VcHld2rJGqcXyjEIN kIzr76qm27sKKR0sbFjbRZqwRD+epKoeQrhLpSMlBnhAJNI7yGs0kSek3UKSTOOJx0A1 7TBQ== ARC-Authentication-Results: i=1; strato.com; arc=none; dkim=pass header.d="duedinghausen.eu" header.s="default" header.a="rsa-sha256" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1751008648; s=strato-dkim-0002; d=hoewweken.de; h=In-Reply-To:From:References:To:Subject:Date:Message-ID:Cc:Date:From: Subject:Sender; bh=4o4ZonFSBxCSFKpix37sRDiNss2LmyDS3c0RmXZ6kus=; b=h0dIVfmLv1z40ymw4LAnwtfMXBP1fqblbRZsHVYS1AtvCb7exz/dvIby4WeHreK1Hv TqSFSPKOzGz9sguFYJeHH6XJzmbHTY+/uj0mBkUpGxxfc+IV/W2WurcO0qRXBSrM/OqF rXPNO9owTr/xO/zTc6TJCqQDki3eF6Is9T6wm+AD7JzpC2+ptLjPJapJuxGiU+NTg/+M tEZR8NFvYGfkVC5DIuD3Q2x8pYCg4wqaYTeDoTGV+1wcx0bJMzExXY3PF7joBhpuaAnX 43AHaAIcTlY1khW2Uc5luAS7REOVbbeaffNQjh0B0wsHJaxhaSd7AwPWyj8udjFK9qab 2UzQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; t=1751008648; s=strato-dkim-0003; d=hoewweken.de; h=In-Reply-To:From:References:To:Subject:Date:Message-ID:Cc:Date:From: Subject:Sender; bh=4o4ZonFSBxCSFKpix37sRDiNss2LmyDS3c0RmXZ6kus=; b=etzp3Qf3+jN8spuoAIn6eFaLpiZnCpLyMQ02l01yVeu77fqoFC2EJ4pL6AshmyEGPr SSKcUhwtAb8MU3KYsMDw== X-RZG-AUTH: ":O2kGeEG7b/pS1EW2TmikjLDsYYueHLp2aWg0q38nsxvThUrH35SBqNi8UP9PRt5KqGVgfw==" X-RZG-CLASS-ID: mo05 Received: from www.duedinghausen.eu by smtp.strato.de (RZmta 51.3.0 DYNA|AUTH) with ESMTPSA id q88bdf15R7HS2hO (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate) for ; Fri, 27 Jun 2025 09:17:28 +0200 (CEST) Received: from [192.168.2.63] (p4fea3d2f.dip0.t-ipconnect.de [79.234.61.47]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) (Authenticated sender: steven) by www.duedinghausen.eu (Postfix) with ESMTPSA id 4D48014F554 for ; Fri, 27 Jun 2025 09:17:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=duedinghausen.eu; s=default; t=1751008648; bh=kW2g3sTOiyuRbaWjREvuAbnHWMO3g2KAP9kDXhSKonY=; h=Date:Subject:To:References:From:In-Reply-To; b=Hhbc7VTSDrhpnM2AvEcj4UsxU519AViySDKCCscucs/CHi+P6bgCEEAz9CjOMHGmd y0S+B8EWc8dD1ZdwiJppzS818lr4IUiBjkfYqT7h5DL4lCcWiR7x2dNPQJAT7Ve+FH YjuglBDHpF7w01iy6p6Vrek2/NOl81rTLOG3Sh61NTp4y3qCyuBfrTVu9UxRjPePwo IGM8VdjutmQN6RVe/010fsPwulzQJzKsZfnxy1uLB9ErBudB82muh0woT0uyvJhxQQ RPe6VcLg0UOGbsZhONAovQ1wbxTO/83KfW+Ir6hA0eoNG52MWm5O+IzQCOc0xx9ZH8 ItY2/tA1twJ1P1hHvTYggxFyhYJWvv/J0rBB/Nibv+wLiC9dvg8zi749lr9pK7rzAN WhvP9uc1okyNjCnZRlcHbD6PoLUeg6L77vKyRyzcgcJL5P/EmRQU9frU1lO4F5TPoH TBc9JqqZosAHC4iz1slemHc0cKpwdlJmPBllFaBSiJLyFBmL0uyEgTJPHc3LH2wb4A P3PdZPqBx9UGuraxGzJiCd3rArguFMg8OBP+jCfQdMSBwR1Tc6h38YNduY4Uw5H1JF 4MqYYUMXmBKgGDddx9aze5iturDlYO5evBQS07uZVK5ZtPZFW3Vdb32gwTkccRtgjA xdey0rOoFDEsAxeuZlQ5FwPs= Message-ID: <1e7c671d-8297-4dc2-bc10-9cc66aac28f1@Duedinghausen.eu> Date: Fri, 27 Jun 2025 09:17:27 +0200 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bhyve: GPU passthrough for Nvidia cards To: virtualization@freebsd.org References: <4785074a005d66ed92e5670b9fa4ce45a5d65eba.camel@FreeBSD.org> Content-Language: en-US From: Stephan Althaus In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.23 / 15.00]; ARC_ALLOW(-1.00)[strato.com:s=strato-dkim-0002:i=1]; NEURAL_HAM_SHORT(-0.90)[-0.900]; NEURAL_HAM_MEDIUM(-0.82)[-0.824]; DMARC_POLICY_ALLOW(-0.50)[duedinghausen.eu,reject]; NEURAL_SPAM_LONG(0.30)[0.299]; R_DKIM_ALLOW(-0.20)[hoewweken.de:s=strato-dkim-0002,hoewweken.de:s=strato-dkim-0003,duedinghausen.eu:s=default]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_POSSIBLE(0.00)[81.169.146.181:from]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6724, ipnet:81.169.144.0/22, country:DE]; RCVD_TLS_ALL(0.00)[]; MLMMJ_DEST(0.00)[virtualization@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[virtualization@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; R_SPF_NA(0.00)[no SPF record]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[hoewweken.de:+,duedinghausen.eu:+] X-Rspamd-Queue-Id: 4bT6MH1wnrz3p1N X-Spamd-Bar: --- On 6/27/25 08:38, Corvin Köhne wrote: > On Thu, 2025-06-26 at 16:49 +0200, Stephan Althaus wrote: >> On 6/26/25 15:07, Corvin Köhne wrote: >>> Hi, >>> >>> I'm currently experimenting with GPU passthrough for Nvidia cards. All I >>> have to >>> do to make it work with Ubuntu guests, is setting the INTPIN register to any >>> value instead of just 0x00. Unfortunately, I'm unfamiliar with PCI legacy >>> interrupts and how they work. The INTLINE register is set to 0xFF >>> (unconnected). >>> So, is it safe to set the INTPIN register or what has to be done for it to >>> work >>> as expected? >>> >>> Any help would be appreciated. >>> >>> >> Hi! >> >> You surely mean 'newer' NVidia cards, as my GT1030 works with FreeBSD >> 14.2 Host and ubuntu guest, NVIdia driver 570, (and Win guest btw) >> out-of-the-box, no kernel/driver modding... >> >> Regards, >> >> Stephan >> >> > FYI, I'm working on a desktop system with an RTX A500 on 15.0-CURRENT. > > Ubuntu dmesg reports "NVRM: Can't find an IRQ for your NVIDIA card!" by default. > Setting the INTPIN register solve this issue and the card can be detected by the > nvidia-smi command. However, as mentioned in my initial post, I'm not sure what > has to be emulated to be able to set the INTPIN in a sane way. > > Hi! I am sorry, i have no knowledge on that topic, how that works in detail. What i can say from my recent tests on various hosts/guests, with my GT1030, that the GPU mostly has INT_PIN 0x01, and the audio device always INT_PIN 0x02. I can't say for sure if it is 'always' because i don't think i have log files for each combination :-/ 00:08.0 VGA compatible controller: NVIDIA Corporation GP108 [GeForce GT 1030] (rev a1) (prog-if 00 [VGA controller])     Subsystem: Micro-Star International Co., Ltd. [MSI] Device 8c98     Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+     Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- SERR-