From nobody Mon Sep 15 18:58:05 2025 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cQZ6z5WYqz67nS9 for ; Mon, 15 Sep 2025 18:58:19 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cQZ6z18K6z48Qq for ; Mon, 15 Sep 2025 18:58:19 +0000 (UTC) (envelope-from ctuffli@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=ctuffli@gmail.com Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-267a5aeb9f1so6320025ad.1 for ; Mon, 15 Sep 2025 11:58:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1757962698; x=1758567498; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=/AT2BT1fNFXzzneCdigxgtIMHi0aLlTohxvKQhUpUVk=; b=DPiRqtVo3QuwC//VvzF/IlA4sgNEmud62SVXrl8q/ucnuYl1x7f/CBHofuhrWdkw6U V32t6CwHw2yhZCh3qziz2RsvTNpvadKUiii3+A8tdqOpotIYwxxtXeou1dSeJJbiFKvS irfW32j4sYwR2u7QXbPROkW0gwzawuKaDM6UipEeC2Okb9WEPhBwDBxdvM/CYom5WS8b +Ess72Mki6vqOyBafp1/ZerEKM62wO9OWlKInId3vYfW/7jZRScknZZLVAyHf+wdrGOV VFtIwAPzwXa8UORN07YxDnKzeTcZnGbHh3rlsG5yea1yyLAmhwMXgq03uWW67ghkZmnT mvLw== X-Gm-Message-State: AOJu0YxEu3X11X9Ct+zf9TE8I9regx9RL0y0O6gwfnvstBZGahR8U8Ih Yy7CevUJunZZT5QO832r8scJHS6llZEyQuU7Sl9mXz//daJr/IDHwpDSvR6WWA== X-Gm-Gg: ASbGnculZl06T7O0Uj9aB0hEL/NnZ6J4/o8u1XPngy9cVYHsDLcwZ55/1RWx0lgOXMt 87el7dD0sBHLPCeuQYOewr6RdNFRBIkWDt5IypEsl09df4+PQSBrTgcuTKmvA1cEy8tPFIGm+LL PhQpXd93m90+NUc5F9M41SoHOmc75LSts+k860SKZ77rGe3DDIdzWTkfEU173jSbaJ7k2iiBzZ3 AylajTOVaV5ETY1H+7q0egY3CQKRuhGuNGDQpFIFzNXrRYkm5xdKLmPcTfeuHH5mBMbOkK1jpz3 riKELKCcJcI+rj9WphH+EZbBnTCtQ7xZWUACuIVq4cpyy9BmUow/LFDU7LePavJ+qF1bF1RkDgX TuahM1fBwsEZuD/ZOwPHtYodKn4jHS354huAqQ+OCwoWXLp+TRh5UE/wAREFA89ff58TMA2c= X-Google-Smtp-Source: AGHT+IGeuBkueLbrtSMVT3w+O/wRaqTUlUEcQXpRl5/Ni36XPsiKBixtHYhpFYi9vli+Ifkb3duZxg== X-Received: by 2002:a17:902:c402:b0:24b:1b82:2acd with SMTP id d9443c01a7336-25d26e481e5mr153912575ad.41.1757962697486; Mon, 15 Sep 2025 11:58:17 -0700 (PDT) Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com. [209.85.215.181]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-26069943776sm80670325ad.136.2025.09.15.11.58.17 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 15 Sep 2025 11:58:17 -0700 (PDT) Received: by mail-pg1-f181.google.com with SMTP id 41be03b00d2f7-b4cb3367d87so2994770a12.3 for ; Mon, 15 Sep 2025 11:58:17 -0700 (PDT) X-Received: by 2002:a17:90a:d607:b0:32e:859:c79 with SMTP id 98e67ed59e1d1-32e08593cd3mr11578641a91.0.1757962696881; Mon, 15 Sep 2025 11:58:16 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 From: Chuck Tuffli Date: Mon, 15 Sep 2025 11:58:05 -0700 X-Gmail-Original-Message-ID: X-Gm-Features: Ac12FXzhYv5kphFcFkQrgrIYJIptceliYcnPboLtUNrGJ2i-dOay1NiTABGz2EI Message-ID: Subject: Max BAR size for passthru? To: freebsd-virtualization@freebsd.org Cc: Mark Johnston , John Baldwin Content-Type: text/plain; charset="UTF-8" X-Spamd-Bar: + X-Spamd-Result: default: False [1.68 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_SPAM_SHORT(0.97)[0.970]; NEURAL_HAM_LONG(-0.78)[-0.782]; NEURAL_SPAM_MEDIUM(0.39)[0.391]; FORGED_SENDER(0.30)[chuck@freebsd.org,ctuffli@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_NONE(0.00)[209.85.215.181:received,209.85.214.176:from]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MISSING_XM_UA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.214.176:from]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; FROM_NEQ_ENVFROM(0.00)[chuck@freebsd.org,ctuffli@gmail.com]; RCVD_COUNT_THREE(0.00)[3]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[] X-Rspamd-Queue-Id: 4cQZ6z18K6z48Qq I'm trying to get PCI passthru of an AMD GPU working, and am seeing what I think is a problem mapping the device's BAR into the guest. The host shows: # pciconf -lbevV pci0:3:0:0 vgapci0@pci0:3:0:0: class=0x030000 rev=0xc0 hdr=0x00 vendor=0x1002 device=0x7590 subvendor=0x148c subdevice=0x2437 vendor = 'Advanced Micro Devices, Inc. [AMD/ATI]' device = 'Navi 44 [Radeon RX 9060 XT]' class = display subclass = VGA bar [10] = type Prefetchable Memory, range 64, base 0x4000000000, size 268435456, enabled bar [18] = type Prefetchable Memory, range 64, base 0x4010000000, size 2097152, enabled bar [20] = type I/O Port, range 32, base 0x6000, size 256, enabled bar [24] = type Memory, range 32, base 0x81100000, size 524288, enabled PCI-e errors = Correctable Error Detected Unsupported Request Detected Non-fatal = Unsupported Request Corrected = Advisory Non-Fatal Error (yes, a 256 MiB BAR). After starting the Linux guest, bhyve spits out the error: bhyve: pci_passthru: map_pptdev_mmio failed. I've traced this a bit and see that vm_map_find returns 3 (KERN_NO_SPACE) with arguments off=0, addr=0x800000000, len=2097152, max_addr=0 In the guest, lspci reports # lspci -s 00:07.0 -v 00:07.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Device 7590 (rev c0) (prog-if 00 [VGA controller]) Subsystem: Tul Corporation / PowerColor Device 2437 Flags: bus master, fast devsel, latency 0 Memory at 800000000 (64-bit, prefetchable) [size=256M] Memory at 810000000 (64-bit, prefetchable) [size=2M] I/O ports at 2000 [size=256] Memory at c1000000 (32-bit, non-prefetchable) [size=512K] Expansion ROM at 000c0000 [virtual] [disabled] [size=128K] Has anyone run into something like this before? Is this a limit I'm bumping into? Suggestions for what to poke next, etc.? TIA! --chuck