From owner-freebsd-virtualization@freebsd.org Mon Mar 18 17:16:10 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 499DF154793F for ; Mon, 18 Mar 2019 17:16:10 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: from mail-oi1-f171.google.com (mail-oi1-f171.google.com [209.85.167.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82A926B718 for ; Mon, 18 Mar 2019 17:16:09 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: by mail-oi1-f171.google.com with SMTP id u12so10733262oiv.1 for ; Mon, 18 Mar 2019 10:16:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=UmOBIWCnN4fQmtkc89a/wEcxCX22OmpnrWaJ1YwT5xA=; b=EWLk1ilIZAFPQW3HRrm8Wfh3FndufX6/rIch5tlpDHGtTod2wgjez04Lfq5m4VkNPo i58KqlExAcydILjqOKsaZ48Ggn1PIJytnYWhRPeIXTtQ4iecb49HbE4QkOVTHPBO3CI1 bq/YLMP7wHFAgWn37TLoBnuy5vH50pnvIQRl4uqTQOAdf9bV3Ya53fBikVpK8ypTzwOE myaOYgAlkGMTxVuHNI297ecXofpv0FC3QQibcoIIP7SufkNSRpikIoHMIT3yF835tI9d bYuuj7V/5Wb1abqei13xIVruu2rk810pwsNsZt+EZ3cm1p/ozBec8aiKUiAyQo53TpOa uNag== X-Gm-Message-State: APjAAAUFLrOyTuX4/UpoMKGFV118l596PtOEkoeXGRwgremhM4dZm20m A3NRMtPRcrCUL97ff/GsCDUBU5ut X-Google-Smtp-Source: APXvYqz7yhK+bGEPEBsjNjPAZpVZuM3sbjw41F9lMFMDCAW13rZlfq+JkGea1THro2glU/52Tm0IsQ== X-Received: by 2002:aca:3a57:: with SMTP id h84mr10099285oia.162.1552929362001; Mon, 18 Mar 2019 10:16:02 -0700 (PDT) Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com. [209.85.167.170]) by smtp.gmail.com with ESMTPSA id n3sm10777ota.81.2019.03.18.10.16.01 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 18 Mar 2019 10:16:01 -0700 (PDT) Received: by mail-oi1-f170.google.com with SMTP id v7so3936570oie.8 for ; Mon, 18 Mar 2019 10:16:01 -0700 (PDT) X-Received: by 2002:aca:407:: with SMTP id 7mr5467206oie.90.1552929361259; Mon, 18 Mar 2019 10:16:01 -0700 (PDT) MIME-Version: 1.0 From: Chuck Tuffli Date: Mon, 18 Mar 2019 10:15:50 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: emulated PCI device BAR size To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 82A926B718 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates 209.85.167.171 as permitted sender) smtp.mailfrom=ctuffli@gmail.com X-Spamd-Result: default: False [-4.22 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; MIME_TRACE(0.00)[0:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.91)[-0.909,0]; RCVD_IN_DNSWL_NONE(0.00)[171.167.85.209.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; FORGED_SENDER(0.30)[chuck@freebsd.org,ctuffli@gmail.com]; DMARC_NA(0.00)[freebsd.org]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[chuck@freebsd.org,ctuffli@gmail.com]; IP_SCORE(-1.30)[ip: (-0.45), ipnet: 209.85.128.0/17(-3.88), asn: 15169(-2.09), country: US(-0.07)]; TO_DOM_EQ_FROM_DOM(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2019 17:16:10 -0000 In debugging why Windows doesn't like bhyve's NVMe device emulation, another developer mentioned that the NVMe specification defines the minimum size of BAR[0] to be 16K bytes. And while most OS don't enforce this requirement, evidently, Windows does. On the surface, a "minimum BAR size" sounds odd, but the MMIO BAR in NVMe includes the doorbell registers for each queue. And since the maximum number of queues is implementation specific, there isn't a fixed size for the BAR. So my question is how to best fix this. The easiest would be to add a check to the BAR size calculation in pci_nvme.c along the lines of: #define NVME_MMIO_SPACE_MIN (1 << 14) ... pci_membar_sz = MAX( , NVME_MMIO_SPACE_MIN); While this makes Windows happy, are there any implications to this approach other than chewing up more memory than the device actually needs? Alternatively, is there a better mechanism to use in this case? --chuck