From owner-freebsd-current@FreeBSD.ORG Tue May 5 13:40:29 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D449014B; Tue, 5 May 2015 13:40:29 +0000 (UTC) Received: from aussmtpmrkps320.us.dell.com (aussmtpmrkps320.us.dell.com [143.166.224.254]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "ausxipmktps31.us.dell.com", Issuer "Verizon Public SureServer CA G14-SHA1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 81E801357; Tue, 5 May 2015 13:40:29 +0000 (UTC) X-Loopcount0: from 64.238.244.148 X-IronPort-AV: E=Sophos;i="5.13,372,1427778000"; d="scan'208";a="269036787" From: Eric Badger To: "current@freebsd.org" , "rstone@FreeBSD.org" Subject: PCI PF memory decode disable when sizing VF BARs Thread-Topic: PCI PF memory decode disable when sizing VF BARs Thread-Index: AdCHMw+ggum3MjVXSzieEZBy1GXG8wABWymg Date: Tue, 5 May 2015 13:37:37 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.31.100.186] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 May 2015 13:40:29 -0000 Hi Ryan and -current, During IOV config, when setting up VF bars, several calls are made to 'pci_= read_bar' (in sys/dev/pci/pci.c) in order to size VF BARs, which causes mem= ory decoding to be turned off temporarily for the PF associated with those = VFs. I'm finding that this can interfere with an already running PF. I've s= everal thoughts about how this might be handled, but I'm not convinced I un= derstand all of the consequences each of them entails, so any thoughts from= others would be appreciated. Here are ideas I've considered: 1. Check the value of the 'reg' arg to 'pci_read_bar' and, if it is outside= a standard BAR range, don't disable memory decoding. This is simple, but f= eels a little hackish and may have consequences I'm missing. 2. Pass some flag/context through such that pci_read_bar knows it is config= uring VF BARs (we might instead disable VF MSE in this case, if it is enabl= ed). It would be necessary to carry this flag/context through several funct= ion calls before reaching pci_read_bar, which might end up being ugly. 3. Rearrange the calls so that VF BARs are sized when the PF is not yet run= ning, and that info saved until VFs are created. Probably it would be done = when the PF BARs are sized for any device supporting IOV, even if that devi= ce never creates VFs. Thanks, Eric