From owner-svn-src-stable-9@freebsd.org Fri Aug 5 17:14:47 2016 Return-Path: Delivered-To: svn-src-stable-9@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 29AC2BB0F1B; Fri, 5 Aug 2016 17:14:47 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D465A1D15; Fri, 5 Aug 2016 17:14:46 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u75HEktv060496; Fri, 5 Aug 2016 17:14:46 GMT (envelope-from jhb@FreeBSD.org) Received: (from jhb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u75HEkKd060495; Fri, 5 Aug 2016 17:14:46 GMT (envelope-from jhb@FreeBSD.org) Message-Id: <201608051714.u75HEkKd060495@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhb set sender to jhb@FreeBSD.org using -f From: John Baldwin Date: Fri, 5 Aug 2016 17:14:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-9@freebsd.org Subject: svn commit: r303777 - stable/9/sys/x86/x86 X-SVN-Group: stable-9 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-9@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for only the 9-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2016 17:14:47 -0000 Author: jhb Date: Fri Aug 5 17:14:45 2016 New Revision: 303777 URL: https://svnweb.freebsd.org/changeset/base/303777 Log: MFC 302181: Add a tunable to disable migration of MSI-X interrupts. The new 'machdep.disable_msix_migration' tunable can be set to 1 to disable migration of MSI-X interrupts. Xen versions prior to 4.6.0 do not properly handle updates to MSI-X table entries after the initial write. In particular, the operation to unmask a table entry after updating it during migration is not propagated to the "real" table for passthrough devices causing the interrupt to remain masked. At least some systems in EC2 are affected by this bug when using SRIOV. The tunable can be set in loader.conf as a workaround. Modified: stable/9/sys/x86/x86/msi.c Directory Properties: stable/9/sys/ (props changed) Modified: stable/9/sys/x86/x86/msi.c ============================================================================== --- stable/9/sys/x86/x86/msi.c Fri Aug 5 17:13:25 2016 (r303776) +++ stable/9/sys/x86/x86/msi.c Fri Aug 5 17:14:45 2016 (r303777) @@ -44,6 +44,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -134,6 +135,16 @@ struct pic msi_pic = { msi_enable_source msi_source_pending, NULL, NULL, msi_config_intr, msi_assign_cpu }; +/* + * Xen hypervisors prior to 4.6.0 do not properly handle updates to + * enabled MSI-X table entries. Allow migration of MSI-X interrupts + * to be disabled via a tunable. + */ +static int msix_disable_migration = 0; +SYSCTL_INT(_machdep, OID_AUTO, disable_msix_migration, CTLFLAG_RDTUN, + &msix_disable_migration, 0, + "Disable migration of MSI-X interrupts between CPUs"); + static int msi_enabled; static int msi_last_irq; static struct mtx msi_lock; @@ -212,6 +223,9 @@ msi_assign_cpu(struct intsrc *isrc, u_in if (msi->msi_first != msi) return (EINVAL); + if (msix_disable_migration && msi->msi_msix) + return (EINVAL); + /* Store information to free existing irq. */ old_vector = msi->msi_vector; old_id = msi->msi_cpu;