From owner-freebsd-current@FreeBSD.ORG Mon Jan 5 13:17:33 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 342B3AF8 for ; Mon, 5 Jan 2015 13:17:33 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EB6A267364 for ; Mon, 5 Jan 2015 13:17:32 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id CFA371FE022; Mon, 5 Jan 2015 14:17:29 +0100 (CET) Message-ID: <54AA8F19.9030300@selasky.org> Date: Mon, 05 Jan 2015 14:18:17 +0100 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: FreeBSD Current , markb@mellanox.com Subject: [RFC] Start SMP subsystem earlier Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jan 2015 13:17:33 -0000 Hi, There is a limitiation on the number of interrupt vectors available when only a single processor is running. To have more interrupts available we need to start SMP earlier when building a monotolith kernel and not loading drivers as modules. The driver in question is a network driver and because it cannot be started after SI_SUB_ROOT_CONF due to PXE support I see no other option than to move SI_SUB_SMP earlier. Suggested patch: > Index: sys/kernel.h > =================================================================== > --- sys/kernel.h (revision 276691) > +++ sys/kernel.h (working copy) > @@ -152,6 +152,7 @@ > SI_SUB_KPROF = 0x9000000, /* kernel profiling*/ > SI_SUB_KICK_SCHEDULER = 0xa000000, /* start the timeout events*/ > SI_SUB_INT_CONFIG_HOOKS = 0xa800000, /* Interrupts enabled config */ > + SI_SUB_SMP = 0xa850000, /* start the APs*/ > SI_SUB_ROOT_CONF = 0xb000000, /* Find root devices */ > SI_SUB_DUMP_CONF = 0xb200000, /* Find dump devices */ > SI_SUB_RAID = 0xb380000, /* Configure GEOM classes */ > @@ -165,7 +166,6 @@ > SI_SUB_KTHREAD_BUF = 0xea00000, /* buffer daemon*/ > SI_SUB_KTHREAD_UPDATE = 0xec00000, /* update daemon*/ > SI_SUB_KTHREAD_IDLE = 0xee00000, /* idle procs*/ > - SI_SUB_SMP = 0xf000000, /* start the APs*/ > SI_SUB_RACCTD = 0xf100000, /* start racctd*/ > SI_SUB_LAST = 0xfffffff /* final initialization */ > }; This fixes a problem for Mellanox drivers in the OFED layer. Possibly we need to move the SMP even earlier to not miss the generic FreeBSD PCI device enumeration or maybe this is not possible. Does anyone know how early we can start SMP? --HPS