From owner-freebsd-arch@FreeBSD.ORG Mon Jul 4 09:08:57 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B869106566C for ; Mon, 4 Jul 2011 09:08:57 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 0DC508FC14 for ; Mon, 4 Jul 2011 09:08:56 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id CE3067300A; Mon, 4 Jul 2011 11:09:12 +0200 (CEST) Date: Mon, 4 Jul 2011 11:09:12 +0200 From: Luigi Rizzo To: src-committers@freebsd.org Message-ID: <20110704090912.GA18578@onelab2.iet.unipi.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Cc: arch@freebsd.org Subject: RFC: Correct location for SUBsystem-wide options ? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jul 2011 09:08:57 -0000 Hi, in the process of deciding where to put the kernel option for NETMAP, i have looked at the current practice, and i see a bit of confusion on how things are done. Looking at the comment at the top of sys/conf/options we have * opt_foo.h for options affecting a single source file foo.[cs] * opt_some_name.h for option SOME_NAME affecting only a few files. for options affecting many but not all files (e.g. all network interfaces) * opt_global.h for system-wide options So, we have no standard place for options affecting all network interfaces, which are exactly the thing i am looking at, and I see all possible varieties in our sys/conf/options: INET, INET6 go into their own headers (opt_inet.h and opt_inet6.h) In my view these are system wide in the sense that INET support may use some specific code also in device drivers, e.g. all the checksum and tcp offloading support DEVICE_POLLING goes into opt_device_polling.h NETGRAPH goes into opt_netgraph.h ALTQ, VIMAGE go into opt_global.h (note that internal ALTQ_* options are in opt_altq.h) Missing a specific header opt_network.h, in my opinion the correct approach is the one used for ALTQ and VIMAGE: they properly go into opt_global.h, and we should move there also INET, INET6, DEVICE_POLLING and NETGRAPH (and DEV_NETMAP). This way we reduce the clutter in all the if_*.c files, and also avoid the risk that one of these files forgets to include a required opt_something.h In any case, it is not my intention to modify the status quo, but at least add a proper suggestion in sys/conf/options to deal with these cases. Comments ? cheers luigi From owner-freebsd-arch@FreeBSD.ORG Mon Jul 4 17:10:02 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61F05106566B for ; Mon, 4 Jul 2011 17:10:02 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id EE4888FC12 for ; Mon, 4 Jul 2011 17:10:01 +0000 (UTC) Received: from julian-mac.elischer.org (home-nat.elischer.org [67.100.89.137]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id p64GldiQ026865 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 4 Jul 2011 09:47:41 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <4E11EEAB.3030005@freebsd.org> Date: Mon, 04 Jul 2011 09:47:39 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.18) Gecko/20110616 Thunderbird/3.1.11 MIME-Version: 1.0 To: Luigi Rizzo References: <20110704090912.GA18578@onelab2.iet.unipi.it> In-Reply-To: <20110704090912.GA18578@onelab2.iet.unipi.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, src-committers@freebsd.org Subject: Re: RFC: Correct location for SUBsystem-wide options ? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jul 2011 17:10:02 -0000 On 7/4/11 2:09 AM, Luigi Rizzo wrote: > Hi, > in the process of deciding where to put the kernel option for NETMAP, > i have looked at the current practice, and i see a bit of confusion > on how things are done. > > Looking at the comment at the top of sys/conf/options we have > * opt_foo.h for options affecting a single source file foo.[cs] > * opt_some_name.h for option SOME_NAME affecting only a few files. > for options affecting many but not all files > (e.g. all network interfaces) > * opt_global.h for system-wide options > > So, we have no standard place for options affecting all network > interfaces, which are exactly the thing i am looking at, and I > see all possible varieties in our sys/conf/options: > > INET, INET6 go into their own headers (opt_inet.h and opt_inet6.h) > In my view these are system wide in the sense that > INET support may use some specific code also in > device drivers, e.g. all the checksum and tcp offloading > support > > DEVICE_POLLING goes into opt_device_polling.h > NETGRAPH goes into opt_netgraph.h > > ALTQ, VIMAGE go into opt_global.h > (note that internal ALTQ_* options are in opt_altq.h) > > > Missing a specific header opt_network.h, in my opinion the correct > approach is the one used for ALTQ and VIMAGE: they properly go > into opt_global.h, and we should move there also INET, INET6, > DEVICE_POLLING and NETGRAPH (and DEV_NETMAP). Netgraph was written to be a completely self standing set of code, completely loadable as modules. So it got ists own ,h VIMAGE needs to affect files all over the place do so we put it in opt_global.h. (we originally had opt_vimage,h) to minimize th work people needed to do to add vimage support. I can not be loaded as a nodule. > This way we reduce the clutter in all the if_*.c files, and > also avoid the risk that one of these files forgets to include > a required opt_something.h > > In any case, it is not my intention to modify the status quo, > but at least add a proper suggestion in sys/conf/options > to deal with these cases. > > Comments ? > > cheers > luigi > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Jul 6 14:49:56 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3102E106564A for ; Wed, 6 Jul 2011 14:49:56 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 865718FC08 for ; Wed, 6 Jul 2011 14:49:55 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA01286 for ; Wed, 06 Jul 2011 17:49:54 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E147611.6060100@FreeBSD.org> Date: Wed, 06 Jul 2011 17:49:53 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: SCHED_ULE preempt_thresh: PRI_MIN_KERN -> PRI_MIN_IDLE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 14:49:56 -0000 I do not have sufficient knowledge of SCHED_ULE, so maybe I shouldn't even talk about this, but I couldn't help but notice that many (many) users have reported in the past heavy interactivity problems with SCHED_ULE under high load, especially I/O-related load. The universal advice has always been to tune preempt_thresh via sysctl kern.sched.preempt_thresh=224. I think that David Xu was the first person that I saw recommending this. In all cases users have reported significant improvements. I must add that I also have the experience and I do use preempt_thresh=224 to this day. Now, I would like to discuss this phenomenon in two veins: 1. Why do we see the interactivity problem with the default setting of preempt_thresh=PRI_MIN_KERN (provided that PREEMPTION is enabled and FULL_PREEMPTION is not)? Could this be a general ULE issue? Or could it be because of some particular hogs (like, purely hypothetically speaking, GEOM threads)? 2. Why don't we change the default (for PREEMPTION and !FULL_PREEMPTION case) to preempt_thresh=PRI_MIN_IDLE? Plus sides of this have been reported via anecdotes. What down sides could there be? Unfortunately somehow I just couldn't grasp ULE priorities and preemption, so I'd like to ask for help of those who already have understood these things. Thank you. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Wed Jul 6 16:25:50 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDD74106564A; Wed, 6 Jul 2011 16:25:49 +0000 (UTC) (envelope-from uebayasi@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 7F1C38FC0C; Wed, 6 Jul 2011 16:25:49 +0000 (UTC) Received: by qyk30 with SMTP id 30so2386781qyk.13 for ; Wed, 06 Jul 2011 09:25:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=HOc3I48gWqmpZ5jB559QLKTjrNIGlt/wVMmdGsKHNa8=; b=Sffag20koqqSop6AOLDebGRKMfQtZAsvtuA5MEFjC8afeBqdXWoAWGpm3Pow52cwA1 hjkpbYBR/dEykjFq8kowpTovqacJ1QM34tfVWFUocQHs0Zyv9n4Ac/Hi5nk8+YNvtLRc xqUpUEmfL/X47gfGzibYoLLKZO/w9r17b5xUI= MIME-Version: 1.0 Received: by 10.229.67.16 with SMTP id p16mr6509719qci.99.1309968182478; Wed, 06 Jul 2011 09:03:02 -0700 (PDT) Received: by 10.229.62.80 with HTTP; Wed, 6 Jul 2011 09:03:02 -0700 (PDT) In-Reply-To: <20110704090912.GA18578@onelab2.iet.unipi.it> References: <20110704090912.GA18578@onelab2.iet.unipi.it> Date: Thu, 7 Jul 2011 01:03:02 +0900 Message-ID: From: Masao Uebayashi To: Luigi Rizzo Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org, src-committers@freebsd.org Subject: Re: RFC: Correct location for SUBsystem-wide options ? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 16:25:50 -0000 I think that conditional build switches belong to modules. If FreeBSD designs kernel carefully so that it can run without network support at all, "network" can be said a module, and network interfaces are modules depending on "network". Then network related build switches don't need to be global. On Mon, Jul 4, 2011 at 6:09 PM, Luigi Rizzo wrote: > Hi, > in the process of deciding where to put the kernel option for NETMAP, > i have looked at the current practice, and i see a bit of confusion > on how things are done. > > Looking at the comment at the top of sys/conf/options we have > * opt_foo.h =A0 =A0 =A0 =A0 =A0 =A0 for options affecting a single source= file foo.[cs] > * opt_some_name.h =A0 =A0 =A0 for option SOME_NAME affecting only a few f= iles. > =A0 =A0 =A0 =A0 =A0 =A0 =A0for options affecting many but not a= ll files > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(e.g. all network interfac= es) > * opt_global.h =A0 =A0 =A0 =A0 =A0for system-wide options > > So, we have no standard place for options affecting all network > interfaces, which are exactly the thing i am looking at, and I > see all possible varieties in our sys/conf/options: > > INET, INET6 =A0 =A0 go into their own headers (opt_inet.h and opt_inet6.h= ) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0In my view these are system wide in the se= nse that > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0INET support may use some specific code al= so in > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0device drivers, e.g. all the checksum and = tcp offloading > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0support > > DEVICE_POLLING =A0goes into opt_device_polling.h > NETGRAPH =A0 =A0 =A0 =A0goes into opt_netgraph.h > > ALTQ, VIMAGE =A0 =A0go into opt_global.h > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(note that internal ALTQ_* options are in = opt_altq.h) > > > Missing a specific header opt_network.h, in my opinion the correct > approach is the one used for ALTQ and VIMAGE: they properly go > into opt_global.h, and we should move there also INET, INET6, > DEVICE_POLLING and NETGRAPH (and DEV_NETMAP). > > This way we reduce the clutter in all the if_*.c files, and > also avoid the risk that one of these files forgets to include > a required opt_something.h > > In any case, it is not my intention to modify the status quo, > but at least add a proper suggestion in sys/conf/options > to deal with these cases. > > Comments ? > > =A0 =A0 =A0 =A0cheers > =A0 =A0 =A0 =A0luigi > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Thu Jul 7 01:37:44 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93F0C106566C; Thu, 7 Jul 2011 01:37:44 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 852628FC08; Thu, 7 Jul 2011 01:37:44 +0000 (UTC) Received: from xyf.my.dom (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p671bf7X070885; Thu, 7 Jul 2011 01:37:42 GMT (envelope-from davidxu@freebsd.org) Message-ID: <4E150DE5.8000801@freebsd.org> Date: Thu, 07 Jul 2011 09:37:41 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.13) Gecko/20110127 Thunderbird/3.1.7 MIME-Version: 1.0 To: Andriy Gapon References: <4E147611.6060100@FreeBSD.org> In-Reply-To: <4E147611.6060100@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: SCHED_ULE preempt_thresh: PRI_MIN_KERN -> PRI_MIN_IDLE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jul 2011 01:37:44 -0000 On 2011/07/06 22:49, Andriy Gapon wrote: > > I do not have sufficient knowledge of SCHED_ULE, so maybe I shouldn't even talk > about this, but I couldn't help but notice that many (many) users have reported in > the past heavy interactivity problems with SCHED_ULE under high load, especially > I/O-related load. > > The universal advice has always been to tune preempt_thresh via sysctl > kern.sched.preempt_thresh=224. I think that David Xu was the first person that I > saw recommending this. In all cases users have reported significant improvements. > I must add that I also have the experience and I do use preempt_thresh=224 to > this day. > > Now, I would like to discuss this phenomenon in two veins: > 1. Why do we see the interactivity problem with the default setting of > preempt_thresh=PRI_MIN_KERN (provided that PREEMPTION is enabled and > FULL_PREEMPTION is not)? Could this be a general ULE issue? Or could it be > because of some particular hogs (like, purely hypothetically speaking, GEOM threads)? > > 2. Why don't we change the default (for PREEMPTION and !FULL_PREEMPTION case) to > preempt_thresh=PRI_MIN_IDLE? Plus sides of this have been reported via anecdotes. > What down sides could there be? > > Unfortunately somehow I just couldn't grasp ULE priorities and preemption, so I'd > like to ask for help of those who already have understood these things. > > Thank you. I think people must have found full preemption hurts performance for some benchmarks, normally batch-like scheduler (FIFO) have best peformance for server applications. But for desktop, you want to tune preempt_thresh to higher value, this should reduce interactivity jitter. Regards, From owner-freebsd-arch@FreeBSD.ORG Thu Jul 7 06:53:18 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D9881065672 for ; Thu, 7 Jul 2011 06:53:18 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id AACC38FC08 for ; Thu, 7 Jul 2011 06:53:17 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA13306; Thu, 07 Jul 2011 09:53:15 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1QeiSZ-00092A-Ak; Thu, 07 Jul 2011 09:53:15 +0300 Message-ID: <4E1557D8.7040604@FreeBSD.org> Date: Thu, 07 Jul 2011 09:53:12 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.18) Gecko/20110626 Lightning/1.0b2 Thunderbird/3.1.11 MIME-Version: 1.0 To: David Xu References: <4E147611.6060100@FreeBSD.org> <4E150DE5.8000801@freebsd.org> In-Reply-To: <4E150DE5.8000801@freebsd.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org Subject: Re: SCHED_ULE preempt_thresh: PRI_MIN_KERN -> PRI_MIN_IDLE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jul 2011 06:53:18 -0000 on 07/07/2011 04:37 David Xu said the following: > On 2011/07/06 22:49, Andriy Gapon wrote: >> >> I do not have sufficient knowledge of SCHED_ULE, so maybe I shouldn't even talk >> about this, but I couldn't help but notice that many (many) users have reported in >> the past heavy interactivity problems with SCHED_ULE under high load, especially >> I/O-related load. >> >> The universal advice has always been to tune preempt_thresh via sysctl >> kern.sched.preempt_thresh=224. I think that David Xu was the first person that I >> saw recommending this. In all cases users have reported significant improvements. >> I must add that I also have the experience and I do use preempt_thresh=224 to >> this day. >> >> Now, I would like to discuss this phenomenon in two veins: >> 1. Why do we see the interactivity problem with the default setting of >> preempt_thresh=PRI_MIN_KERN (provided that PREEMPTION is enabled and >> FULL_PREEMPTION is not)? Could this be a general ULE issue? Or could it be >> because of some particular hogs (like, purely hypothetically speaking, GEOM threads)? >> >> 2. Why don't we change the default (for PREEMPTION and !FULL_PREEMPTION case) to >> preempt_thresh=PRI_MIN_IDLE? Plus sides of this have been reported via anecdotes. >> What down sides could there be? >> >> Unfortunately somehow I just couldn't grasp ULE priorities and preemption, so I'd >> like to ask for help of those who already have understood these things. >> >> Thank you. > > I think people must have found full preemption hurts performance for > some benchmarks, normally batch-like scheduler (FIFO) have best > peformance for server applications. But for desktop, you want to > tune preempt_thresh to higher value, this should reduce interactivity > jitter. Quite possible, but some data/reports wouldn't hurt. Please also note that preempt_thresh=PRI_MIN_IDLE is not exactly FULL_PREEMPTION, which sets preempt_thresh=PRI_MAX_IDLE. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Jul 8 16:08:25 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 03FDF106566B; Fri, 8 Jul 2011 16:08:25 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id A70E18FC12; Fri, 8 Jul 2011 16:08:24 +0000 (UTC) Received: by gwb15 with SMTP id 15so1050322gwb.13 for ; Fri, 08 Jul 2011 09:08:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=7uuuLM6zLi16lLNfwG3t3sBGHvjjxj4DWbM8V1mPBoA=; b=L2mmWkKFYvhr8sD2gQrvNgS7xO7IxFGegcR+y1JRCwua9W57apttzL6woLU4hX85jb YP71Akjf/GO+0EDtUILvrzGYYgeB9VxQYwXnv8RP4Jx5tfXPy1U/9indtR73i5rd6ToP vTCsfeeTbkuytcsJeyg/GzwaSvkB5IomWJV7Q= MIME-Version: 1.0 Received: by 10.236.200.133 with SMTP id z5mr1052886yhn.272.1310139437367; Fri, 08 Jul 2011 08:37:17 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.236.31.4 with HTTP; Fri, 8 Jul 2011 08:37:17 -0700 (PDT) Date: Fri, 8 Jul 2011 17:37:17 +0200 X-Google-Sender-Auth: K_Ms-7-DyzQ1KBz-zfNB6Bv6zL8 Message-ID: From: Attilio Rao To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=UTF-8 Cc: Sergey Kandaurov , Peter Wemm Subject: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 16:08:25 -0000 I've made this patch for making MAXCPU a kernel config option: http://www.freebsd.org/~attilio/maxcpu_kernel_opt.diff Besides if this is a good idea or not (which I think it is) I want to discuss this implementation and similar related problems. In this case I've been forced to include opt_maxcpu.h in all the MD param.h implementations. A similar case, KSTACK_PAGES, includes the opt_kstack_pages.h only in the consumers. While this is possible for KSTACK_PAGES, because there are very little consumers, it would be impratical for MAXCPU. Besides, this is a very dangerous practice IMHO: if a consumer fails to add opt_kstack_pages it may end up with a faulty value, introducing a breakage that would go unnoticed. In my case, I think that including opt_maxcpu is a viable panacea, but in general, after discussing with peter@, probabilly the better idea would be having a centralized script that does pre-processing before to start compiling and set with the right values all those constants (something like genassym.c, but of course with a different purpose). What are your ideas on that? Do you think that including opt_maxcpu.h would be acceptable for the time being? Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Fri Jul 8 16:43:15 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C6AD1065670 for ; Fri, 8 Jul 2011 16:43:15 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3B2248FC15 for ; Fri, 8 Jul 2011 16:43:14 +0000 (UTC) Received: by qwc9 with SMTP id 9so1378471qwc.13 for ; Fri, 08 Jul 2011 09:43:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=YunFjcVI0mC+067jE4bafIYqAzkje6/zzQZdKYa+Y9E=; b=rFzBGNMoXVYeuMozW62ngan0znZuJkTYcxKL9UC7Bxm1ObFotqK8OnnkzLNoGVqrta OpZ+N+XeFru5if/J15t4JeB/jNLdQ8lPOD20QuL58F2I/I4Ce0bGEiQP7GNKqy9sFXjL raYumu+E0xqddVdO7AoSkpeusju7hiBuncBro= MIME-Version: 1.0 Received: by 10.224.180.10 with SMTP id bs10mr1657863qab.318.1310143394229; Fri, 08 Jul 2011 09:43:14 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.229.62.229 with HTTP; Fri, 8 Jul 2011 09:43:14 -0700 (PDT) In-Reply-To: References: Date: Fri, 8 Jul 2011 09:43:14 -0700 X-Google-Sender-Auth: MLaUQoxb82SFWWye1VzuDpOKsjU Message-ID: From: mdf@FreeBSD.org To: Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 Cc: Sergey Kandaurov , Peter Wemm , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 16:43:15 -0000 On Fri, Jul 8, 2011 at 8:37 AM, Attilio Rao wrote: > I've made this patch for making MAXCPU a kernel config option: > http://www.freebsd.org/~attilio/maxcpu_kernel_opt.diff > > Besides if this is a good idea or not (which I think it is) I want to > discuss this implementation and similar related problems. > In this case I've been forced to include opt_maxcpu.h in all the MD > param.h implementations. A similar case, KSTACK_PAGES, includes the > opt_kstack_pages.h only in the consumers. While this is possible for > KSTACK_PAGES, because there are very little consumers, it would be > impratical for MAXCPU. Besides, this is a very dangerous practice > IMHO: if a consumer fails to add opt_kstack_pages it may end up with a > faulty value, introducing a breakage that would go unnoticed. > > In my case, I think that including opt_maxcpu is a viable panacea, but > in general, after discussing with peter@, probabilly the better idea > would be having a centralized script that does pre-processing before > to start compiling and set with the right values all those constants > (something like genassym.c, but of course with a different purpose). > > What are your ideas on that? Do you think that including opt_maxcpu.h > would be acceptable for the time being? Isn't MAXCPU used frequently enough that it should be in opt_global.h? Or are we discouraging the use of opt_global.h and of MAXCPU such that that's not a practical solution? Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Fri Jul 8 16:48:49 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05BBF106566C; Fri, 8 Jul 2011 16:48:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 5A4BA8FC1C; Fri, 8 Jul 2011 16:48:48 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p68GmioX002441 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 8 Jul 2011 19:48:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p68GmiI2058851; Fri, 8 Jul 2011 19:48:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p68Gmi0F058850; Fri, 8 Jul 2011 19:48:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 8 Jul 2011 19:48:44 +0300 From: Kostik Belousov To: Attilio Rao Message-ID: <20110708164844.GZ48734@deviant.kiev.zoral.com.ua> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="I3A9QrhXlID9zuAO" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Sergey Kandaurov , Peter Wemm , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 16:48:49 -0000 --I3A9QrhXlID9zuAO Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 08, 2011 at 05:37:17PM +0200, Attilio Rao wrote: > I've made this patch for making MAXCPU a kernel config option: > http://www.freebsd.org/~attilio/maxcpu_kernel_opt.diff >=20 > Besides if this is a good idea or not (which I think it is) I want to > discuss this implementation and similar related problems. > In this case I've been forced to include opt_maxcpu.h in all the MD > param.h implementations. A similar case, KSTACK_PAGES, includes the > opt_kstack_pages.h only in the consumers. While this is possible for > KSTACK_PAGES, because there are very little consumers, it would be > impratical for MAXCPU. Besides, this is a very dangerous practice > IMHO: if a consumer fails to add opt_kstack_pages it may end up with a > faulty value, introducing a breakage that would go unnoticed. >=20 > In my case, I think that including opt_maxcpu is a viable panacea, but > in general, after discussing with peter@, probabilly the better idea > would be having a centralized script that does pre-processing before > to start compiling and set with the right values all those constants > (something like genassym.c, but of course with a different purpose). >=20 > What are your ideas on that? Do you think that including opt_maxcpu.h > would be acceptable for the time being? I vote for putting MAXCPU in opt_global.h. Why did you choosed separate opt header ? --I3A9QrhXlID9zuAO Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk4XNOsACgkQC3+MBN1Mb4hLjgCfZgu0akl8CeDFWdzbp36q1sGt hqMAn0wq+6tChd4GJFbwBn38TVRSvaJq =hqJW -----END PGP SIGNATURE----- --I3A9QrhXlID9zuAO-- From owner-freebsd-arch@FreeBSD.ORG Fri Jul 8 17:43:54 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 80A481065675; Fri, 8 Jul 2011 17:43:54 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 186088FC15; Fri, 8 Jul 2011 17:43:53 +0000 (UTC) Received: by gyf3 with SMTP id 3so1093422gyf.13 for ; Fri, 08 Jul 2011 10:43:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=SDRuEgRO3CG4smBThwnH8iQYj0uPojp2/2rjm9VWd9g=; b=G35CAupuT6cAXnAPGQdxcUYhcU8wnqoEUg7alB/ueiKYvHmq/FJe0lOU6v5PBvcA4D gWO9xKlypppL3v5h3yTA2vVEysIMFEbjr85rFg1qwoS+wT2A74oZ/XGPBIrv08UbOiCU i9nR/Kg1r9e4ELpbeKNOX+52oQxFTRJnk/CmU= MIME-Version: 1.0 Received: by 10.90.200.1 with SMTP id x1mr2270314agf.58.1310147033193; Fri, 08 Jul 2011 10:43:53 -0700 (PDT) Received: by 10.91.183.18 with HTTP; Fri, 8 Jul 2011 10:43:52 -0700 (PDT) In-Reply-To: <20110708164844.GZ48734@deviant.kiev.zoral.com.ua> References: <20110708164844.GZ48734@deviant.kiev.zoral.com.ua> Date: Fri, 8 Jul 2011 10:43:52 -0700 Message-ID: From: Peter Wemm To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 Cc: Attilio Rao , Sergey Kandaurov , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 17:43:54 -0000 On Fri, Jul 8, 2011 at 9:48 AM, Kostik Belousov wrote: > On Fri, Jul 08, 2011 at 05:37:17PM +0200, Attilio Rao wrote: >> I've made this patch for making MAXCPU a kernel config option: >> http://www.freebsd.org/~attilio/maxcpu_kernel_opt.diff >> >> Besides if this is a good idea or not (which I think it is) I want to >> discuss this implementation and similar related problems. >> In this case I've been forced to include opt_maxcpu.h in all the MD >> param.h implementations. A similar case, KSTACK_PAGES, includes the >> opt_kstack_pages.h only in the consumers. While this is possible for >> KSTACK_PAGES, because there are very little consumers, it would be >> impratical for MAXCPU. Besides, this is a very dangerous practice >> IMHO: if a consumer fails to add opt_kstack_pages it may end up with a >> faulty value, introducing a breakage that would go unnoticed. >> >> In my case, I think that including opt_maxcpu is a viable panacea, but >> in general, after discussing with peter@, probabilly the better idea >> would be having a centralized script that does pre-processing before >> to start compiling and set with the right values all those constants >> (something like genassym.c, but of course with a different purpose). >> >> What are your ideas on that? Do you think that including opt_maxcpu.h >> would be acceptable for the time being? > > I vote for putting MAXCPU in opt_global.h. That isn't right either. Among other things, it makes the problem for kernel modules even worse because it makes non-updated 3rd party modules that happen to be impacted by a MAXCPU change to silently fail by successfully compiling. Part of the problem is that we put default values in include files, and that those defaults can cause side effects if the correct override isn't brought in. Sometimes the defaults are fail-safe, and other times they're not "safe" at all. A more correct solution would be something vaguely along these lines: - remove the #ifndef MAXCPU / #define MAXCPU 32 / #endif from the include files - put "options MAXCPU=32" into the machine/conf/DEFAULTS files - put MAXCPU into opt_maxcpu.h - replace code constructs in include files that will fail with things that will safely fail: eg: sys/pcpu.h: #define PCPU_NAME_LEN (sizeof("CPU ") + sizeof(__XSTRING(MAXCPU) + 1)) becomes.. #ifdef MAXCPU #define PCPU_NAME_LEN (sizeof("CPU ") + sizeof(__XSTRING(MAXCPU) + 1)) #endif and extern struct pcpu *cpuid_to_pcpu[MAXCPU]; should have been: extern struct pcpu *cpuid_to_pcpu[]; or amd64/include/intr_machdep.h: #define INTRCNT_COUNT (1 + NUM_IO_INTS * 2 + (1 + 7) * MAXCPU) becomes #ifdef MAXCPU #define INTRCNT_COUNT (1 + NUM_IO_INTS * 2 + (1 + 7) * MAXCPU) #endif and so on. The idea being that if the requisite opt_maxcpu.h file (or whatever it ends up in) isn't correctly pulled into scope, then you correctly get a compile failure. And if a kernel module somehow depends on the value of INTRCNT_COUNT, then it will correctly fail to compile unless specific kld-safety measures are taken care of. The second point of doing it that way is that you get to have a single source of truth for values. If a value can be set in config(8) then the default value should come from config(8). The other issue is opt_global.h causes massive multiplication of compile dependencies. There really is no need to force a recompile of if_bge.o or ffs_softdep.o just because somebody changes MAXCPU. Dependencies are still supported and work - eg, My home machine: FreeBSD 9.0-CURRENT #1143: Fri Jun 12 20:34:47 PDT 2011 However, detecting that if_bge.o needed to be recompiled is crucial. The alternative I suggested above would solve that. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Fri Jul 8 17:46:59 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7CBA51065670; Fri, 8 Jul 2011 17:46:59 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 4D1CA8FC0C; Fri, 8 Jul 2011 17:46:59 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id D635C46B06; Fri, 8 Jul 2011 13:46:58 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 60E968A02E; Fri, 8 Jul 2011 13:46:58 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Fri, 8 Jul 2011 13:14:43 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110617; KDE/4.5.5; amd64; ; ) References: <20110708164844.GZ48734@deviant.kiev.zoral.com.ua> In-Reply-To: <20110708164844.GZ48734@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201107081314.44128.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Fri, 08 Jul 2011 13:46:58 -0400 (EDT) Cc: Kostik Belousov , Attilio Rao , Sergey Kandaurov , Peter Wemm Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 17:46:59 -0000 On Friday, July 08, 2011 12:48:44 pm Kostik Belousov wrote: > On Fri, Jul 08, 2011 at 05:37:17PM +0200, Attilio Rao wrote: > > I've made this patch for making MAXCPU a kernel config option: > > http://www.freebsd.org/~attilio/maxcpu_kernel_opt.diff > > > > Besides if this is a good idea or not (which I think it is) I want to > > discuss this implementation and similar related problems. > > In this case I've been forced to include opt_maxcpu.h in all the MD > > param.h implementations. A similar case, KSTACK_PAGES, includes the > > opt_kstack_pages.h only in the consumers. While this is possible for > > KSTACK_PAGES, because there are very little consumers, it would be > > impratical for MAXCPU. Besides, this is a very dangerous practice > > IMHO: if a consumer fails to add opt_kstack_pages it may end up with a > > faulty value, introducing a breakage that would go unnoticed. > > > > In my case, I think that including opt_maxcpu is a viable panacea, but > > in general, after discussing with peter@, probabilly the better idea > > would be having a centralized script that does pre-processing before > > to start compiling and set with the right values all those constants > > (something like genassym.c, but of course with a different purpose). > > > > What are your ideas on that? Do you think that including opt_maxcpu.h > > would be acceptable for the time being? > > I vote for putting MAXCPU in opt_global.h. > Why did you choosed separate opt header ? I agree this should just go into opt_global.h. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Jul 8 17:51:32 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B689A1065679; Fri, 8 Jul 2011 17:51:32 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 52FB78FC13; Fri, 8 Jul 2011 17:51:32 +0000 (UTC) Received: by gyf3 with SMTP id 3so1096591gyf.13 for ; Fri, 08 Jul 2011 10:51:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=JBRKAxeJZFIDDhU0J1/dsfk/sLbknKW1P7O+jsnz0vk=; b=eIPmyhDeFaVu/ub8Cr7jy7m3cf7Df+sWQDMntdRxb3L5KVJ1VV5CKcp+xujxgGa7Zr xmDUM9PKD7b+z+4aZlWoyvVqh1hbbp72/6uq490UbEZheMmZzAwQTwTSihts149hqvd+ uUYahAoxh28QUx3O+w6cz2c2TOlJ/jyT481DA= MIME-Version: 1.0 Received: by 10.91.178.5 with SMTP id f5mr2206063agp.85.1310145710348; Fri, 08 Jul 2011 10:21:50 -0700 (PDT) Received: by 10.91.183.18 with HTTP; Fri, 8 Jul 2011 10:21:50 -0700 (PDT) In-Reply-To: References: Date: Fri, 8 Jul 2011 10:21:50 -0700 Message-ID: From: Peter Wemm To: Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 Cc: Sergey Kandaurov , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 17:51:32 -0000 On Fri, Jul 8, 2011 at 8:37 AM, Attilio Rao wrote: > I've made this patch for making MAXCPU a kernel config option: > http://www.freebsd.org/~attilio/maxcpu_kernel_opt.diff The problem with this is now you have to update all the kernel module build glue to create opt_maxcpu.h, including 3rd party modules. What about userland? We have constructs like this: amd64/include/reg.h: #if defined(_KERNEL) && !defined(_STANDALONE) #include "opt_compat.h" #endif Or even: #if defined(SMP) || defined(KLD_MODULE) #define MAXCPU 32 #else #define MAXCPU 1 #endif /* SMP || KLD_MODULE */ Putting opt_*.h references in include files is an invasive change and something we've gone to a great deal of effort to avoid. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Fri Jul 8 17:53:02 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DF1A1065670; Fri, 8 Jul 2011 17:53:02 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id D5DFC8FC0C; Fri, 8 Jul 2011 17:53:01 +0000 (UTC) Received: by yxl31 with SMTP id 31so1102870yxl.13 for ; Fri, 08 Jul 2011 10:53:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=c91TLDtYKcR8ueXQrGZapmd4sSQmN/ZDK8t04WQEAJY=; b=XUtSZYk0ZVyNn8WeUBI4CeNrYQGCEDRNsLLZitn0wpyCA7YtDy591pcvcwR9AXCAVJ tvBwff9dBbDWRfX8sA7LZsr5OHUPBJucgzwCX51wEXWVcYiLT7pOuiQFC4rePED3rQo2 uZmzC7XDvuz8Hrhg0slVohWyP5qIjiseuHBOk= MIME-Version: 1.0 Received: by 10.90.255.18 with SMTP id c18mr2281461agi.31.1310147580828; Fri, 08 Jul 2011 10:53:00 -0700 (PDT) Received: by 10.91.183.18 with HTTP; Fri, 8 Jul 2011 10:53:00 -0700 (PDT) In-Reply-To: References: Date: Fri, 8 Jul 2011 10:53:00 -0700 Message-ID: From: Peter Wemm To: Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 Cc: Sergey Kandaurov , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 17:53:02 -0000 On Fri, Jul 8, 2011 at 8:37 AM, Attilio Rao wrote: > In my case, I think that including opt_maxcpu is a viable panacea, but > in general, after discussing with peter@, probabilly the better idea > would be having a centralized script that does pre-processing before > to start compiling and set with the right values all those constants > (something like genassym.c, but of course with a different purpose). At the risk of fragmenting the thread, I wanted to clarify the goals of what I was talking about last night. What makes me happy is when I can change a value in my config file, and after running 'config FOO; make depend; make', the 6 files affected by the value get recompiled and that's it. That's the point of keeping the dependencies limited. Putting stuff in opt_global.h defeats this. Putting opt_maxcpu.h into a widely included file also defeats this. The genassym/centralized script approach also defeats it. I think the issue you're mainly worried about is correctly detecting when the config override value failed to be pulled in for something that is important. What I suggested earlier will do that and will keep me happy by not adding false dependencies. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Fri Jul 8 17:56:24 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1F528106566B; Fri, 8 Jul 2011 17:56:24 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id A9EF38FC13; Fri, 8 Jul 2011 17:56:23 +0000 (UTC) Received: by gxk28 with SMTP id 28so1099618gxk.13 for ; Fri, 08 Jul 2011 10:56:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=IaD/6y7bO9QK9/DI52iTN3lt58j/9Go8lfuQ410OseA=; b=YRpQJy7lJJRL1qUEYraGq+nEIlY9w/Ll7HdaMnGI1fFCudf7bI4hUnZQHvX75g4Kj0 P9EwHotUkiSxj7AD979nAsdk/0IwvWShdBuoNEBWh+6ARje5jEFt+PJcBBYGZ1Cmorsq o9oKmvxDNngMFD8rvFNLH7iaOOtrXWhJEFi4M= MIME-Version: 1.0 Received: by 10.90.200.1 with SMTP id x1mr2282645agf.58.1310147780977; Fri, 08 Jul 2011 10:56:20 -0700 (PDT) Received: by 10.91.183.18 with HTTP; Fri, 8 Jul 2011 10:56:20 -0700 (PDT) In-Reply-To: <201107081314.44128.jhb@freebsd.org> References: <20110708164844.GZ48734@deviant.kiev.zoral.com.ua> <201107081314.44128.jhb@freebsd.org> Date: Fri, 8 Jul 2011 10:56:20 -0700 Message-ID: From: Peter Wemm To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: Kostik Belousov , Attilio Rao , Sergey Kandaurov , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 17:56:24 -0000 On Fri, Jul 8, 2011 at 10:14 AM, John Baldwin wrote: > On Friday, July 08, 2011 12:48:44 pm Kostik Belousov wrote: >> On Fri, Jul 08, 2011 at 05:37:17PM +0200, Attilio Rao wrote: >> > I've made this patch for making MAXCPU a kernel config option: >> > http://www.freebsd.org/~attilio/maxcpu_kernel_opt.diff >> > >> > Besides if this is a good idea or not (which I think it is) I want to >> > discuss this implementation and similar related problems. >> > In this case I've been forced to include opt_maxcpu.h in all the MD >> > param.h implementations. A similar case, KSTACK_PAGES, includes the >> > opt_kstack_pages.h only in the consumers. While this is possible for >> > KSTACK_PAGES, because there are very little consumers, it would be >> > impratical for MAXCPU. Besides, this is a very dangerous practice >> > IMHO: if a consumer fails to add opt_kstack_pages it may end up with a >> > faulty value, introducing a breakage that would go unnoticed. >> > >> > In my case, I think that including opt_maxcpu is a viable panacea, but >> > in general, after discussing with peter@, probabilly the better idea >> > would be having a centralized script that does pre-processing before >> > to start compiling and set with the right values all those constants >> > (something like genassym.c, but of course with a different purpose). >> > >> > What are your ideas on that? Do you think that including opt_maxcpu.h >> > would be acceptable for the time being? >> >> I vote for putting MAXCPU in opt_global.h. >> Why did you choosed separate opt header ? > > I agree this should just go into opt_global.h. I disagree. That makes it even harder to track down KLD's that depend on the correct value of MAXCPU. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Fri Jul 8 18:14:37 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8041106574C; Fri, 8 Jul 2011 18:14:37 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 690108FC1D; Fri, 8 Jul 2011 18:14:37 +0000 (UTC) Received: by gwb15 with SMTP id 15so1106996gwb.13 for ; Fri, 08 Jul 2011 11:14:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ns6jgb9B6q+63uZRLDL6s0s/ffbYQFMpbW99wU8R8Vg=; b=PZNofiG006j8tt9nysB9xabZd7JrpmhZsOSpcWcCcSPJKVYQXQ04GmY7vnxqeflfVR nheNYAV3XyzHEbPEJOcCcqehQiGHeAikmkfDj/yC9cBYHXilJ/9KmcjIUoweK4kJIayX ONE+raCgCPWxjwLyfzW5xvxKY1aRQyklZxwdA= MIME-Version: 1.0 Received: by 10.91.172.19 with SMTP id z19mr1620934ago.139.1310148876512; Fri, 08 Jul 2011 11:14:36 -0700 (PDT) Received: by 10.91.183.18 with HTTP; Fri, 8 Jul 2011 11:14:36 -0700 (PDT) In-Reply-To: References: <20110708164844.GZ48734@deviant.kiev.zoral.com.ua> <201107081314.44128.jhb@freebsd.org> Date: Fri, 8 Jul 2011 11:14:36 -0700 Message-ID: From: Peter Wemm To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: Kostik Belousov , Attilio Rao , Sergey Kandaurov , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 18:14:38 -0000 On Fri, Jul 8, 2011 at 10:56 AM, Peter Wemm wrote: > On Fri, Jul 8, 2011 at 10:14 AM, John Baldwin wrote: >> On Friday, July 08, 2011 12:48:44 pm Kostik Belousov wrote: >>> On Fri, Jul 08, 2011 at 05:37:17PM +0200, Attilio Rao wrote: >>> > I've made this patch for making MAXCPU a kernel config option: >>> > http://www.freebsd.org/~attilio/maxcpu_kernel_opt.diff >>> > >>> > Besides if this is a good idea or not (which I think it is) I want to >>> > discuss this implementation and similar related problems. >>> > In this case I've been forced to include opt_maxcpu.h in all the MD >>> > param.h implementations. A similar case, KSTACK_PAGES, includes the >>> > opt_kstack_pages.h only in the consumers. While this is possible for >>> > KSTACK_PAGES, because there are very little consumers, it would be >>> > impratical for MAXCPU. Besides, this is a very dangerous practice >>> > IMHO: if a consumer fails to add opt_kstack_pages it may end up with a >>> > faulty value, introducing a breakage that would go unnoticed. >>> > >>> > In my case, I think that including opt_maxcpu is a viable panacea, but >>> > in general, after discussing with peter@, probabilly the better idea >>> > would be having a centralized script that does pre-processing before >>> > to start compiling and set with the right values all those constants >>> > (something like genassym.c, but of course with a different purpose). >>> > >>> > What are your ideas on that? Do you think that including opt_maxcpu.h >>> > would be acceptable for the time being? >>> >>> I vote for putting MAXCPU in opt_global.h. >>> Why did you choosed separate opt header ? >> >> I agree this should just go into opt_global.h. > > I disagree. That makes it even harder to track down KLD's that depend > on the correct value of MAXCPU. Ahh crap. I forgot about this: #ifdef _KERNEL #define CPU_SETSIZE MAXCPU #endif MAXCPU does have to be in opt_global.h and potentially affects kernel modules. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Fri Jul 8 18:22:42 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87874106564A; Fri, 8 Jul 2011 18:22:42 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 03D158FC1A; Fri, 8 Jul 2011 18:22:40 +0000 (UTC) Received: by yxl31 with SMTP id 31so1116310yxl.13 for ; Fri, 08 Jul 2011 11:22:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=WX+SU9bSdHw6EbZPkyef9/RG5fCOCjSuiQSV4LuPe6A=; b=Smo09sPQC9T7l11FO+g0U0iprHXviP+g9G317vQfTdzwvP82dV63Z/P3hYOY34G9SC ORSU3k4s8SHpve2ETm6AgsX3uPvcy+pzWQQG7JDBCQ1nWBn3noCWQh5X4fyWjp8ctG7A jfqkT4uzXG2zYq/yQLmN8VP++R73RfUayejB0= MIME-Version: 1.0 Received: by 10.91.4.9 with SMTP id g9mr2317702agi.112.1310149357908; Fri, 08 Jul 2011 11:22:37 -0700 (PDT) Received: by 10.91.183.18 with HTTP; Fri, 8 Jul 2011 11:22:37 -0700 (PDT) In-Reply-To: References: <20110708164844.GZ48734@deviant.kiev.zoral.com.ua> <201107081314.44128.jhb@freebsd.org> Date: Fri, 8 Jul 2011 11:22:37 -0700 Message-ID: From: Peter Wemm To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Kostik Belousov , Attilio Rao , Sergey Kandaurov , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 18:22:42 -0000 On Fri, Jul 8, 2011 at 11:14 AM, Peter Wemm wrote: > On Fri, Jul 8, 2011 at 10:56 AM, Peter Wemm wrote: >> On Fri, Jul 8, 2011 at 10:14 AM, John Baldwin wrote: >>> On Friday, July 08, 2011 12:48:44 pm Kostik Belousov wrote: >>>> On Fri, Jul 08, 2011 at 05:37:17PM +0200, Attilio Rao wrote: >>>> > I've made this patch for making MAXCPU a kernel config option: >>>> > http://www.freebsd.org/~attilio/maxcpu_kernel_opt.diff >>>> > >>>> > Besides if this is a good idea or not (which I think it is) I want t= o >>>> > discuss this implementation and similar related problems. >>>> > In this case I've been forced to include opt_maxcpu.h in all the MD >>>> > param.h implementations. A similar case, KSTACK_PAGES, includes the >>>> > opt_kstack_pages.h only in the consumers. While this is possible for >>>> > KSTACK_PAGES, because there are very little consumers, it would be >>>> > impratical for MAXCPU. Besides, this is a very dangerous practice >>>> > IMHO: if a consumer fails to add opt_kstack_pages it may end up with= a >>>> > faulty value, introducing a breakage that would go unnoticed. >>>> > >>>> > In my case, I think that including opt_maxcpu is a viable panacea, b= ut >>>> > in general, after discussing with peter@, probabilly the better idea >>>> > would be having a centralized script that does pre-processing before >>>> > to start compiling and set with the right values all those constants >>>> > (something like genassym.c, but of course with a different purpose). >>>> > >>>> > What are your ideas on that? Do you think that including opt_maxcpu.= h >>>> > would be acceptable for the time being? >>>> >>>> I vote for putting MAXCPU in opt_global.h. >>>> Why did you choosed separate opt header ? >>> >>> I agree this should just go into opt_global.h. >> >> I disagree. That makes it even harder to track down KLD's that depend >> on the correct value of MAXCPU. > > Ahh crap. I forgot about this: > #ifdef _KERNEL > #define CPU_SETSIZE =A0 =A0 MAXCPU > #endif > > MAXCPU does have to be in opt_global.h and potentially affects kernel mod= ules. Yes, having a default value for sizing cpuset_t does look harmful if a kld module tries to use them. What is currently in the tree will silently fail by successfully compiling for kld modules that try to use anything affected by that. --=20 Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Fri Jul 8 19:26:31 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B9851065674; Fri, 8 Jul 2011 19:26:31 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 46BD38FC1A; Fri, 8 Jul 2011 19:26:29 +0000 (UTC) Received: by ewy1 with SMTP id 1so986139ewy.13 for ; Fri, 08 Jul 2011 12:26:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=DdZQTKpSzHKHtBK84tQ0YcGy9yXq2UfECMYaTandLn0=; b=Ljo2OKM4wvvtMlfwlnDN2RP39rlUTfn80qJ6eW7xM4bfdUb9AFUkGI3TEh9UAYy8+R 0ETTs9VN6+yvu56Mio/LqmVe1CsDpbh8XfvYndV2VDgIMSxW3hZAraOAzDorTO0cuqyt owtn3VlAlC+xyF9er6crFO2SVIsswz5/Vt4gQ= MIME-Version: 1.0 Received: by 10.213.27.204 with SMTP id j12mr806185ebc.25.1310151674190; Fri, 08 Jul 2011 12:01:14 -0700 (PDT) Received: by 10.213.7.142 with HTTP; Fri, 8 Jul 2011 12:01:14 -0700 (PDT) In-Reply-To: References: <20110708164844.GZ48734@deviant.kiev.zoral.com.ua> <201107081314.44128.jhb@freebsd.org> Date: Fri, 8 Jul 2011 15:01:14 -0400 Message-ID: From: Ryan Stone To: Peter Wemm Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Kostik Belousov , Attilio Rao , Sergey Kandaurov , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jul 2011 19:26:31 -0000 On Fri, Jul 8, 2011 at 2:22 PM, Peter Wemm wrote: > Yes, having a default value for sizing cpuset_t does look harmful if a > kld module tries to use them. =A0What is currently in the tree will > silently fail by successfully compiling for kld modules that try to > use anything affected by that. Could we implicitly add something like MODULE_DEPEND(maxcpu, MAXCPU, MAXCPU, MAXCPU); to every kld module so that we reject modules compiled with the wrong MAXCPU value? Maybe add it to DECLARE_MODULE or something. From owner-freebsd-arch@FreeBSD.ORG Sat Jul 9 05:58:40 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 426E1106564A; Sat, 9 Jul 2011 05:58:40 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail09.syd.optusnet.com.au (mail09.syd.optusnet.com.au [211.29.132.190]) by mx1.freebsd.org (Postfix) with ESMTP id B695E8FC0C; Sat, 9 Jul 2011 05:58:39 +0000 (UTC) Received: from c122-106-165-191.carlnfd1.nsw.optusnet.com.au (c122-106-165-191.carlnfd1.nsw.optusnet.com.au [122.106.165.191]) by mail09.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p695wYdx021085 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 9 Jul 2011 15:58:37 +1000 Date: Sat, 9 Jul 2011 15:58:34 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Peter Wemm In-Reply-To: Message-ID: <20110709151800.Q1090@besplex.bde.org> References: <20110708164844.GZ48734@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Kostik Belousov , Attilio Rao , Sergey Kandaurov , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jul 2011 05:58:40 -0000 On Fri, 8 Jul 2011, Peter Wemm wrote: > On Fri, Jul 8, 2011 at 9:48 AM, Kostik Belousov wrote: >> On Fri, Jul 08, 2011 at 05:37:17PM +0200, Attilio Rao wrote: >>> ... >>> What are your ideas on that? Do you think that including opt_maxcpu.h >>> would be acceptable for the time being? >> >> I vote for putting MAXCPU in opt_global.h. > > That isn't right either. Nothing will work :-). > ... > A more correct solution would be something vaguely along these lines: > > - remove the #ifndef MAXCPU / #define MAXCPU 32 / #endif from the include files > - put "options MAXCPU=32" into the machine/conf/DEFAULTS files > - put MAXCPU into opt_maxcpu.h > - replace code constructs in include files that will fail with things > that will safely fail: > > eg: sys/pcpu.h: > #define PCPU_NAME_LEN (sizeof("CPU ") + sizeof(__XSTRING(MAXCPU) + 1)) > > becomes.. > > #ifdef MAXCPU > #define PCPU_NAME_LEN (sizeof("CPU ") + sizeof(__XSTRING(MAXCPU) + 1)) > #endif That won't work at all, since PCPU_NAME_LEN is only used (in sys/pcpu.h) in a critical struct declaration: % #define PCPU_NAME_LEN (sizeof("CPU ") + sizeof(__XSTRING(MAXCPU) + 1)) % % struct pcpu { % struct thread *pc_curthread; /* Current thread */ % ... % #ifdef KTR % char pc_name[PCPU_NAME_LEN]; /* String name for KTR */ % #endif % } __aligned(CACHE_LINE_SIZE); The KTR ifdef is another source of errors. It is defined in opt_global.h. Modules that are actually modular cannot depend on options, so KTR is never defined for modules. Thus all modules are that use pcpu are likely to be broken if the kernel is compiled with KTR. We often avoid this problem by declaring fields in structs without ifdefs. But if MAXCPU is an option, even declaring the fields depends on including the options header. The above ifdef will mainly detect the dependency on the options header and show that it is nearely 100% global. There is massive namespace pollution involving sys/pcpu.h. It is included nested in even more critical headers like mutex.h and proc.h :-(. > and > extern struct pcpu *cpuid_to_pcpu[MAXCPU]; > should have been: > extern struct pcpu *cpuid_to_pcpu[]; As you noticed later, at least CPU_SETSIZE depends on MAXCPU in a more global way than this. Avoiding this might take significant changes to the API. There can be no structs or arrays with fixed sizes, or APIs that pass pointers to such. > or amd64/include/intr_machdep.h: > #define INTRCNT_COUNT (1 + NUM_IO_INTS * 2 + (1 + 7) * MAXCPU) > becomes > #ifdef MAXCPU > #define INTRCNT_COUNT (1 + NUM_IO_INTS * 2 + (1 + 7) * MAXCPU) > #endif Works better because it is localized, and also because the userland API never involve fixed sizes (it uses either kmem pointer to the start and end, with the end pointer constructed from statically configured sizes, or the sysctl interface which essentially involves dynamic arrays). I think recent changes for cpusets involve using dynamic arrays for the userland API. This might be OK for the kernel too. We have some experience expanding the static array limit for fd sets. FD_SETSIZE = 1024 once seemed large enough for anyone, and it still isn't small enough to be efficient as possible when only a few fd's are used, so APIs and implementations got mainly negative benefits from it having a fixed size even when that size was large enough. E.g., select() has an nfds count so that the whole array doesn't always have to be looked at every time, but the whole array still has to be allocated. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Jul 9 06:28:37 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A8FB21065675; Sat, 9 Jul 2011 06:28:37 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail09.syd.optusnet.com.au (mail09.syd.optusnet.com.au [211.29.132.190]) by mx1.freebsd.org (Postfix) with ESMTP id 7FF288FC17; Sat, 9 Jul 2011 06:28:36 +0000 (UTC) Received: from c122-106-165-191.carlnfd1.nsw.optusnet.com.au (c122-106-165-191.carlnfd1.nsw.optusnet.com.au [122.106.165.191]) by mail09.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p696SWZq005012 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 9 Jul 2011 16:28:33 +1000 Date: Sat, 9 Jul 2011 16:28:32 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Peter Wemm In-Reply-To: Message-ID: <20110709155856.D1090@besplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Attilio Rao , Sergey Kandaurov , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jul 2011 06:28:37 -0000 On Fri, 8 Jul 2011, Peter Wemm wrote: > On Fri, Jul 8, 2011 at 8:37 AM, Attilio Rao wrote: >> In my case, I think that including opt_maxcpu is a viable panacea, but >> in general, after discussing with peter@, probabilly the better idea >> would be having a centralized script that does pre-processing before >> to start compiling and set with the right values all those constants >> (something like genassym.c, but of course with a different purpose). > > At the risk of fragmenting the thread, I wanted to clarify the goals > of what I was talking about last night. > > What makes me happy is when I can change a value in my config file, > and after running 'config FOO; make depend; make', the 6 files > affected by the value get recompiled and that's it. > > That's the point of keeping the dependencies limited. Putting stuff > in opt_global.h defeats this. Putting opt_maxcpu.h into a widely > included file also defeats this. The genassym/centralized script > approach also defeats it. Unless the centralized script is smarter about dependencies than mkdep (or make, or we) are. > I think the issue you're mainly worried about is correctly detecting > when the config override value failed to be pulled in for something > that is important. What I suggested earlier will do that and will > keep me happy by not adding false dependencies. In another thread, someone wanted to move networking options from properly implemented headers like opt_device_polling.h to omnibus headers like opt_net.h. That would expand dependencies significantly. The dependencies on opt_net.h alone might grow larger than all the global dependencies were when the kernel was smaller and before you moved the -D's from global command line to options files. I consider having multiple options per options file to be a style bug at best. Having 1 option per file allows automatic detection of certain errors. E.g., if a .c source file has an ifdef involving option FOO, then it is easy to check that it includes the correct options header for defining FOO, and vice versa, iff the correct options header is named opt_foo.h. Otherwise, or if the ifdef of the options header is included nested in another header, then there is a maze of dependencies to untangle. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Jul 9 07:02:43 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CE7AF1065670; Sat, 9 Jul 2011 07:02:43 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from fallbackmx08.syd.optusnet.com.au (fallbackmx08.syd.optusnet.com.au [211.29.132.10]) by mx1.freebsd.org (Postfix) with ESMTP id E1C0F8FC13; Sat, 9 Jul 2011 07:02:42 +0000 (UTC) Received: from mail01.syd.optusnet.com.au (mail01.syd.optusnet.com.au [211.29.132.182]) by fallbackmx08.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p694qGIZ009713; Sat, 9 Jul 2011 14:52:16 +1000 Received: from c122-106-165-191.carlnfd1.nsw.optusnet.com.au (c122-106-165-191.carlnfd1.nsw.optusnet.com.au [122.106.165.191]) by mail01.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p694qBaM014651 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 9 Jul 2011 14:52:13 +1000 Date: Sat, 9 Jul 2011 14:52:11 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Peter Wemm In-Reply-To: Message-ID: <20110709143840.X873@besplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Attilio Rao , Sergey Kandaurov , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add MAXCPU as a kernel config option and quality discussion on this X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jul 2011 07:02:43 -0000 On Fri, 8 Jul 2011, Peter Wemm wrote: > On Fri, Jul 8, 2011 at 8:37 AM, Attilio Rao wrote: >> I've made this patch for making MAXCPU a kernel config option: >> http://www.freebsd.org/~attilio/maxcpu_kernel_opt.diff > > The problem with this is now you have to update all the kernel module > build glue to create opt_maxcpu.h, including 3rd party modules. > > What about userland? > > We have constructs like this: > > amd64/include/reg.h: > #if defined(_KERNEL) && !defined(_STANDALONE) > #include "opt_compat.h" > #endif > > Or even: > #if defined(SMP) || defined(KLD_MODULE) > #define MAXCPU 32 > #else > #define MAXCPU 1 > #endif /* SMP || KLD_MODULE */ > > Putting opt_*.h references in include files is an invasive change and > something we've gone to a great deal of effort to avoid. Putting them in opt_global.h is less invasive, but works little better since (actually modular) modules can't depend on options but opt_global.h gives options. This is handled well mainly for the SMP option since it is so important, and I'm not sure if it still works for that either. E.g., on amd64 and i386, modules use pessimized non-inline versions of all atomic ops. Other arches don't bother with the non-inlining. They also don't use SMP ifdefs in atomic.h, so this doesn't seem to break modules but just loses any possible optimizations for the non-SMP case. Bruce