From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 16 08:16:55 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B2BA106564A; Fri, 16 Apr 2010 08:16:55 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 9EFB08FC13; Fri, 16 Apr 2010 08:16:54 +0000 (UTC) Received: from lstewart.caia.swin.edu.au (lstewart.caia.swin.edu.au [136.186.229.95]) by lauren.room52.net (Postfix) with ESMTPSA id C9A587E84A; Fri, 16 Apr 2010 18:16:52 +1000 (EST) Message-ID: <4BC81CF4.6060006@freebsd.org> Date: Fri, 16 Apr 2010 18:16:52 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100409 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4BC711DA.4080207@freebsd.org> In-Reply-To: <4BC711DA.4080207@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: rrs@freebsd.org, tuexen@freebsd.org Subject: Re: A cookie for anyone who can solve this C/macro problem... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2010 08:16:55 -0000 On 04/15/10 23:17, Lawrence Stewart wrote: > Hi All, > > I've hit a road block that I'm hoping people with better C/macro fu can > help me with. It also may be the case that what I'm trying to do can't > be done, so I welcome alternative suggestions as well. > > The problem: > > I'm working on a KPI to modularise congestion control support in the > kernel (overview of KPI is visible in [1]). So far, I've been focused > solely on TCP, but I want to have the framework generalise to be able to > share congestion control algorithms between other congestion aware > transports e.g. SCTP, DCCP. > > To achieve this, I need to pass congestion control information into the > hook functions in a protocol agnostic way. The current KPI passes the > tcpcb ptr and this works nicely, but obviously doesn't help. > > The cleanest option as I see it is to embed a new CC specific struct in > the appropriate TCP and SCTP structs, and pass this in to the KPI. I > started down this road and quickly stopped - the amount of code churn > this will cause in the TCP and SCTP stacks is to my mind far to > intrusive, at least for an initial import of the framework. > > So the next idea I've been trying to prototype was to pass a union of > the various protocol struct types into the KPI and then use macro magic > to allow the CC algorithms to access commonly named variables across the > different transport struct types. It would require vars to have the same > name, which would still mean some code churn, but less than the first > option. Here are the basic bits I'm currently working with: > > struct cc_var { > uint16_t type; > union { > struct tcpcb *tcp; > struct sctp_nets *sctp; > } ccvc; > }; > > All function ptrs in struct cc_algo [1] are modified to accept a "struct > cc_var *ccv". > > In an algorithm implementation e.g. NewReno [2], I then want to be able > to do things like this with the fictitious CCVC() macro: > > void > newreno_ack_received(struct cc_var *ccv) > { > u_int cw = CCVC(ccv)->snd_cwnd; > ... > CCVC(ccv)->snd_cwnd = blah; > } > > > So far I haven't found a way to achieve the above, and the best I've > come up (with help) is this: > > #define CCV_DO(ccv, what) \ > ( \ > (ccv)->type == IPPROTO_TCP ? (ccv)->ccvc.tcp->what : \ > (ccv)->ccvc.sctp->what \ > ) > > which can be used like this: > > void > newreno_ack_received(struct cc_var *ccv) > { > u_int cw = CCV_DO(ccv, snd_cwnd); > ... > CCVC(ccv, snd_cwnd = blah); > } > > Of course, this falls apart if you try do this for example: > > CCVC(ccv, snd_cwnd = min(blah, bleh)); > > > So... I'm sure there are some good ideas out there and would really > appreciate hearing about them. Thanks to those who replied on and off list. The cookie reward goes to Hans and Daniel. The key insight I was missing is that the C ternary statement doesn't return something suitable for use as an lvalue. Dereferencing a ptr returned by the ternary does though, and this provides me with a fairly neat way of dealing with the problem. For posterity's sake, relevant snippets of working code that I'm now using are as follows: struct cc_var { union ccv_container { struct tcpcb *tcp; struct sctp_nets *sctp; } ccvc; int type; }; #define CCV(ccv, what) \ (*( \ (ccv)->type == IPPROTO_TCP ? &(ccv)->ccvc.tcp->what : \ &(ccv)->ccvc.sctp->what \ )) In future, the CCV macro could also grow support for DCCP using a nested ternary e.g. (untested) #define CCV(ccv, what) \ (*( \ (ccv)->type == IPPROTO_TCP ? &(ccv)->ccvc.tcp->what : \ ((ccv)->type == IPPROTO_SCTP ? &(ccv)->ccvc.sctp->what : \ &(ccv)->ccvc.dccp->what) \ )) And finally, an example use of the current code in the NewReno CC module: void newreno_ack_received(struct cc_var *ccv) { u_int cw = CCV(ccv, snd_cwnd); u_int incr = CCV(ccv, t_maxseg); if (cw > CCV(ccv, snd_ssthresh)) incr = max((incr * incr / cw), 1); CCV(ccv, snd_cwnd) = min(cw+incr, TCP_MAXWIN << CCV(ccv, snd_scale)); } Whilst we're here and on an educational roll, does anyone have thoughts on what the compiler actually does with "*(blah ? &1 : &2)"? As this TCP code is called in the fast path, I'm curious to know if there is a runtime penalty for taking the ref and then immediately derefing it like this, or if the only overhead is the test+branch and the compiler is smart enough to collapse *(&1 or &2) to (1 or 2). I don't have a good mental model of how this stuff works under the hood... Thanks again. Cheers, Lawrence