From owner-freebsd-net@freebsd.org Thu Mar 10 07:20:13 2016
Return-Path: <owner-freebsd-net@freebsd.org>
Delivered-To: freebsd-net@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
[IPv6:2001:1900:2254:206a::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E3F6ACAA62;
Thu, 10 Mar 2016 07:20:13 +0000 (UTC)
(envelope-from truckman@FreeBSD.org)
Received: from gw.catspoiler.org (unknown [IPv6:2602:304:b010:ef20::f2])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "gw.catspoiler.org", Issuer "gw.catspoiler.org" (not verified))
by mx1.freebsd.org (Postfix) with ESMTPS id 2A5B0F3D;
Thu, 10 Mar 2016 07:20:13 +0000 (UTC)
(envelope-from truckman@FreeBSD.org)
Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2])
by gw.catspoiler.org (8.15.2/8.15.2) with ESMTP id u2A7K1v4013479;
Wed, 9 Mar 2016 23:20:06 -0800 (PST)
(envelope-from truckman@FreeBSD.org)
Message-Id: <201603100720.u2A7K1v4013479@gw.catspoiler.org>
Date: Wed, 9 Mar 2016 23:20:01 -0800 (PST)
From: Don Lewis <truckman@FreeBSD.org>
Subject: Re: Dummynet AQM v0.1- CoDel and FQ-CoDel for FreeBSD's ipfw/dummynet
To: ralsaadi@swin.edu.au
cc: freebsd-net@FreeBSD.org, freebsd-ipfw@FreeBSD.org, garmitage@swin.edu.au
In-Reply-To: <6545444AE21C2749939E637E56594CEA3C1B0A7C@gsp-ex02.ds.swin.edu.au>
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-net>,
<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-net>,
<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 10 Mar 2016 07:20:13 -0000
On 10 Mar, Rasool Al-Saadi wrote:
>
>
> On Wednesday, 9 March 2016, Don Lewis wrote:
>>
>> On 26 Feb, Rasool Al-Saadi wrote:
>> > Dear all,
>> >
>> > I would like to announce that we (myself and Grenville Armitage)
>> > released
>> Dummynet AQM v0.1, which is an independent implementation of CoDel
>> and FQ-CoDel for FreeBSD's ipfw/dummynet framework, based on the IETF
>> CoDel [1] and FQ-CoDel [2] Internet-Drafts.
>> > We prepared patches for FreeBSD11-CURRENT-r295345 and FreeBSD 10.x-
>> RELEASE (10.0, 10.1, 10.2), and a technical report of our
>> implementation.
>> >
>> > Patches and documentation can be found in:
>> > http://caia.swin.edu.au/freebsd/aqm
>>
>> Without the patch below, the dummynet module fails to load
>>
>> # kldload dummynet.ko
>> kldload: can't load dummynet.ko: No such file or directory
>>
>> and the following is printed to /var/log/messages:
>>
>> link_elf: symbol sysctl__net_inet_ip_dummynet_children undefined
>
> Thanks again for testing the patch and for providing feedback.
>
> It seems that this error (and the compilation error in your previous
> email) appears in i386 versions of FreeBSD.
I was testing on FreeBSD 10.3-PRERELEASE and would have been willing to
bet that the difference in behavior was due to a recent change in
FreeBSD. After testing, this second bug is also a difference between
i386 and amd64.
Digging into the problem, this line in ip_dn_io.c:
static SYSCTL_NODE(_net_inet_ip, OID_AUTO, dummynet, CTLFLAG_RW,
0, "Dummynet");
gets translated into:
static struct sysctl_oid_list sysctl__net_inet_ip_dummynet_children; static struct sysctl_oid sysctl___net_inet_ip_dummynet = { &sysctl__net_inet_ip_children, { ((void *)0) }, (-1), 1|((0x80000000|0x40000000)), (void*)&sysctl__net_inet_ip_dummynet_children, 0, "dummynet", 0, "N", 0, 0, "Dummynet" }; __asm__(".globl " "__start_set_sysctl_set"); __asm__(".globl " "__stop_set_sysctl_set"); static void const * const __set_sysctl_set_sym_sysctl___net_inet_ip_dummynet __attribute__((__section__("set_" "sysctl_set"))) __attribute__((__used__)) = &sysctl___net_inet_ip_dummynet; _Static_assert((((0x80000000|0x40000000)) & 0xf) == 0 || (((0x80000000|0x40000000)) & 0) == 1, "compile-time assertion failed");
by the C preprocessor, and it shows up in the symbol table of the .o
file (on amd64):
0000000000000140 b sysctl__net_inet_ip_dummynet_children
where "b" indicates that it is a local symbol in the uninitialized data
section.
The dn_aqm_codel.c and dn_sched_fq_codel.c files also what access to
this SYSCTL_NODE, which they indicated by using:
SYSCTL_DECL(_net_inet_ip_dummynet);
which gets pre-processed to:
extern struct sysctl_oid_list sysctl__net_inet_ip_dummynet_children;
which results in an undefined symbol in the compiled .o file:
U sysctl__net_inet_ip_dummynet_children
A symbol declared as an extern in one compilation until should not
get linked to a matching symbol defined as a static in another
compilation unit.
Looking at the symbol table for the module dummynet.ko, I see both the
static and undefined versions of the symbol:
00000000000001b0 b sysctl__net_inet_ip_dummynet_children
U sysctl__net_inet_ip_dummynet_children
For some reason, the kernel linker on amd64 accepts this module, whereas
on i386 it is (correctly in my opinion) rejected. I'll try to put
together a simple example that I can use to file a FreeBSD bug report.
With the patch (on i386), the symbol is defined as global in ip_dn_io.o:
000000b8 B sysctl__net_inet_ip_dummynet_children
and there are no undefined versions of this symbol in the .ko file:
00010e4c b sysctl__net_inet_ip_dummynet_children
however I don't know why it is no longer a global in the .ko file.