Date: Fri, 31 Jul 2015 20:23:07 +1000 (EST) From: Ian Smith <smithi@nimnet.asn.au> To: hiren panchasara <hiren@FreeBSD.org> Cc: freebsd-ipfw@FreeBSD.org Subject: Re: Traffic not going through dummynet Message-ID: <20150731171616.J17327@sola.nimnet.asn.au> In-Reply-To: <20150730182551.GF39365@strugglingcoder.info> References: <20150730182551.GF39365@strugglingcoder.info>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 30 Jul 2015 11:25:51 -0700, hiren panchasara wrote:
> (For various reason's I didn't get/see Ian's message. Trying to do the
> right thing by setting "In-Reply-To".)
No problem, thanks.
> On 07/27/15 at 01:07P, Ian Smith wrote:
> > On Sun, 19 Jul 2015 21:05:53 -0700, hiren panchasara wrote:
> > > Bah.
> > >
> > > So I removed ipfw and dummynet from kernconf and loaded them manually
> > > after machine came up and it worked as expected.
> >
> > In your previous post, you'd said you were using 11-current, and:
> >
> > > And GENERIC has:
> > > options IPFIREWALL
> > > options DUMMYNET
> > > options HZ=1000
> >
> > Are you sure this was a 11 GENERIC kernconf? Those options haven't
> > been in GENERIC for ages (if ever?), though they haven't needed to be
> > since (perhaps) 8.0. I guess people just follow the handbook :(
>
> I modified GENERIC and added those options. I should have been more
> clear here.
Yeah, the handbook tends to lead one that way. I 'include GENERIC' and
use no{options,device} as needed; saves me getting even more confused :)
> > > Looks like some ordering issue between ipfw and dummynet. Fwiw, for
> > > working setup, kldstat shows:
> > >
> > > 13 2 0xffffffff81e21000 21490 ipfw.ko
> > > 14 1 0xffffffff81e43000 d0f6 dummynet.ko
> >
> > Indeed. If you load ipfw and dummynet by the usual means, being
> > firewall_enable=YES and dummynet_enable=YES in rc.conf, you'll notice
> > that /etc/rc.d/ipfw, in ipfw_prestart, loads dummynet if enabled, and
> > natd and/or firewall_nat if enabled, in that order.
> >
> > The downside to doing that is that you have to have specified a type for
> > rc.firewall or pointed to a custom ruleset so it's sane on startup.
>
> Didn't know the usual mean of rc.conf modifications.
Checking /etc/defaults/rc.conf /firewall_, all you'd need for that is:
firewall_enable=YES
firewall_type=OPEN # permit all, regardless of default_to_accept
dummynet_anable=YES
which would at least load those modules in the right order, however as
shown below, you didn't need to be so fussy, up to 9.3 anyway.
> > Regarding the related(?) Bug 201488 - dummynet appears broken in
> > 10.0-RELEASE and onwards (can't traffic shape on bridges)
> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=201488
> > it does seem likely to be the same issue as you noted.
> >
> > Did you ever hear back from James Rice (for whom I seem to have seen no
> > other messages for an email address) as to whether your advice about
> > loading these in the other order helped there?
>
> I haven't heard back yet.
> >
> > As to whether this is a regression, or it would have ever worked loading
> > dummynet and then ipfw, I don't know, but I have a vague feeling that
> > I've seen other issues regarding loading a module that's already in
> > kernel in recent times .. sorry I can't be any more exact.
Actually the other one was someone loading urtwn with it already in
kernel getting bad results; I suggested trying without it in kernel, but
haven't heard back either.
> Yeah, good point about whether this is a regression or not. I am not
> aware of any such loading issues wrt modules.
In the olden days, attempting to load a module already in kernel would
fail gracefully with a message early on in dmesg. I'm still wondering
if there's some issue there. I'll try loading dummynet before ipfw on
my 9.3 system using a similar setup to yours such as noticeable delay.
> > Maybe dummynet needs a check that ipfw is loaded before starting?
>
> That'd be logical, imo.
Will I ever learn to not suggest things I can't actually fix myself? :)
Ok, here's a near-duplicate on 9.3-R of your procedure on 11.0
root@x200:~ # kldstat -v | egrep 'ipfw|dummynet'
root@x200:~ # kldload dummynet
root@x200:~ # kldstat -v | egrep 'ipfw|dummynet'
9 1 0xffffffff81a3b000 d910 dummynet.ko (/boot/kernel/dummynet.ko)
511 dummynet
10 1 0xffffffff81a49000 12200 ipfw.ko (/boot/kernel/ipfw.ko)
510 ipfw
[ note that loading dummynet also loads ipfw - as expected I guess - and
that they're here listed in that reverse order numerically, FWIW ]
root@x200:~ # ipfw -t show
65535 0 0 deny ip from any to any
root@x200:~ # ipfw add 65000 allow ip from any to any
65000 allow ip from any to any
root@x200:~ # ipfw -t show
65000 0 0 allow ip from any to any
65535 2 560 Fri Jul 31 19:28:47 2015 deny ip from any to any
[ I didn't reboot, so added the allow-all rule. I'd normally use the
method of running 'kldload ipfw && ipfw add 32000 ip from any to any'
from a remote ssh session, but 'kldload dummynet', since it auto-loads
ipfw with default-deny rule, naturally killed my login ]
root@x200:~ # ipfw add 32000 pipe 1 ip from any to any
32000 pipe 1 ip from any to any
root@x200:~ # ipfw pipe 1 config delay 100ms
root@x200:~ # ipfw -t show
32000 0 0 pipe 1 ip from any to any
65000 1 112 Fri Jul 31 19:28:59 2015 allow ip from any to any
65535 2 560 Fri Jul 31 19:28:47 2015 deny ip from any to any
root@x200:~ # ipfw pipe show
00001: unlimited 100 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x0 0 buckets 0 active
root@x200:~ # sysctl kern.features | grep ipfw # not on 9.3
root@x200:~ # sysctl net.inet.ip.dummynet | grep io # if relevant
net.inet.ip.dummynet.io_pkt_drop: 0
net.inet.ip.dummynet.io_pkt_fast: 0
net.inet.ip.dummynet.io_pkt: 7
net.inet.ip.dummynet.io_fast: 0
root@x200:~ # ping t23
PING t23 (192.168.7.7): 56 data bytes
64 bytes from 192.168.7.7: icmp_seq=0 ttl=64 time=198.792 ms
64 bytes from 192.168.7.7: icmp_seq=1 ttl=64 time=199.997 ms
64 bytes from 192.168.7.7: icmp_seq=2 ttl=64 time=199.884 ms
^C
--- t23 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 198.792/199.558/199.997/0.543 ms
root@x200:~ # ipfw disable one_pass # no diff in this context
root@x200:~ # ping t23
PING t23 (192.168.7.7): 56 data bytes
64 bytes from 192.168.7.7: icmp_seq=0 ttl=64 time=198.273 ms
64 bytes from 192.168.7.7: icmp_seq=1 ttl=64 time=200.964 ms
64 bytes from 192.168.7.7: icmp_seq=2 ttl=64 time=200.041 ms
^C
--- t23 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 198.273/199.759/200.964/1.117 ms
root@x200:~ # ipfw pipe show
00001: unlimited 100 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail
sched 65537 type FIFO flags 0x0 0 buckets 0 active
root@x200:~ # ipfw -t show
32000 25 4060 Fri Jul 31 19:37:59 2015 pipe 1 ip from any to any
65000 8 728 Fri Jul 31 19:37:59 2015 allow ip from any to any
65535 2 560 Fri Jul 31 19:28:47 2015 deny ip from any to any
root@x200:~ # ipfw add 2000 skipto 40000 ip from any to any
02000 skipto 40000 ip from any to any
root@x200:~ # ping t23
PING t23 (192.168.7.7): 56 data bytes
64 bytes from 192.168.7.7: icmp_seq=0 ttl=64 time=0.447 ms
64 bytes from 192.168.7.7: icmp_seq=1 ttl=64 time=0.392 ms
64 bytes from 192.168.7.7: icmp_seq=2 ttl=64 time=0.549 ms
64 bytes from 192.168.7.7: icmp_seq=3 ttl=64 time=0.607 ms
^C
--- t23 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.392/0.499/0.607/0.084 ms
[ note here I'd bypassed the pipe rule, and unloading dummynet is ok,
ie it doesn't just then unload ipfw too ]
root@x200:~ # kldunload dummynet
root@x200:~ # ping t23
PING t23 (192.168.7.7): 56 data bytes
64 bytes from 192.168.7.7: icmp_seq=0 ttl=64 time=0.512 ms
64 bytes from 192.168.7.7: icmp_seq=1 ttl=64 time=0.392 ms
^C
--- t23 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.392/0.452/0.512/0.060 ms
[ but as soon as I delete that bypass rule, it's all over .. ]
root@x200:~ # ipfw delete 2000
ipfw: rule 2000: setsockopt(IP_FW_DEL): Protocol not available
root@x200:~ # ipfw -t show
ipfw: getsockopt(IP_FW_GET): Protocol not available
root@x200:~ # kldstat -v | egrep 'ipfw|dummynet'
root@x200:~ #
=======
So in summary: the behaviour you demonstrated on 11.0 does not occur on
9.3 - assuming I did things in the same order that you did - so it does
indeed seem to be a regression. I have no 10.x system to test.
cheers, Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150731171616.J17327>