Date: Wed, 16 Sep 2009 07:18:58 +0100 From: xorquewasp@googlemail.com To: Juergen Lock <nox@jelal.kn-bremen.de> Cc: freebsd-emulation@freebsd.org Subject: Re: Problems with qemu networking on 7.2-RELEASE-amd64? Message-ID: <20090916061858.GA70047@logik.internal.network> In-Reply-To: <20090915223039.GA46462@triton8.kn-bremen.de> References: <20090914051402.GB44046@logik.internal.network> <20090915173935.GA34173@logik.internal.network> <20090915223039.GA46462@triton8.kn-bremen.de>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2009-09-16 00:30:39, Juergen Lock wrote:
> Hi!
'Lo.
> > Again:
> >
> > NetBSD x86 sees no NIC at all.
> >
> > OpenBSD x86 sees a NIC but it doesn't work ("ne3: device timeout").
> >
> Well thats unrelated, I'd say these guests just don't like most of qemu's
> emulated nic choices. I don't remember about OpenBSD, but NetBSD seemed
> to only really like the `pcnet' one, and that only after I patched qemu's
> bios, see this thread:
> http://lists.freebsd.org/pipermail/freebsd-emulation/2009-May/006207.html
>
That's good to know at least... if you can call it that. I'll try the new
BIOS.
> Well if the guest's packets reach the tap interface and come back I'd
> say that mostly rules out qemu to be the cause of the problem, and also
> this works just fine for me on stable/7 and stable/8, and I guess I did
> it on 7.2 as well...
> Hmm have you tried disabling pf to completely rule out any issues with
> it? Also you may need to enable ip forwarding (net.inet.ip.forwarding
> sysctl, also settable via gateway_enable in rc.conf) and/or play with
> the net.link.bridge.pfil_* sysctls. (see the if_bridge manpage.)
Hm!
Some quite disturbing results with the various sysctls.
I tried different combinations of the net.link.bridge.pfil_* controls and
got some strange results. All of the results below are the output of tcpdump
watching pflog0 when I execute 'telnet www.google.com 80' in the guest VM
(NetBSD SPARC):
net.link.bridge.ipfw: 0
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 0
net.link.bridge.pfil_member: 0
net.link.bridge.pfil_bridge: 1
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_onlyip: 0
Connection does appear to work.
Note the strange 'block in on bridge0' at the end...
000009 rule 23/0(match): pass out on bridge0: 10.1.3.12.65531 > 10.2.1.7.53: UDP, length 32
004923 rule 23/0(match): pass in on bridge0: 10.1.3.12.65530 > 10.2.1.7.53: UDP, length 32
000004 rule 23/0(match): pass out on bridge0: 10.1.3.12.65530 > 10.2.1.7.53: UDP, length 32
004191 rule 25/0(match): pass in on bridge0: 10.1.3.12.65533 > 216.239.59.99.80: tcp 0
000008 rule 25/0(match): pass out on bridge0: 10.1.3.12.65533 > 216.239.59.99.80: tcp 0
6. 193397 rule 0/0(match): block in on bridge0: 10.1.3.12.65534 > 216.239.59.99.80: tcp 0
----------------------------------------------------------------------
net.link.bridge.ipfw: 0
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 0
net.link.bridge.pfil_member: 1
net.link.bridge.pfil_bridge: 0
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_onlyip: 0
Connection doesn't work but is correctly blocked by pf, anyway.
38. 968850 rule 22/0(match): pass in on tap0: 10.1.3.12.65529 > 10.2.1.7.53: UDP, length 32
000008 rule 1/0(match): block out on re0: 10.1.3.12.65529 > 10.2.1.7.53: UDP, length 32
5. 001455 rule 1/0(match): block out on re0: 10.1.3.12.65529 > 10.2.1.7.53: UDP, length 32
----------------------------------------------------------------------
net.link.bridge.ipfw: 0
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 1
net.link.bridge.pfil_member: 0
net.link.bridge.pfil_bridge: 0
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_onlyip: 0
(works, no log)
----------------------------------------------------------------------
net.link.bridge.ipfw: 0
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 1
net.link.bridge.pfil_member: 0
net.link.bridge.pfil_bridge: 1
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_onlyip: 0
Works (explicitly allowed by pf rules).
408422 rule 23/0(match): pass in on bridge0: 10.1.3.12.65525 > 10.2.1.7.53: UDP, length 32
000007 rule 23/0(match): pass out on bridge0: 10.1.3.12.65525 > 10.2.1.7.53: UDP, length 32
003908 rule 23/0(match): pass in on bridge0: 10.1.3.12.65524 > 10.2.1.7.53: UDP, length 32
000005 rule 23/0(match): pass out on bridge0: 10.1.3.12.65524 > 10.2.1.7.53: UDP, length 32
004390 rule 25/0(match): pass in on bridge0: 10.1.3.12.65531 > 216.239.59.99.80: tcp 0
000008 rule 25/0(match): pass out on bridge0: 10.1.3.12.65531 > 216.239.59.99.80: tcp 0
----------------------------------------------------------------------
net.link.bridge.ipfw: 0
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 1
net.link.bridge.pfil_member: 1
net.link.bridge.pfil_bridge: 1
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_onlyip: 0
Note that suddenly, outbound TCP no longer works (but isn't blocked,
according to pf) and seems to go no further than the tap0 device:
18. 884422 rule 22/0(match): pass in on tap0: 10.1.3.12.65523 > 10.2.1.7.53: UDP, length 32
000008 rule 23/0(match): pass out on bridge0: 10.1.3.12.65523 > 10.2.1.7.53: UDP, length 32
003237 rule 22/0(match): pass in on tap0: 10.1.3.12.65522 > 10.2.1.7.53: UDP, length 32
000005 rule 23/0(match): pass out on bridge0: 10.1.3.12.65522 > 10.2.1.7.53: UDP, length 32
187188 rule 24/0(match): pass in on tap0: 10.1.3.12.65530 > 216.239.59.147.80: tcp 0
This suggests there's some sort of spooky interaction between pf and bridge
when those three sysctls are all enabled - connections are apparently dropped
and not logged.
That said, I can get something that works and is correctly filtered with
pfil_local_phys and pfil_bridge enabled.
> > I've tried the recent qemu-devel patch but it's so unstable that it seems
> > nearly any execution path results in a segmentation fault.
> >
> Oops :) Backtraces may be interesting there. Also which one you tried,
> the 0.11 rc2 one or the git head snapshot?
I will get to the backtraces. The version I used was the patched port in
your other thread:
http://lists.freebsd.org/pipermail/freebsd-emulation/2009-September/006760.html
Thanks for the minor enlightenment so far!
xw
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090916061858.GA70047>