From owner-freebsd-net@FreeBSD.ORG Wed Jan 16 19:47:39 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0EB1616A46D for ; Wed, 16 Jan 2008 19:47:39 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outU.internet-mail-service.net (outU.internet-mail-service.net [216.240.47.244]) by mx1.freebsd.org (Postfix) with ESMTP id E31EB13C4F3 for ; Wed, 16 Jan 2008 19:47:38 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.40) with ESMTP; Wed, 16 Jan 2008 11:47:38 -0800 Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 9E47F126F45; Wed, 16 Jan 2008 11:47:37 -0800 (PST) Message-ID: <478E5F5B.8090602@elischer.org> Date: Wed, 16 Jan 2008 11:47:39 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Michael MacLeod References: <478DAF82.8010702@elischer.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Multilink PPP Download Speeds With Round-Robin Packets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jan 2008 19:47:39 -0000 Michael MacLeod wrote: > On Jan 16, 2008 2:17 AM, Julian Elischer wrote: >> 1/ when downloading, does the load on each incoming interface >> (I assume you have one ethernet to each modem) match? > > This is a pretty rough way to estimate it, but starting/stopping > tcpdump on each of the interfaces at the same time suggests the load > is balanced down each link pretty evenly. There's a lot of inbound > traffic right now, as a friend is sending me a file right now. Most of > the other tools I'm used to working with to watch traffic only want to > work on the tun0 interface created by ppp. > >> 2/ do the IP stats show a lot of out-of order packets? >> (netstat -s -ptcp I think) >> in fact ip reassembly problems might be of interest. >> (netstat -s -pip (?)) >> 3/are there many retransmissions? > > # netstat -s -ptcp > tcp: > 144824 packets sent > 88543 data packets (117925386 bytes) > 34 data packets (45103 bytes) retransmitted > 6 data packets unnecessarily retransmitted > 0 resends initiated by MTU discovery > 49084 ack-only packets (1022 delayed) > 0 URG only packets > 0 window probe packets > 7156 window update packets > 42 control packets > 140315 packets received > 42211 acks (for 117742680 bytes) > 7009 duplicate acks > 0 acks for unsent data > 90610 packets (117590676 bytes) received in-sequence > 5665 completely duplicate packets (0 bytes) > 0 old duplicate packets > 0 packets with some dup. data (0 bytes duped) > 45 out-of-order packets (64530 bytes) > 0 packets (0 bytes) of data after window > 0 window probes > 640 window update packets > 0 packets received after close > 0 discarded for bad checksums > 0 discarded for bad header offset fields > 0 discarded because packet too short it would be interesting to see how fast the dup packet and dup ack numbers are changing.. > < snip > > > # netstat -s -pip > ip: > 2143677 total packets received > 0 bad header checksums > 0 with size smaller than minimum > 0 with data size < data length > 0 with ip length > max ip packet size > 0 with header length < data size > 0 with data length < header length > 0 with bad options > 0 with incorrect version number > 0 fragments received > 0 fragments dropped (dup or out of space) > 0 fragments dropped after timeout > 0 packets reassembled ok > 149764 packets for this host > 0 packets for unknown/unsupported protocol > 1986854 packets forwarded (0 packets fast forwarded) > 651 packets not forwardable > 0 packets received for unknown multicast group > 0 redirects sent > 156851 packets sent from this host > 0 packets sent with fabricated ip header > 0 output packets dropped due to no bufs, etc. > 0 output packets discarded due to no route > 0 output datagrams fragmented > 0 fragments created > 0 datagrams that can't be fragmented > 0 tunneling packets that can't find gif > 0 datagrams with bad address in header > > So it looks like the connection is pretty decent, though admittedly I > haven't seen what these stats should look like on a non-multilink ppp > connection. However, empirically my VoIP traffic doesn't suffer from > jitter or choppiness, so I'm going to go out on a limb and say the > connection is pretty decent. > >> 4/ what is the outgoing data bandwidth when you are downloading? > > I've been tuning my pf firewall to do outbound traffic shaping, so I'm > not swamping the ack packets in either direction. I can basically run > the pipe at 5000kb down and 1350kb up consistently. > >> 5/ have you tried mpd? (in ports (multilink ppp daemon).) >> it may have different operating characteristics.. (it does it all >> in the kernel using netgraph). Sounds like you need to fix it >> at the other end but mpd might trigger different behaviour >> in the router. > > I looked into mpd first. I tried several of the ports (mpd, mpd4, and > mpd5) all without much success. There isn't much documentation for mpd > (at least compared to the standard userland ppp). Also, both of the > individuals who I know have successfully used multilink ppp and > TekSavvy are using userland ppp. I'd be happy to use mpd, except that > I'm *almost* there with userland ppp, and I'm guessing it's something > I've overlooked or misconfigered somewhere and once I find and correct > it, it'll work like a charm. > > A bit more info about the connection: I'm using pf to do my > firewalling. My pf.conf includes some packet reassembly bits that one > of the other successful guys at dslreports sent me. > > The wan nic's are 3com (xl) cards, and my lan and dmz cards are both > Intel gigabit (em) cards. The router itself is an older dual PIII > 550Mhz rackmount server I got from my place of employment with plenty > of ram. Previously I'd used an older PIII 933Mhz computer I had lying > around, with two realtek (rl) cards for the wan nic's, and I ran into > the same problems, so I can assume it's not the hardware either. hmm I 'm not seeing much.. What happens with multiple downloads at once vs one big one..