From owner-freebsd-net@FreeBSD.ORG Wed Jan 16 07:44:08 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DF1A16A417 for ; Wed, 16 Jan 2008 07:44:08 +0000 (UTC) (envelope-from mikemacleod@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9316413C457 for ; Wed, 16 Jan 2008 07:44:06 +0000 (UTC) (envelope-from mikemacleod@gmail.com) Received: by py-out-1112.google.com with SMTP id u52so252786pyb.10 for ; Tue, 15 Jan 2008 23:44:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=LE1QR3SBpIKr7C/lu6KD/SW3nTY4Rc7r+m5GFYVPpu4=; b=Q3Vt914UUKLXIXqiAIbaA2G4TvN26SpufyIYmq0oyDxzxUAgx5DsP4pTB3CIEcZQtxN4q45R027c4yMG5Pv1ChjmVUDVMwOF3W02SY6mrpppHIMkGW3MWGuowFRJ1aeqAV1QZ4QZyugGdmSPcelr3ttksMFYX3V+GE/T3mYyUHs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=FjCPxYLUtTaXOvgAN0yWqXuCBQa1CMR5Iz5Z573pb83+LwyZbDYOaEAPGSjT4WOz01imK6+N5yPlG4mqAsRVuNUSGwMKXjmkVRRhWBwoiT6FH5tUMLh8wcoMlcH+pNOR9aF5mPkkhfj1K7F+dg1DLy8ZJ2a/l3PcohtCaOmyFD8= Received: by 10.65.15.19 with SMTP id s19mr1067861qbi.17.1200469445376; Tue, 15 Jan 2008 23:44:05 -0800 (PST) Received: by 10.64.179.20 with HTTP; Tue, 15 Jan 2008 23:44:05 -0800 (PST) Message-ID: Date: Wed, 16 Jan 2008 02:44:05 -0500 From: "Michael MacLeod" To: "Julian Elischer" In-Reply-To: <478DAF82.8010702@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <478DAF82.8010702@elischer.org> Cc: freebsd-net@freebsd.org Subject: Re: Multilink PPP Download Speeds With Round-Robin Packets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jan 2008 07:44:08 -0000 On Jan 16, 2008 2:17 AM, Julian Elischer wrote: > 1/ when downloading, does the load on each incoming interface > (I assume you have one ethernet to each modem) match? This is a pretty rough way to estimate it, but starting/stopping tcpdump on each of the interfaces at the same time suggests the load is balanced down each link pretty evenly. There's a lot of inbound traffic right now, as a friend is sending me a file right now. Most of the other tools I'm used to working with to watch traffic only want to work on the tun0 interface created by ppp. > 2/ do the IP stats show a lot of out-of order packets? > (netstat -s -ptcp I think) > in fact ip reassembly problems might be of interest. > (netstat -s -pip (?)) > 3/are there many retransmissions? # netstat -s -ptcp tcp: 144824 packets sent 88543 data packets (117925386 bytes) 34 data packets (45103 bytes) retransmitted 6 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 49084 ack-only packets (1022 delayed) 0 URG only packets 0 window probe packets 7156 window update packets 42 control packets 140315 packets received 42211 acks (for 117742680 bytes) 7009 duplicate acks 0 acks for unsent data 90610 packets (117590676 bytes) received in-sequence 5665 completely duplicate packets (0 bytes) 0 old duplicate packets 0 packets with some dup. data (0 bytes duped) 45 out-of-order packets (64530 bytes) 0 packets (0 bytes) of data after window 0 window probes 640 window update packets 0 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short < snip > # netstat -s -pip ip: 2143677 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 packets reassembled ok 149764 packets for this host 0 packets for unknown/unsupported protocol 1986854 packets forwarded (0 packets fast forwarded) 651 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 156851 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header So it looks like the connection is pretty decent, though admittedly I haven't seen what these stats should look like on a non-multilink ppp connection. However, empirically my VoIP traffic doesn't suffer from jitter or choppiness, so I'm going to go out on a limb and say the connection is pretty decent. > 4/ what is the outgoing data bandwidth when you are downloading? I've been tuning my pf firewall to do outbound traffic shaping, so I'm not swamping the ack packets in either direction. I can basically run the pipe at 5000kb down and 1350kb up consistently. > 5/ have you tried mpd? (in ports (multilink ppp daemon).) > it may have different operating characteristics.. (it does it all > in the kernel using netgraph). Sounds like you need to fix it > at the other end but mpd might trigger different behaviour > in the router. I looked into mpd first. I tried several of the ports (mpd, mpd4, and mpd5) all without much success. There isn't much documentation for mpd (at least compared to the standard userland ppp). Also, both of the individuals who I know have successfully used multilink ppp and TekSavvy are using userland ppp. I'd be happy to use mpd, except that I'm *almost* there with userland ppp, and I'm guessing it's something I've overlooked or misconfigered somewhere and once I find and correct it, it'll work like a charm. A bit more info about the connection: I'm using pf to do my firewalling. My pf.conf includes some packet reassembly bits that one of the other successful guys at dslreports sent me. The wan nic's are 3com (xl) cards, and my lan and dmz cards are both Intel gigabit (em) cards. The router itself is an older dual PIII 550Mhz rackmount server I got from my place of employment with plenty of ram. Previously I'd used an older PIII 933Mhz computer I had lying around, with two realtek (rl) cards for the wan nic's, and I ran into the same problems, so I can assume it's not the hardware either.