From owner-freebsd-current@FreeBSD.ORG Sun Nov 18 02:04:23 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 130F316A418 for ; Sun, 18 Nov 2007 02:04:23 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.176]) by mx1.freebsd.org (Postfix) with ESMTP id D3A5513C461 for ; Sun, 18 Nov 2007 02:04:22 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: by wa-out-1112.google.com with SMTP id k17so1586871waf for ; Sat, 17 Nov 2007 18:04:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=bIwq/bcd+pffUAqgEWwLMyWhUm5H5GMe4yRWgwpCs1k=; b=pfmQY2+/4VhSNQVzH39sZRO7Y6TDxQkZlYnC6xjj8ybAmKO8nfSODR3aqhDyBNUPMMt96/bAyojfJgJqj3F3/0rY89/764hh3+B0dnwKk643hs8JmWQSpr/j33DcP5aKquD8s3zk4iwMxuAf8oBldwp2M98kP/YOCTZV0VfEkbo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=EAsp3ENxHvaae4we0EqTy++MZUsDIMa+02O9IpO039QKkcmIxFgpe7/pRzLfwvydN/80j95C93pRyQBcVvNMKG1h8im34Cs5fW73Swzp5wJUEb6suNIBiEuIH8Y4JAa0PcNZ+6lfmASe9kexCtiHEPSVj3yskJ5AJuI2fzyQq7U= Received: by 10.115.90.1 with SMTP id s1mr494985wal.1195351448328; Sat, 17 Nov 2007 18:04:08 -0800 (PST) Received: by 10.114.13.15 with HTTP; Sat, 17 Nov 2007 18:04:08 -0800 (PST) Message-ID: Date: Sat, 17 Nov 2007 18:04:08 -0800 From: "Kip Macy" To: "Mike Andrews" In-Reply-To: <473F9552.50402@bit0.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20071117003504.R31357@mindcrime.int.bit0.com> <20071117213316.499be43b@vlink.ru> <20071117170537.F59492@mindcrime.int.bit0.com> <20071117182232.T59492@mindcrime.int.bit0.com> <473F9552.50402@bit0.com> Cc: Denis Shaposhnikov , Mike Silbersack , Andre Oppermann , freebsd-current@freebsd.org Subject: Re: bizarre em + TSO + MSS issue in RELENG_7 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Nov 2007 02:04:23 -0000 On Nov 17, 2007 5:28 PM, Mike Andrews wrote: > > Kip Macy wrote: > > On Nov 17, 2007 3:23 PM, Mike Andrews wrote: > >> On Sat, 17 Nov 2007, Kip Macy wrote: > >> > >>> On Nov 17, 2007 2:33 PM, Mike Andrews wrote: > >>>> On Sat, 17 Nov 2007, Kip Macy wrote: > >>>> > >>>>> On Nov 17, 2007 10:33 AM, Denis Shaposhnikov wrote: > >>>>>> On Sat, 17 Nov 2007 00:42:54 -0500 (EST) > >>>>>> Mike Andrews wrote: > >>>>>> > >>>>>>> Has anyone run into problems with MSS not being respected when using > >>>>>>> TSO, specifically on em cards? > >>>>>> Yes, I wrote about this problem on the beginning of 2007, see > >>>>>> > >>>>>> http://tinyurl.com/3e5ak5 > >>>>>> > >>>>> if_em.c:3502 > >>>>> /* > >>>>> * Payload size per packet w/o any headers. > >>>>> * Length of all headers up to payload. > >>>>> */ > >>>>> TXD->tcp_seg_setup.fields.mss = htole16(mp->m_pkthdr.tso_segsz); > >>>>> TXD->tcp_seg_setup.fields.hdr_len = hdr_len; > >>>>> > >>>>> > >>>>> Please print out the value of tso_segsz here. It appears to be being > >>>>> set correctly. The only thing I can think of is that t_maxopd is not > >>>>> correct. As tso_segsz is correct here: > >>>> > >>>> It repeatedly prints 1368 during a 1 meg file transfer over a connection > >>>> with a 1380 MSS. Any other printf's I can add? I'm working on a web page > >>>> with tcpdump / firewall log output illustrating the issue... > >>> Mike - > >>> Denis' tcpdump output doesn't show oversized segments, something else > >>> appears to be happening there. Can you post your tcpdump output > >>> somewhere? > >> > >> URL sent off-list. > > > > if (tso) { > > m->m_pkthdr.csum_flags = CSUM_TSO; > > m->m_pkthdr.tso_segsz = tp->t_maxopd - optlen; > > } > > > > > > Please print the value of maxopd and optlen under "if (tso)" in > > tcp_output. I think the calculated optlen may be too small. > > > maxopt=1380 - optlen=12 = tso_segsz=1368 > > Weird though, after this reboot, I had to re-copy a 4 meg file 5 times > to start getting the firewall to log any drops. Transfer rate was > around 240KB/sec before the firewall started to drop, then it went down > to about 64KB/sec during the 5th copy, and stayed there for subsequent > copies. The actual packet size the firewall said it was dropping was > varying all over the place still, yet the maxopt/optlen/tso_segsz values > stayed constant. But it's interesting that it didn't start dropping > immediately after the reboot -- though the transfer rate was still > sub-optimal. Ok, next theory :D. You shouldn't be seeing "bad len" packets from tcpdump. I'm wondering if that means you're sending down more than 64k. Can you please print out the value of mp->m_pkthdr.len around the same place that you printed out tso_segsz? 64k is the generally accepted limit for TSO, I'm wondering if the card firmware does something weird if you give it more. Thanks. -Kip