From owner-freebsd-pf@FreeBSD.ORG Tue Feb 16 18:46:17 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF8AF10656A3 for ; Tue, 16 Feb 2010 18:46:17 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 78EF28FC20 for ; Tue, 16 Feb 2010 18:46:17 +0000 (UTC) Received: by vws20 with SMTP id 20so434734vws.13 for ; Tue, 16 Feb 2010 10:46:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=T7lzqGxlcA07Upfmcp5lJ+DReoD3esjudY94WvNZzeE=; b=T163rQByPfN9blW72VAl04TbpdMY+2yeLqk2rDCjVTo1H9rURrR3OYk3Da+gm+OcxU n5PMZyW+8UVWNh0DGuE/XwzgrhbMjlsaJFCzMYQgf+6brOth37I5XDyb3UJe4pvp+RdD /iFxuiHzdMy3XRktKVLeI17W/uL0HgXt9N9Q8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=GQ7bNfdW6PtFsWykW7/XwwK/+dMVtSK+B0QiNJfehn592a+GUp9X4nhjt5y6p6w32L aTSnJHs77odcCze2YQwNc/DKuwCYjneH7pcjcmNSKdvPz62gTKLgh3bDh1OqhA9xfj2S abhuQEM2SAyg6KWTrEsS1ZYFX3Qwlrvi2MX5w= Received: by 10.220.124.135 with SMTP id u7mr1095185vcr.231.1266344628532; Tue, 16 Feb 2010 10:23:48 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 28sm9456522vws.11.2010.02.16.10.23.46 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 16 Feb 2010 10:23:47 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 16 Feb 2010 10:23:06 -0800 From: Pyun YongHyeon Date: Tue, 16 Feb 2010 10:23:06 -0800 To: Albert Shih Message-ID: <20100216182306.GC1394@michelle.cdnetworks.com> References: <20100215211141.GK96648@obspm.fr> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="9jxsPFA5p3P2qPhR" Content-Disposition: inline In-Reply-To: <20100215211141.GK96648@obspm.fr> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, freebsd-pf@freebsd.org Subject: Re: Possible bug in TSO or in pf on bce X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:46:18 -0000 --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Feb 15, 2010 at 10:11:41PM +0100, Albert Shih wrote: > Hi all, > > I'm not a tcp/ip guru, so I don't known if it's a bug or not. > > The situation is little complexe, so I'm going to explain that. > > I've one server with tree interfaces two bce and one bge. All test is on > two bce. > > This server running FreeBSD-7.2-p6 and have lot of jail (but the problem is > the same for one jail, so I assume I've just one jail). The bce0 and bce1 > are in different vlan. > > The jail is on bce1 (meaning the jail IP is on the bce1 subnet). > > The default gateway is on bce0 > > So to make all traffic of the jail pass only throught bce1 and not using > bce0 I'm using pf with something like > > pass out route-to (bce1 bce1_subnet_gw) from jail_IP to ! bce1_subnet keep state > pass in on bce1 reply-to (bce1 bce1_subnet_gw) from ! bce1_subnet to jail_IP keep state > > if I do that all traffic pass through the right interface (bce1), but...the > bandwith drop to ~60kb/s (on gigabit interface). > > So I find the problem is with TSO, if I deactivated the TSO the bandwith is > return to normal. > > I don't knwon if it's a bug in PF (the problem is same if I use scrub or > not) or in the TSO support of bce. > At first I thought you hit one of edge case of TSO on bce(4). But it seems the issue comes from pf's route handling. When I ported pf from OpenBSD, there was no TSO capability in FreeBSD at that time so the pf_route() had no special handling code for TSO. Since it was long time ago I'm not sure whether it's correct or not but try attached patch. Apart from TSO FreeBSD got several new features like fib, flow-table and vnet. We may need to check whether these new features are still working with pf(4). --9jxsPFA5p3P2qPhR Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="pf.routeto.patch" Index: sys/contrib/pf/net/pf.c =================================================================== --- sys/contrib/pf/net/pf.c (revision 203960) +++ sys/contrib/pf/net/pf.c (working copy) @@ -6375,6 +6375,7 @@ m0->m_pkthdr.csum_flags &= ifp->if_hwassist; if (ntohs(ip->ip_len) <= ifp->if_mtu || + (m0->m_pkthdr.csum_flags & ifp->if_hwassist & CSUM_TSO) != 0 || (ifp->if_hwassist & CSUM_FRAGMENT && ((ip->ip_off & htons(IP_DF)) == 0))) { /* @@ -6449,7 +6450,7 @@ * Too large for interface; fragment if possible. * Must be able to put at least 8 bytes per fragment. */ - if (ip->ip_off & htons(IP_DF)) { + if (ip->ip_off & htons(IP_DF) || (m0->m_pkthdr.csum_flags & CSUM_TSO)) { KMOD_IPSTAT_INC(ips_cantfrag); if (r->rt != PF_DUPTO) { #ifdef __FreeBSD__ --9jxsPFA5p3P2qPhR--