From owner-svn-src-head@FreeBSD.ORG Tue Oct 30 02:25:45 2012 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 485849B6; Tue, 30 Oct 2012 02:25:45 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id E91408FC08; Tue, 30 Oct 2012 02:25:44 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id rp8so4914043pbb.13 for ; Mon, 29 Oct 2012 19:25:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=6EzEyETw8/byRhcC1Lwp4LefEWR5u0Zbj8WPps0Mny8=; b=wnN0iDFYAOguuZxWzhhE6GZkn78tHkI1i1Q2pjY15RyQHCxeQNrGitEPxVCfHcZZWZ imZSMT6UGKNOvtiX0t6IeWWixgvcvW+PYP7mrg751Tyeokl117PvffwRdaUEg6PrLMAl 323za9Qif86QUi3oMwL4jd2XlO6aBH3Hu3bvNnl/mCA41pGKPkP0aUHGnJu/1Us4aNvf m49ngqpHrV0JANabR7lGJTkmCduPDw8vV6KzJavFCv5auDcm7UTLCXfTTYHjnonuRhlo OUv7zEooKn+4/zd51GFwP1sr9yRDhANIwQNF3XPmKyaasKyEcslKPwBmOr7Fip31cKCd UYlw== Received: by 10.68.204.2 with SMTP id ku2mr98713956pbc.59.1351563944507; Mon, 29 Oct 2012 19:25:44 -0700 (PDT) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPS id j8sm6885067paz.30.2012.10.29.19.25.38 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 29 Oct 2012 19:25:41 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 30 Oct 2012 11:25:07 +0900 From: YongHyeon PYUN Date: Tue, 30 Oct 2012 11:25:07 +0900 To: Andre Oppermann Subject: Re: svn commit: r242161 - in head/sys: net netinet netpfil/pf Message-ID: <20121030022507.GB4298@michelle.cdnetworks.com> References: <201210262106.q9QL6YgY000943@svn.freebsd.org> <508BBE6C.7010409@freebsd.org> <20121027220137.GJ70741@FreeBSD.org> <20121029204104.GA1431@michelle.cdnetworks.com> <20121029052100.GO70741@FreeBSD.org> <20121029214038.GD1431@michelle.cdnetworks.com> <508E3C6B.2090102@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <508E3C6B.2090102@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, Gleb Smirnoff , src-committers@freebsd.org X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: pyunyh@gmail.com List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 02:25:45 -0000 On Mon, Oct 29, 2012 at 09:20:59AM +0100, Andre Oppermann wrote: > On 29.10.2012 22:40, YongHyeon PYUN wrote: > >On Mon, Oct 29, 2012 at 09:21:00AM +0400, Gleb Smirnoff wrote: > >>On Mon, Oct 29, 2012 at 01:41:04PM -0700, YongHyeon PYUN wrote: > >>Y> On Sun, Oct 28, 2012 at 02:01:37AM +0400, Gleb Smirnoff wrote: > >>Y> > On Sat, Oct 27, 2012 at 12:58:52PM +0200, Andre Oppermann wrote: > >>Y> > A> On 26.10.2012 23:06, Gleb Smirnoff wrote: > >>Y> > A> > Author: glebius > >>Y> > A> > Date: Fri Oct 26 21:06:33 2012 > >>Y> > A> > New Revision: 242161 > >>Y> > A> > URL: http://svn.freebsd.org/changeset/base/242161 > >>Y> > A> > > >>Y> > A> > Log: > >>Y> > A> > o Remove last argument to ip_fragment(), and obtain all > >>needed information > >>Y> > A> > on checksums directly from mbuf flags. This simplifies > >>code. > >>Y> > A> > o Clear CSUM_IP from the mbuf in ip_fragment() if we did > >>checksums in > >>Y> > >>Y> I'm not sure whether ti(4)'s checksum offloading for IP fragmented > >>Y> packets(CSUM_IP_FRAGS) still works after this change. ti(4) > >>Y> requires CSUM_IP should be set for IP fragmented packets. Not sure > >>Y> whether it's a bug or not. I have a ti(4) controller but I don't > >>Y> remember where I can find it and don't have a link > >>Y> parter(1000baseSX) to test it. :-( > >> > >>ti(4) declares both CSUM_IP and CSUM_IP_FRAGS, so ip_fragment() won't do > > > >Because it supports both CSUM_IP and CSUM_IP_FRAGS. Probably ti(4) > >is the only controller that supports TCP/UDP checksum offloading > >for an IP fragmented packet. > > This is a bit weird if it doesn't do the fragmentation itself. > Computing the IP header checksum doesn't differ for normal and > fragmented packets. The protocol checksum (TCP or UDP) stays > the same for in the case of IP level fragmentation. It is only > visible in the first fragment which includes the protocol header. My interpretation for CSUM_IP_FRAGS works like the following. - Only peuso header checksum for TCP/UDP is computed by upper stack. - Controller has no ability to fragment the packet so it should done in upper stack(i.e. ip_output()). - When ip_output() has to fragment the packet, it just fragments the packet without completing TCP/UDP and IP checksum. If controller does not support CSUM_IP_FRAGS feature, ip_output() can't delay TCP/UDP checksum in this stage. - The fragmented packets are sent to driver. Driver sets appropriate bits of DMA descriptor based on fragmentation field of mbuf(M_FRAG, M_LASTFRAG) and issue the frame to controller. - The firmware of controller queues the fragmented frames up in its internal memory and hold off sending out the frames since it has to compute TCP/UDP checksum. When it sees a frame which indicates the end of fragmented frame it finally computes TCP/UDP checksum and send each frame out to wire by computing IP checksum on the fly. The difference is which one(upper stack vs. controller) computes TCP/UDP/IP checksum. > > >>software checksums, and thus won't clear these flags. > >> > >>Potentially a driver that announces one flag in if_hwassist but relies on > >>couple of flags to be set on mbuf is not correct. If a driver can't do > >>single > >>checksum processing independently from others, then it should set or > >>clear > >>appropriate flags in if_hwassist as a group. > > > >Hmm, then what would be best way to achieve CSUM_IP_FRAGS in > >driver? I don't have clear idea how to utilize the hardware > >feature. The stack should tell that the mbuf needs TCP/UDP checksum > >offloading for IP fragmented packet(i.e. CSUM_IP_FRAGS is not set by > >upper stack). > > As I said there can't be fragment checksumming without hardware It's up to controller's firmware. It does not send the fragmented frame until it computes TCP/UDP checksum. > based fragmentation. We have three cases here: > > 1. TSO where the hardware does the segmentation, TCP and IP header > checksums for each generated packet. > 2. IP packet fragmentation where a packet is split, the IP header > checksum is recomputed for each fragment, but the protocol csum > stays the same and is not modified. > 3. UDP fragmentation where a large packet is sent to the hardware > and it generates first the UDP checksum and then splits it into > IP fragments each with its own IP header checksum. > > So we end up with these possible large send hardware offload capabilities: > TSO: including IPv4hdr and TCP checksumming > UDP fragmentation: including IPv4hdr and UDP checksumming > IP fragmentation: including IPv4hdr checksumming > > Besides that we have the packet <= MTU sized offload capabilities: > TCP checksumming > UDP checksumming > SCTP checksumming > IPv4hdr checksumming > > >>Y> > A> > hardware. Some driver may not announce CSUM_IP in theur > >>if_hwassist, > >> ^^^^^^^^ > >> > >>Oh, that was a typo! Software was meant. > > That explains quite a bit of confusion. > > -- > Andre >