From owner-freebsd-xen@freebsd.org Wed Nov 11 17:04:41 2015 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8DD7A2C56D for ; Wed, 11 Nov 2015 17:04:41 +0000 (UTC) (envelope-from prvs=7503460ff=wei.liu2@citrix.com) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Verizon Public SureServer CA G14-SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C8151B82; Wed, 11 Nov 2015 17:04:41 +0000 (UTC) (envelope-from prvs=7503460ff=wei.liu2@citrix.com) X-IronPort-AV: E=Sophos;i="5.20,276,1444694400"; d="scan'208";a="311610458" Date: Wed, 11 Nov 2015 17:04:31 +0000 From: Wei Liu To: Roger Pau =?iso-8859-1?Q?Monn=E9?= CC: Larry Baird , , , Wei Liu , Subject: Re: Checksum forwarding issue on XEN Message-ID: <20151111170431.GG10274@zion.uk.xensource.com> References: <20151103201250.GA92469@gta.com> <563B72B2.6060308@citrix.com> <20151105160057.GA2268@gta.com> <563B840E.1050205@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <563B840E.1050205@citrix.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-DLP: MIA2 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2015 17:04:41 -0000 On Thu, Nov 05, 2015 at 05:30:06PM +0100, Roger Pau Monné wrote: > El 05/11/15 a les 17.00, Larry Baird ha escrit: > > Roger, > > > >> Adding the persons that contributed that code in case they can shed some > >> light. > >> > >> El 03/11/15 a les 21.12, Larry Baird ha escrit: > >>> Has anybody made any progress on "Bug 188261 - [xen] FreeBSD DomU PVHVM > >>> guests cannot 'route' traffic for other Xen PV guests on same Dom0 Host." > >>> (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=188261)? > >>> > >>> The code for checksum calculation in the function xnb_add_mbuf_cksum() looks > >>> suspect. > >>> > >>> switch (iph->ip_p) { > >>> case IPPROTO_TCP: > >>> if (mbufc->m_pkthdr.csum_flags & CSUM_IP_VALID) { > >>> size_t tcplen = ntohs(iph->ip_len) - sizeof(struct ip); > >>> struct tcphdr *th = (struct tcphdr*)(iph + 1); > >>> th->th_sum = in_pseudo(iph->ip_src.s_addr, > >>> iph->ip_dst.s_addr, htons(IPPROTO_TCP + tcplen)); > >>> th->th_sum = in_cksum_skip(mbufc, > >>> sizeof(struct ether_header) + ntohs(iph->ip_len), > >>> sizeof(struct ether_header) + (iph->ip_hl << 2)); > >>> } > >>> break; > >>> case IPPROTO_UDP: > >>> if (mbufc->m_pkthdr.csum_flags & CSUM_IP_VALID) { > >>> size_t udplen = ntohs(iph->ip_len) - sizeof(struct ip); > >>> struct udphdr *uh = (struct udphdr*)(iph + 1); > >>> uh->uh_sum = in_pseudo(iph->ip_src.s_addr, > >>> iph->ip_dst.s_addr, htons(IPPROTO_UDP + udplen)); > >>> uh->uh_sum = in_cksum_skip(mbufc, > >>> sizeof(struct ether_header) + ntohs(iph->ip_len), > >>> sizeof(struct ether_header) + (iph->ip_hl << 2)); > >>> } > >>> break; > >>> default: > >>> break; > >>> } > >>> > >>> > >>> Both in_pseudo() and in_cksum_skip() set the same checksum. Does this > >>> make since to anybody? > >> > >> The bug you are referring to affects FreeBSD when running as a guest > >> using xen-netfront, but the code snipped above and the function > >> referenced (xnb_add_mbuf_cksum) is only used on FreeBSD when running as > >> a host (AKA Dom0) by xen-netback. > >> > >> TBH, I don't know that much about FreeBSD network subsystem to have an > >> opinion, but it certainly looks weird. Patches are welcome :). > > > > Xyper-V has a similar forward issue. I found they were misusing csum_flags > > and were always attempting to do checksum offloading if CSUM_IP_VALID was > > set. I have given them a patch that fixes the issue. I was hoping that > > Xen's issue was similar. I found the issue above by looking at all uses > > of csum_flags in sys/dev/xen. It is hard to tell what the correct fix > > is, without fulling understand the protocal used when communicating between > > backend and frontend of Xen. > > > > > I am sure issue with XEN guest forwarding has to with checksum offloading. > > If I am not misinterpreting your comments, I can ignore code in netback and > > concentrate on code in netfront when trying to understand what is going wrong. > > Yes, this issue is related to netfront (sys/dev/xen/netfront/netfront.c) > only, netback code is not involved. The code related to the checksum > stuff is on line ~1409 for the TX side, and around line 872 for the RX > side AFAICT. > > You can find more information about the protocol itself in > sys/xen/interface/io/netif.h. > > Adding Wei Liu who is also doing some work to improve netfront, and > knows more about the protocol than myself. > Sorry for the late reply. I think the place to look at, as Roger suggested, is netfront checksum setup code. The relevant flags in Xen network protocol are: XEN_NETTXF_csum_blank -- protocol checksum field is blank XEN_NETTXF_data_validated -- data has been validated I'm not entirely sure FreeBSD netfront is mapping its own CSUM_* flags correctly to Xen's. I think at some point I can look into it, but not in the near future. Wei. > Roger.