From owner-freebsd-xen@freebsd.org Thu Nov 5 16:30:13 2015 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 501E5A2650F for ; Thu, 5 Nov 2015 16:30:13 +0000 (UTC) (envelope-from prvs=7445e0bd4=roger.pau@citrix.com) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Verizon Public SureServer CA G14-SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DDECB1C4C; Thu, 5 Nov 2015 16:30:12 +0000 (UTC) (envelope-from prvs=7445e0bd4=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.20,248,1444694400"; d="scan'208";a="310609872" Subject: Re: Checksum forwarding issue on XEN To: Larry Baird References: <20151103201250.GA92469@gta.com> <563B72B2.6060308@citrix.com> <20151105160057.GA2268@gta.com> CC: , , Wei Liu , From: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= X-Enigmail-Draft-Status: N1110 Message-ID: <563B840E.1050205@citrix.com> Date: Thu, 5 Nov 2015 17:30:06 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <20151105160057.GA2268@gta.com> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-DLP: MIA1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 16:30:13 -0000 El 05/11/15 a les 17.00, Larry Baird ha escrit: > Roger, > >> Adding the persons that contributed that code in case they can shed some >> light. >> >> El 03/11/15 a les 21.12, Larry Baird ha escrit: >>> Has anybody made any progress on "Bug 188261 - [xen] FreeBSD DomU PVHVM >>> guests cannot 'route' traffic for other Xen PV guests on same Dom0 Host." >>> (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=188261)? >>> >>> The code for checksum calculation in the function xnb_add_mbuf_cksum() looks >>> suspect. >>> >>> switch (iph->ip_p) { >>> case IPPROTO_TCP: >>> if (mbufc->m_pkthdr.csum_flags & CSUM_IP_VALID) { >>> size_t tcplen = ntohs(iph->ip_len) - sizeof(struct ip); >>> struct tcphdr *th = (struct tcphdr*)(iph + 1); >>> th->th_sum = in_pseudo(iph->ip_src.s_addr, >>> iph->ip_dst.s_addr, htons(IPPROTO_TCP + tcplen)); >>> th->th_sum = in_cksum_skip(mbufc, >>> sizeof(struct ether_header) + ntohs(iph->ip_len), >>> sizeof(struct ether_header) + (iph->ip_hl << 2)); >>> } >>> break; >>> case IPPROTO_UDP: >>> if (mbufc->m_pkthdr.csum_flags & CSUM_IP_VALID) { >>> size_t udplen = ntohs(iph->ip_len) - sizeof(struct ip); >>> struct udphdr *uh = (struct udphdr*)(iph + 1); >>> uh->uh_sum = in_pseudo(iph->ip_src.s_addr, >>> iph->ip_dst.s_addr, htons(IPPROTO_UDP + udplen)); >>> uh->uh_sum = in_cksum_skip(mbufc, >>> sizeof(struct ether_header) + ntohs(iph->ip_len), >>> sizeof(struct ether_header) + (iph->ip_hl << 2)); >>> } >>> break; >>> default: >>> break; >>> } >>> >>> >>> Both in_pseudo() and in_cksum_skip() set the same checksum. Does this >>> make since to anybody? >> >> The bug you are referring to affects FreeBSD when running as a guest >> using xen-netfront, but the code snipped above and the function >> referenced (xnb_add_mbuf_cksum) is only used on FreeBSD when running as >> a host (AKA Dom0) by xen-netback. >> >> TBH, I don't know that much about FreeBSD network subsystem to have an >> opinion, but it certainly looks weird. Patches are welcome :). > > Xyper-V has a similar forward issue. I found they were misusing csum_flags > and were always attempting to do checksum offloading if CSUM_IP_VALID was > set. I have given them a patch that fixes the issue. I was hoping that > Xen's issue was similar. I found the issue above by looking at all uses > of csum_flags in sys/dev/xen. It is hard to tell what the correct fix > is, without fulling understand the protocal used when communicating between > backend and frontend of Xen. > I am sure issue with XEN guest forwarding has to with checksum offloading. > If I am not misinterpreting your comments, I can ignore code in netback and > concentrate on code in netfront when trying to understand what is going wrong. Yes, this issue is related to netfront (sys/dev/xen/netfront/netfront.c) only, netback code is not involved. The code related to the checksum stuff is on line ~1409 for the TX side, and around line 872 for the RX side AFAICT. You can find more information about the protocol itself in sys/xen/interface/io/netif.h. Adding Wei Liu who is also doing some work to improve netfront, and knows more about the protocol than myself. Roger.